Skip to content

Instantly share code, notes, and snippets.

View johnnynunez's full-sized avatar
👨‍💻
@NVIDIA AI Developer

Johnny johnnynunez

👨‍💻
@NVIDIA AI Developer
View GitHub Profile

Install VLLM Spark

  1. Install uv
curl -LsSf https://astral.sh/uv/install.sh | sh
  1. Create environment
sudo apt install python3-dev
title Run VLLM Thor & Spark

Run VLLM Thor & Spark

  1. Install uv
curl -LsSf https://astral.sh/uv/install.sh | sh

Install Nvidia GPU and CUDA

1. Install Graphic Cards Drivers Nvidia:

sudo apt-get remove --purge '^nvidia-.*'
sudo apt-get --purge remove nvidia*
sudo apt-get --purge remove "*cublas*" "cuda*"
sudo apt remove nvidia-*
sudo apt-get --purge remove "*nvidia*"
sudo apt autoremove -y

Control your GPU on Linux

1. NVIDIA

1. Install drivers

2. Configuration

2.1 Generate config File: /etc/X11/xorg.conf

sudo nvidia-xconfig

Running SGLang Multi-Node Cluster on DGX Spark + Jetson Thor

Cluster Layout

Machine Hostname IP GPU Role
DGX Spark 1 spark-ea8e 10.0.0.25 NVIDIA GB10 (sm_120, 128GB) HEAD
DGX Spark 2 spark-4080 10.0.0.22 NVIDIA GB10 (sm_120, 128GB) Worker
Jetson Thor 1 jetson-thor 10.0.0.27 NVIDIA Thor (sm_110, 128GB) Worker
Jetson Thor 2 thorx 10.0.0.26 NVIDIA Thor (sm_110, 128GB) Worker

CUDA 13.2 on Jetson Orin (JetPack 6)

Problem

PyTorch nightly built for CUDA 13.2 (cu132) fails on Jetson Orin running JetPack 6.2.2, because the bundled NVIDIA driver (540.5.0) only supports up to CUDA 12.6:

RuntimeError: The NVIDIA driver on your system is too old (found version 12060).

Docker

1. Install Docker

sudo apt install curl

## Uninstall docker

sudo apt-get remove docker docker-engine docker.io containerd runc

Tutorial: Install IsaacSim 6.0 and IsaacLab adapted to 6.0

sudo apt-get update && sudo apt-get install -y gcc-11 g++-11
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-11 200
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-11 200

Remove .cache if you come from old isaac

cd

Run SGLang Thor & Spark

  1. Install uv
curl -LsSf https://astral.sh/uv/install.sh | sh
  1. Create environment
uv venv .sglang --python 3.12

NemoClaw on Jetson Orin Family -- Setup Guide

System Info

Component Version
Board NVIDIA Jetson Orin
JetPack 6.2.2 (L4T R36.5.0)
Kernel 5.15.185-tegra
OS Ubuntu 22.04.5 LTS (aarch64)