Skip to content

Instantly share code, notes, and snippets.

@a10y
Created April 27, 2025 20:50
Show Gist options
  • Save a10y/eff6b611b2c9ff7c6f18e1d1794a0ea2 to your computer and use it in GitHub Desktop.
Save a10y/eff6b611b2c9ff7c6f18e1d1794a0ea2 to your computer and use it in GitHub Desktop.
Get NanoLLM working on AGX Orin JetPack 6.2
FROM dustynv/nano_llm:r36.4.0
RUN apt update && apt install -y gstreamer1.0-nice
@a10y
Copy link
Author

a10y commented Apr 27, 2025

It turns out that getting NanoLLM working on JetPack 6.2 (latest) is not completely straightforward. I was running into issues of the form

[webrtc] new WebRTC peer connecting (100.79.74.112, peer_id=0)
 window**
ERROR:/opt/jetson-utils/codec/gstEncoder.cpp:876:static void gstEncoder::onWebsocketMessage(WebRTCPeer*, const char*, size_t, void*): 'sinkpad' should not be nullptr
Bail out! ERROR:/opt/jetson-utils/codec/gstEncoder.cpp:876:static void gstEncoder::onWebsocketMessage(WebRTCPeer*, const char*, size_t, void*): 'sinkpad' should not be nullptr
Fatal Python error: Aborted

This apparently stopped working after JetPack 6.1 for some reason.

Someone on GH Issues said that installing gstreamer1.0-nice fixes the issue.

To verify yourself:

  1. Download this `Dockerfile to a directory
  2. docker build . and note the result as IMAGE_SHA
  3. Launch the image using the same command-line that jetson-containers would prepare for the original image:
docker run --runtime nvidia \
	--env NVIDIA_DRIVER_CAPABILITIES=compute,utility,graphics \
	-it \
	--rm \
	--network host \
	--shm-size=8g \
	--volume /tmp/argus_socket:/tmp/argus_socket \
	--volume /etc/enctune.conf:/etc/enctune.conf \
	--volume /etc/nv_tegra_release:/etc/nv_tegra_release \
	--volume /tmp/nv_jetson_model:/tmp/nv_jetson_model \
	--volume /var/run/dbus:/var/run/dbus \
	--volume /var/run/avahi-daemon/socket:/var/run/avahi-daemon/socket \
	--volume /var/run/docker.sock:/var/run/docker.sock \
	--volume /home/artoo/code/jetson-containers/data:/data \
	-v /etc/localtime:/etc/localtime:ro \
	-v /etc/timezone:/etc/timezone:ro \
	--device /dev/snd \
	-e PULSE_SERVER=unix:/run/user/1000/pulse/native \
	-v /run/user/1000/pulse:/run/user/1000/pulse \
	--device /dev/bus/usb \
	--device /dev/video0 \
	--device /dev/video1 \
	--device /dev/i2c-0 \
	--device /dev/i2c-1 \
	--device /dev/i2c-2 \
	--device /dev/i2c-3 \
	--device /dev/i2c-4 \
	--device /dev/i2c-5 \
	--device /dev/i2c-6 \
	--device /dev/i2c-7 \
	--device /dev/i2c-8 \
	--device /dev/i2c-9 \
	--name nanollm \
	$IMAGE_SHA  python3 -m nano_llm.agents.video_query --api=mlc --model Efficient-Large-Model/VILA1.5-3b --max-context-len 256 --max-new-tokens 32 --video-input /dev/video0 --video-output webrtc://@:8554/output

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment