Skip to content

Instantly share code, notes, and snippets.

@insideone

insideone/example.sh

Created November 23, 2024 14:27
Show Gist options
  • Save insideone/f12f80e08daa0b45c828203ae4e54053 to your computer and use it in GitHub Desktop.
Save insideone/f12f80e08daa0b45c828203ae4e54053 to your computer and use it in GitHub Desktop.
Download ZIP
NixOS: Using GPU in docker (use --device=.../gpu=all instead of --gpu=all)
Raw
example.sh
#!/usr/bin/env bash
# key point: use `--device=nvidia.com/gpu=all` instead of `--gpu=all` on nixos!
docker run --rm -ti --name local-ai -p 8080:8080 --device nvidia.com/gpu=all localai/localai:latest-gpu-nvidia-cuda-12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment