This is a full account of the steps I ran to get llama.cpp running on the Nvidia Jetson Nano 2GB. It accumulates multiple different fixes and tutorials, whose contributions are referenced at the bottom of this README.
At a high level, the procedure to install llama.cpp on a Jetson Nano consists of 3 steps.
gcc 8.5 compiler from source.
| deb http://archive.ubuntu.com/ubuntu/ focal main restricted universe multiverse | |
| deb-src http://archive.ubuntu.com/ubuntu/ focal main restricted universe multiverse | |
| deb http://archive.ubuntu.com/ubuntu/ focal-updates main restricted universe multiverse | |
| deb-src http://archive.ubuntu.com/ubuntu/ focal-updates main restricted universe multiverse | |
| deb http://archive.ubuntu.com/ubuntu/ focal-security main restricted universe multiverse | |
| deb-src http://archive.ubuntu.com/ubuntu/ focal-security main restricted universe multiverse | |
| deb http://archive.ubuntu.com/ubuntu/ focal-backports main restricted universe multiverse |
| Key Sublime Text 3.2.1 Build 3207 | |
| ----- BEGIN LICENSE ----- | |
| Member J2TeaM | |
| Single User License | |
| EA7E-1011316 | |
| D7DA350E 1B8B0760 972F8B60 F3E64036 | |
| B9B4E234 F356F38F 0AD1E3B7 0E9C5FAD | |
| FA0A2ABE 25F65BD8 D51458E5 3923CE80 | |
| 87428428 79079A01 AA69F319 A1AF29A4 | |
| A684C2DC 0B1583D4 19CBD290 217618CD |
Intel OpenCL stack enablement on Ubuntu 18.04LTS.
Here's how to build and install Intel's new OpenCL enablement stack, Neo, which replaces the legacy Beignet ICD.
Install the dependencies:
Build dependencies:
sudo apt-get install ccache flex bison clang-4.0 cmake g++ git patch zlib1g-dev autoconf xutils-dev libtool pkg-config libpciaccess-dev
frontend localnodes
bind *:80
reqadd X-Forwarded-Proto:\ http
# Add CORS headers when Origin header is present
capture request header origin len 128
http-response add-header Access-Control-Allow-Origin %[capture.req.hdr(0)] if { capture.req.hdr(0) -m found }
rspadd Access-Control-Allow-Methods:\ GET,\ HEAD,\ OPTIONS,\ POST,\ PUT if { capture.req.hdr(0) -m found }
rspadd Access-Control-Allow-Credentials:\ true if { capture.req.hdr(0) -m found }
Tuning Intel Skylake and beyond for optimal performance and feature level support on Linux:
Note that on Skylake, Kabylake (and the now cancelled "Broxton") SKUs, functionality such as power saving, GPU scheduling and HDMI audio have been moved onto binary-only firmware, and as such, the GuC and the HuC blobs must be loaded at run-time to access this functionality.
Enabling GuC and HuC on Skylake and above requires a few extra parameters be passed to the kernel before boot.
Instructions provided for both Fedora and Ubuntu (including Debian):
Note that the firmware for these GPUs is often packaged by your distributor, and as such, you can confirm the firmware blob's availability by running:
Transcoding FLAC music to Opus:
ffmpeg is a highly useful application for converting music and videos. However, audio transcoding is limited to a a single core. If you have a large FLAC archive and you wanted to compress it into the efficient Opus codec, it would take forever with the fastest processor to complete, unless you were to take advantage of all cores in your CPU.
parallel 'ffmpeg -v 0 -i "{}" -c:a libopus -b:a 128k "{.}.opus"' ::: $(find -type f -name '*.flac')
Transcoding Videos to VP9:
FFMPEG filters provide a powerful way to programmatically enhance or alter videos, and it’s fairly simple to add a watermark to a video using the overlay filter. The easiest way to install ffmpeg is to download a pre-built binary for your specific platform. Then you don’t have to worry about including and installing all the right dependencies and codecs you will be using.
Once you have ffmpeg installed, adding a watermark is as easy as passing your existing source through an overlay filter like so:
ffmpeg -i test.mp4 -i watermark.png -filter_complex "overlay=10:10" test1.mp4
Basically, we’re passing in the original video, and an overlay image as inputs, then passing it through the filter, and saving the output as test1.mp4.
| # input: fullchain.pem and privkey.pem as generated by the "letsencrypt-auto" script when run with | |
| # the "auth" aka "certonly" subcommand | |
| # convert certificate chain + private key to the PKCS#12 file format | |
| openssl pkcs12 -export -out keystore.pkcs12 -in fullchain.pem -inkey privkey.pem | |
| # convert PKCS#12 file into Java keystore format | |
| keytool -importkeystore -srckeystore keystore.pkcs12 -srcstoretype PKCS12 -destkeystore keystore.jks | |
| # don't need the PKCS#12 file anymore |