Skip to content

Instantly share code, notes, and snippets.

@rajivmehtaflex
Last active May 10, 2024 15:34
Show Gist options
  • Save rajivmehtaflex/765f1959cac6703c02341f424dea0aa1 to your computer and use it in GitHub Desktop.
Save rajivmehtaflex/765f1959cac6703c02341f424dea0aa1 to your computer and use it in GitHub Desktop.
LLM on Fire
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source $HOME/.cargo/env
cargo --version
sudo apt install libssl-dev
sudo apt install pkg-config
mkdir ~/.cache/huggingface
touch ~/.cache/huggingface/token
echo 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 > ~/.cache/huggingface/token
git clone https://github.com/EricLBuehler/mistral.rs.git
cd mistral.rs
cargo build --release
cp ./target/release/mistralrs-server ./mistralrs_server
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment