rajivmehtaflex

mkdir ollama-quntization
cd ollama-quntization/
bash <(curl -sSL https://g.bodaay.io/hfd) -h
./hfdownloader -m TinyLlama/TinyLlama-1.1B-Chat-v1.0
sudo apt install tree
tree
touch Modelfile
ollama create -f Modelfile tinyllama
ollama cp tinyllama rajivmehtapy/tinyllama
ollama list

rajivmehtaflex

--model
/workspace/codesandbox-template-blank/llamafile/AutoCoder_S_6.gguf
--server
--host
0.0.0.0
-ngl
100
...

rajivmehtaflex

Converting HuggingFace Models to GGUF/GGML
Downloading a HuggingFace model
Running llama.cpp convert.py on the HuggingFace model
(Optionally) Uploading the model back to HuggingFace
Downloading a HuggingFace model
There are various ways to download models, but in my experience the huggingface_hub library has been the most reliable. The git clone method occasionally results in OOM errors for large models.

Install the huggingface_hub library:

pip install huggingface_hub

rajivmehtaflex

#https://tcude.net/creating-cloudflare-tunnels-on-ubuntu/
#https://github.com/ollama/ollama/blob/main/docs/faq.md#how-can-i-use-ollama-with-ngrok
#------------------------------------
wget -q https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-linux-amd64.deb
dpkg -i cloudflared-linux-amd64.deb
cloudflared tunnel --url http://localhost:11434 --http-host-header="localhost:11434"

rajivmehtaflex

Running a MongoDB container with data export
This script demonstrates how to set up and use a MongoDB container to export data.
Pull the mongo image:
docker pull mongo

docker volume create mymongodata

Activate docker Container
docker container run -d --name some-mongo -e MONGO_INITDB_ROOT_USERNAME=mongoadmin -e MONGO_INITDB_ROOT_PASSWORD=mongopasswd -v mymongodata:/data/db -p 27017:27017 mongo

rajivmehtaflex

apt install mysql-server
service mysql start
mysql -e "ALTER USER 'root'@'localhost' IDENTIFIED WITH 'mysql_native_password' BY 'root';FLUSH PRIVILEGES;"
#username/password:root/root
mysql -u root -p

rajivmehtaflex

curl https://gist.githubusercontent.com/rajivmehtaflex/a1e6de2355e7a16bcd9f38f43a58794b/raw/0636ecab2e4f83c441dbc1b55b562850c3e66859/conda_install.sh | sh
conda config --set auto_activate_base false
conda create --name crewexp python=3.10 -y
conda activate crewexp
pip install poetry
cd gdata/

rajivmehtaflex

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source $HOME/.cargo/env
cargo --version
sudo apt install libssl-dev
sudo apt install pkg-config
mkdir ~/.cache/huggingface
touch ~/.cache/huggingface/token
echo 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 > ~/.cache/huggingface/token
git clone https://github.com/EricLBuehler/mistral.rs.git
cd mistral.rs

rajivmehtaflex

#——Install Package———
pip install -U "huggingface_hub[cli]"
huggingface-cli login
HF_HUB_ENABLE_HF_TRANSFER=1

#———Download Package———
huggingface-cli download QuantFactory/Meta-Llama-3-120B-Instruct-GGUF --local-dir . --local-dir-use-symlinks False --include='*Q4_K*gguf'

#————Merge it————————

rajivmehtaflex

import concurrent.futures
from typing import Any, Callable, List
from tqdm import tqdm

def concurrent_task_executor(task: Callable[[Any], None], data_list: List[Any], max_workers: int = 32, description: str = None) -> None:
    """
    Execute tasks concurrently on a list of data objects using ThreadPoolExecutor.

    Args:
        task (Callable): The function to apply to each data object.