qlora mlx-examples mistral fine tune

gistfile1.txt

#
#
#  Instructions can be found https://github.com/ml-explore/mlx-examples/tree/main/lora
#
#
user@mbp lora % curl -O https://files.mistral-7b-v0-1.mistral.ai/mistral-7B-v0.1.tar

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 13.4G  100 13.4G    0     0  29.1M      0  0:07:53  0:07:53 --:--:-- 31.6M
user@mbp lora % tar -xf mistral-7B-v0.1.tar

user@mbp lora % python3 convert.py --torch-path mistral-7B-v0.1 --mlx-path q_converted_mistral -q
[INFO] Quantizing

user@mbp lora % python3 lora.py --model q_converted_mistral --train --iters 600
Loading pretrained model
Total parameters 1244.041M
Trainable parameters 1.704M
Loading datasets
Training
Iter 1: Val loss 2.439, Val took 66.208s
Iter 10: Train loss 2.415, It/sec 0.253, Tokens/sec 101.631
Iter 20: Train loss 2.266, It/sec 0.238, Tokens/sec 96.615
Iter 30: Train loss 2.146, It/sec 0.228, Tokens/sec 93.642
Iter 40: Train loss 1.914, It/sec 0.253, Tokens/sec 98.129
Iter 50: Train loss 1.642, It/sec 0.247, Tokens/sec 98.293
Iter 60: Train loss 1.444, It/sec 0.248, Tokens/sec 98.317
Iter 70: Train loss 1.435, It/sec 0.216, Tokens/sec 87.692
Iter 80: Train loss 1.440, It/sec 0.230, Tokens/sec 92.337
Iter 90: Train loss 1.399, It/sec 0.264, Tokens/sec 100.153
Iter 100: Train loss 1.409, It/sec 0.250, Tokens/sec 95.332
Iter 110: Train loss 1.298, It/sec 0.262, Tokens/sec 101.486
Iter 120: Train loss 1.289, It/sec 0.241, Tokens/sec 94.295
Iter 130: Train loss 1.283, It/sec 0.242, Tokens/sec 95.730
Iter 140: Train loss 1.305, It/sec 0.246, Tokens/sec 95.735
Iter 150: Train loss 1.296, It/sec 0.226, Tokens/sec 91.946
Iter 160: Train loss 1.230, It/sec 0.240, Tokens/sec 94.467
Iter 170: Train loss 1.237, It/sec 0.241, Tokens/sec 99.233
Iter 180: Train loss 1.226, It/sec 0.263, Tokens/sec 95.467
Iter 190: Train loss 1.241, It/sec 0.244, Tokens/sec 97.599
Iter 200: Train loss 1.326, It/sec 0.250, Tokens/sec 95.227

python3 lora.py --model q_converted_mistral --adapter-file adapters.npz --test
Loading pretrained model
Total parameters 1244.041M
Trainable parameters 1.704M
Loading datasets
Testing
Test loss 1.382, Test ppl 3.984.

python3 lora.py --model q_converted_mistral --adapter-file adapters.npz --num-tokens 50 --prompt "table: 1-10015132-16
columns: Player, No., Nationality, Position, Years in Toronto, School/Club Team
Q: What is terrence ross' nationality
A: "
Loading pretrained model
Total parameters 1244.041M
Trainable parameters 1.704M
Loading datasets
Generating
table: 1-10015132-16
columns: Player, No., Nationality, Position, Years in Toronto, School/Club Team
Q: What is terrence ross' nationality
A: SELECT MAX No. FROM 1-10015132-16 WHERE Nationality = 'Canadian'� columns: Player, No., Nationality, Position, Years in Toronto, School/Club