ccache not found, please consider installing for faster builds### ctest_debug
Runs ctest in debug mode
- status: 0
+ ctest -C Debug --output-on-failure -L main -E 'test-opt|test-backend-ops|test-llama-archs'
Test project /root/llama.me/build-ci-debug
Start 1: test-tokenizer-0-bert-bge
1/42 Test #1: test-tokenizer-0-bert-bge ......... Passed 0.11 sec
Start 2: test-tokenizer-0-command-r
2/42 Test #2: test-tokenizer-0-command-r ........ Passed 1.64 sec
Start 3: test-tokenizer-0-deepseek-coder
3/42 Test #3: test-tokenizer-0-deepseek-coder ... Passed 0.26 sec
Start 4: test-tokenizer-0-deepseek-llm
4/42 Test #4: test-tokenizer-0-deepseek-llm ..... Passed 0.64 sec
Start 5: test-tokenizer-0-falcon
5/42 Test #5: test-tokenizer-0-falcon ........... Passed 0.40 sec
Start 6: test-tokenizer-0-gemma-4
6/42 Test #6: test-tokenizer-0-gemma-4 .......... Passed 1.86 sec
Start 7: test-tokenizer-0-gpt-2
7/42 Test #7: test-tokenizer-0-gpt-2 ............ Passed 0.31 sec
Start 8: test-tokenizer-0-llama-bpe
8/42 Test #8: test-tokenizer-0-llama-bpe ........ Passed 1.02 sec
Start 9: test-tokenizer-0-llama-spm
9/42 Test #9: test-tokenizer-0-llama-spm ........ Passed 0.11 sec
Start 10: test-tokenizer-0-mpt
10/42 Test #10: test-tokenizer-0-mpt .............. Passed 0.31 sec
Start 11: test-tokenizer-0-phi-3
11/42 Test #11: test-tokenizer-0-phi-3 ............ Passed 0.11 sec
Start 12: test-tokenizer-0-qwen2
12/42 Test #12: test-tokenizer-0-qwen2 ............ Passed 0.89 sec
Start 13: test-tokenizer-0-qwen35
13/42 Test #13: test-tokenizer-0-qwen35 ........... Passed 0.89 sec
Start 14: test-tokenizer-0-refact
14/42 Test #14: test-tokenizer-0-refact ........... Passed 0.30 sec
Start 15: test-tokenizer-0-starcoder
15/42 Test #15: test-tokenizer-0-starcoder ........ Passed 0.31 sec
Start 16: test-tokenizers-ggml-vocabs
16/42 Test #16: test-tokenizers-ggml-vocabs ....... Passed 0.50 sec
Start 17: test-sampling
17/42 Test #17: test-sampling ..................... Passed 3.86 sec
Start 18: test-reasoning-budget
18/42 Test #18: test-reasoning-budget ............. Passed 0.01 sec
Start 19: test-grammar-parser
19/42 Test #19: test-grammar-parser ............... Passed 0.00 sec
Start 20: test-grammar-integration
20/42 Test #20: test-grammar-integration .......... Passed 0.07 sec
Start 21: test-llama-grammar
21/42 Test #21: test-llama-grammar ................ Passed 0.01 sec
Start 22: test-chat
22/42 Test #22: test-chat ......................... Passed 19.56 sec
Start 23: test-json-schema-to-grammar
23/42 Test #23: test-json-schema-to-grammar ....... Passed 2.03 sec
Start 24: test-tokenizer-1-llama-spm
24/42 Test #24: test-tokenizer-1-llama-spm ........ Passed 0.84 sec
Start 25: test-chat-peg-parser
25/42 Test #25: test-chat-peg-parser .............. Passed 0.24 sec
Start 26: test-jinja
26/42 Test #26: test-jinja ........................ Passed 0.51 sec
Start 28: test-chat-auto-parser
27/42 Test #28: test-chat-auto-parser ............. Passed 2.51 sec
Start 29: test-chat-template
28/42 Test #29: test-chat-template ................ Passed 0.51 sec
Start 30: test-json-partial
29/42 Test #30: test-json-partial ................. Passed 0.01 sec
Start 31: test-log
30/42 Test #31: test-log .......................... Passed 0.02 sec
Start 32: test-peg-parser
31/42 Test #32: test-peg-parser ................... Passed 0.10 sec
Start 33: test-regex-partial
32/42 Test #33: test-regex-partial ................ Passed 0.01 sec
Start 34: test-download-model
33/42 Test #34: test-download-model ............... Passed 1.97 sec
Start 35: test-thread-safety
34/42 Test #35: test-thread-safety ................ Passed 1.67 sec
Start 36: test-arg-parser
35/42 Test #36: test-arg-parser ................... Passed 0.15 sec
Start 37: test-gguf
36/42 Test #37: test-gguf ......................... Passed 0.21 sec
Start 44: test-barrier
37/42 Test #44: test-barrier ...................... Passed 2.00 sec
Start 45: test-quantize-fns
38/42 Test #45: test-quantize-fns ................. Passed 1.87 sec
Start 46: test-quantize-perf
39/42 Test #46: test-quantize-perf ................ Passed 0.24 sec
Start 47: test-rope
40/42 Test #47: test-rope ......................... Passed 0.06 sec
Start 48: test-mtmd-c-api
41/42 Test #48: test-mtmd-c-api ................... Passed 0.01 sec
Start 49: test-alloc
42/42 Test #49: test-alloc ........................ Passed 0.00 sec
100% tests passed, 0 tests failed out of 42
Label Time Summary:
main = 46.18 sec*proc (41 tests)
Total Test time (real) = 48.17 sec
real 0m48,178s
user 1m45,190s
sys 0m1,528s
ccache not found, please consider installing for faster builds### ctest_release
Runs ctest in release mode
- status: 0
+ ctest -C Release --output-on-failure -L 'main|python'
Test project /root/llama.me/build-ci-release
Start 1: test-tokenizer-0-bert-bge
1/45 Test #1: test-tokenizer-0-bert-bge ......... Passed 0.03 sec
Start 2: test-tokenizer-0-command-r
2/45 Test #2: test-tokenizer-0-command-r ........ Passed 0.43 sec
Start 3: test-tokenizer-0-deepseek-coder
3/45 Test #3: test-tokenizer-0-deepseek-coder ... Passed 0.05 sec
Start 4: test-tokenizer-0-deepseek-llm
4/45 Test #4: test-tokenizer-0-deepseek-llm ..... Passed 0.14 sec
Start 5: test-tokenizer-0-falcon
5/45 Test #5: test-tokenizer-0-falcon ........... Passed 0.08 sec
Start 6: test-tokenizer-0-gemma-4
6/45 Test #6: test-tokenizer-0-gemma-4 .......... Passed 0.57 sec
Start 7: test-tokenizer-0-gpt-2
7/45 Test #7: test-tokenizer-0-gpt-2 ............ Passed 0.06 sec
Start 8: test-tokenizer-0-llama-bpe
8/45 Test #8: test-tokenizer-0-llama-bpe ........ Passed 0.25 sec
Start 9: test-tokenizer-0-llama-spm
9/45 Test #9: test-tokenizer-0-llama-spm ........ Passed 0.03 sec
Start 10: test-tokenizer-0-mpt
10/45 Test #10: test-tokenizer-0-mpt .............. Passed 0.06 sec
Start 11: test-tokenizer-0-phi-3
11/45 Test #11: test-tokenizer-0-phi-3 ............ Passed 0.03 sec
Start 12: test-tokenizer-0-qwen2
12/45 Test #12: test-tokenizer-0-qwen2 ............ Passed 0.20 sec
Start 13: test-tokenizer-0-qwen35
13/45 Test #13: test-tokenizer-0-qwen35 ........... Passed 0.19 sec
Start 14: test-tokenizer-0-refact
14/45 Test #14: test-tokenizer-0-refact ........... Passed 0.06 sec
Start 15: test-tokenizer-0-starcoder
15/45 Test #15: test-tokenizer-0-starcoder ........ Passed 0.06 sec
Start 16: test-tokenizers-ggml-vocabs
16/45 Test #16: test-tokenizers-ggml-vocabs ....... Passed 0.47 sec
Start 17: test-sampling
17/45 Test #17: test-sampling ..................... Passed 0.86 sec
Start 18: test-reasoning-budget
18/45 Test #18: test-reasoning-budget ............. Passed 0.00 sec
Start 19: test-grammar-parser
19/45 Test #19: test-grammar-parser ............... Passed 0.00 sec
Start 20: test-grammar-integration
20/45 Test #20: test-grammar-integration .......... Passed 0.01 sec
Start 21: test-llama-grammar
21/45 Test #21: test-llama-grammar ................ Passed 0.00 sec
Start 22: test-chat
22/45 Test #22: test-chat ......................... Passed 2.11 sec
Start 23: test-json-schema-to-grammar
23/45 Test #23: test-json-schema-to-grammar ....... Passed 1.91 sec
Start 24: test-tokenizer-1-llama-spm
24/45 Test #24: test-tokenizer-1-llama-spm ........ Passed 0.11 sec
Start 25: test-llama-archs
25/45 Test #25: test-llama-archs .................. Passed 4.37 sec
Start 26: test-chat-peg-parser
26/45 Test #26: test-chat-peg-parser .............. Passed 0.02 sec
Start 27: test-jinja
27/45 Test #27: test-jinja ........................ Passed 0.06 sec
Start 28: test-jinja-py
28/45 Test #28: test-jinja-py ..................... Passed 11.41 sec
Start 29: test-chat-auto-parser
29/45 Test #29: test-chat-auto-parser ............. Passed 0.28 sec
Start 30: test-chat-template
30/45 Test #30: test-chat-template ................ Passed 0.06 sec
Start 31: test-json-partial
31/45 Test #31: test-json-partial ................. Passed 0.00 sec
Start 32: test-log
32/45 Test #32: test-log .......................... Passed 0.01 sec
Start 33: test-peg-parser
33/45 Test #33: test-peg-parser ................... Passed 0.01 sec
Start 34: test-regex-partial
34/45 Test #34: test-regex-partial ................ Passed 0.00 sec
Start 35: test-download-model
35/45 Test #35: test-download-model ............... Passed 2.15 sec
Start 36: test-thread-safety
36/45 Test #36: test-thread-safety ................ Passed 0.25 sec
Start 37: test-arg-parser
37/45 Test #37: test-arg-parser ................... Passed 0.08 sec
Start 38: test-gguf
38/45 Test #38: test-gguf ......................... Passed 0.06 sec
Start 39: test-backend-ops
39/45 Test #39: test-backend-ops .................. Passed 0.01 sec
Start 46: test-barrier
40/45 Test #46: test-barrier ...................... Passed 0.27 sec
Start 47: test-quantize-fns
41/45 Test #47: test-quantize-fns ................. Passed 1.05 sec
Start 48: test-quantize-perf
42/45 Test #48: test-quantize-perf ................ Passed 0.05 sec
Start 49: test-rope
43/45 Test #49: test-rope ......................... Passed 0.04 sec
Start 50: test-mtmd-c-api
44/45 Test #50: test-mtmd-c-api ................... Passed 0.00 sec
Start 51: test-alloc
45/45 Test #51: test-alloc ........................ Passed 0.00 sec
100% tests passed, 0 tests failed out of 45
Label Time Summary:
main = 14.34 sec*proc (43 tests)
python = 11.41 sec*proc (1 test)
Total Test time (real) = 27.92 sec
real 0m27,932s
user 0m48,059s
sys 0m3,839s
BGE Small (BERT):
- status: 0
- f16:
+ ./bin/llama-embedding --model ../models-mnt/bge-small/ggml-model-f16.gguf -p 'I believe the meaning of life is' -ngl 99 -c 0 --no-op-offload
warning: no usable GPU found, --gpu-layers option will be ignored
warning: one possible reason is that llama.cpp was compiled without GPU support
warning: consult docs/build.md for compilation instructions
0.00.001.751 I main: n_parallel == 1 -> unified KV cache is enabled
0.00.001.767 I common_init_result: fitting params to device memory ...
0.00.001.768 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.00.037.794 I common_params_fit_impl: projected to use 69 MiB of host memory vs. 77898 MiB of total host memory
0.00.052.091 W load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
0.00.079.876 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.00.093.002 I
0.00.093.087 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.094.049 I batch_decode: n_tokens = 9, n_seq = 1
embedding 0: -0.043960 -0.019931 0.007712 -0.000751 0.001344 -0.036999 0.109418 0.042582 0.092107 -0.015945 0.006784 -0.035670 -0.017907 0.015017 0.018081 0.015926 -0.011260 0.010490 -0.085274 -0.008459 0.091390 -0.017088 -0.060390 -0.024478 0.027519 0.076056 0.027986 -0.014574 0.017650 -0.033289 -0.037877 -0.019006 0.068682 -0.009854 -0.025034 0.072386 -0.046555 0.011033 -0.050286 0.047710 0.032437 -0.011726 0.022050 0.049604 0.010482 0.005781 -0.028890 0.008935 -0.018469 -0.051451 -0.046061 0.030487 -0.035441 0.054225 -0.069635 0.044224 0.029798 0.046302 0.073453 -0.042601 0.076094 0.038833 -0.181188 0.082507 0.042264 -0.064571 -0.060111 -0.017840 0.006475 0.005886 0.017158 -0.026637 0.064610 0.112595 0.035123 -0.067396 0.027069 -0.067289 -0.033462 -0.033188 0.033237 0.013539 -0.003328 -0.037494 -0.052071 0.055166 -0.002017 -0.038255 0.064464 0.028839 -0.043310 -0.029211 -0.039469 0.036347 0.008376 -0.015456 -0.036623 0.018123 0.028586 0.342812 -0.044411 0.056112 0.017662 -0.020885 -0.066804 0.000129 -0.037850 -0.030067 -0.008578 -0.021541 0.000520 -0.003230 0.004019 0.018876 -0.008559 0.025835 0.049417 0.000103 0.050908 -0.042491 -0.031902 0.023618 0.030714 -0.023199 -0.046255 -0.079230 0.115155 0.046709 0.027811 -0.040727 0.067800 -0.022945 0.010306 -0.032945 -0.018295 0.043837 0.024285 0.052386 0.007493 0.008896 0.011247 -0.074669 -0.065547 -0.026802 -0.041168 -0.023887 0.026716 0.006903 0.027724 0.052860 -0.036699 0.057693 -0.000230 0.031761 -0.019740 -0.022072 0.041010 -0.058929 0.019603 0.043151 0.043615 0.041618 -0.022525 0.027063 -0.021866 0.005424 -0.041325 -0.001270 0.024436 0.002108 0.044329 -0.022718 0.043624 0.064766 0.055420 0.037081 -0.000899 0.046123 0.045816 -0.008500 0.063064 -0.073204 -0.011986 0.032123 0.023920 0.014668 -0.033710 0.001100 -0.015855 -0.019007 0.047854 0.110846 0.028421 0.031361 -0.013281 -0.057457 0.006676 0.005131 -0.012261 -0.051530 -0.000929 -0.017667 -0.019422 -0.040945 0.009207 -0.057959 0.050955 0.052410 -0.009574 -0.040242 -0.014044 -0.024869 -0.017278 0.006302 0.006588 -0.026899 0.015624 0.030757 0.002582 0.023228 -0.022214 -0.098589 -0.051112 -0.278050 -0.014963 -0.061568 -0.027236 0.017703 -0.010968 -0.017083 0.035028 0.046984 -0.015453 0.015220 -0.025476 0.047851 -0.005930 -0.000693 -0.061024 -0.068902 -0.060359 -0.035918 0.043342 -0.055018 0.015045 0.000518 -0.058211 -0.010436 0.012567 0.151515 0.127119 -0.013597 0.041975 -0.025644 0.014008 -0.001054 -0.150439 0.044871 0.005309 -0.036256 -0.029792 -0.020171 -0.034870 0.010267 0.033580 -0.048145 -0.051801 -0.017462 -0.023506 0.047393 0.052029 -0.016825 -0.055483 0.025833 -0.005685 0.010733 0.038723 0.008216 -0.009770 -0.105720 -0.027463 -0.096157 0.025080 -0.011272 0.092367 0.056073 0.003696 0.027793 0.002038 -0.051117 -0.039891 -0.013549 -0.044969 -0.015316 0.002913 -0.043493 -0.077962 0.065198 -0.006816 -0.001589 -0.014637 0.071582 0.023727 -0.037145 0.009170 0.001555 -0.032286 0.015427 0.037899 0.000311 -0.053185 0.021337 -0.039844 0.000057 0.013402 0.019822 -0.057872 0.006451 -0.049526 -0.267880 0.039165 -0.067962 0.038210 -0.012318 0.041493 -0.016126 0.052432 -0.071385 0.011312 0.024689 -0.007221 0.082055 0.028516 -0.021516 0.040582 -0.004526 -0.074635 -0.014780 0.019998 0.002250 0.023125 0.197151 -0.043246 -0.025972 -0.004938 -0.019236 0.074234 0.001751 -0.031946 -0.036566 -0.045049 0.000551 -0.011591 0.018178 -0.029435 -0.008457 0.006441 0.050757 -0.014900 0.006181 0.026086 -0.030793 0.048078 0.114039 -0.040842 -0.011476 0.005396 -0.003585 0.025123 -0.059147 0.013750 -0.010379 0.038713 0.051474 0.035410 0.035039 -0.017055 0.026369 -0.014503 -0.050014 0.003251 0.054109 0.039720 -0.039147
real 0m0,109s
user 0m0,179s
sys 0m0,034s
- q8_0:
+ ./bin/llama-embedding --model ../models-mnt/bge-small/ggml-model-q8_0.gguf -p 'I believe the meaning of life is' -ngl 99 -c 0 --no-op-offload
warning: no usable GPU found, --gpu-layers option will be ignored
warning: one possible reason is that llama.cpp was compiled without GPU support
warning: consult docs/build.md for compilation instructions
0.00.001.629 I main: n_parallel == 1 -> unified KV cache is enabled
0.00.001.638 I common_init_result: fitting params to device memory ...
0.00.001.639 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.00.038.033 I common_params_fit_impl: projected to use 40 MiB of host memory vs. 77898 MiB of total host memory
0.00.052.143 W load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
0.00.077.834 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.00.087.543 I
0.00.087.621 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.088.625 I batch_decode: n_tokens = 9, n_seq = 1
embedding 0: -0.044769 -0.020293 0.008312 -0.001913 0.002951 -0.036361 0.109152 0.042860 0.091707 -0.015266 0.006223 -0.036915 -0.018787 0.014956 0.017155 0.014664 -0.013389 0.010356 -0.085124 -0.007760 0.092818 -0.017684 -0.061927 -0.024778 0.027992 0.076285 0.027446 -0.014928 0.017682 -0.033781 -0.038384 -0.017640 0.068602 -0.010417 -0.023592 0.072539 -0.046256 0.010643 -0.051011 0.049985 0.032044 -0.012679 0.021985 0.050196 0.010734 0.005425 -0.027972 0.008380 -0.018950 -0.052749 -0.045274 0.028617 -0.035570 0.052332 -0.068761 0.043724 0.029066 0.045856 0.073537 -0.043214 0.075989 0.038740 -0.182764 0.082396 0.043152 -0.065902 -0.059464 -0.017057 0.006858 0.004254 0.016762 -0.027090 0.066127 0.112956 0.033962 -0.067310 0.026852 -0.066913 -0.034776 -0.034429 0.033462 0.014505 -0.003978 -0.037120 -0.051974 0.054514 -0.003063 -0.037339 0.063217 0.028557 -0.042307 -0.028936 -0.039699 0.036995 0.007952 -0.014911 -0.035936 0.018657 0.030676 0.344795 -0.044852 0.056983 0.017340 -0.021102 -0.063378 -0.000059 -0.037958 -0.029928 -0.008731 -0.020472 0.001127 -0.004184 0.004079 0.018009 -0.009781 0.024267 0.048652 -0.001033 0.051251 -0.041336 -0.030359 0.023049 0.030682 -0.023500 -0.045304 -0.078945 0.114234 0.048049 0.027377 -0.041017 0.067538 -0.022838 0.011250 -0.033847 -0.016738 0.044509 0.022050 0.052398 0.007432 0.006576 0.010436 -0.075216 -0.064472 -0.026449 -0.041538 -0.024660 0.026852 0.006176 0.026528 0.051708 -0.037361 0.058659 0.001662 0.032153 -0.019548 -0.021730 0.041598 -0.059513 0.019495 0.042868 0.042108 0.040890 -0.022224 0.028439 -0.022892 0.006937 -0.041591 0.000471 0.024398 0.001802 0.043274 -0.023550 0.043495 0.064700 0.056079 0.038828 -0.000302 0.047336 0.045703 -0.009004 0.060762 -0.072220 -0.011673 0.033569 0.023710 0.014491 -0.033240 0.001091 -0.016882 -0.017800 0.047999 0.110772 0.028647 0.030924 -0.010618 -0.056152 0.005752 0.004509 -0.011819 -0.051905 -0.002408 -0.017285 -0.020138 -0.040943 0.009639 -0.058830 0.051337 0.051765 -0.010950 -0.039967 -0.015824 -0.024824 -0.015123 0.005632 0.006604 -0.027472 0.015294 0.031295 0.002561 0.023642 -0.021440 -0.098875 -0.050686 -0.277322 -0.013596 -0.060241 -0.027209 0.017223 -0.008998 -0.017215 0.034745 0.048475 -0.016189 0.015411 -0.023170 0.049729 -0.005031 0.000517 -0.060754 -0.069485 -0.060687 -0.035264 0.045340 -0.056031 0.014686 -0.000107 -0.059174 -0.010550 0.010932 0.150441 0.126988 -0.012928 0.043261 -0.026197 0.014554 -0.001012 -0.148888 0.043045 0.005783 -0.035040 -0.029486 -0.019778 -0.034115 0.009486 0.035182 -0.048500 -0.052967 -0.016393 -0.024666 0.048341 0.050784 -0.017477 -0.056926 0.023685 -0.005170 0.011758 0.038203 0.006412 -0.008393 -0.105756 -0.028217 -0.097797 0.024940 -0.011161 0.091588 0.056131 0.004987 0.027713 0.001477 -0.050231 -0.038585 -0.013572 -0.046539 -0.015327 0.002897 -0.043732 -0.077053 0.065700 -0.006396 -0.000981 -0.015114 0.070910 0.024637 -0.036404 0.008641 0.001068 -0.033026 0.016985 0.037254 0.000248 -0.052019 0.020569 -0.038401 -0.000725 0.012999 0.019775 -0.058274 0.004633 -0.049796 -0.268352 0.039090 -0.067125 0.036869 -0.011389 0.043082 -0.016243 0.050856 -0.071612 0.012399 0.024082 -0.007879 0.083606 0.029305 -0.021130 0.042402 -0.003813 -0.073862 -0.014481 0.019215 0.001791 0.023348 0.196663 -0.044314 -0.026124 -0.004931 -0.019014 0.074743 0.001652 -0.032484 -0.037155 -0.045355 0.001014 -0.011051 0.017752 -0.026512 -0.009489 0.005867 0.050566 -0.014158 0.005610 0.027206 -0.031022 0.048065 0.112428 -0.040627 -0.011999 0.004289 -0.002847 0.025315 -0.060347 0.014617 -0.010438 0.037544 0.049858 0.036079 0.036533 -0.017197 0.025771 -0.014269 -0.049203 0.003926 0.053891 0.040036 -0.038903
real 0m0,102s
user 0m0,145s
sys 0m0,036s
Rerank Tiny (Jina):
- status: 0
- f16:
+ ./bin/llama-embedding --model ../models-mnt/rerank-tiny/ggml-model-f16.gguf -p 'what is panda?\thi\nwhat is panda?\tit'\''s a bear\nwhat is panda?\tThe giant panda (Ailuropoda melanoleuca), sometimes called a panda bear or simply panda, is a bear species endemic to China.' -ngl 99 -c 0 --pooling rank --embd-normalize -1 --no-op-offload --verbose-prompt
warning: no usable GPU found, --gpu-layers option will be ignored
warning: one possible reason is that llama.cpp was compiled without GPU support
warning: consult docs/build.md for compilation instructions
0.00.002.494 I main: n_parallel == 1 -> unified KV cache is enabled
0.00.002.505 I common_init_result: fitting params to device memory ...
0.00.002.505 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.00.271.161 I common_params_fit_impl: projected to use 115 MiB of host memory vs. 77898 MiB of total host memory
0.00.286.548 W load: empty token at index 5
0.00.290.813 W load: model vocab missing newline token, using special_pad_id instead
0.00.508.456 W llama_init_from_model: model default pooling_type is [-1], but [4] was specified
0.00.531.030 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.00.540.564 I
0.00.540.650 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.540.829 I main: prompt 0: 'what is panda? hi'
0.00.540.831 I main: number of tokens in prompt = 9
0 -> '<s>'
21381 -> 'what'
21152 -> 'is'
49990 -> 'panda'
61 -> '?'
2 -> '</s>'
2 -> '</s>'
23233 -> 'hi'
2 -> '</s>'
0.00.540.838 I main: prompt 1: 'what is panda? it's a bear'
0.00.540.838 I main: number of tokens in prompt = 13
0 -> '<s>'
21381 -> 'what'
21152 -> 'is'
49990 -> 'panda'
61 -> '?'
2 -> '</s>'
2 -> '</s>'
21153 -> 'it'
37 -> '''
87 -> 's'
69 -> 'a'
25706 -> 'bear'
2 -> '</s>'
0.00.540.840 I main: prompt 2: 'what is panda? The giant panda (Ailuropoda melanoleuca), sometimes called a panda bear or simply panda, is a bear species endemic to China.'
0.00.540.841 I main: number of tokens in prompt = 40
0 -> '<s>'
21381 -> 'what'
21152 -> 'is'
49990 -> 'panda'
61 -> '?'
2 -> '</s>'
2 -> '</s>'
21215 -> 'he'
28390 -> 'giant'
49990 -> 'panda'
38 -> '('
21163 -> 'il'
26237 -> 'uro'
21223 -> 'po'
23179 -> 'da'
36906 -> 'melan'
26791 -> 'ole'
89 -> 'u'
21402 -> 'ca'
21686 -> '),'
23314 -> 'sometimes'
22517 -> 'called'
69 -> 'a'
49990 -> 'panda'
25706 -> 'bear'
21142 -> 'or'
22810 -> 'simply'
49990 -> 'panda'
42 -> ','
21152 -> 'is'
69 -> 'a'
25706 -> 'bear'
25677 -> 'species'
28930 -> 'ende'
22024 -> 'mic'
21148 -> 'to'
76 -> 'h'
22344 -> 'ina'
44 -> '.'
2 -> '</s>'
0.00.540.912 I batch_decode: n_tokens = 62, n_seq = 3
rerank score 0: 0.023
rerank score 1: 0.025
rerank score 2: 0.199
real 0m0,563s
user 0m0,579s
sys 0m0,078s
- rerank score 0 @ 0.023 OK
- rerank score 1 @ 0.025 OK
- rerank score 2 @ 0.199 OK
Runs test scripts
- status: 0
+ cd ./tools/gguf-split
+ bash tests.sh /root/llama.me/build-ci-release/bin /root/llama.me/tmp/mnt/models
+ SPLIT=/root/llama.me/build-ci-release/bin/llama-gguf-split
+ MAIN=/root/llama.me/build-ci-release/bin/llama-completion
+ WORK_PATH=/root/llama.me/tmp/mnt/models/gguf-split
+++ dirname tests.sh
++ realpath ./../../
+ ROOT_DIR=/root/llama.me
+ mkdir -p /root/llama.me/tmp/mnt/models/gguf-split
+ rm -f '/root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split*.gguf' '/root/llama.me/tmp/mnt/models/gguf-split/ggml-model-merge*.gguf'
+ cd /root/llama.me/tmp/mnt/models/gguf-split
+ /root/llama.me/scripts/hf.sh --repo ggml-org/Qwen3-0.6B-GGUF --file Qwen3-0.6B-Q8_0.gguf
[+] attempting to download Qwen3-0.6B-Q8_0.gguf
[+] wget -q -c -O ./Qwen3-0.6B-Q8_0.gguf https://huggingface.co/ggml-org/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B-Q8_0.gguf
./Qwen3-0.6B-Q8_0.gguf
+ echo PASS
PASS
+ /root/llama.me/build-ci-release/bin/llama-gguf-split --split-max-tensors 28 /root/llama.me/tmp/mnt/models/gguf-split/Qwen3-0.6B-Q8_0.gguf /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split
n_split: 12
split 00001: n_tensors = 28, total_size = 371M
split 00002: n_tensors = 28, total_size = 45M
split 00003: n_tensors = 28, total_size = 40M
split 00004: n_tensors = 28, total_size = 43M
split 00005: n_tensors = 28, total_size = 42M
split 00006: n_tensors = 28, total_size = 44M
split 00007: n_tensors = 28, total_size = 39M
split 00008: n_tensors = 28, total_size = 45M
split 00009: n_tensors = 28, total_size = 39M
split 00010: n_tensors = 28, total_size = 44M
split 00011: n_tensors = 28, total_size = 42M
split 00012: n_tensors = 3, total_size = 6M
Writing file /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00001-of-00012.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00002-of-00012.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00003-of-00012.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00004-of-00012.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00005-of-00012.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00006-of-00012.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00007-of-00012.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00008-of-00012.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00009-of-00012.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00010-of-00012.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00011-of-00012.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00012-of-00012.gguf ... gguf_split: 12 gguf split written with a total of 311 tensors.
done
+ echo PASS
PASS
+ echo
+ /root/llama.me/build-ci-release/bin/llama-completion -no-cnv --model /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00001-of-00012.gguf -p 'I believe the meaning of life is' --n-predict 32
0.00.001.768 I llama_completion: llama backend init
0.00.001.773 I llama_completion: load the model and apply lora adapter, if any
0.00.001.779 I common_init_result: fitting params to device memory ...
0.00.001.780 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.00.184.861 I common_params_fit_impl: projected to use 5542 MiB of host memory vs. 77898 MiB of total host memory
0.00.283.379 W load: control-looking token: 128247 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
0.01.968.347 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.02.047.906 I llama_completion: llama threadpool init, n_threads = 8
0.02.047.924 I
0.02.047.984 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.02.047.985 I
0.02.048.054 I sampler seed: 1465155874
0.02.048.062 I sampler params:
repeat_last_n = 64, repeat_penalty = 1,000, frequency_penalty = 0,000, presence_penalty = 0,000
dry_multiplier = 0,000, dry_base = 1,750, dry_allowed_length = 2, dry_penalty_last_n = -1
top_k = 40, top_p = 0,950, min_p = 0,050, xtc_probability = 0,000, xtc_threshold = 0,100, typical_p = 1,000, top_n_sigma = -1,000, temp = 0,800
mirostat = 0, mirostat_lr = 0,100, mirostat_ent = 5,000, adaptive_target = -1,000, adaptive_decay = 0,900
0.02.048.067 I sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
0.02.048.068 I generate: n_ctx = 40960, n_batch = 2048, n_predict = 32, n_keep = 0
0.02.048.068 I
I believe the meaning of life is to find one's true self, and I can't see myself as such. I feel this is a problem of the mind, which is separate from the external
0.02.453.878 I common_perf_print: sampling time = 4,82 ms
0.02.453.879 I common_perf_print: samplers time = 1,68 ms / 39 tokens
0.02.453.883 I common_perf_print: load time = 1824,32 ms
0.02.453.884 I common_perf_print: prompt eval time = 20,72 ms / 7 tokens ( 2,96 ms per token, 337,81 tokens per second)
0.02.453.885 I common_perf_print: eval time = 379,52 ms / 31 runs ( 12,24 ms per token, 81,68 tokens per second)
0.02.453.885 I common_perf_print: total time = 405,98 ms / 38 tokens
0.02.453.885 I common_perf_print: unaccounted time = 0,91 ms / 0,2 % (total - sampling - prompt eval - eval) / (total)
0.02.453.886 I common_perf_print: graphs reused = 30
+ echo PASS
PASS
+ echo
+ /root/llama.me/build-ci-release/bin/llama-gguf-split --merge /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00001-of-00012.gguf /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-merge.gguf
gguf_merge: /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00001-of-00012.gguf -> /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-merge.gguf
gguf_merge: reading metadata /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00001-of-00012.gguf ...�[3Ddone
gguf_merge: reading metadata /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00002-of-00012.gguf ...�[3Ddone
gguf_merge: reading metadata /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00003-of-00012.gguf ...�[3Ddone
gguf_merge: reading metadata /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00004-of-00012.gguf ...�[3Ddone
gguf_merge: reading metadata /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00005-of-00012.gguf ...�[3Ddone
gguf_merge: reading metadata /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00006-of-00012.gguf ...�[3Ddone
gguf_merge: reading metadata /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00007-of-00012.gguf ...�[3Ddone
gguf_merge: reading metadata /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00008-of-00012.gguf ...�[3Ddone
gguf_merge: reading metadata /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00009-of-00012.gguf ...�[3Ddone
gguf_merge: reading metadata /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00010-of-00012.gguf ...�[3Ddone
gguf_merge: reading metadata /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00011-of-00012.gguf ...�[3Ddone
gguf_merge: reading metadata /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00012-of-00012.gguf ...�[3Ddone
gguf_merge: writing tensors /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00001-of-00012.gguf ...�[3Ddone
gguf_merge: writing tensors /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00002-of-00012.gguf ...�[3Ddone
gguf_merge: writing tensors /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00003-of-00012.gguf ...�[3Ddone
gguf_merge: writing tensors /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00004-of-00012.gguf ...�[3Ddone
gguf_merge: writing tensors /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00005-of-00012.gguf ...�[3Ddone
gguf_merge: writing tensors /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00006-of-00012.gguf ...�[3Ddone
gguf_merge: writing tensors /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00007-of-00012.gguf ...�[3Ddone
gguf_merge: writing tensors /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00008-of-00012.gguf ...�[3Ddone
gguf_merge: writing tensors /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00009-of-00012.gguf ...�[3Ddone
gguf_merge: writing tensors /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00010-of-00012.gguf ...�[3Ddone
gguf_merge: writing tensors /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00011-of-00012.gguf ...�[3Ddone
gguf_merge: writing tensors /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00012-of-00012.gguf ...�[3Ddone
gguf_merge: /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-merge.gguf merged from 12 split with 311 tensors.
+ echo PASS
PASS
+ echo
+ /root/llama.me/build-ci-release/bin/llama-completion -no-cnv --model /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-merge.gguf -p 'I believe the meaning of life is' --n-predict 32
0.00.001.773 I llama_completion: llama backend init
0.00.001.777 I llama_completion: load the model and apply lora adapter, if any
0.00.001.784 I common_init_result: fitting params to device memory ...
0.00.001.785 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.00.196.093 I common_params_fit_impl: projected to use 5542 MiB of host memory vs. 77898 MiB of total host memory
0.00.279.046 W load: control-looking token: 128247 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
0.01.976.584 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.02.057.590 I llama_completion: llama threadpool init, n_threads = 8
0.02.057.609 I
0.02.057.665 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.02.057.666 I
0.02.057.733 I sampler seed: 774352655
0.02.057.740 I sampler params:
repeat_last_n = 64, repeat_penalty = 1,000, frequency_penalty = 0,000, presence_penalty = 0,000
dry_multiplier = 0,000, dry_base = 1,750, dry_allowed_length = 2, dry_penalty_last_n = -1
top_k = 40, top_p = 0,950, min_p = 0,050, xtc_probability = 0,000, xtc_threshold = 0,100, typical_p = 1,000, top_n_sigma = -1,000, temp = 0,800
mirostat = 0, mirostat_lr = 0,100, mirostat_ent = 5,000, adaptive_target = -1,000, adaptive_decay = 0,900
0.02.057.749 I sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
0.02.057.750 I generate: n_ctx = 40960, n_batch = 2048, n_predict = 32, n_keep = 0
0.02.057.750 I
I believe the meaning of life is to pursue what is important and achieve what is necessary. That is to say, we should always look for the meaning we find and pursue it. However, the
0.02.462.880 I common_perf_print: sampling time = 4,67 ms
0.02.462.881 I common_perf_print: samplers time = 1,62 ms / 39 tokens
0.02.462.885 I common_perf_print: load time = 1822,33 ms
0.02.462.886 I common_perf_print: prompt eval time = 16,54 ms / 7 tokens ( 2,36 ms per token, 423,11 tokens per second)
0.02.462.887 I common_perf_print: eval time = 383,09 ms / 31 runs ( 12,36 ms per token, 80,92 tokens per second)
0.02.462.887 I common_perf_print: total time = 405,30 ms / 38 tokens
0.02.462.888 I common_perf_print: unaccounted time = 1,00 ms / 0,2 % (total - sampling - prompt eval - eval) / (total)
0.02.462.888 I common_perf_print: graphs reused = 30
+ echo PASS
PASS
+ echo
+ /root/llama.me/build-ci-release/bin/llama-gguf-split --split-max-tensors 32 --no-tensor-first-split /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-merge.gguf /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-32-tensors
n_split: 11
split 00001: n_tensors = 0, total_size = 5M
split 00002: n_tensors = 32, total_size = 370M
split 00003: n_tensors = 32, total_size = 49M
split 00004: n_tensors = 32, total_size = 50M
split 00005: n_tensors = 32, total_size = 47M
split 00006: n_tensors = 32, total_size = 47M
split 00007: n_tensors = 32, total_size = 50M
split 00008: n_tensors = 32, total_size = 50M
split 00009: n_tensors = 32, total_size = 49M
split 00010: n_tensors = 32, total_size = 46M
split 00011: n_tensors = 23, total_size = 36M
Writing file /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-32-tensors-00001-of-00011.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-32-tensors-00002-of-00011.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-32-tensors-00003-of-00011.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-32-tensors-00004-of-00011.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-32-tensors-00005-of-00011.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-32-tensors-00006-of-00011.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-32-tensors-00007-of-00011.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-32-tensors-00008-of-00011.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-32-tensors-00009-of-00011.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-32-tensors-00010-of-00011.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-32-tensors-00011-of-00011.gguf ... gguf_split: 11 gguf split written with a total of 311 tensors.
done
+ echo PASS
PASS
+ echo
+ /root/llama.me/build-ci-release/bin/llama-completion -no-cnv --model /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-32-tensors-00001-of-00011.gguf -p 'I believe the meaning of life is' --n-predict 32
0.00.001.769 I llama_completion: llama backend init
0.00.001.774 I llama_completion: load the model and apply lora adapter, if any
0.00.001.781 I common_init_result: fitting params to device memory ...
0.00.001.782 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.00.175.397 I common_params_fit_impl: projected to use 5542 MiB of host memory vs. 77898 MiB of total host memory
0.00.255.197 W load: control-looking token: 128247 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
0.01.939.205 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.02.018.293 I llama_completion: llama threadpool init, n_threads = 8
0.02.018.310 I
0.02.018.370 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.02.018.371 I
0.02.018.451 I sampler seed: 1386843486
0.02.018.458 I sampler params:
repeat_last_n = 64, repeat_penalty = 1,000, frequency_penalty = 0,000, presence_penalty = 0,000
dry_multiplier = 0,000, dry_base = 1,750, dry_allowed_length = 2, dry_penalty_last_n = -1
top_k = 40, top_p = 0,950, min_p = 0,050, xtc_probability = 0,000, xtc_threshold = 0,100, typical_p = 1,000, top_n_sigma = -1,000, temp = 0,800
mirostat = 0, mirostat_lr = 0,100, mirostat_ent = 5,000, adaptive_target = -1,000, adaptive_decay = 0,900
0.02.018.463 I sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
0.02.018.463 I generate: n_ctx = 40960, n_batch = 2048, n_predict = 32, n_keep = 0
0.02.018.463 I
I believe the meaning of life is to be happy, and I believe that happiness is a result of a great deal of personal effort to be a happy person. This belief is supported by the fact
0.02.427.636 I common_perf_print: sampling time = 5,22 ms
0.02.427.638 I common_perf_print: samplers time = 1,83 ms / 39 tokens
0.02.427.643 I common_perf_print: load time = 1804,39 ms
0.02.427.644 I common_perf_print: prompt eval time = 20,66 ms / 7 tokens ( 2,95 ms per token, 338,75 tokens per second)
0.02.427.645 I common_perf_print: eval time = 382,49 ms / 31 runs ( 12,34 ms per token, 81,05 tokens per second)
0.02.427.645 I common_perf_print: total time = 409,35 ms / 38 tokens
0.02.427.646 I common_perf_print: unaccounted time = 0,98 ms / 0,2 % (total - sampling - prompt eval - eval) / (total)
0.02.427.646 I common_perf_print: graphs reused = 30
+ echo PASS
PASS
+ echo
+ /root/llama.me/build-ci-release/bin/llama-gguf-split --split-max-size 500M /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-merge.gguf /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-500M
n_split: 2
split 00001: n_tensors = 116, total_size = 504M
split 00002: n_tensors = 195, total_size = 299M
Writing file /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-500M-00001-of-00002.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-500M-00002-of-00002.gguf ... gguf_split: 2 gguf split written with a total of 311 tensors.
done
+ echo PASS
PASS
+ echo
+ /root/llama.me/build-ci-release/bin/llama-completion -no-cnv --model /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-500M-00001-of-00002.gguf -p 'I believe the meaning of life is' --n-predict 32
0.00.001.795 I llama_completion: llama backend init
0.00.001.799 I llama_completion: load the model and apply lora adapter, if any
0.00.001.805 I common_init_result: fitting params to device memory ...
0.00.001.805 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.00.198.644 I common_params_fit_impl: projected to use 5542 MiB of host memory vs. 77898 MiB of total host memory
0.00.283.923 W load: control-looking token: 128247 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
0.01.957.175 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.02.034.123 I llama_completion: llama threadpool init, n_threads = 8
0.02.034.147 I
0.02.034.202 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.02.034.202 I
0.02.034.268 I sampler seed: 400587886
0.02.034.278 I sampler params:
repeat_last_n = 64, repeat_penalty = 1,000, frequency_penalty = 0,000, presence_penalty = 0,000
dry_multiplier = 0,000, dry_base = 1,750, dry_allowed_length = 2, dry_penalty_last_n = -1
top_k = 40, top_p = 0,950, min_p = 0,050, xtc_probability = 0,000, xtc_threshold = 0,100, typical_p = 1,000, top_n_sigma = -1,000, temp = 0,800
mirostat = 0, mirostat_lr = 0,100, mirostat_ent = 5,000, adaptive_target = -1,000, adaptive_decay = 0,900
0.02.034.283 I sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
0.02.034.283 I generate: n_ctx = 40960, n_batch = 2048, n_predict = 32, n_keep = 0
0.02.034.283 I
I believe the meaning of life is to be happy, and that it is the way to the spiritual life. I want to find out if there is a way to achieve this. How should I
0.02.446.774 I common_perf_print: sampling time = 5,32 ms
0.02.446.775 I common_perf_print: samplers time = 1,68 ms / 39 tokens
0.02.446.779 I common_perf_print: load time = 1797,11 ms
0.02.446.780 I common_perf_print: prompt eval time = 20,54 ms / 7 tokens ( 2,93 ms per token, 340,82 tokens per second)
0.02.446.781 I common_perf_print: eval time = 385,83 ms / 31 runs ( 12,45 ms per token, 80,35 tokens per second)
0.02.446.794 I common_perf_print: total time = 412,65 ms / 38 tokens
0.02.446.795 I common_perf_print: unaccounted time = 0,96 ms / 0,2 % (total - sampling - prompt eval - eval) / (total)
0.02.446.795 I common_perf_print: graphs reused = 30
+ echo PASS
PASS
+ echo
+ rm -f /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00001-of-00012.gguf /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00002-of-00012.gguf /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00003-of-00012.gguf /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00004-of-00012.gguf /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00005-of-00012.gguf /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00006-of-00012.gguf /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00007-of-00012.gguf /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00008-of-00012.gguf /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00009-of-00012.gguf /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00010-of-00012.gguf /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00011-of-00012.gguf /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-00012-of-00012.gguf /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-32-tensors-00001-of-00011.gguf /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-32-tensors-00002-of-00011.gguf /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-32-tensors-00003-of-00011.gguf /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-32-tensors-00004-of-00011.gguf /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-32-tensors-00005-of-00011.gguf /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-32-tensors-00006-of-00011.gguf /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-32-tensors-00007-of-00011.gguf /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-32-tensors-00008-of-00011.gguf /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-32-tensors-00009-of-00011.gguf /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-32-tensors-00010-of-00011.gguf /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-32-tensors-00011-of-00011.gguf /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-500M-00001-of-00002.gguf /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-split-500M-00002-of-00002.gguf /root/llama.me/tmp/mnt/models/gguf-split/ggml-model-merge.gguf
real 0m14,437s
user 0m16,790s
sys 0m9,612s
+ cd ./tools/quantize
+ bash tests.sh /root/llama.me/build-ci-release/bin /root/llama.me/tmp/mnt/models
+ SPLIT=/root/llama.me/build-ci-release/bin/llama-gguf-split
+ QUANTIZE=/root/llama.me/build-ci-release/bin/llama-quantize
+ MAIN=/root/llama.me/build-ci-release/bin/llama-completion
+ WORK_PATH=/root/llama.me/tmp/mnt/models/quantize
+++ dirname tests.sh
++ realpath ./../../
+ ROOT_DIR=/root/llama.me
+ mkdir -p /root/llama.me/tmp/mnt/models/quantize
+ rm -f '/root/llama.me/tmp/mnt/models/quantize/ggml-model-split*.gguf' '/root/llama.me/tmp/mnt/models/quantize/ggml-model-requant*.gguf'
+ cd /root/llama.me/tmp/mnt/models/quantize
+ /root/llama.me/scripts/hf.sh --repo ggml-org/Qwen3-0.6B-GGUF --file Qwen3-0.6B-Q8_0.gguf
[+] attempting to download Qwen3-0.6B-Q8_0.gguf
[+] wget -q -c -O ./Qwen3-0.6B-Q8_0.gguf https://huggingface.co/ggml-org/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B-Q8_0.gguf
./Qwen3-0.6B-Q8_0.gguf
+ echo PASS
PASS
+ /root/llama.me/build-ci-release/bin/llama-gguf-split --split-max-tensors 28 /root/llama.me/tmp/mnt/models/quantize/Qwen3-0.6B-Q8_0.gguf /root/llama.me/tmp/mnt/models/quantize/ggml-model-split
n_split: 12
split 00001: n_tensors = 28, total_size = 371M
split 00002: n_tensors = 28, total_size = 45M
split 00003: n_tensors = 28, total_size = 40M
split 00004: n_tensors = 28, total_size = 43M
split 00005: n_tensors = 28, total_size = 42M
split 00006: n_tensors = 28, total_size = 44M
split 00007: n_tensors = 28, total_size = 39M
split 00008: n_tensors = 28, total_size = 45M
split 00009: n_tensors = 28, total_size = 39M
split 00010: n_tensors = 28, total_size = 44M
split 00011: n_tensors = 28, total_size = 42M
split 00012: n_tensors = 3, total_size = 6M
Writing file /root/llama.me/tmp/mnt/models/quantize/ggml-model-split-00001-of-00012.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/quantize/ggml-model-split-00002-of-00012.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/quantize/ggml-model-split-00003-of-00012.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/quantize/ggml-model-split-00004-of-00012.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/quantize/ggml-model-split-00005-of-00012.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/quantize/ggml-model-split-00006-of-00012.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/quantize/ggml-model-split-00007-of-00012.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/quantize/ggml-model-split-00008-of-00012.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/quantize/ggml-model-split-00009-of-00012.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/quantize/ggml-model-split-00010-of-00012.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/quantize/ggml-model-split-00011-of-00012.gguf ... done
Writing file /root/llama.me/tmp/mnt/models/quantize/ggml-model-split-00012-of-00012.gguf ... gguf_split: 12 gguf split written with a total of 311 tensors.
done
+ echo PASS
PASS
+ echo
+ /root/llama.me/build-ci-release/bin/llama-quantize --allow-requantize --keep-split /root/llama.me/tmp/mnt/models/quantize/ggml-model-split-00001-of-00012.gguf /root/llama.me/tmp/mnt/models/quantize/ggml-model-requant.gguf Q4_K
llama_print_build_info: build = 9440 (eb91ed029)
llama_print_build_info: built with GNU 13.3.0 for Linux x86_64
llama_quantize: quantizing '/root/llama.me/tmp/mnt/models/quantize/ggml-model-split-00001-of-00012.gguf' to '/root/llama.me/tmp/mnt/models/quantize/ggml-model-requant' as Q4_K
llama_model_loader: additional 11 GGUFs metadata loaded.
llama_model_loader: loaded meta data with 37 key-value pairs and 311 tensors from /root/llama.me/tmp/mnt/models/quantize/ggml-model-split-00001-of-00012.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = qwen3
llama_model_loader: - kv 1: general.type str = model
llama_model_loader: - kv 2: general.name str = Qwen3 0.6B
llama_model_loader: - kv 3: general.basename str = Qwen3
llama_model_loader: - kv 4: general.size_label str = 0.6B
llama_model_loader: - kv 5: general.license str = apache-2.0
llama_model_loader: - kv 6: general.license.link str = https://huggingface.co/Qwen/Qwen3-0.6...
llama_model_loader: - kv 7: general.base_model.count u32 = 1
llama_model_loader: - kv 8: general.base_model.0.name str = Qwen3 0.6B Base
llama_model_loader: - kv 9: general.base_model.0.organization str = Qwen
llama_model_loader: - kv 10: general.base_model.0.repo_url str = https://huggingface.co/Qwen/Qwen3-0.6...
llama_model_loader: - kv 11: general.tags arr[str,1] = ["text-generation"]
llama_model_loader: - kv 12: qwen3.block_count u32 = 28
llama_model_loader: - kv 13: qwen3.context_length u32 = 40960
llama_model_loader: - kv 14: qwen3.embedding_length u32 = 1024
llama_model_loader: - kv 15: qwen3.feed_forward_length u32 = 3072
llama_model_loader: - kv 16: qwen3.attention.head_count u32 = 16
llama_model_loader: - kv 17: qwen3.attention.head_count_kv u32 = 8
llama_model_loader: - kv 18: qwen3.rope.freq_base f32 = 1000000.000000
llama_model_loader: - kv 19: qwen3.attention.layer_norm_rms_epsilon f32 = 0.000001
llama_model_loader: - kv 20: qwen3.attention.key_length u32 = 128
llama_model_loader: - kv 21: qwen3.attention.value_length u32 = 128
llama_model_loader: - kv 22: tokenizer.ggml.model str = gpt2
llama_model_loader: - kv 23: tokenizer.ggml.pre str = qwen2
llama_model_loader: - kv 24: tokenizer.ggml.tokens arr[str,151936] = ["!", "\"", "#", "$", "%", "&", "'", ...
llama_model_loader: - kv 25: tokenizer.ggml.token_type arr[i32,151936] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv 26: tokenizer.ggml.merges arr[str,151387] = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
llama_model_loader: - kv 27: tokenizer.ggml.eos_token_id u32 = 151645
llama_model_loader: - kv 28: tokenizer.ggml.padding_token_id u32 = 151643
llama_model_loader: - kv 29: tokenizer.ggml.bos_token_id u32 = 151643
llama_model_loader: - kv 30: tokenizer.ggml.add_bos_token bool = false
llama_model_loader: - kv 31: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
llama_model_loader: - kv 32: general.quantization_version u32 = 2
llama_model_loader: - kv 33: general.file_type u32 = 7
llama_model_loader: - kv 34: split.no u16 = 0
llama_model_loader: - kv 35: split.tensors.count i32 = 311
llama_model_loader: - kv 36: split.count u16 = 12
llama_model_loader: - type f32: 113 tensors
llama_model_loader: - type q8_0: 198 tensors
[ 1/ 311] output.weight - [ 1024, 151936, 1, 1], type = q8_0, converting to q6_K .. size = 157.65 MiB -> 121.71 MiB
[ 2/ 311] output_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 3/ 311] token_embd.weight - [ 1024, 151936, 1, 1], type = q8_0, converting to q4_K .. size = 157.65 MiB -> 83.46 MiB
[ 4/ 311] blk.0.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 5/ 311] blk.0.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 6/ 311] blk.0.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 7/ 311] blk.0.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 8/ 311] blk.0.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 9/ 311] blk.0.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 10/ 311] blk.0.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 1.06 MiB -> 0.82 MiB
[ 11/ 311] blk.0.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 3.19 MiB -> 2.46 MiB
[ 12/ 311] blk.0.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 13/ 311] blk.0.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 14/ 311] blk.0.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 15/ 311] blk.1.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 16/ 311] blk.1.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 17/ 311] blk.1.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 18/ 311] blk.1.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 19/ 311] blk.1.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 20/ 311] blk.1.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 21/ 311] blk.1.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 1.06 MiB -> 0.82 MiB
[ 22/ 311] blk.1.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 3.19 MiB -> 2.46 MiB
[ 23/ 311] blk.1.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 24/ 311] blk.1.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 25/ 311] blk.1.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 26/ 311] blk.2.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 27/ 311] blk.2.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 28/ 311] blk.2.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 29/ 311] blk.2.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 30/ 311] blk.2.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 31/ 311] blk.2.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 32/ 311] blk.2.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 1.06 MiB -> 0.82 MiB
[ 33/ 311] blk.2.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 3.19 MiB -> 2.46 MiB
[ 34/ 311] blk.2.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 35/ 311] blk.2.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 36/ 311] blk.2.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 37/ 311] blk.3.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 38/ 311] blk.3.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 39/ 311] blk.3.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 40/ 311] blk.3.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 41/ 311] blk.3.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 42/ 311] blk.3.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 43/ 311] blk.3.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 44/ 311] blk.3.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 45/ 311] blk.3.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 46/ 311] blk.3.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 47/ 311] blk.3.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 48/ 311] blk.4.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 49/ 311] blk.4.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 50/ 311] blk.4.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 51/ 311] blk.4.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 52/ 311] blk.4.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 53/ 311] blk.4.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 54/ 311] blk.4.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 55/ 311] blk.4.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 56/ 311] blk.4.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 57/ 311] blk.4.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 58/ 311] blk.4.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 59/ 311] blk.5.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 60/ 311] blk.5.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 61/ 311] blk.5.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 62/ 311] blk.5.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 63/ 311] blk.5.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 64/ 311] blk.5.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 65/ 311] blk.5.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 1.06 MiB -> 0.82 MiB
[ 66/ 311] blk.5.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 3.19 MiB -> 2.46 MiB
[ 67/ 311] blk.5.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 68/ 311] blk.5.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 69/ 311] blk.5.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 70/ 311] blk.6.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 71/ 311] blk.6.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 72/ 311] blk.6.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 73/ 311] blk.6.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 74/ 311] blk.6.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 75/ 311] blk.6.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 76/ 311] blk.6.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 77/ 311] blk.6.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 78/ 311] blk.6.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 79/ 311] blk.6.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 80/ 311] blk.6.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 81/ 311] blk.7.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 82/ 311] blk.7.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 83/ 311] blk.7.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 84/ 311] blk.7.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 85/ 311] blk.7.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 86/ 311] blk.7.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 87/ 311] blk.7.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 88/ 311] blk.7.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 89/ 311] blk.7.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 90/ 311] blk.7.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 91/ 311] blk.7.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 92/ 311] blk.8.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 93/ 311] blk.8.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 94/ 311] blk.8.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 95/ 311] blk.8.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 96/ 311] blk.8.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 97/ 311] blk.8.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 98/ 311] blk.8.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 1.06 MiB -> 0.82 MiB
[ 99/ 311] blk.8.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 3.19 MiB -> 2.46 MiB
[ 100/ 311] blk.8.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 101/ 311] blk.8.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 102/ 311] blk.8.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 103/ 311] blk.9.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 104/ 311] blk.9.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 105/ 311] blk.9.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 106/ 311] blk.9.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 107/ 311] blk.9.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 108/ 311] blk.9.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 109/ 311] blk.9.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 110/ 311] blk.9.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 111/ 311] blk.9.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 112/ 311] blk.9.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 113/ 311] blk.9.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 114/ 311] blk.10.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 115/ 311] blk.10.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 116/ 311] blk.10.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 117/ 311] blk.10.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 118/ 311] blk.10.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 119/ 311] blk.10.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 120/ 311] blk.10.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 121/ 311] blk.10.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 122/ 311] blk.10.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 123/ 311] blk.10.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 124/ 311] blk.10.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 125/ 311] blk.11.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 126/ 311] blk.11.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 127/ 311] blk.11.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 128/ 311] blk.11.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 129/ 311] blk.11.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 130/ 311] blk.11.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 131/ 311] blk.11.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 1.06 MiB -> 0.82 MiB
[ 132/ 311] blk.11.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 3.19 MiB -> 2.46 MiB
[ 133/ 311] blk.11.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 134/ 311] blk.11.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 135/ 311] blk.11.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 136/ 311] blk.12.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 137/ 311] blk.12.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 138/ 311] blk.12.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 139/ 311] blk.12.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 140/ 311] blk.12.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 141/ 311] blk.12.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 142/ 311] blk.12.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 143/ 311] blk.12.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 144/ 311] blk.12.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 145/ 311] blk.12.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 146/ 311] blk.12.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 147/ 311] blk.13.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 148/ 311] blk.13.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 149/ 311] blk.13.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 150/ 311] blk.13.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 151/ 311] blk.13.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 152/ 311] blk.13.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 153/ 311] blk.13.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 154/ 311] blk.13.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 155/ 311] blk.13.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 156/ 311] blk.13.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 157/ 311] blk.13.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 158/ 311] blk.14.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 159/ 311] blk.14.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 160/ 311] blk.14.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 161/ 311] blk.14.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 162/ 311] blk.14.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 163/ 311] blk.14.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 164/ 311] blk.14.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 1.06 MiB -> 0.82 MiB
[ 165/ 311] blk.14.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 3.19 MiB -> 2.46 MiB
[ 166/ 311] blk.14.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 167/ 311] blk.14.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 168/ 311] blk.14.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 169/ 311] blk.15.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 170/ 311] blk.15.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 171/ 311] blk.15.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 172/ 311] blk.15.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 173/ 311] blk.15.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 174/ 311] blk.15.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 175/ 311] blk.15.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 176/ 311] blk.15.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 177/ 311] blk.15.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 178/ 311] blk.15.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 179/ 311] blk.15.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 180/ 311] blk.16.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 181/ 311] blk.16.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 182/ 311] blk.16.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 183/ 311] blk.16.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 184/ 311] blk.16.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 185/ 311] blk.16.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 186/ 311] blk.16.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 187/ 311] blk.16.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 188/ 311] blk.16.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 189/ 311] blk.16.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 190/ 311] blk.16.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 191/ 311] blk.17.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 192/ 311] blk.17.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 193/ 311] blk.17.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 194/ 311] blk.17.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 195/ 311] blk.17.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 196/ 311] blk.17.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 197/ 311] blk.17.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 1.06 MiB -> 0.82 MiB
[ 198/ 311] blk.17.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 3.19 MiB -> 2.46 MiB
[ 199/ 311] blk.17.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 200/ 311] blk.17.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 201/ 311] blk.17.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 202/ 311] blk.18.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 203/ 311] blk.18.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 204/ 311] blk.18.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 205/ 311] blk.18.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 206/ 311] blk.18.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 207/ 311] blk.18.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 208/ 311] blk.18.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 209/ 311] blk.18.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 210/ 311] blk.18.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 211/ 311] blk.18.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 212/ 311] blk.18.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 213/ 311] blk.19.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 214/ 311] blk.19.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 215/ 311] blk.19.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 216/ 311] blk.19.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 217/ 311] blk.19.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 218/ 311] blk.19.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 219/ 311] blk.19.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 220/ 311] blk.19.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 221/ 311] blk.19.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 222/ 311] blk.19.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 223/ 311] blk.19.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 224/ 311] blk.20.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 225/ 311] blk.20.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 226/ 311] blk.20.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 227/ 311] blk.20.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 228/ 311] blk.20.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 229/ 311] blk.20.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 230/ 311] blk.20.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 1.06 MiB -> 0.82 MiB
[ 231/ 311] blk.20.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 3.19 MiB -> 2.46 MiB
[ 232/ 311] blk.20.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 233/ 311] blk.20.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 234/ 311] blk.20.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 235/ 311] blk.21.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 236/ 311] blk.21.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 237/ 311] blk.21.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 238/ 311] blk.21.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 239/ 311] blk.21.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 240/ 311] blk.21.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 241/ 311] blk.21.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 242/ 311] blk.21.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 243/ 311] blk.21.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 244/ 311] blk.21.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 245/ 311] blk.21.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 246/ 311] blk.22.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 247/ 311] blk.22.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 248/ 311] blk.22.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 249/ 311] blk.22.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 250/ 311] blk.22.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 251/ 311] blk.22.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 252/ 311] blk.22.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 253/ 311] blk.22.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 254/ 311] blk.22.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 255/ 311] blk.22.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 256/ 311] blk.22.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 257/ 311] blk.23.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 258/ 311] blk.23.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 259/ 311] blk.23.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 260/ 311] blk.23.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 261/ 311] blk.23.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 262/ 311] blk.23.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 263/ 311] blk.23.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 1.06 MiB -> 0.82 MiB
[ 264/ 311] blk.23.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 3.19 MiB -> 2.46 MiB
[ 265/ 311] blk.23.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 266/ 311] blk.23.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 267/ 311] blk.23.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 268/ 311] blk.24.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 269/ 311] blk.24.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 270/ 311] blk.24.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 271/ 311] blk.24.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 272/ 311] blk.24.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 273/ 311] blk.24.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 274/ 311] blk.24.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 1.06 MiB -> 0.82 MiB
[ 275/ 311] blk.24.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 3.19 MiB -> 2.46 MiB
[ 276/ 311] blk.24.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 277/ 311] blk.24.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 278/ 311] blk.24.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 279/ 311] blk.25.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 280/ 311] blk.25.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 281/ 311] blk.25.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 282/ 311] blk.25.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 283/ 311] blk.25.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 284/ 311] blk.25.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 285/ 311] blk.25.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 1.06 MiB -> 0.82 MiB
[ 286/ 311] blk.25.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 3.19 MiB -> 2.46 MiB
[ 287/ 311] blk.25.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 288/ 311] blk.25.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 289/ 311] blk.25.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 290/ 311] blk.26.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 291/ 311] blk.26.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 292/ 311] blk.26.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 293/ 311] blk.26.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 294/ 311] blk.26.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 295/ 311] blk.26.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 296/ 311] blk.26.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 1.06 MiB -> 0.82 MiB
[ 297/ 311] blk.26.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 3.19 MiB -> 2.46 MiB
[ 298/ 311] blk.26.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 299/ 311] blk.26.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 300/ 311] blk.26.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 301/ 311] blk.27.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 302/ 311] blk.27.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 303/ 311] blk.27.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 304/ 311] blk.27.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 305/ 311] blk.27.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 306/ 311] blk.27.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 307/ 311] blk.27.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 1.06 MiB -> 0.82 MiB
[ 308/ 311] blk.27.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 3.19 MiB -> 2.46 MiB
[ 309/ 311] blk.27.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 310/ 311] blk.27.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 311/ 311] blk.27.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
llama_model_quantize_impl: model size = 761.80 MiB (8.50 BPW)
llama_model_quantize_impl: quant size = 456.11 MiB (5.09 BPW)
llama_quantize: quantize time = 5211.17 ms
llama_quantize: total time = 5211.17 ms
+ echo PASS
PASS
+ echo
+ /root/llama.me/build-ci-release/bin/llama-completion -no-cnv --model /root/llama.me/tmp/mnt/models/quantize/ggml-model-requant-00001-of-00012.gguf -p 'I believe the meaning of life is' --n-predict 32
0.00.001.812 I llama_completion: llama backend init
0.00.001.818 I llama_completion: load the model and apply lora adapter, if any
0.00.001.826 I common_init_result: fitting params to device memory ...
0.00.001.826 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.00.185.525 I common_params_fit_impl: projected to use 5032 MiB of host memory vs. 77898 MiB of total host memory
0.00.267.524 W load: control-looking token: 128247 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
0.02.092.503 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.02.164.145 I llama_completion: llama threadpool init, n_threads = 8
0.02.164.163 I
0.02.164.220 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.02.164.222 I
0.02.164.294 I sampler seed: 615738873
0.02.164.301 I sampler params:
repeat_last_n = 64, repeat_penalty = 1,000, frequency_penalty = 0,000, presence_penalty = 0,000
dry_multiplier = 0,000, dry_base = 1,750, dry_allowed_length = 2, dry_penalty_last_n = -1
top_k = 40, top_p = 0,950, min_p = 0,050, xtc_probability = 0,000, xtc_threshold = 0,100, typical_p = 1,000, top_n_sigma = -1,000, temp = 0,800
mirostat = 0, mirostat_lr = 0,100, mirostat_ent = 5,000, adaptive_target = -1,000, adaptive_decay = 0,900
0.02.164.306 I sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
0.02.164.306 I generate: n_ctx = 40960, n_batch = 2048, n_predict = 32, n_keep = 0
0.02.164.306 I
I believe the meaning of life is to live in the present and to accept the present moment. If I had to summarize this belief, I would say it is the result of the interplay between
0.02.432.326 I common_perf_print: sampling time = 4,71 ms
0.02.432.327 I common_perf_print: samplers time = 1,64 ms / 39 tokens
0.02.432.331 I common_perf_print: load time = 1939,94 ms
0.02.432.332 I common_perf_print: prompt eval time = 14,94 ms / 7 tokens ( 2,13 ms per token, 468,60 tokens per second)
0.02.432.346 I common_perf_print: eval time = 247,69 ms / 31 runs ( 7,99 ms per token, 125,16 tokens per second)
0.02.432.347 I common_perf_print: total time = 268,19 ms / 38 tokens
0.02.432.348 I common_perf_print: unaccounted time = 0,85 ms / 0,3 % (total - sampling - prompt eval - eval) / (total)
0.02.432.348 I common_perf_print: graphs reused = 30
+ echo PASS
PASS
+ echo
+ /root/llama.me/build-ci-release/bin/llama-quantize --allow-requantize /root/llama.me/tmp/mnt/models/quantize/ggml-model-split-00001-of-00012.gguf /root/llama.me/tmp/mnt/models/quantize/ggml-model-requant-merge.gguf Q4_K
llama_print_build_info: build = 9440 (eb91ed029)
llama_print_build_info: built with GNU 13.3.0 for Linux x86_64
llama_quantize: quantizing '/root/llama.me/tmp/mnt/models/quantize/ggml-model-split-00001-of-00012.gguf' to '/root/llama.me/tmp/mnt/models/quantize/ggml-model-requant-merge.gguf' as Q4_K
llama_model_loader: additional 11 GGUFs metadata loaded.
llama_model_loader: loaded meta data with 37 key-value pairs and 311 tensors from /root/llama.me/tmp/mnt/models/quantize/ggml-model-split-00001-of-00012.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = qwen3
llama_model_loader: - kv 1: general.type str = model
llama_model_loader: - kv 2: general.name str = Qwen3 0.6B
llama_model_loader: - kv 3: general.basename str = Qwen3
llama_model_loader: - kv 4: general.size_label str = 0.6B
llama_model_loader: - kv 5: general.license str = apache-2.0
llama_model_loader: - kv 6: general.license.link str = https://huggingface.co/Qwen/Qwen3-0.6...
llama_model_loader: - kv 7: general.base_model.count u32 = 1
llama_model_loader: - kv 8: general.base_model.0.name str = Qwen3 0.6B Base
llama_model_loader: - kv 9: general.base_model.0.organization str = Qwen
llama_model_loader: - kv 10: general.base_model.0.repo_url str = https://huggingface.co/Qwen/Qwen3-0.6...
llama_model_loader: - kv 11: general.tags arr[str,1] = ["text-generation"]
llama_model_loader: - kv 12: qwen3.block_count u32 = 28
llama_model_loader: - kv 13: qwen3.context_length u32 = 40960
llama_model_loader: - kv 14: qwen3.embedding_length u32 = 1024
llama_model_loader: - kv 15: qwen3.feed_forward_length u32 = 3072
llama_model_loader: - kv 16: qwen3.attention.head_count u32 = 16
llama_model_loader: - kv 17: qwen3.attention.head_count_kv u32 = 8
llama_model_loader: - kv 18: qwen3.rope.freq_base f32 = 1000000.000000
llama_model_loader: - kv 19: qwen3.attention.layer_norm_rms_epsilon f32 = 0.000001
llama_model_loader: - kv 20: qwen3.attention.key_length u32 = 128
llama_model_loader: - kv 21: qwen3.attention.value_length u32 = 128
llama_model_loader: - kv 22: tokenizer.ggml.model str = gpt2
llama_model_loader: - kv 23: tokenizer.ggml.pre str = qwen2
llama_model_loader: - kv 24: tokenizer.ggml.tokens arr[str,151936] = ["!", "\"", "#", "$", "%", "&", "'", ...
llama_model_loader: - kv 25: tokenizer.ggml.token_type arr[i32,151936] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv 26: tokenizer.ggml.merges arr[str,151387] = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
llama_model_loader: - kv 27: tokenizer.ggml.eos_token_id u32 = 151645
llama_model_loader: - kv 28: tokenizer.ggml.padding_token_id u32 = 151643
llama_model_loader: - kv 29: tokenizer.ggml.bos_token_id u32 = 151643
llama_model_loader: - kv 30: tokenizer.ggml.add_bos_token bool = false
llama_model_loader: - kv 31: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
llama_model_loader: - kv 32: general.quantization_version u32 = 2
llama_model_loader: - kv 33: general.file_type u32 = 7
llama_model_loader: - kv 34: split.no u16 = 0
llama_model_loader: - kv 35: split.tensors.count i32 = 311
llama_model_loader: - kv 36: split.count u16 = 12
llama_model_loader: - type f32: 113 tensors
llama_model_loader: - type q8_0: 198 tensors
[ 1/ 311] output.weight - [ 1024, 151936, 1, 1], type = q8_0, converting to q6_K .. size = 157.65 MiB -> 121.71 MiB
[ 2/ 311] output_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 3/ 311] token_embd.weight - [ 1024, 151936, 1, 1], type = q8_0, converting to q4_K .. size = 157.65 MiB -> 83.46 MiB
[ 4/ 311] blk.0.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 5/ 311] blk.0.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 6/ 311] blk.0.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 7/ 311] blk.0.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 8/ 311] blk.0.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 9/ 311] blk.0.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 10/ 311] blk.0.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 1.06 MiB -> 0.82 MiB
[ 11/ 311] blk.0.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 3.19 MiB -> 2.46 MiB
[ 12/ 311] blk.0.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 13/ 311] blk.0.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 14/ 311] blk.0.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 15/ 311] blk.1.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 16/ 311] blk.1.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 17/ 311] blk.1.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 18/ 311] blk.1.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 19/ 311] blk.1.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 20/ 311] blk.1.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 21/ 311] blk.1.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 1.06 MiB -> 0.82 MiB
[ 22/ 311] blk.1.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 3.19 MiB -> 2.46 MiB
[ 23/ 311] blk.1.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 24/ 311] blk.1.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 25/ 311] blk.1.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 26/ 311] blk.2.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 27/ 311] blk.2.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 28/ 311] blk.2.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 29/ 311] blk.2.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 30/ 311] blk.2.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 31/ 311] blk.2.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 32/ 311] blk.2.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 1.06 MiB -> 0.82 MiB
[ 33/ 311] blk.2.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 3.19 MiB -> 2.46 MiB
[ 34/ 311] blk.2.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 35/ 311] blk.2.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 36/ 311] blk.2.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 37/ 311] blk.3.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 38/ 311] blk.3.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 39/ 311] blk.3.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 40/ 311] blk.3.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 41/ 311] blk.3.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 42/ 311] blk.3.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 43/ 311] blk.3.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 44/ 311] blk.3.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 45/ 311] blk.3.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 46/ 311] blk.3.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 47/ 311] blk.3.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 48/ 311] blk.4.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 49/ 311] blk.4.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 50/ 311] blk.4.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 51/ 311] blk.4.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 52/ 311] blk.4.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 53/ 311] blk.4.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 54/ 311] blk.4.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 55/ 311] blk.4.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 56/ 311] blk.4.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 57/ 311] blk.4.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 58/ 311] blk.4.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 59/ 311] blk.5.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 60/ 311] blk.5.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 61/ 311] blk.5.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 62/ 311] blk.5.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 63/ 311] blk.5.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 64/ 311] blk.5.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 65/ 311] blk.5.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 1.06 MiB -> 0.82 MiB
[ 66/ 311] blk.5.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 3.19 MiB -> 2.46 MiB
[ 67/ 311] blk.5.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 68/ 311] blk.5.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 69/ 311] blk.5.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 70/ 311] blk.6.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 71/ 311] blk.6.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 72/ 311] blk.6.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 73/ 311] blk.6.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 74/ 311] blk.6.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 75/ 311] blk.6.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 76/ 311] blk.6.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 77/ 311] blk.6.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 78/ 311] blk.6.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 79/ 311] blk.6.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 80/ 311] blk.6.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 81/ 311] blk.7.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 82/ 311] blk.7.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 83/ 311] blk.7.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 84/ 311] blk.7.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 85/ 311] blk.7.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 86/ 311] blk.7.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 87/ 311] blk.7.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 88/ 311] blk.7.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 89/ 311] blk.7.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 90/ 311] blk.7.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 91/ 311] blk.7.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 92/ 311] blk.8.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 93/ 311] blk.8.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 94/ 311] blk.8.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 95/ 311] blk.8.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 96/ 311] blk.8.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 97/ 311] blk.8.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 98/ 311] blk.8.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 1.06 MiB -> 0.82 MiB
[ 99/ 311] blk.8.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 3.19 MiB -> 2.46 MiB
[ 100/ 311] blk.8.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 101/ 311] blk.8.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 102/ 311] blk.8.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 103/ 311] blk.9.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 104/ 311] blk.9.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 105/ 311] blk.9.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 106/ 311] blk.9.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 107/ 311] blk.9.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 108/ 311] blk.9.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 109/ 311] blk.9.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 110/ 311] blk.9.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 111/ 311] blk.9.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 112/ 311] blk.9.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 113/ 311] blk.9.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 114/ 311] blk.10.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 115/ 311] blk.10.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 116/ 311] blk.10.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 117/ 311] blk.10.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 118/ 311] blk.10.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 119/ 311] blk.10.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 120/ 311] blk.10.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 121/ 311] blk.10.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 122/ 311] blk.10.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 123/ 311] blk.10.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 124/ 311] blk.10.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 125/ 311] blk.11.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 126/ 311] blk.11.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 127/ 311] blk.11.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 128/ 311] blk.11.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 129/ 311] blk.11.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 130/ 311] blk.11.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 131/ 311] blk.11.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 1.06 MiB -> 0.82 MiB
[ 132/ 311] blk.11.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 3.19 MiB -> 2.46 MiB
[ 133/ 311] blk.11.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 134/ 311] blk.11.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 135/ 311] blk.11.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 136/ 311] blk.12.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 137/ 311] blk.12.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 138/ 311] blk.12.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 139/ 311] blk.12.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 140/ 311] blk.12.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 141/ 311] blk.12.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 142/ 311] blk.12.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 143/ 311] blk.12.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 144/ 311] blk.12.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 145/ 311] blk.12.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 146/ 311] blk.12.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 147/ 311] blk.13.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 148/ 311] blk.13.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 149/ 311] blk.13.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 150/ 311] blk.13.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 151/ 311] blk.13.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 152/ 311] blk.13.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 153/ 311] blk.13.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 154/ 311] blk.13.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 155/ 311] blk.13.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 156/ 311] blk.13.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 157/ 311] blk.13.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 158/ 311] blk.14.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 159/ 311] blk.14.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 160/ 311] blk.14.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 161/ 311] blk.14.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 162/ 311] blk.14.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 163/ 311] blk.14.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 164/ 311] blk.14.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 1.06 MiB -> 0.82 MiB
[ 165/ 311] blk.14.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 3.19 MiB -> 2.46 MiB
[ 166/ 311] blk.14.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 167/ 311] blk.14.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 168/ 311] blk.14.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 169/ 311] blk.15.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 170/ 311] blk.15.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 171/ 311] blk.15.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 172/ 311] blk.15.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 173/ 311] blk.15.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 174/ 311] blk.15.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 175/ 311] blk.15.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 176/ 311] blk.15.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 177/ 311] blk.15.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 178/ 311] blk.15.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 179/ 311] blk.15.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 180/ 311] blk.16.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 181/ 311] blk.16.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 182/ 311] blk.16.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 183/ 311] blk.16.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 184/ 311] blk.16.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 185/ 311] blk.16.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 186/ 311] blk.16.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 187/ 311] blk.16.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 188/ 311] blk.16.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 189/ 311] blk.16.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 190/ 311] blk.16.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 191/ 311] blk.17.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 192/ 311] blk.17.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 193/ 311] blk.17.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 194/ 311] blk.17.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 195/ 311] blk.17.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 196/ 311] blk.17.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 197/ 311] blk.17.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 1.06 MiB -> 0.82 MiB
[ 198/ 311] blk.17.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 3.19 MiB -> 2.46 MiB
[ 199/ 311] blk.17.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 200/ 311] blk.17.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 201/ 311] blk.17.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 202/ 311] blk.18.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 203/ 311] blk.18.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 204/ 311] blk.18.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 205/ 311] blk.18.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 206/ 311] blk.18.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 207/ 311] blk.18.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 208/ 311] blk.18.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 209/ 311] blk.18.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 210/ 311] blk.18.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 211/ 311] blk.18.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 212/ 311] blk.18.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 213/ 311] blk.19.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 214/ 311] blk.19.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 215/ 311] blk.19.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 216/ 311] blk.19.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 217/ 311] blk.19.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 218/ 311] blk.19.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 219/ 311] blk.19.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 220/ 311] blk.19.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 221/ 311] blk.19.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 222/ 311] blk.19.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 223/ 311] blk.19.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 224/ 311] blk.20.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 225/ 311] blk.20.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 226/ 311] blk.20.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 227/ 311] blk.20.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 228/ 311] blk.20.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 229/ 311] blk.20.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 230/ 311] blk.20.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 1.06 MiB -> 0.82 MiB
[ 231/ 311] blk.20.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 3.19 MiB -> 2.46 MiB
[ 232/ 311] blk.20.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 233/ 311] blk.20.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 234/ 311] blk.20.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 235/ 311] blk.21.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 236/ 311] blk.21.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 237/ 311] blk.21.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 238/ 311] blk.21.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 239/ 311] blk.21.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 240/ 311] blk.21.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 241/ 311] blk.21.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 242/ 311] blk.21.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 243/ 311] blk.21.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 244/ 311] blk.21.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 245/ 311] blk.21.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 246/ 311] blk.22.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 247/ 311] blk.22.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 248/ 311] blk.22.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 249/ 311] blk.22.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 250/ 311] blk.22.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 251/ 311] blk.22.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 252/ 311] blk.22.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 253/ 311] blk.22.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 254/ 311] blk.22.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 255/ 311] blk.22.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 256/ 311] blk.22.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 257/ 311] blk.23.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 258/ 311] blk.23.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 259/ 311] blk.23.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 260/ 311] blk.23.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 261/ 311] blk.23.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 262/ 311] blk.23.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 263/ 311] blk.23.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 1.06 MiB -> 0.82 MiB
[ 264/ 311] blk.23.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 3.19 MiB -> 2.46 MiB
[ 265/ 311] blk.23.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 266/ 311] blk.23.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 267/ 311] blk.23.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 268/ 311] blk.24.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 269/ 311] blk.24.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 270/ 311] blk.24.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 271/ 311] blk.24.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 272/ 311] blk.24.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 273/ 311] blk.24.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 274/ 311] blk.24.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 1.06 MiB -> 0.82 MiB
[ 275/ 311] blk.24.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 3.19 MiB -> 2.46 MiB
[ 276/ 311] blk.24.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 277/ 311] blk.24.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 278/ 311] blk.24.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 279/ 311] blk.25.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 280/ 311] blk.25.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 281/ 311] blk.25.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 282/ 311] blk.25.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 283/ 311] blk.25.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 284/ 311] blk.25.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 285/ 311] blk.25.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 1.06 MiB -> 0.82 MiB
[ 286/ 311] blk.25.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 3.19 MiB -> 2.46 MiB
[ 287/ 311] blk.25.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 288/ 311] blk.25.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 289/ 311] blk.25.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 290/ 311] blk.26.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 291/ 311] blk.26.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 292/ 311] blk.26.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 293/ 311] blk.26.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 294/ 311] blk.26.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 295/ 311] blk.26.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 296/ 311] blk.26.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 1.06 MiB -> 0.82 MiB
[ 297/ 311] blk.26.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 3.19 MiB -> 2.46 MiB
[ 298/ 311] blk.26.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 299/ 311] blk.26.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 300/ 311] blk.26.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 301/ 311] blk.27.attn_k.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 1.06 MiB -> 0.56 MiB
[ 302/ 311] blk.27.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 303/ 311] blk.27.attn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 304/ 311] blk.27.attn_output.weight - [ 2048, 1024, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 305/ 311] blk.27.attn_q.weight - [ 1024, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 2.12 MiB -> 1.12 MiB
[ 306/ 311] blk.27.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 307/ 311] blk.27.attn_v.weight - [ 1024, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 1.06 MiB -> 0.82 MiB
[ 308/ 311] blk.27.ffn_down.weight - [ 3072, 1024, 1, 1], type = q8_0, converting to q6_K .. size = 3.19 MiB -> 2.46 MiB
[ 309/ 311] blk.27.ffn_gate.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
[ 310/ 311] blk.27.ffn_norm.weight - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB
[ 311/ 311] blk.27.ffn_up.weight - [ 1024, 3072, 1, 1], type = q8_0, converting to q4_K .. size = 3.19 MiB -> 1.69 MiB
llama_model_quantize_impl: model size = 761.80 MiB (8.50 BPW)
llama_model_quantize_impl: quant size = 456.11 MiB (5.09 BPW)
llama_quantize: quantize time = 5188.91 ms
llama_quantize: total time = 5188.91 ms
+ echo PASS
PASS
+ echo
+ /root/llama.me/build-ci-release/bin/llama-completion -no-cnv --model /root/llama.me/tmp/mnt/models/quantize/ggml-model-requant-merge.gguf -p 'I believe the meaning of life is' --n-predict 32
0.00.001.835 I llama_completion: llama backend init
0.00.001.841 I llama_completion: load the model and apply lora adapter, if any
0.00.001.849 I common_init_result: fitting params to device memory ...
0.00.001.850 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.00.176.375 I common_params_fit_impl: projected to use 5032 MiB of host memory vs. 77898 MiB of total host memory
0.00.257.352 W load: control-looking token: 128247 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
0.02.093.128 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.02.164.422 I llama_completion: llama threadpool init, n_threads = 8
0.02.164.449 I
0.02.164.507 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.02.164.509 I
0.02.164.595 I sampler seed: 1205069089
0.02.164.606 I sampler params:
repeat_last_n = 64, repeat_penalty = 1,000, frequency_penalty = 0,000, presence_penalty = 0,000
dry_multiplier = 0,000, dry_base = 1,750, dry_allowed_length = 2, dry_penalty_last_n = -1
top_k = 40, top_p = 0,950, min_p = 0,050, xtc_probability = 0,000, xtc_threshold = 0,100, typical_p = 1,000, top_n_sigma = -1,000, temp = 0,800
mirostat = 0, mirostat_lr = 0,100, mirostat_ent = 5,000, adaptive_target = -1,000, adaptive_decay = 0,900
0.02.164.615 I sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
0.02.164.615 I generate: n_ctx = 40960, n_batch = 2048, n_predict = 32, n_keep = 0
0.02.164.615 I
I believe the meaning of life is to live in the present moment and to create and share with others what can be done in the present. How do I get from today to tomorrow in the context
0.02.432.656 I common_perf_print: sampling time = 5,04 ms
0.02.432.658 I common_perf_print: samplers time = 1,64 ms / 39 tokens
0.02.432.664 I common_perf_print: load time = 1949,26 ms
0.02.432.665 I common_perf_print: prompt eval time = 14,98 ms / 7 tokens ( 2,14 ms per token, 467,13 tokens per second)
0.02.432.666 I common_perf_print: eval time = 247,27 ms / 31 runs ( 7,98 ms per token, 125,37 tokens per second)
0.02.432.666 I common_perf_print: total time = 268,25 ms / 38 tokens
0.02.432.666 I common_perf_print: unaccounted time = 0,94 ms / 0,4 % (total - sampling - prompt eval - eval) / (total)
0.02.432.667 I common_perf_print: graphs reused = 30
+ echo PASS
PASS
+ echo
+ rm -f /root/llama.me/tmp/mnt/models/quantize/ggml-model-split-00001-of-00012.gguf /root/llama.me/tmp/mnt/models/quantize/ggml-model-split-00002-of-00012.gguf /root/llama.me/tmp/mnt/models/quantize/ggml-model-split-00003-of-00012.gguf /root/llama.me/tmp/mnt/models/quantize/ggml-model-split-00004-of-00012.gguf /root/llama.me/tmp/mnt/models/quantize/ggml-model-split-00005-of-00012.gguf /root/llama.me/tmp/mnt/models/quantize/ggml-model-split-00006-of-00012.gguf /root/llama.me/tmp/mnt/models/quantize/ggml-model-split-00007-of-00012.gguf /root/llama.me/tmp/mnt/models/quantize/ggml-model-split-00008-of-00012.gguf /root/llama.me/tmp/mnt/models/quantize/ggml-model-split-00009-of-00012.gguf /root/llama.me/tmp/mnt/models/quantize/ggml-model-split-00010-of-00012.gguf /root/llama.me/tmp/mnt/models/quantize/ggml-model-split-00011-of-00012.gguf /root/llama.me/tmp/mnt/models/quantize/ggml-model-split-00012-of-00012.gguf /root/llama.me/tmp/mnt/models/quantize/ggml-model-requant-00001-of-00012.gguf /root/llama.me/tmp/mnt/models/quantize/ggml-model-requant-00002-of-00012.gguf /root/llama.me/tmp/mnt/models/quantize/ggml-model-requant-00003-of-00012.gguf /root/llama.me/tmp/mnt/models/quantize/ggml-model-requant-00004-of-00012.gguf /root/llama.me/tmp/mnt/models/quantize/ggml-model-requant-00005-of-00012.gguf /root/llama.me/tmp/mnt/models/quantize/ggml-model-requant-00006-of-00012.gguf /root/llama.me/tmp/mnt/models/quantize/ggml-model-requant-00007-of-00012.gguf /root/llama.me/tmp/mnt/models/quantize/ggml-model-requant-00008-of-00012.gguf /root/llama.me/tmp/mnt/models/quantize/ggml-model-requant-00009-of-00012.gguf /root/llama.me/tmp/mnt/models/quantize/ggml-model-requant-00010-of-00012.gguf /root/llama.me/tmp/mnt/models/quantize/ggml-model-requant-00011-of-00012.gguf /root/llama.me/tmp/mnt/models/quantize/ggml-model-requant-00012-of-00012.gguf /root/llama.me/tmp/mnt/models/quantize/ggml-model-requant-merge.gguf
real 0m16,949s
user 2m2,505s
sys 0m6,519s
Qwen3 0.6B:
-
status: 1
-
perplexity:
-
imatrix:
- f16:
+ ./bin/llama-completion -no-cnv --model ../models-mnt/qwen3/0.6B/ggml-model-f16.gguf -ngl 99 -c 1024 -s 1234 -n 64 --ignore-eos -p 'I believe the meaning of life is'
warning: no usable GPU found, --gpu-layers option will be ignored
warning: one possible reason is that llama.cpp was compiled without GPU support
warning: consult docs/build.md for compilation instructions
0.00.001.909 I llama_completion: llama backend init
0.00.001.913 I llama_completion: load the model and apply lora adapter, if any
0.00.001.920 I common_init_result: fitting params to device memory ...
0.00.001.921 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.00.185.993 I common_params_fit_impl: projected to use 1558 MiB of host memory vs. 77898 MiB of total host memory
0.00.273.239 W load: control-looking token: 128247 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
0.00.401.484 W llama_context: n_ctx_seq (1024) < n_ctx_train (32768) -- the full capacity of the model will not be utilized
0.00.444.381 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.00.474.552 I llama_completion: llama threadpool init, n_threads = 8
0.00.474.573 I
0.00.474.643 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.474.644 I
0.00.474.716 I sampler seed: 1234
0.00.474.725 I sampler params:
repeat_last_n = 64, repeat_penalty = 1,000, frequency_penalty = 0,000, presence_penalty = 0,000
dry_multiplier = 0,000, dry_base = 1,750, dry_allowed_length = 2, dry_penalty_last_n = -1
top_k = 40, top_p = 0,950, min_p = 0,050, xtc_probability = 0,000, xtc_threshold = 0,100, typical_p = 1,000, top_n_sigma = -1,000, temp = 0,800
mirostat = 0, mirostat_lr = 0,100, mirostat_ent = 5,000, adaptive_target = -1,000, adaptive_decay = 0,900
0.00.474.731 I sampler chain: logits -> logit-bias -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
0.00.474.732 I generate: n_ctx = 1024, n_batch = 2048, n_predict = 64, n_keep = 0
0.00.474.732 I
I believe the meaning of life is to have fun, so I have made a list of activities that I love doing. I have a lot of them but some of them are just more fun than others so I am going to list them one by one. One of my favourite places to go is to a park or a beach. I love going to these
0.01.884.667 I common_perf_print: sampling time = 8,24 ms
0.01.884.669 I common_perf_print: samplers time = 3,32 ms / 71 tokens
0.01.884.675 I common_perf_print: load time = 249,29 ms
0.01.884.676 I common_perf_print: prompt eval time = 24,02 ms / 7 tokens ( 3,43 ms per token, 291,48 tokens per second)
0.01.884.677 I common_perf_print: eval time = 1376,15 ms / 63 runs ( 21,84 ms per token, 45,78 tokens per second)
0.01.884.678 I common_perf_print: total time = 1410,12 ms / 70 tokens
0.01.884.678 I common_perf_print: unaccounted time = 1,73 ms / 0,1 % (total - sampling - prompt eval - eval) / (total)
0.01.884.678 I common_perf_print: graphs reused = 62
real 0m1,963s
user 0m11,850s
sys 0m0,218s
+ ./bin/llama-perplexity --model ../models-mnt/qwen3/0.6B/ggml-model-f16.gguf -f ../models-mnt/wikitext/wikitext-2-raw/wiki.test.raw -ngl 99 -c 1024 -b 512 --chunks 2
error while handling argument "-f": error: failed to open file '../models-mnt/wikitext/wikitext-2-raw/wiki.test.raw'
usage:
-f, --file FNAME a file containing the prompt (default: none)
to show complete usage, run with -h
real 0m0,005s
user 0m0,003s
sys 0m0,002s
- bf16:
+ ./bin/llama-completion -no-cnv --model ../models-mnt/qwen3/0.6B/ggml-model-bf16.gguf -ngl 99 -c 1024 -s 1234 -n 64 --ignore-eos -p 'I believe the meaning of life is'
warning: no usable GPU found, --gpu-layers option will be ignored
warning: one possible reason is that llama.cpp was compiled without GPU support
warning: consult docs/build.md for compilation instructions
0.00.001.802 I llama_completion: llama backend init
0.00.001.806 I llama_completion: load the model and apply lora adapter, if any
0.00.001.814 I common_init_result: fitting params to device memory ...
0.00.001.814 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.00.179.409 I common_params_fit_impl: projected to use 1558 MiB of host memory vs. 77898 MiB of total host memory
0.00.261.991 W load: control-looking token: 128247 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
0.00.379.053 W llama_context: n_ctx_seq (1024) < n_ctx_train (32768) -- the full capacity of the model will not be utilized
0.00.423.666 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.00.453.866 I llama_completion: llama threadpool init, n_threads = 8
0.00.453.900 I
0.00.454.001 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.454.004 I
0.00.454.125 I sampler seed: 1234
0.00.454.138 I sampler params:
repeat_last_n = 64, repeat_penalty = 1,000, frequency_penalty = 0,000, presence_penalty = 0,000
dry_multiplier = 0,000, dry_base = 1,750, dry_allowed_length = 2, dry_penalty_last_n = -1
top_k = 40, top_p = 0,950, min_p = 0,050, xtc_probability = 0,000, xtc_threshold = 0,100, typical_p = 1,000, top_n_sigma = -1,000, temp = 0,800
mirostat = 0, mirostat_lr = 0,100, mirostat_ent = 5,000, adaptive_target = -1,000, adaptive_decay = 0,900
0.00.454.148 I sampler chain: logits -> logit-bias -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
0.00.454.151 I generate: n_ctx = 1024, n_batch = 2048, n_predict = 64, n_keep = 0
0.00.454.151 I
I believe the meaning of life is to give, not to receive. Do you agree with this view?
Yes, I agree with this view. The concept of giving as the foundation of life reflects a deeper spiritual principle. It suggests that the purpose of existence is to contribute to the well-being of others and society, rather than seeking personal gain. This perspective
0.01.887.219 I common_perf_print: sampling time = 8,23 ms
0.01.887.221 I common_perf_print: samplers time = 3,12 ms / 71 tokens
0.01.887.227 I common_perf_print: load time = 235,29 ms
0.01.887.228 I common_perf_print: prompt eval time = 24,25 ms / 7 tokens ( 3,46 ms per token, 288,60 tokens per second)
0.01.887.229 I common_perf_print: eval time = 1398,91 ms / 63 runs ( 22,20 ms per token, 45,04 tokens per second)
0.01.887.229 I common_perf_print: total time = 1433,36 ms / 70 tokens
0.01.887.229 I common_perf_print: unaccounted time = 1,97 ms / 0,1 % (total - sampling - prompt eval - eval) / (total)
0.01.887.229 I common_perf_print: graphs reused = 62
real 0m1,962s
user 0m12,018s
sys 0m0,209s
- q8_0:
+ ./bin/llama-completion -no-cnv --model ../models-mnt/qwen3/0.6B/ggml-model-q8_0.gguf -ngl 99 -c 1024 -s 1234 -n 64 --ignore-eos -p 'I believe the meaning of life is'
warning: no usable GPU found, --gpu-layers option will be ignored
warning: one possible reason is that llama.cpp was compiled without GPU support
warning: consult docs/build.md for compilation instructions
0.00.001.685 I llama_completion: llama backend init
0.00.001.689 I llama_completion: load the model and apply lora adapter, if any
0.00.001.696 I common_init_result: fitting params to device memory ...
0.00.001.696 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.00.181.291 I common_params_fit_impl: projected to use 1025 MiB of host memory vs. 77898 MiB of total host memory
0.00.262.973 W load: control-looking token: 128247 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
0.00.346.508 W llama_context: n_ctx_seq (1024) < n_ctx_train (32768) -- the full capacity of the model will not be utilized
0.00.391.419 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.00.414.329 I llama_completion: llama threadpool init, n_threads = 8
0.00.414.349 I
0.00.414.416 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.414.417 I
0.00.414.491 I sampler seed: 1234
0.00.414.498 I sampler params:
repeat_last_n = 64, repeat_penalty = 1,000, frequency_penalty = 0,000, presence_penalty = 0,000
dry_multiplier = 0,000, dry_base = 1,750, dry_allowed_length = 2, dry_penalty_last_n = -1
top_k = 40, top_p = 0,950, min_p = 0,050, xtc_probability = 0,000, xtc_threshold = 0,100, typical_p = 1,000, top_n_sigma = -1,000, temp = 0,800
mirostat = 0, mirostat_lr = 0,100, mirostat_ent = 5,000, adaptive_target = -1,000, adaptive_decay = 0,900
0.00.414.505 I sampler chain: logits -> logit-bias -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
0.00.414.505 I generate: n_ctx = 1024, n_batch = 2048, n_predict = 64, n_keep = 0
0.00.414.506 I
I believe the meaning of life is to give back. That's why I'm a volunteer. I am a volunteer because I believe that life is important to give back. I work in a nursing home to help people. I also help with the medical staff at a clinic. I am also a volunteer for the Red Cross. I volunteer because I think giving
0.01.218.670 I common_perf_print: sampling time = 9,11 ms
0.01.218.672 I common_perf_print: samplers time = 3,60 ms / 71 tokens
0.01.218.677 I common_perf_print: load time = 194,18 ms
0.01.218.681 I common_perf_print: prompt eval time = 16,42 ms / 7 tokens ( 2,35 ms per token, 426,28 tokens per second)
0.01.218.682 I common_perf_print: eval time = 777,29 ms / 63 runs ( 12,34 ms per token, 81,05 tokens per second)
0.01.218.682 I common_perf_print: total time = 804,35 ms / 70 tokens
0.01.218.686 I common_perf_print: unaccounted time = 1,52 ms / 0,2 % (total - sampling - prompt eval - eval) / (total)
0.01.218.686 I common_perf_print: graphs reused = 62
real 0m1,277s
user 0m6,927s
sys 0m0,166s
- q4_0:
+ ./bin/llama-completion -no-cnv --model ../models-mnt/qwen3/0.6B/ggml-model-q4_0.gguf -ngl 99 -c 1024 -s 1234 -n 64 --ignore-eos -p 'I believe the meaning of life is'
warning: no usable GPU found, --gpu-layers option will be ignored
warning: one possible reason is that llama.cpp was compiled without GPU support
warning: consult docs/build.md for compilation instructions
0.00.001.703 I llama_completion: llama backend init
0.00.001.707 I llama_completion: load the model and apply lora adapter, if any
0.00.001.715 I common_init_result: fitting params to device memory ...
0.00.001.715 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.00.181.396 I common_params_fit_impl: projected to use 543 MiB of host memory vs. 77898 MiB of total host memory
0.00.265.419 W load: control-looking token: 128247 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
0.00.461.489 W llama_context: n_ctx_seq (1024) < n_ctx_train (32768) -- the full capacity of the model will not be utilized
0.00.506.112 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.00.520.832 I llama_completion: llama threadpool init, n_threads = 8
0.00.520.852 I
0.00.520.922 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.520.923 I
0.00.520.998 I sampler seed: 1234
0.00.521.007 I sampler params:
repeat_last_n = 64, repeat_penalty = 1,000, frequency_penalty = 0,000, presence_penalty = 0,000
dry_multiplier = 0,000, dry_base = 1,750, dry_allowed_length = 2, dry_penalty_last_n = -1
top_k = 40, top_p = 0,950, min_p = 0,050, xtc_probability = 0,000, xtc_threshold = 0,100, typical_p = 1,000, top_n_sigma = -1,000, temp = 0,800
mirostat = 0, mirostat_lr = 0,100, mirostat_ent = 5,000, adaptive_target = -1,000, adaptive_decay = 0,900
0.00.521.013 I sampler chain: logits -> logit-bias -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
0.00.521.013 I generate: n_ctx = 1024, n_batch = 2048, n_predict = 64, n_keep = 0
0.00.521.013 I
I believe the meaning of life is to make the most out of my life, and to be happy. This is what I have been doing. I'm not sure I have reached my full potential, and I'm not sure if I have given myself enough freedom to grow spiritually.
What's the point of life?
I'm not sure I have reached my
0.01.026.627 I common_perf_print: sampling time = 9,60 ms
0.01.026.628 I common_perf_print: samplers time = 3,90 ms / 71 tokens
0.01.026.633 I common_perf_print: load time = 299,50 ms
0.01.026.635 I common_perf_print: prompt eval time = 14,61 ms / 7 tokens ( 2,09 ms per token, 479,26 tokens per second)
0.01.026.636 I common_perf_print: eval time = 480,01 ms / 63 runs ( 7,62 ms per token, 131,25 tokens per second)
0.01.026.636 I common_perf_print: total time = 505,80 ms / 70 tokens
0.01.026.637 I common_perf_print: unaccounted time = 1,59 ms / 0,3 % (total - sampling - prompt eval - eval) / (total)
0.01.026.637 I common_perf_print: graphs reused = 62
real 0m1,101s
user 0m4,548s
sys 0m0,221s
- q4_1:
+ ./bin/llama-completion -no-cnv --model ../models-mnt/qwen3/0.6B/ggml-model-q4_1.gguf -ngl 99 -c 1024 -s 1234 -n 64 --ignore-eos -p 'I believe the meaning of life is'
warning: no usable GPU found, --gpu-layers option will be ignored
warning: one possible reason is that llama.cpp was compiled without GPU support
warning: consult docs/build.md for compilation instructions
0.00.001.728 I llama_completion: llama backend init
0.00.001.740 I llama_completion: load the model and apply lora adapter, if any
0.00.001.748 I common_init_result: fitting params to device memory ...
0.00.001.748 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.00.181.720 I common_params_fit_impl: projected to use 806 MiB of host memory vs. 77898 MiB of total host memory
0.00.262.978 W load: control-looking token: 128247 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
0.00.333.540 W llama_context: n_ctx_seq (1024) < n_ctx_train (32768) -- the full capacity of the model will not be utilized
0.00.378.316 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.00.394.368 I llama_completion: llama threadpool init, n_threads = 8
0.00.394.389 I
0.00.394.458 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.394.459 I
0.00.394.535 I sampler seed: 1234
0.00.394.544 I sampler params:
repeat_last_n = 64, repeat_penalty = 1,000, frequency_penalty = 0,000, presence_penalty = 0,000
dry_multiplier = 0,000, dry_base = 1,750, dry_allowed_length = 2, dry_penalty_last_n = -1
top_k = 40, top_p = 0,950, min_p = 0,050, xtc_probability = 0,000, xtc_threshold = 0,100, typical_p = 1,000, top_n_sigma = -1,000, temp = 0,800
mirostat = 0, mirostat_lr = 0,100, mirostat_ent = 5,000, adaptive_target = -1,000, adaptive_decay = 0,900
0.00.394.550 I sampler chain: logits -> logit-bias -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
0.00.394.551 I generate: n_ctx = 1024, n_batch = 2048, n_predict = 64, n_keep = 0
0.00.394.551 I
I believe the meaning of life is to find your own voice. I will never be able to hear my own voice. I will never be able to listen to a song. I will never be able to see my own face. I will never be able to share my own thoughts. I will never be able to share my own voice. I will never
0.00.957.264 I common_perf_print: sampling time = 9,97 ms
0.00.957.265 I common_perf_print: samplers time = 4,08 ms / 71 tokens
0.00.957.270 I common_perf_print: load time = 173,96 ms
0.00.957.271 I common_perf_print: prompt eval time = 20,90 ms / 7 tokens ( 2,99 ms per token, 334,94 tokens per second)
0.00.957.271 I common_perf_print: eval time = 530,36 ms / 63 runs ( 8,42 ms per token, 118,79 tokens per second)
0.00.957.272 I common_perf_print: total time = 562,90 ms / 70 tokens
0.00.957.272 I common_perf_print: unaccounted time = 1,67 ms / 0,3 % (total - sampling - prompt eval - eval) / (total)
0.00.957.272 I common_perf_print: graphs reused = 62
real 0m1,008s
user 0m4,952s
sys 0m0,134s
- q5_0:
+ ./bin/llama-completion -no-cnv --model ../models-mnt/qwen3/0.6B/ggml-model-q5_0.gguf -ngl 99 -c 1024 -s 1234 -n 64 --ignore-eos -p 'I believe the meaning of life is'
warning: no usable GPU found, --gpu-layers option will be ignored
warning: one possible reason is that llama.cpp was compiled without GPU support
warning: consult docs/build.md for compilation instructions
0.00.001.705 I llama_completion: llama backend init
0.00.001.709 I llama_completion: load the model and apply lora adapter, if any
0.00.001.716 I common_init_result: fitting params to device memory ...
0.00.001.716 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.00.181.683 I common_params_fit_impl: projected to use 832 MiB of host memory vs. 77898 MiB of total host memory
0.00.270.458 W load: control-looking token: 128247 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
0.00.345.237 W llama_context: n_ctx_seq (1024) < n_ctx_train (32768) -- the full capacity of the model will not be utilized
0.00.387.599 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.00.404.291 I llama_completion: llama threadpool init, n_threads = 8
0.00.404.310 I
0.00.404.378 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.404.380 I
0.00.404.452 I sampler seed: 1234
0.00.404.460 I sampler params:
repeat_last_n = 64, repeat_penalty = 1,000, frequency_penalty = 0,000, presence_penalty = 0,000
dry_multiplier = 0,000, dry_base = 1,750, dry_allowed_length = 2, dry_penalty_last_n = -1
top_k = 40, top_p = 0,950, min_p = 0,050, xtc_probability = 0,000, xtc_threshold = 0,100, typical_p = 1,000, top_n_sigma = -1,000, temp = 0,800
mirostat = 0, mirostat_lr = 0,100, mirostat_ent = 5,000, adaptive_target = -1,000, adaptive_decay = 0,900
0.00.404.466 I sampler chain: logits -> logit-bias -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
0.00.404.468 I generate: n_ctx = 1024, n_batch = 2048, n_predict = 64, n_keep = 0
0.00.404.468 I
I believe the meaning of life is to give a chance for others to feel the joy of life. Is this statement correct?
Yes, your statement is correct. Many people believe that the purpose of life is to inspire and uplift others, allowing them to experience happiness and fulfillment. This perspective emphasizes the importance of compassion, empathy, and the impact we have on
0.00.992.428 I common_perf_print: sampling time = 8,09 ms
0.00.992.431 I common_perf_print: samplers time = 3,21 ms / 71 tokens
0.00.992.434 I common_perf_print: load time = 183,44 ms
0.00.992.435 I common_perf_print: prompt eval time = 14,92 ms / 7 tokens ( 2,13 ms per token, 469,04 tokens per second)
0.00.992.436 I common_perf_print: eval time = 563,63 ms / 63 runs ( 8,95 ms per token, 111,77 tokens per second)
0.00.992.436 I common_perf_print: total time = 588,14 ms / 70 tokens
0.00.992.437 I common_perf_print: unaccounted time = 1,50 ms / 0,3 % (total - sampling - prompt eval - eval) / (total)
0.00.992.437 I common_perf_print: graphs reused = 62
real 0m1,044s
user 0m5,176s
sys 0m0,124s
- q5_1:
+ ./bin/llama-completion -no-cnv --model ../models-mnt/qwen3/0.6B/ggml-model-q5_1.gguf -ngl 99 -c 1024 -s 1234 -n 64 --ignore-eos -p 'I believe the meaning of life is'
warning: no usable GPU found, --gpu-layers option will be ignored
warning: one possible reason is that llama.cpp was compiled without GPU support
warning: consult docs/build.md for compilation instructions
0.00.001.731 I llama_completion: llama backend init
0.00.001.736 I llama_completion: load the model and apply lora adapter, if any
0.00.001.744 I common_init_result: fitting params to device memory ...
0.00.001.744 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.00.184.383 I common_params_fit_impl: projected to use 858 MiB of host memory vs. 77898 MiB of total host memory
0.00.266.357 W load: control-looking token: 128247 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
0.00.339.606 W llama_context: n_ctx_seq (1024) < n_ctx_train (32768) -- the full capacity of the model will not be utilized
0.00.384.268 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.00.401.346 I llama_completion: llama threadpool init, n_threads = 8
0.00.401.365 I
0.00.401.425 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.401.427 I
0.00.401.495 I sampler seed: 1234
0.00.401.502 I sampler params:
repeat_last_n = 64, repeat_penalty = 1,000, frequency_penalty = 0,000, presence_penalty = 0,000
dry_multiplier = 0,000, dry_base = 1,750, dry_allowed_length = 2, dry_penalty_last_n = -1
top_k = 40, top_p = 0,950, min_p = 0,050, xtc_probability = 0,000, xtc_threshold = 0,100, typical_p = 1,000, top_n_sigma = -1,000, temp = 0,800
mirostat = 0, mirostat_lr = 0,100, mirostat_ent = 5,000, adaptive_target = -1,000, adaptive_decay = 0,900
0.00.401.507 I sampler chain: logits -> logit-bias -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
0.00.401.510 I generate: n_ctx = 1024, n_batch = 2048, n_predict = 64, n_keep = 0
0.00.401.510 I
I believe the meaning of life is to give and to be given. I have been given a lot of responsibility, and I have been given the ability to use that responsibility to help others. I have always been very passionate about helping others. I have seen a positive effect from my actions and I believe that this is the meaning of life. I have never
0.01.026.156 I common_perf_print: sampling time = 9,28 ms
0.01.026.156 I common_perf_print: samplers time = 3,70 ms / 71 tokens
0.01.026.160 I common_perf_print: load time = 178,23 ms
0.01.026.161 I common_perf_print: prompt eval time = 24,06 ms / 7 tokens ( 3,44 ms per token, 290,96 tokens per second)
0.01.026.161 I common_perf_print: eval time = 590,06 ms / 63 runs ( 9,37 ms per token, 106,77 tokens per second)
0.01.026.174 I common_perf_print: total time = 624,81 ms / 70 tokens
0.01.026.175 I common_perf_print: unaccounted time = 1,41 ms / 0,2 % (total - sampling - prompt eval - eval) / (total)
0.01.026.176 I common_perf_print: graphs reused = 62
real 0m1,080s
user 0m5,472s
sys 0m0,126s
- q2_k:
+ ./bin/llama-completion -no-cnv --model ../models-mnt/qwen3/0.6B/ggml-model-q2_k.gguf -ngl 99 -c 1024 -s 1234 -n 64 --ignore-eos -p 'I believe the meaning of life is'
warning: no usable GPU found, --gpu-layers option will be ignored
warning: one possible reason is that llama.cpp was compiled without GPU support
warning: consult docs/build.md for compilation instructions
0.00.002.477 I llama_completion: llama backend init
0.00.002.486 I llama_completion: load the model and apply lora adapter, if any
0.00.002.501 I common_init_result: fitting params to device memory ...
0.00.002.502 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.00.186.936 I common_params_fit_impl: projected to use 615 MiB of host memory vs. 77898 MiB of total host memory
0.00.276.361 W load: control-looking token: 128247 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
0.00.393.253 W llama_context: n_ctx_seq (1024) < n_ctx_train (32768) -- the full capacity of the model will not be utilized
0.00.438.008 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.00.450.969 I llama_completion: llama threadpool init, n_threads = 8
0.00.451.004 I
0.00.451.073 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.451.075 I
0.00.451.156 I sampler seed: 1234
0.00.451.166 I sampler params:
repeat_last_n = 64, repeat_penalty = 1,000, frequency_penalty = 0,000, presence_penalty = 0,000
dry_multiplier = 0,000, dry_base = 1,750, dry_allowed_length = 2, dry_penalty_last_n = -1
top_k = 40, top_p = 0,950, min_p = 0,050, xtc_probability = 0,000, xtc_threshold = 0,100, typical_p = 1,000, top_n_sigma = -1,000, temp = 0,800
mirostat = 0, mirostat_lr = 0,100, mirostat_ent = 5,000, adaptive_target = -1,000, adaptive_decay = 0,900
0.00.451.174 I sampler chain: logits -> logit-bias -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
0.00.451.177 I generate: n_ctx = 1024, n_batch = 2048, n_predict = 64, n_keep = 0
0.00.451.177 I
I believe the meaning of life is not of the soul is of the soul
A. is of the body
B. is of the brain
C. is of the brain
C. is of the mind
C. is of the mind
C. is of the mind
C. is of the soul
C. is of the soul
0.00.879.249 I common_perf_print: sampling time = 9,28 ms
0.00.879.250 I common_perf_print: samplers time = 3,73 ms / 71 tokens
0.00.879.254 I common_perf_print: load time = 223,40 ms
0.00.879.255 I common_perf_print: prompt eval time = 15,96 ms / 7 tokens ( 2,28 ms per token, 438,62 tokens per second)
0.00.879.256 I common_perf_print: eval time = 401,44 ms / 63 runs ( 6,37 ms per token, 156,94 tokens per second)
0.00.879.256 I common_perf_print: total time = 428,29 ms / 70 tokens
0.00.879.257 I common_perf_print: unaccounted time = 1,61 ms / 0,4 % (total - sampling - prompt eval - eval) / (total)
0.00.879.257 I common_perf_print: graphs reused = 62
real 0m0,929s
user 0m3,878s
sys 0m0,163s
- q3_k:
+ ./bin/llama-completion -no-cnv --model ../models-mnt/qwen3/0.6B/ggml-model-q3_k.gguf -ngl 99 -c 1024 -s 1234 -n 64 --ignore-eos -p 'I believe the meaning of life is'
warning: no usable GPU found, --gpu-layers option will be ignored
warning: one possible reason is that llama.cpp was compiled without GPU support
warning: consult docs/build.md for compilation instructions
0.00.001.753 I llama_completion: llama backend init
0.00.001.757 I llama_completion: load the model and apply lora adapter, if any
0.00.001.764 I common_init_result: fitting params to device memory ...
0.00.001.765 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.00.204.313 I common_params_fit_impl: projected to use 655 MiB of host memory vs. 77898 MiB of total host memory
0.00.286.487 W load: control-looking token: 128247 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
0.00.437.765 W llama_context: n_ctx_seq (1024) < n_ctx_train (32768) -- the full capacity of the model will not be utilized
0.00.483.458 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.00.498.255 I llama_completion: llama threadpool init, n_threads = 8
0.00.498.277 I
0.00.498.349 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.498.350 I
0.00.498.429 I sampler seed: 1234
0.00.498.438 I sampler params:
repeat_last_n = 64, repeat_penalty = 1,000, frequency_penalty = 0,000, presence_penalty = 0,000
dry_multiplier = 0,000, dry_base = 1,750, dry_allowed_length = 2, dry_penalty_last_n = -1
top_k = 40, top_p = 0,950, min_p = 0,050, xtc_probability = 0,000, xtc_threshold = 0,100, typical_p = 1,000, top_n_sigma = -1,000, temp = 0,800
mirostat = 0, mirostat_lr = 0,100, mirostat_ent = 5,000, adaptive_target = -1,000, adaptive_decay = 0,900
0.00.498.447 I sampler chain: logits -> logit-bias -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
0.00.498.448 I generate: n_ctx = 1024, n_batch = 2048, n_predict = 64, n_keep = 0
0.00.498.448 I
I believe the meaning of life is to serve God, but I don’t know how to live it. I don’t know how to live it, so how can I serve God? And I don’t know if I am to serve God. What can I serve God, and where will I serve God? Also, I know that I am to serve
0.00.982.197 I common_perf_print: sampling time = 9,62 ms
0.00.982.198 I common_perf_print: samplers time = 3,99 ms / 71 tokens
0.00.982.201 I common_perf_print: load time = 254,85 ms
0.00.982.202 I common_perf_print: prompt eval time = 17,17 ms / 7 tokens ( 2,45 ms per token, 407,74 tokens per second)
0.00.982.203 I common_perf_print: eval time = 455,75 ms / 63 runs ( 7,23 ms per token, 138,23 tokens per second)
0.00.982.203 I common_perf_print: total time = 483,95 ms / 70 tokens
0.00.982.204 I common_perf_print: unaccounted time = 1,41 ms / 0,3 % (total - sampling - prompt eval - eval) / (total)
0.00.982.204 I common_perf_print: graphs reused = 62
real 0m1,038s
user 0m4,392s
sys 0m0,161s
- q4_k:
+ ./bin/llama-completion -no-cnv --model ../models-mnt/qwen3/0.6B/ggml-model-q4_k.gguf -ngl 99 -c 1024 -s 1234 -n 64 --ignore-eos -p 'I believe the meaning of life is'
warning: no usable GPU found, --gpu-layers option will be ignored
warning: one possible reason is that llama.cpp was compiled without GPU support
warning: consult docs/build.md for compilation instructions
0.00.001.878 I llama_completion: llama backend init
0.00.001.888 I llama_completion: load the model and apply lora adapter, if any
0.00.001.901 I common_init_result: fitting params to device memory ...
0.00.001.901 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.00.190.336 I common_params_fit_impl: projected to use 589 MiB of host memory vs. 77898 MiB of total host memory
0.00.275.111 W load: control-looking token: 128247 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
0.00.533.423 W llama_context: n_ctx_seq (1024) < n_ctx_train (32768) -- the full capacity of the model will not be utilized
0.00.575.479 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.00.590.624 I llama_completion: llama threadpool init, n_threads = 8
0.00.590.645 I
0.00.590.709 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.590.710 I
0.00.590.778 I sampler seed: 1234
0.00.590.786 I sampler params:
repeat_last_n = 64, repeat_penalty = 1,000, frequency_penalty = 0,000, presence_penalty = 0,000
dry_multiplier = 0,000, dry_base = 1,750, dry_allowed_length = 2, dry_penalty_last_n = -1
top_k = 40, top_p = 0,950, min_p = 0,050, xtc_probability = 0,000, xtc_threshold = 0,100, typical_p = 1,000, top_n_sigma = -1,000, temp = 0,800
mirostat = 0, mirostat_lr = 0,100, mirostat_ent = 5,000, adaptive_target = -1,000, adaptive_decay = 0,900
0.00.590.792 I sampler chain: logits -> logit-bias -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
0.00.590.793 I generate: n_ctx = 1024, n_batch = 2048, n_predict = 64, n_keep = 0
0.00.590.793 I
I believe the meaning of life is to do what is good and right and to be a good person. I believe that the only way to be good and right is to do what is good and right. I believe that by doing what is good and right, I will become a good and right person. The world would be a better place if I were
0.01.116.936 I common_perf_print: sampling time = 9,01 ms
0.01.116.936 I common_perf_print: samplers time = 3,62 ms / 71 tokens
0.01.116.941 I common_perf_print: load time = 360,91 ms
0.01.116.942 I common_perf_print: prompt eval time = 13,12 ms / 7 tokens ( 1,87 ms per token, 533,66 tokens per second)
0.01.116.943 I common_perf_print: eval time = 502,86 ms / 63 runs ( 7,98 ms per token, 125,28 tokens per second)
0.01.116.943 I common_perf_print: total time = 526,32 ms / 70 tokens
0.01.116.944 I common_perf_print: unaccounted time = 1,34 ms / 0,3 % (total - sampling - prompt eval - eval) / (total)
0.01.116.944 I common_perf_print: graphs reused = 62
real 0m1,181s
user 0m4,785s
sys 0m0,210s
- q5_k:
+ ./bin/llama-completion -no-cnv --model ../models-mnt/qwen3/0.6B/ggml-model-q5_k.gguf -ngl 99 -c 1024 -s 1234 -n 64 --ignore-eos -p 'I believe the meaning of life is'
warning: no usable GPU found, --gpu-layers option will be ignored
warning: one possible reason is that llama.cpp was compiled without GPU support
warning: consult docs/build.md for compilation instructions
0.00.001.626 I llama_completion: llama backend init
0.00.001.630 I llama_completion: load the model and apply lora adapter, if any
0.00.001.637 I common_init_result: fitting params to device memory ...
0.00.001.638 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.00.191.223 I common_params_fit_impl: projected to use 839 MiB of host memory vs. 77898 MiB of total host memory
0.00.275.972 W load: control-looking token: 128247 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
0.00.347.684 W llama_context: n_ctx_seq (1024) < n_ctx_train (32768) -- the full capacity of the model will not be utilized
0.00.393.684 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.00.410.142 I llama_completion: llama threadpool init, n_threads = 8
0.00.410.164 I
0.00.410.231 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.410.233 I
0.00.410.308 I sampler seed: 1234
0.00.410.316 I sampler params:
repeat_last_n = 64, repeat_penalty = 1,000, frequency_penalty = 0,000, presence_penalty = 0,000
dry_multiplier = 0,000, dry_base = 1,750, dry_allowed_length = 2, dry_penalty_last_n = -1
top_k = 40, top_p = 0,950, min_p = 0,050, xtc_probability = 0,000, xtc_threshold = 0,100, typical_p = 1,000, top_n_sigma = -1,000, temp = 0,800
mirostat = 0, mirostat_lr = 0,100, mirostat_ent = 5,000, adaptive_target = -1,000, adaptive_decay = 0,900
0.00.410.321 I sampler chain: logits -> logit-bias -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
0.00.410.322 I generate: n_ctx = 1024, n_batch = 2048, n_predict = 64, n_keep = 0
0.00.410.322 I
I believe the meaning of life is to have fun, and to enjoy it. I think it is good to have fun, and to enjoy it, especially when it is the pursuit of something worthwhile. My life is filled with the pursuit of happiness. I spend a great deal of my time working on projects, reading books, and listening to music. I
0.01.001.021 I common_perf_print: sampling time = 9,76 ms
0.01.001.022 I common_perf_print: samplers time = 3,93 ms / 71 tokens
0.01.001.027 I common_perf_print: load time = 180,11 ms
0.01.001.028 I common_perf_print: prompt eval time = 20,33 ms / 7 tokens ( 2,90 ms per token, 344,27 tokens per second)
0.01.001.029 I common_perf_print: eval time = 559,29 ms / 63 runs ( 8,88 ms per token, 112,64 tokens per second)
0.01.001.029 I common_perf_print: total time = 590,89 ms / 70 tokens
0.01.001.030 I common_perf_print: unaccounted time = 1,51 ms / 0,3 % (total - sampling - prompt eval - eval) / (total)
0.01.001.031 I common_perf_print: graphs reused = 62
real 0m1,053s
user 0m5,198s
sys 0m0,128s
- q6_k:
+ ./bin/llama-completion -no-cnv --model ../models-mnt/qwen3/0.6B/ggml-model-q6_k.gguf -ngl 99 -c 1024 -s 1234 -n 64 --ignore-eos -p 'I believe the meaning of life is'
warning: no usable GPU found, --gpu-layers option will be ignored
warning: one possible reason is that llama.cpp was compiled without GPU support
warning: consult docs/build.md for compilation instructions
0.00.002.554 I llama_completion: llama backend init
0.00.002.559 I llama_completion: load the model and apply lora adapter, if any
0.00.002.566 I common_init_result: fitting params to device memory ...
0.00.002.566 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.00.203.740 I common_params_fit_impl: projected to use 888 MiB of host memory vs. 77898 MiB of total host memory
0.00.300.657 W load: control-looking token: 128247 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
0.00.375.800 W llama_context: n_ctx_seq (1024) < n_ctx_train (32768) -- the full capacity of the model will not be utilized
0.00.420.335 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.00.437.505 I llama_completion: llama threadpool init, n_threads = 8
0.00.437.525 I
0.00.437.593 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.437.596 I
0.00.437.677 I sampler seed: 1234
0.00.437.687 I sampler params:
repeat_last_n = 64, repeat_penalty = 1,000, frequency_penalty = 0,000, presence_penalty = 0,000
dry_multiplier = 0,000, dry_base = 1,750, dry_allowed_length = 2, dry_penalty_last_n = -1
top_k = 40, top_p = 0,950, min_p = 0,050, xtc_probability = 0,000, xtc_threshold = 0,100, typical_p = 1,000, top_n_sigma = -1,000, temp = 0,800
mirostat = 0, mirostat_lr = 0,100, mirostat_ent = 5,000, adaptive_target = -1,000, adaptive_decay = 0,900
0.00.437.692 I sampler chain: logits -> logit-bias -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
0.00.437.693 I generate: n_ctx = 1024, n_batch = 2048, n_predict = 64, n_keep = 0
0.00.437.693 I
I believe the meaning of life is to give back, and I have given back to the world. My life is full of love, and I love love. I love the way the world is beautiful, and I love the people I love, and I love all who love me. I love the beauty of the world, and I love the love that
0.01.076.365 I common_perf_print: sampling time = 9,47 ms
0.01.076.366 I common_perf_print: samplers time = 3,84 ms / 71 tokens
0.01.076.370 I common_perf_print: load time = 194,56 ms
0.01.076.372 I common_perf_print: prompt eval time = 16,94 ms / 7 tokens ( 2,42 ms per token, 413,32 tokens per second)
0.01.076.374 I common_perf_print: eval time = 610,85 ms / 63 runs ( 9,70 ms per token, 103,14 tokens per second)
0.01.076.374 I common_perf_print: total time = 638,87 ms / 70 tokens
0.01.076.375 I common_perf_print: unaccounted time = 1,61 ms / 0,3 % (total - sampling - prompt eval - eval) / (total)
0.01.076.375 I common_perf_print: graphs reused = 62
real 0m1,133s
user 0m5,610s
sys 0m0,139s
- save-load-state: