llama.cpp demo

output.txt

C:\pub\llama.cpp\build>.\bin\Release\main.exe -m C:\pub\llama\llama-2-7b-chat\ggml-model-q4_K_M.bin -c 2048 -p "translate in japanese: The pen is mightier than the sword."
main: build = 852 (294f424)
main: seed  = 1690113032
llama.cpp: loading model from C:\pub\llama\llama-2-7b-chat\ggml-model-q4_K_M.bin
llama_model_load_internal: format     = ggjt v3 (latest)
llama_model_load_internal: n_vocab    = 32000
llama_model_load_internal: n_ctx      = 2048
llama_model_load_internal: n_embd     = 4096
llama_model_load_internal: n_mult     = 256
llama_model_load_internal: n_head     = 32
llama_model_load_internal: n_layer    = 32
llama_model_load_internal: n_rot      = 128
llama_model_load_internal: freq_base  = 10000.0
llama_model_load_internal: freq_scale = 1
llama_model_load_internal: ftype      = 15 (mostly Q4_K - Medium)
llama_model_load_internal: n_ff       = 11008
llama_model_load_internal: model size = 7B
llama_model_load_internal: ggml ctx size =    0.08 MB
llama_model_load_internal: mem required  = 5563.32 MB (+ 1026.00 MB per state)
llama_new_context_with_model: kv self size  = 1024.00 MB

system_info: n_threads = 12 / 24 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 |
sampling: repeat_last_n = 64, repeat_penalty = 1.100000, presence_penalty = 0.000000, frequency_penalty = 0.000000, top_k = 40, tfs_z = 1.000000, top_p = 0.950000, typical_p = 1.000000, temp = 0.800000, mirostat = 0, mirostat_lr = 0.100000, mirostat_ent = 5.000000
generate: n_ctx = 2048, n_batch = 512, n_predict = -1, n_keep = 0


 translate in japanese: The pen is mightier than the sword. Japanese people often use this proverb to encourage others to use their words and ideas to make a difference in the world, rather than resorting to violence. In this translation, the word for "pen" (筆) is written with two characters, 筆 and 遊, which together mean "to write."
Here are some other ways the phrase "The pen is mightier than the sword" can be translated in Japanese:
* 筆は剣よりも強い (Hookup wa ken yori mo tsuyoi) - This translation uses the word for "hookup" (筆) instead of "pen," which is more commonly used in Japan.
* 文字は剣の力を凌ぐ (Monji wa ken no chikara o furu) - This translation uses the word for "character" (文字) to emphasize the idea that words can be just as powerful as a sword.
* 評論は剣に勝る (Hyoryon wa ken ni katuru) - This translation uses the word for "criticism" (評論) to emphasize the idea that the power of words can be more effective than physical strength.
In Japan, this proverb is often used in a variety of contexts, including politics, literature, and education. It is a reminder that ideas and words have the power to shape society and make a positive impact on the world, and that violence should never be seen as a solution to any problem. [end of text]

llama_print_timings:        load time =   602.31 ms
llama_print_timings:      sample time =    44.31 ms /   340 runs   (    0.13 ms per token,  7673.73 tokens per second)
llama_print_timings: prompt eval time =   742.60 ms /    16 tokens (   46.41 ms per token,    21.55 tokens per second)
llama_print_timings:        eval time = 35378.33 ms /   339 runs   (  104.36 ms per token,     9.58 tokens per second)
llama_print_timings:       total time = 36209.68 ms