benchmark_default_prompt.text

llm_benchmark run --custombenchmark=custom_models.yml --no-sendinfo

-------Linux----------

{'id': '0', 'name': 'NVIDIA GeForce RTX 4060', 'driver': '535.309.01', 'gpu_memory_total': '8188.0 MB', 'gpu_memory_free': '7257.0 MB', 'gpu_memory_used': '674.0 MB', 'gpu_load': '16.0%', 'gpu_temperature': '50.0°C'}

Only one GPU card

Total memory size : 15.51 GB

cpu_info: Intel(R) Core(TM) i5-10400F CPU @ 2.90GHz

gpu_info: NVIDIA GeForce RTX 4060

os_version: Ubuntu 22.04.5 LTS

ollama_version: 0.24.0

----------

running custom benchmark from models_file_path: custom_models.yml

Disabling sendinfo for custom benchmark

LLM models file path：custom_models.yml

Checking and pulling the following LLM models

llama3.1:latest

qwen3.5:latest

gemma4:e4b

----------

Running custom-model

model_name =    llama3.1:latest

prompt = Summarize the key differences between classical and operant conditioning in psychology.

eval rate:            48.06 tokens/s

prompt = Translate the following English paragraph into Chinese and elaborate more -> Artificial intelligence is transforming various industries by enhancing efficiency and enabling new capabilities.

eval rate:            50.16 tokens/s

prompt = What are the main causes of the American Civil War?

eval rate:            49.16 tokens/s

prompt = How does photosynthesis contribute to the carbon cycle?

eval rate:            49.67 tokens/s

prompt = Develop a python function that solves the following problem, sudoku game.

eval rate:            46.89 tokens/s

--------------------

Average of eval rate:  48.788  tokens/s

----------------------------------------

model_name =    qwen3.5:latest

prompt = Summarize the key differences between classical and operant conditioning in psychology.

eval rate:            19.99 tokens/s

prompt = Translate the following English paragraph into Chinese and elaborate more -> Artificial intelligence is transforming various industries by enhancing efficiency and enabling new capabilities.

eval rate:            19.94 tokens/s

prompt = What are the main causes of the American Civil War?

eval rate:            20.15 tokens/s

prompt = How does photosynthesis contribute to the carbon cycle?

eval rate:            20.15 tokens/s

prompt = Develop a python function that solves the following problem, sudoku game.

eval rate:            20.18 tokens/s

--------------------

Average of eval rate:  20.082  tokens/s

----------------------------------------

model_name =    gemma4:e4b

prompt = Summarize the key differences between classical and operant conditioning in psychology.

eval rate:            25.83 tokens/s

prompt = Translate the following English paragraph into Chinese and elaborate more -> Artificial intelligence is transforming various industries by enhancing efficiency and enabling new capabilities.

eval rate:            25.85 tokens/s

prompt = What are the main causes of the American Civil War?

eval rate:            25.75 tokens/s

prompt = How does photosynthesis contribute to the carbon cycle?

eval rate:            25.88 tokens/s

prompt = Develop a python function that solves the following problem, sudoku game.

eval rate:            25.70 tokens/s

--------------------

Average of eval rate:  25.802  tokens/s