Rob Bos rajbos
rajbos
/ Measure-LocalAIThroughput.ps1
Last active
May 9, 2026 11:54
Benchmarks output throughput (tokens/sec) for local AI inference: Ollama, LM Studio, Foundry Local, and vLLM (HuggingFace via Docker).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #Requires -Version 7 | |
| <# | |
| .SYNOPSIS | |
| Benchmarks output throughput (tokens/sec) for local AI inference providers. | |
| .DESCRIPTION | |
| Sends a standardised prompt to Ollama, LM Studio, Foundry Local, and/or | |
| vLLM (HuggingFace), measures tokens generated per second using the | |
| OpenAI-compatible /v1/chat/completions API, and outputs a Markdown summary table. | |
| Each provider is tested $Runs times; results are averaged. |
OlderNewer