gistfile1.txt

% ./ds4 \
  -m ./ds4flash.gguf \
  --ssd-streaming \
  --ssd-streaming-cache-experts 32GB \
  --ctx 32768 \
  --nothink
ds4: Metal SSD streaming cache budget 32.00 GiB / 6.75 MiB per expert = 4854 experts
ds4: Metal device Apple M1 Max, 64.00 GiB RAM
ds4: Metal 4 tensor API disabled for pre-M5/pre-A19 devices
ds4: drift-patch flags hc_stable=on norm_unify=on kv_raw_f32=off rope_exp2_log2=off math_safe=off tensor_matmul=off
ds4: Metal SSD streaming mode enabled; full model residency and warmup are skipped
ds4: SSD streaming initial metal model map restricted to token embedding (1 spans, 0.99 GiB tensor span)
ds4: metal backend initialized for graph diagnostics
ds4: context buffers 880.67 MiB (ctx=32768, backend=metal, prefill_chunk=4096, raw_kv_rows=4352, compressed_kv_rows=8194)
Commands:
  /help          Show this help.
  /think         Use normal thinking mode.
  /think-max     Use Think Max only when context is at least 393216 tokens.
  /nothink       Disable thinking mode.
  /ctx N         Set context size for following prompts.
  /power N       Set GPU duty cycle percentage, 1..100.
  /read FILE     Read a prompt from FILE and run it.
  /quit, /exit   Leave the prompt.
  Ctrl+C         Stop generation and return to the prompt.
ds4> /read README.md
processing 15740 input tokens: 15740/15740 (100.0%)
This is a comprehensive README for DwarfStar, a specialized inference engine for DeepSeek V4 Flash and PRO models. The document covers:

- **Project scope**: A standalone native engine (not a generic GGUF runner) optimized for DeepSeek V4 models
- **Hardware targets**: MacBooks (96-512GB RAM), NVIDIA DGX Spark, with AMD ROCm support in a separate branch
- **Key features**:
  - SSD streaming for models larger than RAM
  - Distributed inference across multiple machines
  - Disk-based KV cache persistence
  - Native coding agent with session management
  - Multiple backends (Metal, CUDA, CPU)
  - Thinking modes and steering capabilities
- **Practical usage**: Download scripts, build instructions, CLI/server setup, agent client configuration
- **Technical details**: GGUF quantization, distributed protocol, KV cache format, benchmarking tools

Would you like help with any specific aspect of this project, such as setup, configuration, or understanding a particular feature?
ds4: prefill: 112.09 t/s, generation: 7.15 t/s
ds4>