Skip to content

Instantly share code, notes, and snippets.

@alvarobartt

alvarobartt/estimate-deepseek-vram.py

Created January 31, 2025 15:52
Show Gist options
  • Save alvarobartt/264bfcb0e7b8abe584cfcd13ae9a29ed to your computer and use it in GitHub Desktop.
Save alvarobartt/264bfcb0e7b8abe584cfcd13ae9a29ed to your computer and use it in GitHub Desktop.
Download ZIP
Calculates the required VRAM for DeepSeek R1 (pulled from the Hugging Face Hub Safetensors metadata)
Raw
estimate-deepseek-vram.py
from huggingface_hub import get_safetensors_metadata
model_id = "deepseek-ai/DeepSeek-R1"
dtype_bytes = {"F32": 4, "F16": 2, "F8": 1}
metadata = get_safetensors_metadata(model_id)
memory = (
sum(count * dtype_bytes[key.split("_")[0]] for key, count in metadata.parameter_count.items())
/ (1024**3)
* 1.18
)
print(f"{model_id=} requires {memory=}GB")
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment