Touhidul Alam touhi99

🎧

Focusing

NLP Developer | Agent is all I seek

touhi99 / get_memory_size.py

Created January 30, 2025 17:07 — forked from philschmid/get_memory_size.py

Get needed GPU per precision for a Hugging Face Model Id

	from typing import Dict, Union
	from huggingface_hub import get_safetensors_metadata
	import argparse
	import sys

	# Example:
	# python get_gpu_memory.py Qwen/Qwen2.5-7B-Instruct

	# Dictionary mapping dtype strings to their byte sizes
	bytes_per_dtype: Dict[str, float] = {