Skip to content

Instantly share code, notes, and snippets.

@AmosLewis

AmosLewis/castf16.py

Last active January 27, 2025 19:33
Show Gist options
  • Save AmosLewis/404230fb1c51c903214c360df4d63284 to your computer and use it in GitHub Desktop.
Save AmosLewis/404230fb1c51c903214c360df4d63284 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
castf16.py
# Download the f32 npy, then use this script to cast 32 to 16
# wget https://gist.github.com/aviator19941/380acabc77aeb4749fac14262e17db69
# wget https://sharkpublic.blob.core.windows.net/sharkpublic/halo-models/llm-dev/llama3_8b/prefill_args_bs4_128_stride_32/cs_f16.npy
# wget https://sharkpublic.blob.core.windows.net/sharkpublic/halo-models/llm-dev/llama3_8b/prefill_args_bs4_128_stride_32/seq_block_ids.npy
# wget https://sharkpublic.blob.core.windows.net/sharkpublic/halo-models/llm-dev/llama3_8b/prefill_args_bs4_128_stride_32/seq_lens.npy
# wget https://sharkpublic.blob.core.windows.net/sharkpublic/halo-models/llm-dev/llama3_8b/prefill_args_bs4_128_stride_32/tokens.npy
# pip install numpy==1.26
# pip install bfloat16
import numpy as np
from bfloat16 import bfloat16
prefill = ['cs_f16.npy','seq_block_ids.npy','seq_lens.npy','tokens.npy']
for prefillnpy in prefill:
data = np.load(prefillnpy)
data_converted = data.astype(bfloat16)
bf16prefillnpy = 'bf16_'+prefillnpy
np.save(bf16prefillnpy, data_converted)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment