Skip to content

Instantly share code, notes, and snippets.

View yazdanbakhsh's full-sized avatar
💭
I may be slow to respond.

Amir Yazdanbakhsh yazdanbakhsh

💭
I may be slow to respond.
View GitHub Profile
@yazdanbakhsh
yazdanbakhsh / vit_base_patch16_224
Created April 3, 2023 17:54
Hyperparameters for ViT-Base-P16-224
aa: rand-m15-n2-p1-mstd0
amp: true
amp_dtype: bfloat16
amp_impl: native
aot_autograd: false
aug_repeats: 0
aug_splits: 0
batch_size: 256
bce_loss: true
bce_target_thresh: null
@yazdanbakhsh
yazdanbakhsh / ViT-B-16-Pretrain
Created December 26, 2022 20:01
Configuration used to train ViT-B/16 from scratch using timm library
aa: rand-m15-n2-p1-mstd0
amp: true
amp_dtype: float16
amp_impl: native
aot_autograd: false
aug_repeats: 0
aug_splits: 0
batch_size: 256
bce_loss: true
bce_target_thresh: null