Skip to content

Instantly share code, notes, and snippets.

@awni
Last active July 8, 2025 23:35
Show Gist options
  • Save awni/773e2a12079da40a1cbc566686c84c8f to your computer and use it in GitHub Desktop.
Save awni/773e2a12079da40a1cbc566686c84c8f to your computer and use it in GitHub Desktop.
Display the source blob
Display the rendered blob
Raw
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@crack93
Copy link

crack93 commented Jul 6, 2025

remove the tokenizer from there and use train_dataset=CacheDataset(ds_train),
val_dataset=CacheDataset(ds_valid),

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment