Skip to content

Instantly share code, notes, and snippets.

@eustin

eustin/letor-part-1-model-vocab-size.py

Created May 25, 2020 22:37
Show Gist options
  • Save eustin/6819799eac96fea7288caeb7de62aa14 to your computer and use it in GitHub Desktop.
Save eustin/6819799eac96fea7288caeb7de62aa14 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
letor-part-1-model-vocab-size.py
VOCAB_SIZE = max(tokeniser.index_word) + 1
print(f"VOCAB_SIZE: {VOCAB_SIZE}")
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment