Skip to content

Instantly share code, notes, and snippets.

@akash-ch2812
Created July 24, 2020 06:31
Show Gist options
  • Save akash-ch2812/0596e461ed0f6fd3201a86453d326a02 to your computer and use it in GitHub Desktop.
Save akash-ch2812/0596e461ed0f6fd3201a86453d326a02 to your computer and use it in GitHub Desktop.
# compute length of vocabulary and maximum length of a caption (for padding)
vocab_len = len(tokenizer.word_counts) + 1
print(f"Vocabulary length - {vocab_len}")
max_caption_len = max([len(x.split(" ")) for x in all_captions])
print(f"Maximum length of caption - {max_caption_len}")
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment