Skip to content

Instantly share code, notes, and snippets.

@pythonlessons

pythonlessons/transformers_introduction_4.py

Created August 10, 2023 13:19
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save pythonlessons/77595ee90832982bc69b70220b9b53af to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save pythonlessons/77595ee90832982bc69b70220b9b53af to your computer and use it in GitHub Desktop.
Download ZIP
transformers_introduction
Raw
transformers_introduction_4.py
vocab_size = 1000
d_model = 512
embedding_layer = PositionalEmbedding(vocab_size, d_model)
random_input = np.random.randint(0, vocab_size, size=(1, 100))
output = embedding_layer(random_input)
print("random_input shape", random_input.shape)
print("PositionalEmbedding output", output.shape)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment