Skip to content

Instantly share code, notes, and snippets.

@matthewchung74
Created February 5, 2021 23:48
Show Gist options
  • Save matthewchung74/2635bd1e1563ccb44291d6790857dffb to your computer and use it in GitHub Desktop.
Save matthewchung74/2635bd1e1563ccb44291d6790857dffb to your computer and use it in GitHub Desktop.
embed_dim = 768
num_heads = 8
block = Block(embed_dim, 8)
batch_size = 1
class_token = nn.Parameter(torch.zeros(1, 1, embed_dim))
class_tokens = class_token.expand(batch_size, -1, -1)
pos_embed = nn.Parameter(torch.zeros(1, num_patches + 1, embed_dim))
x = torch.cat((class_tokens, patch_output), dim=1)
x = x + pos_embed
block(x)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
<ipython-input-15-b5f1798f46a9> in <module>()
1 embed_dim = 768
2 num_heads = 8
----> 3 block = Block(embed_dim, 8)
4
5 batch_size = 1
NameError: name 'Mlp' is not defined
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment