Skip to content

Instantly share code, notes, and snippets.

@shubham0204

shubham0204/mlp_mixer_3.py

Created June 13, 2021 01:46
Show Gist options
  • Save shubham0204/a734b13d0533f1ee3f98b6e27b1b3013 to your computer and use it in GitHub Desktop.
Save shubham0204/a734b13d0533f1ee3f98b6e27b1b3013 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
mlp_mixer_3.py
# Channel Mixing MLPs : Allow communication within channels ( features of embeddings )
def channel_mixing( x , channel_mixing_mlp_dims ):
# x is a tensor of shape ( batch_size , num_patches , channels )
x = tf.keras.layers.LayerNormalization( epsilon=1e-6 )( x )
x = mlp( x , channel_mixing_mlp_dims )
return x
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment