Skip to content

Instantly share code, notes, and snippets.

@ragulpr

ragulpr/zero-padding-batchnorm-keras.ipynb

Created February 27, 2019 00:22
Show Gist options
  • Save ragulpr/50b7011e7348944bee1ee160db2fbe0a to your computer and use it in GitHub Desktop.
Save ragulpr/50b7011e7348944bee1ee160db2fbe0a to your computer and use it in GitHub Desktop.
Download ZIP
Display the source blob
Display the rendered blob
Raw
zero-padding-batchnorm-keras.ipynb
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@ragulpr
Copy link
Author

ragulpr commented Aug 28, 2019

@AnirudhDagar, unless things have happened over in the Torch (and in particular, the CUDA)-community since last year when I checked, Batchnorm does not support masking. Unfortunately, there's many numerical gotchas and BatchNorm has highly optimized low level implementations, so it seems like it wouldn't be very feasible to just write it using the basic Pytorch python API (I've tried, it was slow).

@AnirudhDagar
Copy link

Thanks for the explanation :)

@drussellmrichie
Copy link

This is old but I found this post from reddit, and this is just to say that Keras BatchNormalization does support masking now:

https://keras.io/api/layers/normalization_layers/batch_normalization/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment