Skip to content

Instantly share code, notes, and snippets.

@Lexie88rus

Lexie88rus/silu_inplace_implementation.py

Created July 10, 2019 08:16
Show Gist options
  • Save Lexie88rus/01e81cc442afc498070b97a5dffcef2e to your computer and use it in GitHub Desktop.
Save Lexie88rus/01e81cc442afc498070b97a5dffcef2e to your computer and use it in GitHub Desktop.
Download ZIP
Example of implementation of in-place SiLU activation function
Raw
silu_inplace_implementation.py
def silu_inplace_2(input):
'''
Example of implementation of in-place SiLU activation function using torch.sigmoid_
https://arxiv.org/pdf/1606.08415.pdf
'''
result = input.clone()
torch.sigmoid_(input)
input *= result
return input
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment