Skip to content

Instantly share code, notes, and snippets.

@thomwolf

thomwolf/comparing-hidden-states.py

Created August 9, 2019 11:18
Show Gist options
  • Save thomwolf/7826d82ace410429d5d3d11e161af905 to your computer and use it in GitHub Desktop.
Save thomwolf/7826d82ace410429d5d3d11e161af905 to your computer and use it in GitHub Desktop.
Download ZIP
Compare the hidden-states of the TensorFlow and PyTorch models
Raw
comparing-hidden-states.py
# Get the tensorflow and pytorch hidden-states as NumPy arrays
tensorflow_hidden_states = sess.run(feed_dict)
pytorch_hidden_states = pytorch_model(inputs)
pytorch_hidden_states = pytorch_hidden_states.cpu().detach().numpy()
# Compute the maximum absolute difference between hidden-states.
# Should be less than 1e-3. Typically around 1e-5/1e-6.
max_absolute_diff = np.amax(np.abs(tensorflow_hidden_states - pytorch_hidden_states))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment