Dot-product and Multi-head attention from the paper "Attention is all you need" (2017). Implementation in modern Tensorflow 2 using the Keras API.
Example use of the implementations below:
batch_size = 10
n_vectors = 150
d_model = 512
Problem: When running a conda environment and opening tmux on macOS, a utility called path_helper
is run again. Essentially, the shell is initialized twice which messes up the ${PATH}
so that the wrong Python version shows up within tmux
.
If using bash, edit /etc/profile
and add one line. (For zsh
, edit /etc/zprofile
)
...
#!/bin/bash | |
# | |
# curl -O https://gist.githubusercontent.com/ekreutz/b245da335cc10b0d270136b35eb22d1f/raw/43c6d07b0b3d3165af9c10cef18e8709a07ce680/resetXcode.sh && bash ./resetXcode.sh ; rm ./resetXcode.sh | |
# | |
if [[ $(whoami) != jenkins ]]; then | |
echo "Run as jenkins" | |
exit 1 | |
fi |