Some bugs exist in the newest version but I found a checkout point that works well! Please follow the instructions below.
sudo apt-get install libboost-all-dev
conda install pybind11
Note: pip install pybind11
will cause error later, make sure you use conda.