-
Install or upgrade
pythonandpip:# Install sudo apt install python3 # Upgrade - better upgrading all packages since many are connected to Python sudo apt update sudo apt upgrade# Install sudo apt install python3-pip # Upgrade python3 -m pip install --upgrade pip- Make sure to have at least Python 3.6+. To check your version:
Python 2 has been deprecatedpython3 --version
- Make sure to have at least Python 3.6+. To check your version:
-
Download the MuJoCo version 2.1 binary for Linux
- If the previous link doesn't work, go to the Github release page version 2.1 and download mujoco210-linux-x86_64.tar.gz
- Note: new MuJoCo versions may be available, but make sure to download version 2.1
-
cdwhere you downloaded the file and extract the downloadedmujoco210directory into~/.mujoco/mujoco210, either manually or by running the following command:mkdir -p ~/.mujoco && tar -zxvf mujoco210-linux-x86_64.tar.gz --directory ~/.mujoco -
Please check that path and directory
~/.mujoco/mujoco210exist and contain the downloaded files -
Depending on what shell terminal you use, it is necessary to add an
exportcommand to the shell configuration file. As a rule of thumb, if you do not know what shell you are using, it means is probablybash-
For
bashshell (usually the standard pre-installed one)-
Add the line
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/.mujoco/mujoco210/binat the end of file~/.bashrc, either manually or by running the following command:echo 'export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/.mujoco/mujoco210/bin' >> ~/.bashrc -
Open file
~/.bashrcand check if the line was correctly appended at the end:cat ~/.bashrc -
Run the following to apply the latest
exportcommand:source ~/.bashrc
-
-
For other shells (like
zsh) follow the steps above described forbash, but instead of editing~/.bashrc, modify the correspondent configuration file for your shell- For example, if you are using
zsh, edit~/.zshrclike described before
- For example, if you are using
-
-
Install the following:
sudo apt install python3-dev sudo apt install build-essential sudo apt install patchelf -
For Ubuntu distributions - may be optional
- Install the following:
sudo apt install libosmesa6-dev libgl1-mesa-glx libglfw3 - Create a symbolic link - it may already exists
sudo ln -s /usr/lib/x86_64-linux-gnu/libGL.so.1 /usr/lib/x86_64-linux-gnu/libGL.so
- Install the following:
-
Install using
pip3:pip3 install -U 'mujoco-py<2.2,>=2.1'
- Run
python3:python3 - In
python3shell import MuJoCo - we are going to compile the library:import mujoco_py
If the import is clean and Python doesn't throw any error: congrats! You just installed MuJoCo 2.1 successfully!
- Run
python3:python3 - Paste the following in
python3shell:import os import mujoco_py mj_path = mujoco_py.utils.discover_mujoco() xml_path = os.path.join(mj_path, 'model', 'humanoid.xml') model = mujoco_py.load_model_from_path(xml_path) sim = mujoco_py.MjSim(model) print(sim.data.qpos) sim.step() print(sim.data.qpos) - If MuJoCo works properly, the script will successfully print data, like:
First print: # [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] Second print: # [-2.09531783e-19 2.72130735e-05 6.14480786e-22 -3.45474715e-06 # 7.42993721e-06 -1.40711141e-04 -3.04253586e-04 -2.07559344e-04 # 8.50646247e-05 -3.45474715e-06 7.42993721e-06 -1.40711141e-04 # -3.04253586e-04 -2.07559344e-04 -8.50646247e-05 1.11317030e-04 # -7.03465386e-05 -2.22862221e-05 -1.11317030e-04 7.03465386e-05 # -2.22862221e-05]
If you need to include mujoco-py in your own package, add it to your requirements like so:
mujoco-py<2.2,>=2.1
-
brewis required. If you do not have it, install it:/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" -
Install or upgrade
pythonandpip:# Install & upgrade brew install python3pip3is automatically installed withpython3- Make sure to have at least Python 3.6+. To check your version:
Python 2 has been deprecatedpython3 --version
- Make sure to have at least Python 3.6+. To check your version:
-
Download the MuJoCo version 2.1 binary for OSX
- If the previous link doesn't work, go to the Github release page version 2.1 and download mujoco210-macos-x86_64.tar.gz
- Note: new MuJoCo versions may be available, but make sure to download version 2.1
-
cdwhere you downloaded the file and extract the downloadedmujoco210directory into~/.mujoco/mujoco210, either manually or by running the following command:mkdir -p ~/.mujoco && tar -zxvf mujoco210-macos-x86_64.tar.gz --directory ~/.mujoco -
Please check that path and directory
~/.mujoco/mujoco210exist and contain the downloaded files -
Depending on what shell terminal you use, it is necessary to add an
exportcommand to the shell configuration file. As a rule of thumb, if you do not know what shell you are using, it means is probablyzsh-
For
zshshell (usually the standard pre-installed one)-
Add the line
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/.mujoco/mujoco210/binat the end of file~/.zshrc, either manually or by running the following command:echo 'export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/.mujoco/mujoco210/bin' >> ~/.zshrc -
Open file
~/.zshrcand check if the line was correctly appended at the end:cat ~/.zshrc -
Run the following to apply the latest
exportcommand:source ~/.zshrc
-
-
For other shells follow the steps above described for
zsh, but instead of editing~/.zshrc, modify the correspondent configuration file for your shell
-
-
Install the following:
brew install gcc@9 brew install patchelf -
Install using
pip3:pip3 install -U 'mujoco-py<2.2,>=2.1'
- Run
python3:python3 - In
python3shell import MuJoCo - we are going to compile the library:import mujoco_py
If the import is clean and Python doesn't throw any error: congrats! You just installed MuJoCo 2.1 successfully!
- Run
python3:python3 - Paste the following in
python3shell:import os import mujoco_py mj_path = mujoco_py.utils.discover_mujoco() xml_path = os.path.join(mj_path, 'model', 'humanoid.xml') model = mujoco_py.load_model_from_path(xml_path) sim = mujoco_py.MjSim(model) print(sim.data.qpos) sim.step() print(sim.data.qpos) - If MuJoCo works properly, the script will successfully print data, like:
First print: # [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] Second print: # [-2.09531783e-19 2.72130735e-05 6.14480786e-22 -3.45474715e-06 # 7.42993721e-06 -1.40711141e-04 -3.04253586e-04 -2.07559344e-04 # 8.50646247e-05 -3.45474715e-06 7.42993721e-06 -1.40711141e-04 # -3.04253586e-04 -2.07559344e-04 -8.50646247e-05 1.11317030e-04 # -7.03465386e-05 -2.22862221e-05 -1.11317030e-04 7.03465386e-05 # -2.22862221e-05]
If you need to include mujoco-py in your own package, add it to your requirements like so:
mujoco-py<2.2,>=2.1
Please note that MuJoCo support for Windows has been deprecated and removed in 2.0.2.0. One known good past version is 1.50.1.68, so this is the version we are going to install
Note: please use Powershell or Powershell Core (also available on Chocolatey) terminal in Admin mode for this tutorial
Using Anaconda or Miniconda is not recommended
-
Install or upgrade
python, either using Chocolatey package manager or manually:-
Manually: https://www.python.org/downloads/windows/ Download the latest stable release compatible with your system, at least Python 3.6+
-
Using Chocolatey:
# Install choco install python3 # Upgrade choco upgrade python3
-
-
Make sure to have at least Python 3.6+. To check your version:
python --versionPython 2 has been deprecated
Note: if you have Anaconda/Miniconda installed on your system, please make sure to check the order of your Envrironment Variables.
Having a wrong path order leads to problems when opening a command line.
If that is the case:
-
Run
pythonon a terminal:python -
If you see a message like:
This Python interpreter is in a conda environment, but the environment has not been activated. Libraries may fail to load.Then your
pathEnvironment Variable needs to be fixed -
In the System Environment Variable
PATH, your Python installation paths (usuallyC:\Python310\andC:\Python310\Scripts\) should be on top of Anaconda/Miniconda paths (usuallyC:\tools\miniconda3and all the other folders referring toconda). To check and fix your Environment Variables: -
Open Environment Variables panel, either manually or by running the following (in Admin mode):
start-process rundll32.exe sysdm.cpl,EditEnvironmentVariables -verb RunAs -
In the System Variables (not for current user) section, open the variable
PATH -
Check if the paths referring to python installation (like
C:\Python310\andC:\Python310\Scripts\) are under the Anaconda/Miniconda paths -
If that is true, bring python's paths on top of the Anaconda/miniconda ones
- Press
Okto exit the panel andOkagain to save changes
- Press
Continue MuJoCo installation:
-
Install or upgrade
pip:# Install python get-pip.py # Upgrade python -m pip install --upgrade pip -
Download the MuJoCo version 1.50 binary for Windows
-
cdwhere you downloaded the file and extract the downloadedmjpro150directory into$home\.mujoco\mjpro150, either manually or by running the following command:Expand-Archive -LiteralPath "mjpro150_win64.zip" -DestinationPath "$home\.mujoco" -
Please check that path and directory
$home\.mujoco\mjpro150exist and contain the downloaded files -
Add the path to
$home\.mujoco\mjpro150to the SystempathEnvironment Variable, either manually or by running the following (Admin mode):# Run this to save your current Path variable in a file just in case $Env:PATH >> Env_Path.txt # Make sure to include the plus and the semicolon [Environment]::SetEnvironmentVariable("PATH", $Env:PATH + ";$home\.mujoco\mjpro150\bin", [EnvironmentVariableTarget]::Machine) -
Close and reopen a new Powershell in admin mode (it is necessary to print new Environment Variables, just running
refreshenvwon't work) -
Check if the line was correctly appended at the end of the
pathvariable:$Env:PATH -
Download the
mjkey.txtfree licence activation key from the Roboti licence page- You need an activation key because MuJoCo was a paid software before version 2.0. In this case DeepMind has granted Roboti LLC license to provide an unlocked activation key for legacy versions of MuJoCo (2.0 and earlier).
-
cdwhere you downloaded the file and placemjkey.txtin$home\.mujoco, either manually or by running the following:Copy-Item "mjkey.txt" -Destination "$home\.mujoco" -
Download the MuJoCo 1.50.1.68 repository in
.zip- Do not attempt to directly clone the repository since it will download the latest version
-
cdwhere you downloaded the file and extract the downloadedmujoco-py-9ea9bb000d6b8551b99f9aa440862e0c7f7b4191.zipfile in the same directory, either manually or by running the following:Expand-Archive -LiteralPath "mujoco-py-9ea9bb000d6b8551b99f9aa440862e0c7f7b4191.zip" -
Making sure to stay in the same directory, open
mujoco-py-9ea9bb000d6b8551b99f9aa440862e0c7f7b4191folder and move the inner folder (which has the same name) outside. The purpose is to accessmujoco-py-9ea9bb000d6b8551b99f9aa440862e0c7f7b4191folder contents directly (and not having a sub folder). Then rename the just moved folder using a shorter name, likemujoco-py-repo. Do all of this either manually or by running the following, making sure to stay in the directory you extracted the folder:cd mujoco-py-9ea9bb000d6b8551b99f9aa440862e0c7f7b4191 Move-Item -Path mujoco-py-9ea9bb000d6b8551b99f9aa440862e0c7f7b4191 -Destination ..\mujoco-py-repo cd .. Remove-Item mujoco-py-9ea9bb000d6b8551b99f9aa440862e0c7f7b4191- The purpose is to shorten the folder path, since a long path will throw an error when compiling later
-
Download Microsoft C++ Build Tools and install them (about 100MB)
-
Once installed you should have a new terminal called
x64 Native Tools Command Prompt for VS 2019(the version year may change). You should be able to find it from the Windows Menu, or in the following path:C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Visual Studio 2019\Visual Studio Tools\VCNote: it is important to run it in Admin mode and from the Windows menu or the path shown above, since the link includes some dependencies when executed
-
Run
x64 Native Tools Command Prompt for VS 2019from the Windows Menu in Admin mode -
cdinto the previously extracted and renamed foldermujoco-py-repo -
Run the following:
pip install -r requirements.txt pip install -r requirements.dev.txt python setup.py install -
Run
python:python -
If your
pythonversion is above 3.8 (python3 --versionto check):- Since Python 3.8, only "trusted dlls" are loaded. For this reason, Windows users having Python 3.8+ need to to add the location of the installed MuJoCo version to the list of allowed DLLs at the beginning of every Python script that uses
mujoco-py. Otherwise it will throw a similar error:ImportError: DLL load failed while importing cymj: The specified module could not be found.. - To do this, in
pythonshell importos:import os - In
pythonshell add MuJoCo dlls to the trusted dlls:
Note: on Windows you have to do this for every Python script that useshome = os.path.expanduser("~") os.add_dll_directory(home + "\\.mujoco\\mjpro150\\bin")mujoco-py.
- Since Python 3.8, only "trusted dlls" are loaded. For this reason, Windows users having Python 3.8+ need to to add the location of the installed MuJoCo version to the list of allowed DLLs at the beginning of every Python script that uses
-
In
pythonshell import MuJoCo - we are going to compile the library:import mujoco_py
If the import is clean and Python doesn't throw any error: congrats! You just installed MuJoCo 1.50.1.68 successfully!
-
Run
python:python -
Paste the following in
pythonshell:import os home = os.path.expanduser("~") os.add_dll_directory(home + "\\.mujoco\\mjpro150\\bin") import mujoco_py mj_path = mujoco_py.utils.discover_mujoco()[0] xml_path = os.path.join(mj_path, 'model', 'humanoid.xml') model = mujoco_py.load_model_from_path(xml_path) sim = mujoco_py.MjSim(model) print(sim.data.qpos) sim.step() print(sim.data.qpos)Note: the instruction
mj_path = mujoco_py.utils.discover_mujoco()[0]has a[0]at the end becausemujoco_py.utils.discover_mujoco()returns atupleof this form:('$home\\.mujoco\\mjpro150', '$home\\.mujoco\\mjkey.txt'). Since MuJoCo version 1.50.1.68 needs a key in$home\.mujoco\mjkey.txt, the previous instruction retires both foldermjpro150and filemjkey.txt. We only need the folder path when coding, so make sure to include[0]inmj_path = mujoco_py.utils.discover_mujoco()[0]. This tip is only for Windows -
If MuJoCo works properly, the script will successfully print data, like:
First print: # [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] Second print: # [-2.09531783e-19 2.72130735e-05 6.14480786e-22 -3.45474715e-06 # 7.42993721e-06 -1.40711141e-04 -3.04253586e-04 -2.07559344e-04 # 8.50646247e-05 -3.45474715e-06 7.42993721e-06 -1.40711141e-04 # -3.04253586e-04 -2.07559344e-04 -8.50646247e-05 1.11317030e-04 # -7.03465386e-05 -2.22862221e-05 -1.11317030e-04 7.03465386e-05 # -2.22862221e-05]
Once you compiled mujoco-py, you can run your Python script on any terminal (not necessarilly x64 Native Tools Command Prompt for VS 2019 terminal), as long as you place the subfolder /mujoco-py-repo/mujoco-py in the same folder of your Python script (or run python right outside folder mujoco-py)
In this tutorial, we used folder mujoco-py-repo, placed in the directory where you downloaded mujoco-py-9ea9bb000d6b8551b99f9aa440862e0c7f7b4191.zip. Inside you can find mujoco-py and copy it into whatever directory you want to run your Python script.
Directory tree example:
your-folder
|-- mujoco-py
| |--__pycache__
| |-- generated
| |-- gl
| |-- etc...
|
|-- your-python-script.py (or run here python from a terminal)
If you need to include mujoco-py in your own package, add it to your requirements like so:
mujoco-py<1.50.2,>=1.50.1
Troubleshooting - from the mujoco-py Github repository
If this happend during installation or just running python -c "import mujoco_py" then the issue seems to be related to this and the TL;DR is that for macOS the default compiler Apple clang LLVM does not support openmp. So you can try to install another clang/llvm installation. For example (requires brew):
brew install llvm
brew install boost
brew install hdf5
# Add this to your .bashrc/.zshrc:
export PATH="/usr/local/opt/llvm/bin:$PATH"
export CC="/usr/local/opt/llvm/bin/clang"
export CXX="/usr/local/opt/llvm/bin/clang++"
export CXX11="/usr/local/opt/llvm/bin/clang++"
export CXX14="/usr/local/opt/llvm/bin/clang++"
export CXX17="/usr/local/opt/llvm/bin/clang++"
export CXX1X="/usr/local/opt/llvm/bin/clang++"
export LDFLAGS="-L/usr/local/opt/llvm/lib"
export CPPFLAGS="-I/usr/local/opt/llvm/include"
Note: Don't forget to source your .bashrc/.zshrc after editing it and try to install mujoco-py again:
# Make sure your python environment is activated
pip install -U 'mujoco-py<2.2,>=2.1'
A common error when installing is:
raise ImportError("Failed to load GLFW3 shared library.")
Which happens when the glfw python package fails to find a GLFW dynamic library.
MuJoCo ships with its own copy of this library, which can be used during installation.
Add the path to the mujoco bin directory to your dynamic loader:
LD_LIBRARY_PATH=$HOME/.mujoco/mujoco210/bin pip install mujoco-py
This is particularly useful on Ubuntu 14.04, which does not have a GLFW package.
For more information visit the mujoco-py repository
To test mujoco-py and try its functionalities we are going to train a sample gym environment based on MuJoCo.
The training will be performed using BasicRL, which is a basic implementation of the standard reinforcement learning algorithms in TensorFlow 2, designed for gym-like environments.
In particular, in this example we will use the DDPG algorithm
-
Install
gym:pip install gym pip install gym[mujoco]- Some shells such as Zsh require quotation marks around brackets, i.e.
pip install 'gym[mujoco]'
- Some shells such as Zsh require quotation marks around brackets, i.e.
-
Install
tensorflow:pip install tensorflow -
Clone the BasicRL repository, either using Git Bash or by downloading it in .zip:
# Using git bash git clone https://github.com/d-corsi/BasicRL- If you choose to download it in .zip, then extract the folder and rename it
BasicRL
- If you choose to download it in .zip, then extract the folder and rename it
Choose a MuJoCo environment from here. Clicking on a thumbnail will take you to a more detailed page. In this page scroll down in the panel that contains an animated gif, until you find the Import label. Use the corresponding Import instruction in the next steps
In our example, we will use the environment Ant-v2
-
cdinto theBasicRLfolder -
On a new Python script file or on a Python shell, paste the following (if you are using Windows, see also below):
import os import mujoco_py import gym from basic_rl.main import BasicRL #PLACE YOUR CHOSEN ENVIRIONMENT HERE env = gym.make("Ant-v2") basic_rl = BasicRL("DDPG", env, verbose=2) # 100 episodes just to test the code basic_rl.train(num_episode=100)- Replace your chosen environment in the
makeinstruction - On Windows make sure to:
cdright outside of themujoco-pyfolder as described here. In this case, it means you should copy themujoco-pyfolder into theBasicRLfolder andcdinsideBasicRL- Add the following lines after the
import osand before theimport mujoco-pyinstructions:home = os.path.expanduser("~") os.add_dll_directory(home + "\\.mujoco\\mjpro150\\bin")
- Replace your chosen environment in the
-
Please note that in this example we will use 100 episodes to train our model just to prove that it works. Usually you should use at least 2000+ episodes
-
Run the script (if it is a file,
python name_script.py)
When calling the function BasicRL using argument verbose = 2 , the script saves in a file some information: for each episode, the reward and step
- Files are saved in
.csvformat under the directorydata
We are going to use this information to draw a plot
To plot the result, we will use plot.py placed in the plotter folder of BasicRL.
Before running it, we have to change something in the code to adapt it to our environment and generated .csv files.
-
Open
BasicRL/plotter/plot.pyin a text editor -
Edit it like the following:
from reinforcement_plotter import ReinforcementPlotter import glob # Collect the data from the CSV file data = [ glob.glob("data/DDPG_*.csv") ] # Plot The Results plotter = ReinforcementPlotter( x_label="episode", y_label="reward", title="Ant v2" ) plotter.load_array( data, key="reward", ref_line=0 ) plotter.process_data( rolling_window=100 ) plotter.render_std_log( labels=["DDPG"], colors=["b"], styles=['-'] )
Using seed 664 (see the repository to know more), the plot is the following:
As you can see, the rewards are negative and the plot decreases over time. This is because we didn't train the model enough.
In this section we are going to show a plot that combines 3 different runs of the same algorithm DDPG on the same model ant-v2, but changing seeds.
The implemented function ReinforcementPlotter.render_std_log - in the BasicRL repository - merges the 3 different runs in one plot, by calculating the variance creating a standard plot.
To show a real improvement of the trained model, we have to increase the number of episodes:
-
Using the same code shown in the Use section, change the number of episode from 100 to 5000:
basic_rl.train(num_episode=5000) -
Run the script 3 times (if it is a file,
python name_script.py) -
To draw the functions into one plot, you can run the same
plotter/plot.pyas before. It will automatically merge them
The final plot of model Ant-v2, trained with 5000 episodes 3 times (seeds 484, 661 and 905) and using algorithm DDPG is the following:
Deborah Pintani: DebbyX3

