Skip to content

Instantly share code, notes, and snippets.

@DebbyX3
Created March 26, 2025 14:05
Show Gist options
  • Select an option

  • Save DebbyX3/d55f0d93077e8960f1e5925be9ed7fe5 to your computer and use it in GitHub Desktop.

Select an option

Save DebbyX3/d55f0d93077e8960f1e5925be9ed7fe5 to your computer and use it in GitHub Desktop.

mujoco-py guide

Install mujoco-py

Install on Linux based systems - MuJoCo version 2.1

Install MuJoCo

  • Install or upgrade python and pip:

     # Install
     sudo apt install python3
     
     # Upgrade - better upgrading all packages since many are connected to Python
     sudo apt update
     sudo apt upgrade
    
     # Install
     sudo apt install python3-pip
     
     # Upgrade
     python3 -m pip install --upgrade pip
    
    • Make sure to have at least Python 3.6+. To check your version:
       python3 --version
      
      Python 2 has been deprecated
  • Download the MuJoCo version 2.1 binary for Linux

  • cd where you downloaded the file and extract the downloaded mujoco210 directory into ~/.mujoco/mujoco210, either manually or by running the following command:

     mkdir -p ~/.mujoco && tar -zxvf mujoco210-linux-x86_64.tar.gz --directory ~/.mujoco
    
  • Please check that path and directory ~/.mujoco/mujoco210 exist and contain the downloaded files

  • Depending on what shell terminal you use, it is necessary to add an export command to the shell configuration file. As a rule of thumb, if you do not know what shell you are using, it means is probably bash

    • For bash shell (usually the standard pre-installed one)

      • Add the line export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/.mujoco/mujoco210/bin at the end of file ~/.bashrc, either manually or by running the following command:

         echo 'export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/.mujoco/mujoco210/bin' >> ~/.bashrc
        
      • Open file ~/.bashrc and check if the line was correctly appended at the end:

         cat ~/.bashrc
        
      • Run the following to apply the latest export command:

         source ~/.bashrc
        
    • For other shells (like zsh) follow the steps above described for bash , but instead of editing ~/.bashrc, modify the correspondent configuration file for your shell

      • For example, if you are using zsh, edit ~/.zshrc like described before
  • Install the following:

     sudo apt install python3-dev
     sudo apt install build-essential
     sudo apt install patchelf
    
  • For Ubuntu distributions - may be optional

    • Install the following:
       sudo apt install libosmesa6-dev libgl1-mesa-glx libglfw3
      
    • Create a symbolic link - it may already exists
       sudo ln -s /usr/lib/x86_64-linux-gnu/libGL.so.1 /usr/lib/x86_64-linux-gnu/libGL.so
      
  • Install using pip3:

     pip3 install -U 'mujoco-py<2.2,>=2.1'
    

Check if MuJoCo is correctly installed

  • Run python3:
     python3
    
  • In python3 shell import MuJoCo - we are going to compile the library:
     import mujoco_py
    

If the import is clean and Python doesn't throw any error: congrats! You just installed MuJoCo 2.1 successfully!

Check MuJoCo functionalities by using a simple script

  • Run python3:
     python3
    
  • Paste the following in python3 shell:
     import os
     import mujoco_py
     
     mj_path = mujoco_py.utils.discover_mujoco()
     xml_path = os.path.join(mj_path, 'model', 'humanoid.xml')
     model = mujoco_py.load_model_from_path(xml_path)
     sim = mujoco_py.MjSim(model)
    
     print(sim.data.qpos)
     
     sim.step()
     
     print(sim.data.qpos)
    
  • If MuJoCo works properly, the script will successfully print data, like:
     First print:
     # [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
    
     Second print:
     # [-2.09531783e-19  2.72130735e-05  6.14480786e-22 -3.45474715e-06
     #   7.42993721e-06 -1.40711141e-04 -3.04253586e-04 -2.07559344e-04
     #   8.50646247e-05 -3.45474715e-06  7.42993721e-06 -1.40711141e-04
     #  -3.04253586e-04 -2.07559344e-04 -8.50646247e-05  1.11317030e-04
     #  -7.03465386e-05 -2.22862221e-05 -1.11317030e-04  7.03465386e-05
     #  -2.22862221e-05]
    

Include MuJoCo in your own package

If you need to include mujoco-py in your own package, add it to your requirements like so:

mujoco-py<2.2,>=2.1

Install on OSX - MuJoCo version 2.1

Install MuJoCo

  • brew is required. If you do not have it, install it:

    /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
    
  • Install or upgrade python and pip:

     # Install & upgrade
     brew install python3
    

    pip3 is automatically installed with python3

    • Make sure to have at least Python 3.6+. To check your version:
       python3 --version
      
      Python 2 has been deprecated
  • Download the MuJoCo version 2.1 binary for OSX

  • cd where you downloaded the file and extract the downloaded mujoco210 directory into ~/.mujoco/mujoco210, either manually or by running the following command:

     mkdir -p ~/.mujoco && tar -zxvf mujoco210-macos-x86_64.tar.gz --directory ~/.mujoco
    
  • Please check that path and directory ~/.mujoco/mujoco210 exist and contain the downloaded files

  • Depending on what shell terminal you use, it is necessary to add an export command to the shell configuration file. As a rule of thumb, if you do not know what shell you are using, it means is probably zsh

    • For zsh shell (usually the standard pre-installed one)

      • Add the line export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/.mujoco/mujoco210/bin at the end of file ~/.zshrc, either manually or by running the following command:

         echo 'export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/.mujoco/mujoco210/bin' >> ~/.zshrc
        
      • Open file ~/.zshrc and check if the line was correctly appended at the end:

         cat ~/.zshrc
        
      • Run the following to apply the latest export command:

         source ~/.zshrc
        
    • For other shells follow the steps above described for zsh , but instead of editing ~/.zshrc, modify the correspondent configuration file for your shell

  • Install the following:

     brew install gcc@9
     brew install patchelf
    
  • Install using pip3:

     pip3 install -U 'mujoco-py<2.2,>=2.1'
    

Check if MuJoCo is correctly installed

  • Run python3:
     python3
    
  • In python3 shell import MuJoCo - we are going to compile the library:
     import mujoco_py
    

If the import is clean and Python doesn't throw any error: congrats! You just installed MuJoCo 2.1 successfully!

Check MuJoCo functionalities by using a simple script

  • Run python3:
     python3
    
  • Paste the following in python3 shell:
     import os
     import mujoco_py
     
     mj_path = mujoco_py.utils.discover_mujoco()
     xml_path = os.path.join(mj_path, 'model', 'humanoid.xml')
     model = mujoco_py.load_model_from_path(xml_path)
     sim = mujoco_py.MjSim(model)
    
     print(sim.data.qpos)
     
     sim.step()
     
     print(sim.data.qpos)
    
  • If MuJoCo works properly, the script will successfully print data, like:
     First print:
     # [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
    
     Second print:
     # [-2.09531783e-19  2.72130735e-05  6.14480786e-22 -3.45474715e-06
     #   7.42993721e-06 -1.40711141e-04 -3.04253586e-04 -2.07559344e-04
     #   8.50646247e-05 -3.45474715e-06  7.42993721e-06 -1.40711141e-04
     #  -3.04253586e-04 -2.07559344e-04 -8.50646247e-05  1.11317030e-04
     #  -7.03465386e-05 -2.22862221e-05 -1.11317030e-04  7.03465386e-05
     #  -2.22862221e-05]
    

Include MuJoCo in your own package

If you need to include mujoco-py in your own package, add it to your requirements like so:

mujoco-py<2.2,>=2.1

Install on Windows - MuJoCo version 1.50.1.68

Please note that MuJoCo support for Windows has been deprecated and removed in 2.0.2.0. One known good past version is 1.50.1.68, so this is the version we are going to install

Note: please use Powershell or Powershell Core (also available on Chocolatey) terminal in Admin mode for this tutorial

Using Anaconda or Miniconda is not recommended

  • Install or upgrade python, either using Chocolatey package manager or manually:

    • Manually: https://www.python.org/downloads/windows/ Download the latest stable release compatible with your system, at least Python 3.6+

    • Using Chocolatey:

       # Install
       choco install python3
       
       # Upgrade
       choco upgrade python3
      
  • Make sure to have at least Python 3.6+. To check your version:

     python --version
    

    Python 2 has been deprecated

Note: if you have Anaconda/Miniconda installed on your system, please make sure to check the order of your Envrironment Variables. Having a wrong path order leads to problems when opening a command line. If that is the case:

  • Run python on a terminal:

     python
    
  • If you see a message like:

     This Python interpreter is in a conda environment, but the environment has not been activated. Libraries may fail to load.
    

    Then your path Environment Variable needs to be fixed

  • In the System Environment Variable PATH, your Python installation paths (usually C:\Python310\ and C:\Python310\Scripts\) should be on top of Anaconda/Miniconda paths (usually C:\tools\miniconda3 and all the other folders referring to conda). To check and fix your Environment Variables:

  • Open Environment Variables panel, either manually or by running the following (in Admin mode):

     start-process rundll32.exe sysdm.cpl,EditEnvironmentVariables -verb RunAs
    
  • In the System Variables (not for current user) section, open the variable PATH

  • Check if the paths referring to python installation (like C:\Python310\ and C:\Python310\Scripts\) are under the Anaconda/Miniconda paths

  • If that is true, bring python's paths on top of the Anaconda/miniconda ones

    • Press Ok to exit the panel and Ok again to save changes

Continue MuJoCo installation:

  • Install or upgrade pip:

     # Install
     python get-pip.py
     
     # Upgrade
     python -m pip install --upgrade pip
    
  • Download the MuJoCo version 1.50 binary for Windows

  • cd where you downloaded the file and extract the downloaded mjpro150 directory into $home\.mujoco\mjpro150, either manually or by running the following command:

     Expand-Archive -LiteralPath  "mjpro150_win64.zip" -DestinationPath "$home\.mujoco"
    
  • Please check that path and directory $home\.mujoco\mjpro150 exist and contain the downloaded files

  • Add the path to $home\.mujoco\mjpro150 to the System path Environment Variable, either manually or by running the following (Admin mode):

     # Run this to save your current Path variable in a file just in case
     $Env:PATH >> Env_Path.txt
     
     # Make sure to include the plus and the semicolon 
     [Environment]::SetEnvironmentVariable("PATH", $Env:PATH + ";$home\.mujoco\mjpro150\bin", [EnvironmentVariableTarget]::Machine)
    
  • Close and reopen a new Powershell in admin mode (it is necessary to print new Environment Variables, just running refreshenv won't work)

  • Check if the line was correctly appended at the end of the path variable:

     $Env:PATH
    
  • Download the mjkey.txt free licence activation key from the Roboti licence page

    • You need an activation key because MuJoCo was a paid software before version 2.0. In this case DeepMind has granted Roboti LLC license to provide an unlocked activation key for legacy versions of MuJoCo (2.0 and earlier).
  • cd where you downloaded the file and place mjkey.txt in $home\.mujoco, either manually or by running the following:

     Copy-Item  "mjkey.txt" -Destination  "$home\.mujoco"
    
  • Download the MuJoCo 1.50.1.68 repository in .zip

    • Do not attempt to directly clone the repository since it will download the latest version
  • cd where you downloaded the file and extract the downloaded mujoco-py-9ea9bb000d6b8551b99f9aa440862e0c7f7b4191.zip file in the same directory, either manually or by running the following:

     Expand-Archive -LiteralPath  "mujoco-py-9ea9bb000d6b8551b99f9aa440862e0c7f7b4191.zip"
    
  • Making sure to stay in the same directory, open mujoco-py-9ea9bb000d6b8551b99f9aa440862e0c7f7b4191 folder and move the inner folder (which has the same name) outside. The purpose is to access mujoco-py-9ea9bb000d6b8551b99f9aa440862e0c7f7b4191 folder contents directly (and not having a sub folder). Then rename the just moved folder using a shorter name, like mujoco-py-repo. Do all of this either manually or by running the following, making sure to stay in the directory you extracted the folder:

     cd mujoco-py-9ea9bb000d6b8551b99f9aa440862e0c7f7b4191
     Move-Item -Path mujoco-py-9ea9bb000d6b8551b99f9aa440862e0c7f7b4191 -Destination ..\mujoco-py-repo
     cd ..
     Remove-Item mujoco-py-9ea9bb000d6b8551b99f9aa440862e0c7f7b4191
    
    • The purpose is to shorten the folder path, since a long path will throw an error when compiling later
  • Download Microsoft C++ Build Tools and install them (about 100MB)

  • Once installed you should have a new terminal called x64 Native Tools Command Prompt for VS 2019 (the version year may change). You should be able to find it from the Windows Menu, or in the following path:

     C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Visual Studio 2019\Visual Studio Tools\VC
    

    Note: it is important to run it in Admin mode and from the Windows menu or the path shown above, since the link includes some dependencies when executed

  • Run x64 Native Tools Command Prompt for VS 2019 from the Windows Menu in Admin mode

  • cd into the previously extracted and renamed folder mujoco-py-repo

  • Run the following:

     pip install -r requirements.txt
     pip install -r requirements.dev.txt
     python setup.py install
    
  • Run python:

     python
    
  • If your python version is above 3.8 (python3 --version to check):

    • Since Python 3.8, only "trusted dlls" are loaded. For this reason, Windows users having Python 3.8+ need to to add the location of the installed MuJoCo version to the list of allowed DLLs at the beginning of every Python script that uses mujoco-py. Otherwise it will throw a similar error: ImportError: DLL load failed while importing cymj: The specified module could not be found..
    • To do this, in python shell import os :
       import os
      
    • In python shell add MuJoCo dlls to the trusted dlls:
       home = os.path.expanduser("~")
       os.add_dll_directory(home + "\\.mujoco\\mjpro150\\bin")
      
      Note: on Windows you have to do this for every Python script that uses mujoco-py.
  • In python shell import MuJoCo - we are going to compile the library:

     import mujoco_py
    

If the import is clean and Python doesn't throw any error: congrats! You just installed MuJoCo 1.50.1.68 successfully!

Check MuJoCo functionalities by using a simple script

  • Run python:

     python
    
  • Paste the following in python shell:

     import os
     home = os.path.expanduser("~")
     os.add_dll_directory(home + "\\.mujoco\\mjpro150\\bin")
     
     import mujoco_py
     	
     mj_path = mujoco_py.utils.discover_mujoco()[0]
     xml_path = os.path.join(mj_path, 'model', 'humanoid.xml')
     model = mujoco_py.load_model_from_path(xml_path)
     sim = mujoco_py.MjSim(model)
    
     print(sim.data.qpos)
     
     sim.step()
     
     print(sim.data.qpos)
    

    Note: the instruction mj_path = mujoco_py.utils.discover_mujoco()[0] has a [0] at the end because mujoco_py.utils.discover_mujoco() returns a tuple of this form: ('$home\\.mujoco\\mjpro150', '$home\\.mujoco\\mjkey.txt'). Since MuJoCo version 1.50.1.68 needs a key in $home\.mujoco\mjkey.txt, the previous instruction retires both folder mjpro150 and file mjkey.txt. We only need the folder path when coding, so make sure to include [0] in mj_path = mujoco_py.utils.discover_mujoco()[0]. This tip is only for Windows

  • If MuJoCo works properly, the script will successfully print data, like:

     First print:
     # [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
    
     Second print:
     # [-2.09531783e-19  2.72130735e-05  6.14480786e-22 -3.45474715e-06
     #   7.42993721e-06 -1.40711141e-04 -3.04253586e-04 -2.07559344e-04
     #   8.50646247e-05 -3.45474715e-06  7.42993721e-06 -1.40711141e-04
     #  -3.04253586e-04 -2.07559344e-04 -8.50646247e-05  1.11317030e-04
     #  -7.03465386e-05 -2.22862221e-05 -1.11317030e-04  7.03465386e-05
     #  -2.22862221e-05]
    

Using MuJoCo

Once you compiled mujoco-py, you can run your Python script on any terminal (not necessarilly x64 Native Tools Command Prompt for VS 2019 terminal), as long as you place the subfolder /mujoco-py-repo/mujoco-py in the same folder of your Python script (or run python right outside folder mujoco-py)

In this tutorial, we used folder mujoco-py-repo, placed in the directory where you downloaded mujoco-py-9ea9bb000d6b8551b99f9aa440862e0c7f7b4191.zip. Inside you can find mujoco-py and copy it into whatever directory you want to run your Python script.

Directory tree example:

your-folder
	|-- mujoco-py
	|	|--__pycache__
	|	|-- generated
	|	|-- gl
	|	|-- etc...
	|
	|-- your-python-script.py (or run here python from a terminal)

Include MuJoCo in your own package

If you need to include mujoco-py in your own package, add it to your requirements like so:

mujoco-py<1.50.2,>=1.50.1

Troubleshooting - from the mujoco-py Github repository

You're on MacOS and you see clang: error: unsupported option '-fopenmp'

If this happend during installation or just running python -c "import mujoco_py" then the issue seems to be related to this and the TL;DR is that for macOS the default compiler Apple clang LLVM does not support openmp. So you can try to install another clang/llvm installation. For example (requires brew):

brew install llvm
brew install boost
brew install hdf5

# Add this to your .bashrc/.zshrc:
export PATH="/usr/local/opt/llvm/bin:$PATH"

export CC="/usr/local/opt/llvm/bin/clang"
export CXX="/usr/local/opt/llvm/bin/clang++"
export CXX11="/usr/local/opt/llvm/bin/clang++"
export CXX14="/usr/local/opt/llvm/bin/clang++"
export CXX17="/usr/local/opt/llvm/bin/clang++"
export CXX1X="/usr/local/opt/llvm/bin/clang++"

export LDFLAGS="-L/usr/local/opt/llvm/lib"
export CPPFLAGS="-I/usr/local/opt/llvm/include"

Note: Don't forget to source your .bashrc/.zshrc after editing it and try to install mujoco-py again:

# Make sure your python environment is activated
pip install -U 'mujoco-py<2.2,>=2.1'

Missing GLFW

A common error when installing is:

raise ImportError("Failed to load GLFW3 shared library.")

Which happens when the glfw python package fails to find a GLFW dynamic library. MuJoCo ships with its own copy of this library, which can be used during installation. Add the path to the mujoco bin directory to your dynamic loader:

LD_LIBRARY_PATH=$HOME/.mujoco/mujoco210/bin pip install mujoco-py

This is particularly useful on Ubuntu 14.04, which does not have a GLFW package.

mujoco-py repository

For more information visit the mujoco-py repository

Test and train a MuJoCo environment using BasicRL

To test mujoco-py and try its functionalities we are going to train a sample gym environment based on MuJoCo. The training will be performed using BasicRL, which is a basic implementation of the standard reinforcement learning algorithms in TensorFlow 2, designed for gym-like environments.

In particular, in this example we will use the DDPG algorithm

Install

  • Install gym:

     pip install gym
     pip install gym[mujoco]
    
    • Some shells such as Zsh require quotation marks around brackets, i.e. pip install 'gym[mujoco]'
  • Install tensorflow:

     pip install tensorflow
    
  • Clone the BasicRL repository, either using Git Bash or by downloading it in .zip:

     # Using git bash
     git clone https://github.com/d-corsi/BasicRL
    
    • If you choose to download it in .zip, then extract the folder and rename it BasicRL

Use

Choose a MuJoCo environment from here. Clicking on a thumbnail will take you to a more detailed page. In this page scroll down in the panel that contains an animated gif, until you find the Import label. Use the corresponding Import instruction in the next steps

In our example, we will use the environment Ant-v2

  • cd into the BasicRL folder

  • On a new Python script file or on a Python shell, paste the following (if you are using Windows, see also below):

     import os
     import mujoco_py
     import gym
    
     from basic_rl.main import BasicRL
    
     #PLACE YOUR CHOSEN ENVIRIONMENT HERE
     env = gym.make("Ant-v2")
     
     basic_rl = BasicRL("DDPG", env, verbose=2)
     # 100 episodes just to test the code
     basic_rl.train(num_episode=100)
    
    • Replace your chosen environment in the make instruction
    • On Windows make sure to:
      • cd right outside of the mujoco-py folder as described here. In this case, it means you should copy the mujoco-py folder into the BasicRL folder and cd inside BasicRL
      • Add the following lines after the import os and before the import mujoco-py instructions:
         home = os.path.expanduser("~")
         os.add_dll_directory(home + "\\.mujoco\\mjpro150\\bin")
        
  • Please note that in this example we will use 100 episodes to train our model just to prove that it works. Usually you should use at least 2000+ episodes

  • Run the script (if it is a file, python name_script.py)

When calling the function BasicRL using argument verbose = 2 , the script saves in a file some information: for each episode, the reward and step

  • Files are saved in .csv format under the directory data

We are going to use this information to draw a plot

Plot

To plot the result, we will use plot.py placed in the plotter folder of BasicRL.

Before running it, we have to change something in the code to adapt it to our environment and generated .csv files.

  • Open BasicRL/plotter/plot.py in a text editor

  • Edit it like the following:

     from reinforcement_plotter import ReinforcementPlotter
     import glob
    
     # Collect the data from the CSV file
     data = [
     		glob.glob("data/DDPG_*.csv")
     ]
    
     # Plot The Results
     plotter = ReinforcementPlotter( x_label="episode", y_label="reward", title="Ant v2" )
     plotter.load_array( data, key="reward", ref_line=0 )
     plotter.process_data( rolling_window=100 )
     plotter.render_std_log( labels=["DDPG"], colors=["b"], styles=['-'] )
    

Using seed 664 (see the repository to know more), the plot is the following:

Plot 100 episodes

As you can see, the rewards are negative and the plot decreases over time. This is because we didn't train the model enough.

Plot using different seeds and training using more episodes

In this section we are going to show a plot that combines 3 different runs of the same algorithm DDPG on the same model ant-v2, but changing seeds. The implemented function ReinforcementPlotter.render_std_log - in the BasicRL repository - merges the 3 different runs in one plot, by calculating the variance creating a standard plot.

To show a real improvement of the trained model, we have to increase the number of episodes:

  • Using the same code shown in the Use section, change the number of episode from 100 to 5000:

    basic_rl.train(num_episode=5000)
    
  • Run the script 3 times (if it is a file, python name_script.py)

    • Please note that the computation time required to solve 5000 episodes is significantly higher than solving only 100 episodes (about 4/5 hours)
    • If you don't specify a seed, the script will pick one at random (more on that here or in this example)
  • To draw the functions into one plot, you can run the same plotter/plot.py as before. It will automatically merge them

The final plot of model Ant-v2, trained with 5000 episodes 3 times (seeds 484, 661 and 905) and using algorithm DDPG is the following:

Plot 5000 episoded 3 times

Author:

Deborah Pintani: DebbyX3

Repository BasicRL: Davide Corsi - d-corsi

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment