Working with Python on the Orchestra cluster

notes.md

Create the venv (virtual environment)

You only need to do this one time.

module avail  # look for dev/python/3
module load dev/python/3.4.2
python -m venv ~/venv   # You can replace ~/venv with any directory if you wish

Activate the venv

Do this each time you log in, before running python code.

module load dev/python/3.4.2
source ~/venv/bin/activate

Install a package

One time per package. Your venv must be activated before running this command.

pip install modulename

Display a list of packages installed in your environment

pip list

Here is a variant that shows the exact version numbers of each package. The output can be saved to a file and passed to pip install in another venv to replicate this one.

pip freeze

Run Python itself

python myscript.py

Or run IPython for interactive work:

ipython

Note that you should do these in an interactive job so as not to use resources on the login nodes.

Can you please give the list of suggested libraries we started to install but it took too long?

numpy
scipy
pandas
matplotlib
seaborn

I just tested on the new "O2" cluster and they have the more modern Python 3.6 there, for which pre-compiled versions of these libraries are available. This makes the install process take literally seconds. Once O2 is ready for production I'll do a session on this same procedure on O2, which only varies in the module commands.