Troubleshooting why Jupyter notebook wasn't finding installed python packages

jupyter-not-finding-packages.md

Tried creating and installing needed packages into a virtual python environment, but Jupyter notebook would still throw Module not found errors, even after pip, pip3 on base installation, as well as pip, pip3 install method into the virtual environment.

conda create -n my-conda-env         # creates new virtual env
source activate my-conda-env         # activate environment in terminal
conda install jupyter                # install jupyter + notebook
jupyter notebook                     # start server + kernel

To solve, in jupyter notebook, run:

import sys
print(sys.executable)

Use the path to install the package via terminal
/usr/local/Cellar/jupyterlab/3.0.14/libexec/bin/python3.9 -m pip install pandas

---

Problems arising from packages that appear to be installed

Add your virtual environment as Python kernel in this way (Make sure it's activated):

(venv) $ ipython kernel install --name "local-venv-kernel" --user

Now, you can select the created kernel "local-venv-kernel" when you start Jupyter notebook or lab.

You could check the installed libraries using this code in a notebook cell:

!pip freeze

• need to look further into how to target and select which jupyter

Installing packages with pip to same location as conda and not system

If you have a non-conda pip as your default pip but conda python is your default python (as below)

>which -a pip
/home/<user>/.local/bin/pip   
/home/<user>/.conda/envs/newenv/bin/pip
/usr/bin/pip

>which -a python
/home/<user>/.conda/envs/newenv/bin/python
/usr/bin/python

Then instead of just calling pip install <package>, you can use the module flag -m with python so that it uses the anaconda python for the installation

python -m pip install <package>

This installs the package to the anaconda library directory rather than to the library directory associated with (the non-anaconda) pip

EDIT: The reason this works is as follows: the command pip references a specific pip file/shortcut (which -a pip tells you which one). Similarly, the command python references a specific python file (which -a python tells you which one). For one reason or another these two commands can become unsynchronized, so that your 'default' pip is in a different folder than your default python, and therefore is associated with a different version of python.

In contrast, the python -m pip construction does not use the shortcut that the pip command points to. Instead, it asks python to find its version of pip and use that version to install a package.

Thanks for the note, work for me.