Publish a complex package to PyPi

gistfile2.md

Publish a Complex Package to PyPi

By a complex package, I mean a package with pure Python code as well as code in other languages like C, Fortran, etc.

Note: I am assuming that your setup.py already works, and I will show you how to create a manylinux Python wheel. You will need Docker for this.

The manylinux Wheel 🛞

To create a distribution of your Python package with statically linked libraries (i.e., a self-contained package), you need to use a Linux distribution that provides all necessary dependencies. This is where manylinux comes in handy.

To get started, pull the container where you will install your Python package:

docker pull quay.io/pypa/manylinux2010_x86_64

Then, log into it:

docker run -ti -v ~/MyPackage:/io quay.io/pypa/manylinux2010_x86_64 /bin/bash

Notes:

1. Make sure to install the correct version of numpy to ensure compatibility with the setuptools version:

   /opt/python/cp37-cp37m/bin/python -m pip install numpy
   
   /opt/python/cp39-cp39/bin/python -m pip install numpy==1.22

2. Navigate to the /io directory and clone your project repository:

   cd /io/
   git clone https://github.com/user.or.org/MyPackage.git

3. Enter the repository folder and run the setup.py script with the bdist_wheel and sdist options:

   cd MyPackage/
   /opt/python/cp37-cp37m/bin/python setup.py bdist_wheel sdist
   
   /opt/python/cp39-cp39/bin/python setup.py bdist_wheel sdist
   cd dist/

4. To create a Linux-agnostic wheel, use the auditwheel tool:

   auditwheel repair MyPackage-x.y.z-cp37-cp37m-linux_x86_64.whl
   auditwheel repair MyPackage-x.y.z-cp39-cp39-linux_x86_64.whl

5. The final wheel file will be in the wheelhouse directory:

   cd wheelhouse/
   ls MyPackage-x.y.z-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
   ls MyPackage-x.y.z-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl

6. Finally, copy the wheel file from the Docker container to your local host:

   First, get your container ID:

   docker ps

   The container ID should look something like 73dc6928ec89. Next, copy the wheel files you want:

   docker cp 73dc6928ec89:/io/MyPackage/dist/wheelhouse/MyPackage-x.y.z-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl .
   docker cp 73dc6928ec89:/io/MyPackage/dist/wheelhouse/MyPackage-x.y.z-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl .
   docker cp 73dc6928ec89:/io/MyPackage/dist/MyPackage-x.y.z.tar.gz .

Publishing

Use twine to publish MyPackage to https://pypi.org (if you don't have twine installed, simply run pip install twine):

twine upload dist/*

Now, MyPackage can be easily installed with:

pip install MyPackage