Python build tools introduction #python

python_build_tools.md

Python Build Tools

In modern Python package project, it often includes such files:

• pyproject.toml
• setup.cfg
• setup.py
• MANIFEST.in

What's the relations among them and the purpose from them?

In brief,

• pyproject.toml is used by Python package builder such as build
• setup.cfg and setup.py are used by packaging tool setuptools
• MANIFEST.in specifies set of files are included into source distribution

for package builder build to build bdist_wheel, it will read MANIFEST.in to copy set of files to build folder.

In PEP 517, it specified a new standard way to build and package Python modules. Under the new introduction, the modern Python Build System design the forefront part and backend part.

1. The forefront part. The build, it can understand pyproject.toml that it will install needed packages for building from build-system.requires and decide to use the backend part from build-system.build-backend.

2. The backend part. If you were to use a different build system, such as flit or poetry, those would go here and the configuration details would be different that setuptools configuration described below in pyproject.toml:

   [build-system]
    requires = [
        "setuptools>=42",
        "wheel"
    ]
    build-backend = "setuptools.build_meta" # for `poetry` or `flit`, chage configuration here

Build

A simple, correct PEP 517 package builder.

Setuptools

Setuptools Documentation

Here are various Python building tools. In past, we always use setuptools to build a Python pakcage that need some required files such as setup.py. In modern Python, setuptools introduce another file setup.cfg which configures metadata statically rather than dynamicly configured by setup.py, and prefers to use setup.cfg. Here some rules adopted by setuptools,

1. both setup.cfg and setup.py. It reads setup.cfg to configure first and then reads setup.py to overwrite or supplement building extensions configuration dynamicly.
2. only setup.cfg.
3. only setup.py.

So we can move most metadata configartions from setup.py to setup.cfg.

setup.py file:

setup(
    # name="plyvel",
    # description="Plyvel, a fast and feature-rich Python interface to LevelDB",
    # long_description=get_file_contents("README.rst"),
    # url="https://github.com/wbolster/plyvel",
    # version=__version__,  # noqa: F821
    # author="Wouter Bolsterlee",
    # author_email="[email protected]",
    ext_modules = cythonize(ext_modules),
    # package_data = {"plyvel":['*.pxd', '*.h']},
    # packages=["plyvel"],
    # license="BSD License",
    # classifiers=[
    #     "Development Status :: 5 - Production/Stable",
    #     "Intended Audience :: Developers",
    #     "Intended Audience :: Information Technology",
    #     "Intended Audience :: Science/Research",
    #     "License :: OSI Approved :: BSD License",
    #     "Operating System :: POSIX",
    #     "Operating System :: Windows",
    #     "Programming Language :: C++",
    #     "Programming Language :: Cython",
    #     "Programming Language :: Python",
    #     "Programming Language :: Python :: 2",
    #     "Programming Language :: Python :: 3",
    #     "Topic :: Database",
    #     "Topic :: Database :: Database Engines/Servers",
    #     "Topic :: Software Development :: Libraries :: Python Modules",
    # ],
)

setup.cfg file:

[metadata]
name = plyvel
description = Plyvel, a fast and feature-rich Python interface to LevelDB
long_description = file: README.rst
url = https://github.com/wbolster/plyvel
version = 1.3.0  # noqa: F821
author = Wouter Bolsterlee
author_email = [email protected]
license = BSD License
license_files = LICENSE.rst
classifiers = 
    Development Status :: 5 - Production/Stable
    Intended Audience :: Developers
    Intended Audience :: Information Technology
    Intended Audience :: Science/Research
    License :: OSI Approved :: BSD License
    Operating System :: POSIX
    Operating System :: Windows
    Programming Language :: C++
    Programming Language :: Cython
    Programming Language :: Python
    Programming Language :: Python :: 2
    Programming Language :: Python :: 3
    Topic :: Database
    Topic :: Database :: Database Engines/Servers
    Topic :: Software Development :: Libraries :: Python Modules
keywords = plyvel, leveldb

[options]
package_dir =
    = src
packages = find:
install_requires = 
python_requires = >=2.7, >=3.7

[options.packages.find]
where = src

Cython

Source Files and Compilation

Cython Tricks

Poetry

Flit