How to write Rust libraries for the Python scientific computing ecosystem

Writing Rust libraries for the Python scientific computing ecosystem.md

Source: https://www.reddit.com/r/rust/comments/rk12bg/writing_rust_libraries_for_the_python_scientific/

One of Rust's many strengths is that it can be seamlessly integrated with Python and speed up critical code sections. I recently wrote a small library with an efficient ragged array datatype, and I figured it would make for a good example of how to set up a Rust Python package with PyO3 and maturin that interoperates with numpy. There are a lot of little details that took me quite a while to figure out:

• Publishing to PyPI:
  • GitHub action that builds and publishes wheels for multiple Python versions and operating systems
  • Package dependencies and metadata must be specified in a pyproject.toml
  • To get the README to show up on the PyPI website, it needs to be set explicitly in Cargo.toml (maturin#552)
  • Badge that displays the latest package version on PyPI
• Exporting MyPy type annotations:
  • In most cases, you will only need a type stubs file placed in the root project directory, in which case the name of the file should be the same as the package name
  • Since this is a mixed Python/Rust project, you also need to place a py.typed marker file in the Python module
  • There is some extra Python code to export a generic version of the RaggedBuffer datatype (generics are not supported by PyO3)
• Integration with numpy uses the rust-numpy crate:
  • Example of method that accepts numpy arrays as arguments
  • Example of a method that returns a numpy array to Python (this performs a copy, there ought to be a way to avoid it but the current implementation has been plenty fast for my use case so far)
• Implementing special Python methods such as __str__, __repr__, and __cmp__, multiplication and addition operators, and indexing operations