Copy-less bindings of C-generated arrays with Cython

00README.rst

Cython example of exposing C-computed arrays in Python without data copies

The goal of this example is to show how an existing C codebase for numerical computing (here c_code.c) can be wrapped in Cython to be exposed in Python.

The meat of the example is that the data is allocated in C, but exposed in Python without a copy using the PyArray_SimpleNewFromData numpy function in the Cython file cython_wrapper.pyx.

Note that the ownership of the memory is then handed off to the Python VM, and there is no control of when Python will deallocate the memory. If the memory is still being used by the C code, please refer to the following blog post by Travis Oliphant:

http://blog.enthought.com/python/numpy-arrays-with-pre-allocated-memory

You will need Cython, numpy, and a C compiler.

To build the C extension in-place run:

$ python setup.py build_ext --i

To test the C-Python bindings, run the test.py file.

Files
c_code.c	The C code to bind. Knows nothing about Python
cython_wrapper.c	The Cython code implementing the binding
setup.py	The configure/make/install script
test.py	Python code using the C extension

---

Author:	Gael Varoquaux
License:	BSD

c_code.c

/* Small C file creating an array to demo C -> Python data passing 
 * 
 * Author: Gael Varoquaux
 * License: BSD
 */

#include <stdlib.h>

float *compute(int size)
{
    int* array;
    array = malloc(sizeof(int)*size);
    int i;
    for (i=0; i<size; i++)
    {
	array[i] = i;
    }
    return array;
}

cython_wrapper.pyx

""" Small Cython file to demonstrate the use of PyArray_SimpleNewFromData
in Cython to create an array from already allocated memory.

Cython enables mixing C-level calls and Python-level calls in the same
file with a Python-like syntax and easy type cohersion. See 
http://cython.org for more information
"""

# Author: Gael Varoquaux
# License: BSD

# Declare the prototype of the C function we are interested in calling
cdef extern from "c_code.c":
    float *compute(int size)

# Import the Python-level symbols of numpy
import numpy as np

# Import the C-level symbols of numpy
cimport numpy as np

# Numpy must be initialized. When using numpy from C or Cython you must
# _always_ do that, or you will have segfaults
np.import_array()


def py_compute(int size):
    """ Python binding of the 'compute' function in 'c_code.c' that does
        not copy the data allocated in C.
    """
    cdef float *array
    # Call the C function
    array = compute(size)

    # Create a C array to describe the shape of the ndarray
    cdef np.npy_intp shape[1]
    shape[0] = <np.npy_intp> size

    # Use the PyArray_SimpleNewFromData function from numpy to create a
    # new Python object pointing to the existing data
    ndarray = np.PyArray_SimpleNewFromData(1, shape,
                    np.NPY_INT, <void *> array)

    # Tell Python that it can deallocate the memory when the ndarray
    # object gets garbage collected
    # As the OWNDATA flag of an array is read-only in Python, we need to
    # call the C function PyArray_UpdateFlags
    np.PyArray_UpdateFlags(ndarray, ndarray.flags.num | np.NPY_OWNDATA)
    return ndarray

setup.py

""" Example of building a module with a Cython file. See the distutils
and numpy distutils documentations for more info:
http://docs.scipy.org/doc/numpy/reference/distutils.html
"""
# Author: Gael Varoquaux
# License: BSD

import numpy
from Cython.Distutils import build_ext

def configuration(parent_package='', top_path=None):
    """ Function used to build our configuration.
    """
    from numpy.distutils.misc_util import Configuration

    # The configuration object that hold information on all the files
    # to be built.
    config = Configuration('', parent_package, top_path)
    config.add_extension('cython_wrapper',
                         sources=['cython_wrapper.pyx'],
                         # libraries=['m'],
                         depends=['c_code.c'],
                         include_dirs=[numpy.get_include()])
    return config


if __name__ == '__main__':
    # Retrieve the parameters of our local configuration
    params = configuration(top_path='').todict()

    # Override the C-extension building so that it knows about '.pyx'
    # Cython files
    params['cmdclass'] = dict(build_ext=build_ext)

    # Call the actual building/packaging function (see distutils docs)
    from numpy.distutils.core import setup
    setup(**params)

test.py

""" Script to smoke-test our Cython wrappers
"""
# Author: Gael Varoquaux
# License: BSD

import cython_wrapper
print cython_wrapper.py_compute(10)