Benchmarking for scikit-image

skimage-benchmarking.md

Writing benchmarks for scikit-image

If you are requesting a change to be made for the sake of performance, it helps to provide a benchmark showing the improvement of the algorithm in the usecase that you are considering.

To create a benchmark, you should.

1. Install airspeed velocity

pip install asv

or

 conda install -c conda-forge asv 

from your development environment.

2. Add a new file to the folder benchmarks. Here a short example that shows the structure of the file.

import numpy as np
from skimage.feature import greycomatrix
from skimage import img_as_ubyte


class GreyCoMatrixSuite:
    """Benchmark for the greycomatrix in scikit-image."""
    
    # All parameters combinations will be tests.
    params = [[(50, 50), (100, 100)],  # shape
              [True, False],           # symmetric
              [True, False],           # normed
             ]
              
    # These are friendly names that will appear on the graphs.
    param_names = ['shape', 'symmetric', 'normed']

    def setup(self, shape, symmetric, normed):
        # Unless you need random to show the performance of a
        # particula algorithm, it is probably fastest to
        # allocate the array as ``full``.
        # This ensures that the memory is directly available to
        # routine without continuously pagefaulting
        # in this case, I want to make sure that we are hitting all
        # combinations of distances in the covariance matrix.
        # self.image - np.full(shape, fill_value=1)
        self.image = img_as_ubyte(np.random.random(shape))

    # You need to include the shape parameter even if you don't use it
    # in your function
    def time_greycomatrix(self, shape, symmetric, normed):
        greycomatrix(self.image, [1], [0, np.pi/4, np.pi/2, 3*np.pi/4],
                     symmetric=symmetric, normed=normed)

3. Run asv machine from within the skimage directory.
4. Update the version of master in your cloned repository

git pull upstream master

5. Run your benchmark in development mode to catch any errors

asv dev -b time_greycomatrix

6. Run your benchmarks in your existing environment to track your progress

asv run -E existing -b time_greycomatrix

7. Compare your results to those of the master branch

asv continuous -E conda:3.6 -b time_greycomatrix master HEAD

Creating the environments for the first time takes a long time.

I'm not sure it actually takes up that much space. They are mostly hard links and windows has a hard time detecting that they don't all take a distinct amount of space.