Timer Wrapper Feedback!

api-feedback.md

I'd like some feedback on this li'l thing. If you want to have a look, the source is here, but I'm more interested in help on the API and naming stuff, so don't worry about that unless you want to!

I've written a wrapper for timeit.Timer. It behaves a lot like the command line python -m timeit ... interface, but makes comparison easier by providing a Python API for specifying functions to compare and arguments to use.

python -m timeit -s 'import my_function' 'my_function(10)'

You specify 10 as the arguments to the function and decorate my_function with the provided decorator. When you run the comparisons, it prints a little table showing the results.

This makes comparisons more fun, if you ask me. Consider primeexample.py and the resulting table, primeexample.txt. If you called all those calls form the command line, you'd have to run

$ python -m timeit -s 'from example import check_all' 'check_all(10)'
...
$ python -m timeit -s 'from example import memozied' 'memoized(10)'
...
$ python -m timeit -s 'from example import sieve' 'sieve(10)'
...
$ python -m timeit -s 'from example import check_all' 'check_all(100)'
...

And so forth, and then collate all the results from the runs. This may be a strawman, exaggerated version of how hard and unpleasent it is, but I posit that the CLI requires lots of repetition for this use case, and the Timer Python interface is more complicated than necessary (see here for examples of both). There should be a simpler interface for this.

classexample.py and .txt show the use of the class interface for specifying timings, and demonstrates that they classes can be used independently. primeexample.py shows use of the module-level API, which (I think) makes casual use easier by avoiding setup. (Just so you know, the calls are just aliases for an already-instantiated SimpleTimeIt object.) multiargs.py demonstrates registering arguments for multi-argument functions, and its output is in multiargs.txt. withsetup.py and .txt show how to use the _setup parameter to do pre-timing setup.

So, while things aren't set in stone, I was hoping that y'all could give me some feedback on this thing!

• Is there anything I can do to make it more understandable? More Pythonic? More fun?
• Would you prefer any alternate interfaces for specifying arguments or functions?
• What kinds of timing, running, or printing configuration options would you use if provided?
• Is the provided module-level API a good idea? In principle I try to avoid giving modules state when possible, but since this is for scripts that fire off and finish quickly, I don't think there's a huge chance for silent/mysterious error when people don't anticipate the state hanging around.

And of course, most of all,

• Would you actually use this? When, and for what?

bike.shed

And of course, the worst but best part of this feedback request: what should things be named?

Since this is kind of a Pokémon-style battle between your functions, I've been thinking thought a synonym for 'battle' would be appropriate name for the module and its primary object. Under consideration:

• dustup
• fracas
• rhubarb (which is apparently a synonym for scuffle!)
• rumble
• scuffle

Rumble, as in "Let's Get Ready to Rumble!", is my current favorite. And I think a fun name for the function-registering decorator would be contender, so

from rumble import Rumble
rumble = Rumble()

@rumble.contender
def memoized(n):
    ...

I am not super-attached to this "battle" idea, so other suggestions are welcome no matter how far afield!

I don't really like call_with as the name for the other function, but I don't know what else to call it. Maybe just arguments?

classexample.py

from functools import wraps

from simpletimeit import stimeit

fib_timer = stimeit.SimpleTimeIt()
for x in [3, 9, 17]:
    fib_timer.call_with(x)

def memoize(f):
    '''memoizer for single-argument functions'''
    _cache = {}
    @wraps(f)
    def wrapper(x):
        try:
            return _cache[x]
        except KeyError:
            _cache[x] = f(x)
            return _cache[x]
    return wrapper

@fib_timer.time_this
def recursive(n):
    if n == 0:
        return 0
    if n in (1, 2):
        return 1
    return recursive(n - 1) + recursive(n - 2)

@fib_timer.time_this
@memoize
def memoized(n):
    if n == 0:
        return 0
    if n in (1, 2):
        return 1
    return memoized(n - 1) + memoized(n - 2)

prime_timer = stimeit.SimpleTimeIt()
prime_timer.call_with(100)
prime_timer.call_with(500)

@prime_timer.time_this
def sieve(n):
    flags = [True for _ in range(n + 1)]
    flags[0] = flags[1] = False
    for i in range(len(flags)):
        if flags[i]:
            for j in range(i + 1, len(flags)):
                if flags[j] and j % i == 0:
                    flags[j] = False

    return [i for i, f in enumerate(flags) if f]

@prime_timer.time_this
@memoize
def memoized(n, _primes={}):
    result = []
    for i in range(2, n + 1):
        if i not in _primes:
            _primes[i] = not any(i % x == 0 for x in range(2, i))
        if _primes[i]:
            result.append(i)

    return result

print('fibonacci!')
fib_timer.run()

print('ready for prime time!')
prime_timer.run()

classexample.txt

fibonacci!
args: 3      usec     loops    best of
---------  ------  --------  ---------
recursive    0.55   1000000          3
memoized     0.16  10000000          3

args: 9      usec     loops    best of
---------  ------  --------  ---------
recursive   13.40    100000          3
memoized     0.16  10000000          3

args: 17      usec    loops    best of
----------  ------  -------  ---------
recursive   598.17     1000          3
memoized      0.19  1000000          3

ready for prime time!
args: 100      usec     loops    best of
-----------  ------  --------  ---------
sieve        130.00     10000          3
memoized       0.16  10000000          3

args: 500       usec     loops    best of
-----------  -------  --------  ---------
sieve        2003.86      1000          3
memoized        0.16  10000000          3

multiargs.py

from functools import wraps

from simpletimeit import stimeit

st = stimeit.SimpleTimeIt()

st.call_with(10, 100)
st.call_with(8000, 92833)
st.call_with(898989, 1000000000001)

@st.time_this
def divide(a, b):
    while b != 0:
        a, b = b, a % b
    return a

@st.time_this
def subtract(a, b):
    while a != b:
        if a > b:
            a -= b
        else:
            b -= a
    return a

@st.time_this
def recurse(a, b):
    if b == 0:
        return a
    else:
        return recurse(b, a % b)

if __name__ == '__main__':
    st.run()

multiargs.txt

args: 10, 100      usec    loops    best of
---------------  ------  -------  ---------
divide             0.38  1000000          3
subtract           1.25  1000000          3
recurse            0.48  1000000          3

args: 8000, 92833      usec    loops    best of
-------------------  ------  -------  ---------
divide                 1.48  1000000          3
subtract               5.69   100000          3
recurse                2.07   100000          3

args: 898989, 1000000000001         usec    loops    best of
-----------------------------  ---------  -------  ---------
divide                              1.57  1000000          3
subtract                       155705.35       10          3
recurse                             2.20   100000          3

primeexample.py

from simpletimeit import stimeit

stimeit.call_with(100)
stimeit.call_with(500)


@stimeit.time_this
def check_all(n):
    result = []
    for i in range(2, n + 1):
        if not any((i % x) == 0 for x in range(2, i)):
            result.append(i)
    return result

@stimeit.time_this
def sieve(n):
    flags = [True for _ in range(n + 1)]
    flags[0] = flags[1] = False
    for i in range(len(flags)):
        if flags[i]:
            for j in range(i + 1, len(flags)):
                if flags[j] and j % i == 0:
                    flags[j] = False

    return [i for i, f in enumerate(flags) if f]

@stimeit.time_this
def memoized(n, _primes={}):
    result = []
    for i in range(2, n + 1):
        if i not in _primes:
            _primes[i] = not any(i % x == 0 for x in range(2, i))
        if _primes[i]:
            result.append(i)

    return result

if __name__ == '__main__':
    stimeit.run()

primeexample.txt

args: 100      usec    loops    best of
-----------  ------  -------  ---------
check_all    242.35     1000          3
sieve        138.30    10000          3
memoized      12.61   100000          3

args: 500       usec    loops    best of
-----------  -------  -------  ---------
check_all    3539.53      100          3
sieve        2019.23      100          3
memoized       65.86    10000          3

withsetup.py

from simpletimeit import stimeit
import gc

st = stimeit.SimpleTimeIt()

demo_var = 1.1

st.call_with(101, _setup='from __main__ import demo_var')

@st.time_this
def demonstrate_setup(n):
    raise ValueError(n, demo_var)

st.run()

withsetup.txt

Traceback (most recent call last):
  File "withsetup.py", line 14, in <module>
    st.run()
  File "/Users/jwshephe/dev/simpletimeit/simpletimeit/stimeit.py", line 145, in run
    results = self._get_results(setup, args)
  File "/Users/jwshephe/dev/simpletimeit/simpletimeit/stimeit.py", line 126, in _get_results
    for func in self._functions)
  File "/Users/jwshephe/dev/simpletimeit/simpletimeit/stimeit.py", line 126, in <genexpr>
    for func in self._functions)
  File "/Users/jwshephe/dev/simpletimeit/simpletimeit/stimeit.py", line 121, in _run_setup_and_func_with_args
    setup=self._prepared_setup(setup, func))
  File "/Users/jwshephe/dev/simpletimeit/simpletimeit/adaptiverun.py", line 26, in adaptiverun
    x = t.timeit(number)
  File "/Users/jwshephe/miniconda/envs/timeit-scratch/lib/python3.4/timeit.py", line 178, in timeit
    timing = self.inner(it, self.timer)
  File "<timeit-src>", line 7, in inner
  File "withsetup.py", line 12, in demonstrate_setup
    raise ValueError(n, demo_var)
ValueError: (101, 1.1)