methodmemoize.py

import functools

class memoize(object):
    def __init__(self, func):
        self.func = func
        self.cache = {}
    def __call__(self, *args):
        return self.cache_get(args, lambda: self.func(*args))
    def __get__(self, obj, objtype):
        return self.cache_get(obj, lambda: self.__class__(functools.partial(self.func, obj)))
    def cache_get(self, key, gen_callback):
        try:
            return self.cache[key]
        except KeyError:
            self.cache[key] = gen_callback()
            return self.cache[key]

class Adder(object):
    @memoize
    def add(self, arg1, arg2):
        print 'CALCULATING', arg1, '+', arg2
        return arg1 + arg2


@memoize
def subtract(arg1, arg2):
    print 'CALCULATING', arg1, '-', arg2
    return arg1 - arg2

def main():
    print subtract(10, 5)
    print subtract(10, 5)

    adder1 = Adder()
    adder2 = Adder()
    print adder1.add(5, 5)
    print adder1.add(5, 5)
    print adder2.add(5, 5)
    print adder2.add(5, 5)

if __name__ == '__main__':
    main()

Thanks for this!

However, no need to do a dictionary lookup twice if there is a cache miss:

        v = func()
        cache[key] = v
        return v

Thanks very much for this! It's far more useful than the more well-known memoize decorator in the Python Decorator Library wiki. Dropping it into my code led to an order-of-magnitude improvement!

Thanks for the comment. I should note that the cache never gets cleaned, therefore never dereferences any object that has run any of its memoized methods. So if your application is long-running and creates lots of objects, they will never get garbage-collected. This caveat applies to all memoization decorators I know of (for example calling a memoized function with objects as arguments), but ideally a method-memoization decorator would clear the cache for any object that is garbage-collected. The only way I know to do this would be to store the cached result dictionary on each object in a unique attribute.

At first glance, it looks like this one does the right thing (doesn't store object references): http://code.activestate.com/recipes/577452-a-memoize-decorator-for-instance-methods/

I'm trying to understand your memoize example. From the point of view of being able to wrap both plain functions and class methods, what exactly does it do that http://wiki.python.org/moin/PythonDecoratorLibrary#Memoize doesn't?
(I'm still learning decorators and descriptors, so please bear with me.)

I can't remember, to be honest. I'm pretty sure that the more well-known memoizer in the Python decorator library was hitting the TypeError exception a lot when I tried it. It may have been because I was passing lists in (which are uncacheable). If you're curious, I'd modify the TypeError to print out the exception rather than just call the function and pretend that nothing happened. I think that it's possibly better behaviour to re-raise the TypeError in that case... if some code assumes that a function is memoized and is therefore O(1), performance can degrade by orders of magnitude if memoization is failing, to the point where the program does not function as expected. There's arguments leaning both ways on this matter, of course, but I think it's a policy question, and should be customizable (e.g. via a decorator argument).

…

On Sep 16, 2011, at 6:09 AM, Dominique wrote: I'm trying to understand your memoize example. From the point of view of being able to wrap both plain functions and class methods, what exactly does it do that http://wiki.python.org/moin/PythonDecoratorLibrary#Memoize doesn't? (I'm still learning decorators and descriptors, so please bear with me.) ## Reply to this email directly or view it on GitHub: https://gist.github.com/267733

Thanks for the response. While trying to work this out I wrote my own, pieced together from various sources and my understanding of how things work. One thing I wanted to do is cache numpy arrays by computing their sha1 digest. My code is at https://gist.github.com/1222577. I'd be interested in any comments.