Python Decorators

python_decorators.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A guide to Python's function decorators#\n",
    "\n",
    "(Converted to Jupyter from http://thecodeship.com/patterns/guide-to-python-function-decorators/)\n",
    "\n",
    "Python is rich with powerful features and expressive syntax. One of my favorites is decorators. In the context of design patterns, decorators dynamically alter the functionality of a function, method or class without having to directly use subclasses. This is ideal when you need to extend the functionality of functions that you don't want to modify. We can implement the decorator pattern anywhere, but Python facilitates the implementation by providing much more expressive features and syntax for that.\n",
    "\n",
    "In this post I will be discussing Python's function decorators in depth, accompanied by a bunch of examples on the way to clear up the concepts. All examples are in Python 2.7 but the same concepts should apply to Python 3 with some change in the syntax.\n",
    "\n",
    "Essentially, decorators work as wrappers, modifying the behavior of the code before and after a target function execution, without the need to modify the function itself, augmenting the original functionality, thus decorating it.\n",
    "\n",
    "### What you need to know about functions\n",
    "\n",
    "Before diving in, there are some prerequisites that should be clear. In Python, functions are first class citizens, they are objects and that means we can do a lot of useful stuff with them."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Assign functions to variables #"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "hello John\n"
     ]
    }
   ],
   "source": [
    "def greet(name):\n",
    "    return \"hello \"+name\n",
    "\n",
    "greet_someone = greet\n",
    "print greet_someone(\"John\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define functions inside other functions #"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hello John\n"
     ]
    }
   ],
   "source": [
    "def greet(name):\n",
    "    def get_message():\n",
    "        return \"Hello \"\n",
    "    result = get_message() + name\n",
    "    return result\n",
    "\n",
    "print greet(\"John\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Functions can be passed as parameters to other functions #"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hello John\n"
     ]
    }
   ],
   "source": [
    "def greet(name):\n",
    "    return \"Hello \" + name \n",
    "\n",
    "def call_func(func):\n",
    "    other_name = \"John\"\n",
    "    return func(other_name)  \n",
    "\n",
    "print call_func(greet)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Functions can return other functions #\n",
    "\n",
    "In other words, functions generating other functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hello there!\n"
     ]
    }
   ],
   "source": [
    "def compose_greet_func():\n",
    "    def get_message():\n",
    "        return \"Hello there!\"\n",
    "\n",
    "    return get_message\n",
    "\n",
    "greet = compose_greet_func()\n",
    "print greet()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Inner functions have access to the enclosing scope #\n",
    "\n",
    "More commonly known as a closure. A very powerful pattern that we will come across while building decorators. Another thing to note, Python only allows read access to the outer scope and not assignment. Notice how we modified the example above to read a \"name\" argument from the enclosing scope of the inner function and return the new function.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hello there John!\n"
     ]
    }
   ],
   "source": [
    "def compose_greet_func(name):\n",
    "    def get_message():\n",
    "        return \"Hello there \"+name+\"!\"\n",
    "    return get_message\n",
    "\n",
    "greet = compose_greet_func(\"John\")\n",
    "print greet()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Composition of Decorators #\n",
    "\n",
    "Function decorators are simply wrappers to existing functions. Putting the ideas mentioned above together, we can build a decorator. In this example let's consider a function that wraps the string output of another function by p tags."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<p>lorem ipsum, John dolor sit amet</p>\n"
     ]
    }
   ],
   "source": [
    "def get_text(name):\n",
    "    return \"lorem ipsum, {0} dolor sit amet\".format(name)\n",
    "\n",
    "def p_decorate(func):\n",
    "    def func_wrapper(name):\n",
    "        return \"<p>{0}</p>\".format(func(name))\n",
    "    return func_wrapper\n",
    "\n",
    "my_get_text = p_decorate(get_text)\n",
    "\n",
    "print my_get_text(\"John\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That was our first decorator. A function that takes another function as an argument, generates a new function, augmenting the work of the original function, and returning the generated function so we can use it anywhere. To have get_text itself be decorated by p_decorate, we just have to assign get_text to the result of p_decorate."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<p>lorem ipsum, John dolor sit amet</p>\n"
     ]
    }
   ],
   "source": [
    "get_text = p_decorate(get_text)\n",
    "\n",
    "print get_text(\"John\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another thing to notice is that our decorated function takes a name argument. All what we had to do in the decorator is to let the wrapper of get_text pass that argument.\n",
    "\n",
    "### Python's Decorator Syntax #\n",
    "\n",
    "Python makes creating and using decorators a bit cleaner and nicer for the programmer through some **syntactic sugar**.\n",
    "\n",
    "To decorate get_text we don't have to ``get_text = p_decorator(get_text)`` There is a neat shortcut for that, which is to mention the name of the decorating function before the function to be decorated. The name of the decorator should be perpended with an @ symbol.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<p>lorem ipsum, John dolor sit amet</p>\n"
     ]
    }
   ],
   "source": [
    "def p_decorate(func):\n",
    "   def func_wrapper(name):\n",
    "       return \"<p>{0}</p>\".format(func(name))\n",
    "   return func_wrapper\n",
    "\n",
    "@p_decorate\n",
    "def get_text(name):\n",
    "   return \"lorem ipsum, {0} dolor sit amet\".format(name)\n",
    "\n",
    "print get_text(\"John\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's consider we wanted to decorate our get_text function by 2 other functions to wrap a div and strong tag around the string output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<div><p><strong>lorem ipsum, John dolor sit amet</strong></p></div>\n"
     ]
    }
   ],
   "source": [
    "def p_decorate(func):\n",
    "   def func_wrapper(name):\n",
    "       return \"<p>{0}</p>\".format(func(name))\n",
    "   return func_wrapper\n",
    "\n",
    "def strong_decorate(func):\n",
    "    def func_wrapper(name):\n",
    "        return \"<strong>{0}</strong>\".format(func(name))\n",
    "    return func_wrapper\n",
    "\n",
    "def div_decorate(func):\n",
    "    def func_wrapper(name):\n",
    "        return \"<div>{0}</div>\".format(func(name))\n",
    "    return func_wrapper\n",
    "\n",
    "\n",
    "@div_decorate\n",
    "@p_decorate\n",
    "@strong_decorate\n",
    "def get_text(name):\n",
    "    return \"lorem ipsum, {0} dolor sit amet\".format(name)\n",
    "\n",
    "# Basic approach would be:\n",
    "# get_text = div_decorate(p_decorate(strong_decorate(get_text)))\n",
    "\n",
    "print get_text(\"John\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Decorating Methods#\n",
    "\n",
    "In Python, methods are functions that expect their first parameter to be a reference to the current object. We can build decorators for methods the same way, while taking ``self`` into consideration in the wrapper function.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<p>John Doe</p>\n"
     ]
    }
   ],
   "source": [
    "def p_decorate(func):\n",
    "   def func_wrapper(self):\n",
    "       return \"<p>{0}</p>\".format(func(self))\n",
    "   return func_wrapper\n",
    "\n",
    "class Person(object):\n",
    "    def __init__(self):\n",
    "        self.name = \"John\"\n",
    "        self.family = \"Doe\"\n",
    "\n",
    "    @p_decorate\n",
    "    def get_fullname(self):\n",
    "        return self.name+\" \"+self.family\n",
    "\n",
    "my_person = Person()\n",
    "print my_person.get_fullname()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A much better approach would be to make our decorator useful for functions and methods alike. This can be done by putting ``*args`` and ``**kwargs`` as parameters for the wrapper, then it can accept any arbitrary number of arguments and keyword arguments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<p>John Doe</p>\n"
     ]
    }
   ],
   "source": [
    "def p_decorate(func):\n",
    "    def func_wrapper(*args, **kwargs):\n",
    "       return \"<p>{0}</p>\".format(func(*args, **kwargs))\n",
    "    return func_wrapper\n",
    "\n",
    "class Person(object):\n",
    "    def __init__(self):\n",
    "        self.name = \"John\"\n",
    "        self.family = \"Doe\"\n",
    "\n",
    "    @p_decorate\n",
    "    def get_fullname(self):\n",
    "        return self.name+\" \"+self.family\n",
    "\n",
    "my_person = Person()\n",
    "\n",
    "print my_person.get_fullname()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Passing arguments to decorators\n",
    "\n",
    "Looking back at the example before the one above, you can notice how redundant the decorators in the example are. 3 decorators(div_decorate, p_decorate, strong_decorate) each with the same functionality but wrapping the string with different tags. We can definitely do much better than that. Why not have a more general implementation for one that takes the tag to wrap with as a string? Yes please!\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<p>Hello John</p>\n"
     ]
    }
   ],
   "source": [
    "def tags(tag_name):\n",
    "    def tags_decorator(func):\n",
    "        def func_wrapper(name):\n",
    "            return \"<{0}>{1}</{0}>\".format(tag_name, func(name))\n",
    "        return func_wrapper\n",
    "    return tags_decorator\n",
    "\n",
    "@tags(\"p\")\n",
    "def get_text(name):\n",
    "    return \"Hello \"+name\n",
    "\n",
    "print get_text(\"John\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It took a bit more work in this case. Decorators expect to receive a function as an argument, that is why we will have to build a function that takes those extra arguments and generate our decorator on the fly. In the example above, ``tags`` is our decorator generator."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Debugging decorated functions#\n",
    "\n",
    "At the end of the day decorators are just wrapping our functions, in case of debugging that can be problematic since the wrapper function does not carry the name, module and docstring of the original function. Based on the example above if we do:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "func_wrapper\n"
     ]
    }
   ],
   "source": [
    "print get_text.__name__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The output was expected to be get_text yet, the attributes ``__name__``, ``__doc__``, and ``__module__`` of ``get_text`` got overridden by those of the wrapper (func_wrapper). Obviously we can re-set them within func_wrapper but Python provides a much nicer way."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Functools to the rescue\n",
    "\n",
    "Fortunately Python (as of version 2.5) includes the functools module which contains functools.wraps. Wraps is a decorator for updating the attributes of the wrapping function(func_wrapper) to those of the original function(get_text). This is as simple as decorating func_wrapper by @wraps(func). Here is the updated example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "get_text\n",
      "returns some text\n",
      "__main__\n"
     ]
    }
   ],
   "source": [
    "from functools import wraps\n",
    "\n",
    "def tags(tag_name):\n",
    "    def tags_decorator(func):\n",
    "        @wraps(func)\n",
    "        def func_wrapper(name):\n",
    "            return \"<{0}>{1}</{0}>\".format(tag_name, func(name))\n",
    "        return func_wrapper\n",
    "    return tags_decorator\n",
    "\n",
    "@tags(\"p\")\n",
    "def get_text(name):\n",
    "    \"\"\"returns some text\"\"\"\n",
    "    return \"Hello \"+name\n",
    "\n",
    "print get_text.__name__ # get_text\n",
    "print get_text.__doc__ # returns some text\n",
    "print get_text.__module__ # __main__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Where to use decorators #\n",
    "\n",
    "The examples in this post are pretty simple relative to how much you can do with decorators. They can give so much power and elegance to your program. In general, decorators are ideal for extending the behavior of functions that we don't want to modify. For a great list of useful decorators I suggest you check out the [Python Decorator Library](https://wiki.python.org/moin/PythonDecoratorLibrary)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}

Thanks a lot! Super helpful