async await の仕組みについて、 yield from から await へと見ていく.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# python の async/await 構文について調べる。\n",
    "\n",
    "関連する PEP はこれ: https://www.python.org/dev/peps/pep-0492/\n",
    "\n",
    "色々読んだり試したりした今のところの理解としては、\n",
    "\n",
    "* 動作としては `yield from` と同じようなもの\n",
    "* 構文やら要求するインターフェースやら出てくるクラスやらは別物\n",
    "  * 今までの generator ベースの coroutine は、普通に iterator が作りたいだけなのか見分けがつけづらいとか、色々紛らわしかった。\n",
    "* `with` や `for` のための構文やら機能やらを足した\n",
    "\n",
    "今回は、 async/await がどういうものなのかを見ていきたい。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## まずは yield from について\n",
    "\n",
    "元となったのは `yield from` (generator) だ。\n",
    "まずはこちらの動作を確認する。\n",
    "\n",
    "1. `yield from` は後ろに `Iterable` (sub generator) をとり、そこから `Iterator` を取り出す (`.__iter__()` を呼び出す)\n",
    "2. `Iterator` を進めて、出てきた要素を外側 (generator を進めている側) に戻す\n",
    "3. 外側が generator を進める (`.__next__()` や `.send()` を呼び出す) と、 `Iterator` の止まっていた部分から動作を再開する\n",
    "4. `Iterator` が generator の場合、そこから `return` された値が `yield from` の結果として generator に渡る\n",
    "\n",
    "実際のコードでも見てみる。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "create generator\n",
      "... and start it\n",
      "  start generator\n",
      "  ... and yield from sub generator\n",
      "    start sub generator\n",
      "    ... and yield value\n",
      "got value 1\n",
      "resume it\n",
      "    sub generator resumed with value 2\n",
      "    ... and return value\n",
      "  generator resumed with value 42\n",
      "generator finished\n"
     ]
    }
   ],
   "source": [
    "def sub_generator():\n",
    "    print('    start sub generator')\n",
    "    print('    ... and yield value')\n",
    "    ret = yield 1\n",
    "    print('    sub generator resumed with value', ret)\n",
    "    print('    ... and return value')\n",
    "    return 42\n",
    "\n",
    "def generator():\n",
    "    print('  start generator')\n",
    "    print('  ... and yield from sub generator')\n",
    "    ret = yield from sub_generator()\n",
    "    print('  generator resumed with value', ret)\n",
    "\n",
    "print('create generator')\n",
    "g = generator()\n",
    "print('... and start it')\n",
    "ret = g.send(None)\n",
    "print('got value', ret)\n",
    "print('resume it')\n",
    "try:\n",
    "    # it raise StopIteration which indicate generator finished\n",
    "    g.send(2)\n",
    "except StopIteration:\n",
    "    print('generator finished')\n",
    "else:\n",
    "    assert False"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## yield from その2\n",
    "\n",
    "さて、上記では sub generator を作るのに普通に generator を使ったが、ここはユーザー定義の `Iterable` でも構わない。\n",
    "\n",
    "以下で自分で定義した `Iterable` で sub generator を置き換えてみる。\n",
    "上記と同様の動作をするはずだ。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "create generator\n",
      "... and start it\n",
      "  start generator\n",
      "  ... and yield from my iterable\n",
      "    start my iterable\n",
      "    ... and yield value\n",
      "got value 1\n",
      "resume it\n",
      "    my iterable resumed with value 2\n",
      "    ... and return value\n",
      "  generator resumed with value 42\n",
      "generator finished\n"
     ]
    }
   ],
   "source": [
    "class MyIterable(object):\n",
    "    def __iter__(self):\n",
    "        print('    start my iterable')\n",
    "        print('    ... and yield value')\n",
    "        ret = yield 1\n",
    "        print('    my iterable resumed with value', ret)\n",
    "        print('    ... and return value')\n",
    "        return 42\n",
    "\n",
    "def generator():\n",
    "    print('  start generator')\n",
    "    print('  ... and yield from my iterable')\n",
    "    ret = yield from MyIterable()\n",
    "    print('  generator resumed with value', ret)\n",
    "\n",
    "print('create generator')\n",
    "g = generator()\n",
    "print('... and start it')\n",
    "ret = g.send(None)\n",
    "print('got value', ret)\n",
    "print('resume it')\n",
    "try:\n",
    "    # it raise StopIteration which indicate generator finished\n",
    "    g.send(2)\n",
    "except StopIteration:\n",
    "    print('generator finished')\n",
    "else:\n",
    "    assert False"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## そして async/await へ\n",
    "\n",
    "では続いて async/await を見る。\n",
    "\n",
    "上述したように、 async/await は `yield from` と動作は同じだが、\n",
    "\n",
    "* `def` の代わりに `async def`, `yield from` の代わりに `await` と書く\n",
    "* `Iterable` の代わりに `Awaitable` をとる\n",
    "* `Awaitable` は `.__iter__()` の代わりに `.__await__()` が呼ばれる\n",
    "  * (`.__await__()` が返すものは `Iterable` 同様 `Iterator`.)\n",
    "* 呼び出し結果の coroutine は `__iter__`, `__next__` を持たない (`Iterable`/`Iterator` ではない)\n",
    "\n",
    "という違いがある。\n",
    "\n",
    "では、上記 `yield from` の例をこれにしたがって書き換えてみよう。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "create coroutine\n",
      "... and start it\n",
      "  start coroutine\n",
      "  ... and await my awaitable\n",
      "    start my awaitable\n",
      "    ... and yield value\n",
      "got value 1\n",
      "resume it\n",
      "    my awaitable resumed with value 2\n",
      "    ... and return value\n",
      "  coroutine resumed with value 42\n",
      "coroutine finished\n"
     ]
    }
   ],
   "source": [
    "class MyAwaitable(object):\n",
    "    # __iter__ -> __await__\n",
    "    def __await__(self):\n",
    "        print('    start my awaitable')\n",
    "        print('    ... and yield value')\n",
    "        ret = yield 1\n",
    "        print('    my awaitable resumed with value', ret)\n",
    "        print('    ... and return value')\n",
    "        return 42\n",
    "\n",
    "# def -> async def\n",
    "async def coroutine():\n",
    "    print('  start coroutine')\n",
    "    print('  ... and await my awaitable')\n",
    "    # yield from -> await\n",
    "    ret = await MyAwaitable()\n",
    "    print('  coroutine resumed with value', ret)\n",
    "\n",
    "print('create coroutine')\n",
    "coro = coroutine()\n",
    "print('... and start it')\n",
    "ret = coro.send(None)\n",
    "print('got value', ret)\n",
    "print('resume it')\n",
    "try:\n",
    "    # it raise StopIteration which indicate coroutine finished\n",
    "    coro.send(2)\n",
    "except StopIteration:\n",
    "    print('coroutine finished')\n",
    "else:\n",
    "    assert False"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## まとめ\n",
    "\n",
    "以上、 generator からの変遷を見ることによって、 async/await を理解することを試みた。\n",
    "\n",
    "`asyncio` ではこの仕組みを利用し、 `Future` と組み合わせることによって、非同期処理を同期的処理のように書き下せる仕組みを提供している。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}