Comparing data formats in Keras ("channels_first" vs "channels_last") for VGG style ConvNets.

NHWC vs. NCHW.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# NHWC vs. NCHW\n",
    ">\"On GPU, NCHW is faster. But on CPU, NHWC is sometimes faster.\" [(source)](https://www.tensorflow.org/performance/performance_models#build_the_model_with_both_nhwc_and_nchw)\n",
    "\n",
    "When computing convolutions, we can consider each tensor element as a struct with multiple features (e.g. one per image color) but we could also view each feature as part of an individual feature map. Depending on how we order the input tensor we'll get slightly different execution paths. Essentially it comes down to the order of computation and cache locality (in rough pseudo code):\n",
    "\n",
    "```python\n",
    "# NHWC\n",
    "for i in rows\n",
    "    for j in columns\n",
    "        for k in filters\n",
    "\n",
    "# NCHW\n",
    "for k in filters\n",
    "    for i in rows\n",
    "        for j in columns\n",
    "```\n",
    "However, due to such a convoluted and deep toolchain (puns intended) it is really hard to know what the end result will be in terms of actual wall time. Benchmarking is the best way of getting a feel for how much we need to care about designing models that work with both data formats. The type of network that should benefit the most from the right ordering is something like [VGG16](https://arxiv.org/abs/1409.1556) that has a huge amount of convolution kernels compared to more modern architectures, so let's benchmark that one, because it will give us a decent upper bound on what performance differences we should expect."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-25T09:47:17.120450Z",
     "start_time": "2018-06-25T09:47:15.433154Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.9.0-dev20180607'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "tf.VERSION"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-25T09:47:17.132610Z",
     "start_time": "2018-06-25T09:47:17.124143Z"
    }
   },
   "outputs": [],
   "source": [
    "from tensorflow.python.keras import *\n",
    "from tensorflow.python.keras.models import *\n",
    "from tensorflow.python.keras.layers import *\n",
    "from tensorflow.python.keras.applications import *\n",
    "\n",
    "\n",
    "def create_model():\n",
    "    backend.clear_session()\n",
    "\n",
    "    base_model = VGG16(include_top=False)\n",
    "    inputs = base_model.inputs\n",
    "\n",
    "    x = base_model.output\n",
    "    x = GlobalAveragePooling2D()(x)\n",
    "    x = Dense(1024, activation='relu')(x)\n",
    "    x = Dense(100, activation='softmax')(x)\n",
    "\n",
    "    outputs = [x]\n",
    "    model = Model(inputs, outputs)\n",
    "    model.compile('adam', 'sparse_categorical_crossentropy')\n",
    "    return model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-25T09:55:43.073936Z",
     "start_time": "2018-06-25T09:47:17.135120Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/1\n",
      "50000/50000 [==============================] - 50s 1ms/step - loss: 15.9570\n",
      "Epoch 1/1\n",
      "50000/50000 [==============================] - 44s 878us/step - loss: 15.9569\n",
      "Epoch 1/1\n",
      "50000/50000 [==============================] - 44s 878us/step - loss: 15.9569\n",
      "Epoch 1/1\n",
      "50000/50000 [==============================] - 44s 877us/step - loss: 15.9569\n",
      "Epoch 1/1\n",
      "50000/50000 [==============================] - 44s 879us/step - loss: 15.9569\n",
      "Epoch 1/1\n",
      "50000/50000 [==============================] - 44s 886us/step - loss: 15.9569\n",
      "Epoch 1/1\n",
      "50000/50000 [==============================] - 44s 883us/step - loss: 15.9569\n",
      "Epoch 1/1\n",
      "50000/50000 [==============================] - 44s 883us/step - loss: 15.9569\n",
      "Epoch 1/1\n",
      "50000/50000 [==============================] - 44s 879us/step - loss: 15.9569\n",
      "Epoch 1/1\n",
      "50000/50000 [==============================] - 44s 881us/step - loss: 15.9569\n",
      "Epoch 1/1\n",
      "50000/50000 [==============================] - 44s 878us/step - loss: 15.9569\n",
      "44 s ± 141 ms per loop (mean ± std. dev. of 10 runs, 1 loop each)\n"
     ]
    }
   ],
   "source": [
    "backend.set_image_data_format('channels_first')\n",
    "x_train, y_train = datasets.cifar100.load_data()[0]\n",
    "model = create_model()\n",
    "nchw = %timeit -o -r10 model.fit(x_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-25T10:04:12.371955Z",
     "start_time": "2018-06-25T09:55:43.078257Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/1\n",
      "50000/50000 [==============================] - 49s 976us/step - loss: 15.9564\n",
      "Epoch 1/1\n",
      "50000/50000 [==============================] - 45s 902us/step - loss: 15.9569\n",
      "Epoch 1/1\n",
      "50000/50000 [==============================] - 45s 902us/step - loss: 15.9569\n",
      "Epoch 1/1\n",
      "50000/50000 [==============================] - 45s 905us/step - loss: 15.9569\n",
      "Epoch 1/1\n",
      "50000/50000 [==============================] - 45s 904us/step - loss: 15.9569\n",
      "Epoch 1/1\n",
      "50000/50000 [==============================] - 45s 901us/step - loss: 15.9569\n",
      "Epoch 1/1\n",
      "50000/50000 [==============================] - 45s 900us/step - loss: 15.9569\n",
      "Epoch 1/1\n",
      "50000/50000 [==============================] - 45s 901us/step - loss: 15.9569\n",
      "Epoch 1/1\n",
      "50000/50000 [==============================] - 45s 900us/step - loss: 15.9569\n",
      "Epoch 1/1\n",
      "50000/50000 [==============================] - 45s 900us/step - loss: 15.9569\n",
      "Epoch 1/1\n",
      "50000/50000 [==============================] - 45s 896us/step - loss: 15.9569\n",
      "45.1 s ± 118 ms per loop (mean ± std. dev. of 10 runs, 1 loop each)\n"
     ]
    }
   ],
   "source": [
    "backend.set_image_data_format('channels_last')\n",
    "x_train, y_train = datasets.cifar100.load_data()[0]\n",
    "model = create_model()\n",
    "nhwc = %timeit -o -r10 model.fit(x_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-25T10:55:12.438612Z",
     "start_time": "2018-06-25T10:55:12.208193Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Channels first is 2.39% faster.'"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAD8CAYAAACGsIhGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAFxhJREFUeJzt3X2QZXV95/H3JwPyjEwW04ojDlli7egQjfZaCrNuD9TqrsxONLtsGMWH7FhjKkGJFiBTbCmydnxAhMW4WzXLqKjJpCwTt0ZwwRi4WUe01hlQBmwTiRgfwCJGgzYgwvDdP+6deOnt7nN7pk/3ne73q6pr7vmd8zv325cz8+H3Ow83VYUkSbP5pcUuQJI0/AwLSVIjw0KS1MiwkCQ1MiwkSY0MC0lSI8NCktTIsJAkNTIsJEmNDlvsAubLiSeeWKtXr17sMpaMBx98kGOOOWaxy5Cm5fE5f/bs2fPDqnpK03ZLJixWr17N7t27F7uMJaPT6TA2NrbYZUjT8vicP0n+bpDtnIaSJDUyLCRJjQwLSVIjw0KS1MiwkCQ1MiwkHTJ27NjB2rVrOeuss1i7di07duxY7JKWjSVz6aykpW3Hjh1ceumlbN++nX379rFixQo2b94MwKZNmxa5uqXPkYWkQ8L4+Djbt29n/fr1HHbYYaxfv57t27czPj6+2KUtC4aFpEPCxMQE69ate0LbunXrmJiYWKSKlhfDQtIhYc2aNezatesJbbt27WLNmjWLVNHyYlhIOiRceumlbN68mVtuuYXHHnuMW265hc2bN3PppZcudmnLgie4JR0S9p/EftOb3sTExARr1qxhfHzck9sLxLCQdMjYtGkTmzZt8kGCi8BpKElSI8NCktSo9bBIsiLJ7Umu7y1/NMk9Sb7a+3neNH2el+RLSe5KckeS3267TknSzBbinMUFwARwfF/bRVX1qVn6PAS8tqq+meQkYE+Sm6rqH9ssVJI0vVZHFklWAWcD186lX1X9TVV9s/f6XuB+oPFr/yRJ7Wh7Gupq4GLg8Snt473ppauSHDHbDpK8EHgS8Lct1ShJatDaNFSSDcD9VbUnyVjfqq3AD+gGwDbgbcDlM+zjacDHgddV1dTAIckWYAvAyMgInU5nPn+FZW1yctLPU0PL43Phpara2XHybuA1wGPAkXTPWfx5VZ3Xt80YcGFVbZim//FAB/jDhvMbAIyOjtbu3bvnp3h5HbsWXZID6tfWv2lLVZI9VTXatF1r01BVtbWqVlXVauBc4OaqOq83WiDdI+EVwJ1T+yZ5EvBp4GODBIWkpaeqZvx55tuun3Gd2rEY91n8cZK9wF7gROBdAElGk+w/Ef6fgJcAr5/tEltJ0sJYkMd9VFWH7pQSVXXmDNvsBt7Qe/0J4BMLUZskqZl3cEuSGhkWkqRGhoUkqZFhIUlqZFhIkhoZFpKkRoaFJKmRYSFJamRYSJIaLcgd3JI0k+e+83M88PCjc+63+pIbBt72yUcdztfe8dI5v4d+wbCQtKgeePhRvv2es+fUZ65PRZ5LsGh6TkNJkhoZFpKkRoaFJKmRYSFJauQJbkmL6rg1l3DadZfMveN1c3kPgLmdRNcTGRaSFtVPJ97j1VCHAKehJEmNWg+LJCuS3J7k+t7yR5Pc0/Td2klel+SbvZ/XtV2nJGlmCzENdQEwARzf13ZRVX1qpg5Jfhl4BzAKFLAnyc6q+nGrlUqSptXqyCLJKrpnla6dY9eXAX9RVT/qBcRfAP92vuuTJA2m7Wmoq4GLgcentI8nuSPJVUmOmKbf04Hv9i1/r9cmSVoErU1DJdkA3F9Ve5KM9a3aCvwAeBKwDXgbcPkBvscWYAvAyMgInU7nYEpWn8nJST9PLZi5HmsHcnx6PB+cNs9ZnAFsTPJy4Ejg+CSfqKrzeusfSfIR4MJp+n4fGOtbXgV0pm5UVdvoBg6jo6M1l0vpNLu5XpooHbAbb+D1Nz44x04BBu/z5KMO93g+SK2FRVVtpTuKoDeyuLCqzkvytKq6L0mAVwB3TtP9JuAPk6zsLb90/74kLS0z3WPR/Sdi7qrqYMrRDBbjPos/TrIX2AucCLwLIMlokmsBqupHwH8FvtL7ubzXJmmZqKoZf2655ZYZ16kdC3IHd1V16E0jVdWZM2yzG3hD3/KHgQ8vQHmSpAbewS1JamRYSJIaGRaSpEaGhSSpkWEhSWpkWEiSGhkWkqRGhoUkqZFhIUlqZFhIkhoZFpKkRoaFJKmRYSFJamRYSJIaGRaSpEaGhSSpkWEhSWpkWEg6ZOzYsYO1a9dy1llnsXbtWnbs2LHYJS0bC/K1qpJ0sHbs2MGll17K9u3b2bdvHytWrGDz5s0AbNq0aZGrW/paH1kkWZHk9iTXT2m/JsnkDH0OT3Jdkr1JJpJsbbtOScNtfHyc7du3s379eg477DDWr1/P9u3bGR8fX+zSloWFmIa6AJjob0gyCqycpc85wBFVdRrwAuCNSVa3VaCk4TcxMcG6deue0LZu3TomJiZm6KH51GpYJFkFnA1c29e2ArgCuHiWrgUck+Qw4Cjg58BPWixV0pBbs2YNu3btekLbrl27WLNmzSJVtLy0fc7iarqhcFxf2/nAzqq6L8lM/T4F/CZwH3A08Jaq+tHUjZJsAbYAjIyM0Ol05q/yZW5yctLPU0Plla98Ja9+9au56KKLOOWUU7jqqqu44oor2Lx5s8fqAmgtLJJsAO6vqj1JxnptJ9GdYhpr6P5CYB9wEt3pqi8k+XxVfat/o6raBmwDGB0drbGxpt1qUJ1OBz9PDZOxsTGe/exnMz4+zsTEBGvWrOHKK6/05PYCaXNkcQawMcnLgSOB44G7gEeAu3ujiqOT3F1Vp07p+yrgxqp6FLg/yReBUeBbSFq2Nm3axKZNm/yfmUXQ2jmLqtpaVauqajVwLnBzVa2sqqdW1epe+0PTBAXAd4AzAZIcA7wI+EZbtUqSZjc0N+Ul2Zjk8t7ih4Bjk9wFfAX4SFXdsXjVSdLytiA35VVVB+hM035s3+udwM7e60m65zYkSUNgaEYWkqThZVhIkhoZFpKkRoaFJKmRYSFJamRYSJIaGRaSpEaGhSSpkWEhSWo06x3cSX55gH08XlX/OE/1SJKGUNPjPu7t/cz4xRPACuDkeatIkjR0msJioqp+Y7YNktw+j/VIkoZQ0zmLFw+wj0G2kSQdwmYNi6r6GUCSf57kiN7rsSRvTnJC/zaSpKVr0Kuh/gzYl+RUul9j+gzgT1qrSpI0VAYNi8er6jHglcAHq+oi4GntlSVJGiaDhsWjSTYBrwOu77Ud3k5JkqRhM2hY/A7dE9njVXVPklOAj7dXliRpmAwUFlX19ap6c1Xt6C3fU1XvHaRvkhVJbk9y/ZT2a5JMztLv15N8KcldSfYmOXKQ95Mkzb9ZwyLJtqYdDLDNBcDElD6jwMpZ9nkY8Angd6vqOcAY8GhTLZKkdjTdlPeKJLNdGhtg/Ywrk1XA2cA48NZe2wrgCuBVdE+YT+elwB1V9TWAqvqHhjolSS1qCouLBtjHF2ZZdzVwMXBcX9v5wM6qui+Z8SkizwIqyU3AU4A/rar3DVCLJKkFs4ZFVV13oDtOsgG4v6r2JBnrtZ0EnEN3WqmprnXAvwQeAv4yyZ6q+ssp77EF2AIwMjJCp9M50HI1xeTkpJ+nhpbH58JrGlkcjDOAjUleDhwJHA/cBTwC3N0bVRyd5O6qOnVK3+8B/6eqfgiQ5LPA84EnhEVVbaN7kyCjo6M1NjbW3m+zzHQ6Hfw8Naw8Phdea99nUVVbq2pVVa0GzgVurqqVVfXUqlrda39omqAAuAk4LcnRvZPd/xr4elu1SpJmN6ewSHJ0W4Uk2ZjkcoCq+jHwAeArwFeB26rqhrbeW5I0u4GmoZKcDlwLHAucnOS5wBur6vcG6V9VHaAzTfuxfa93Ajv7lj9B9/JZSdIiG3RkcRXwMuAfAHqXtL6kraIkScNl4GmoqvrulKZ981yLJGlIDXo11Hd7U1GV5HCmuStbkrR0DTqy+F3g94GnA98HntdbliQtAwONLHr3O7y65VokSUNq0KuhTgHeBKzu71NVG9spS5I0TAY9Z/G/gO3AZ4DH2ytHkjSMBg2Ln1XVNa1WIkkaWoOGxX9L8g7gc3Sf7QRAVd3WSlWSpKEyaFicBrwGOJNfTENVb1mStMQNGhbnAL9aVT9vsxhJ0nAa9D6LO4ET2ixEkjS8Bh1ZnAB8I8lXeOI5Cy+dlaRlYNCweEerVUiShtqgd3D/VduFSJKG16xhkWRXVa1L8lO6Vz/90yqgqur4VquTJA2FppHFMQBVddwC1CJJGlJNV0NVw3pJ0jLQNLL4lSRvnWllVX2g6Q2SrAB2A9+vqg197dcA/7n/q1Wn6Xsy8HXgsqp6f9N7SZLa0RQWK+h+73YO4j32f1HSP53fSDIKrByg7weA/30Q7y1JmgdNYXFfVV1+oDtPsgo4GxgH3tprWwFcAbwKeOUsfV8B3AM8eKDvL0maH03nLA5mRAFwNXAxT3ys+fnAzqq6b8Y3TY4F3ga88yDfX5I0D5pGFmcd6I6TbADur6o9ScZ6bSfRfc7UWEP3y4CrqmoymTmvkmwBtgCMjIzQ6XQOtFxNMTk56eepoeXxufBS1c4FT0neTfdJtY8BR9I9Z/FI7+dnvc1OBr5VVadO6fsF4Bm9xRPojkzeXlV/NNP7jY6O1u7du+f1d1jOOp0OY2Nji12GNC2Pz/mTZE9VjTZtN+jjPuasqrYCW3vFjAEX9l8N1WufnBoUvb7/qm+by4DJ2YJCktSuQZ8627okG5Mc8Ml0SVJ7WhtZ9KuqDtCZpv3Yvtc7gZ3TbHNZi6VJkgYwNCMLSdLwMiwkSY0MC0lSI8NCktTIsJAkNTIsJEmNDAtJUiPDQpLUaEFuytPwmu1BjTNp63likoaXI4tlrqqm/Xnm266fcZ2k5cewkCQ1MiwkSY0MC0lSI8NCktTIsJAkNTIsJEmNDAtJUiPDQpLUyLCQJDVqPSySrEhye5Lrp7Rfk2Ryhj7/JsmeJHt7f57Zdp2SpJktxLOhLgAmgOP3NyQZBVbO0ueHwL+vqnuTrAVuAp7eapWSpBm1OrJIsgo4G7i2r20FcAVw8Uz9qur2qrq3t3gXcFSSI9qsVZI0s7ZHFlfTDYXj+trOB3ZW1X0DPvH0PwC3VdUjU1ck2QJsARgZGaHT6Rx0wfoFP08Nq8nJSY/PBdZaWCTZANxfVXuSjPXaTgLOAcYG3MdzgPcCL51ufVVtA7YBjI6O1tjYQLvVIG68AT9PDatOp+PxucDaHFmcAWxM8nLgSLrnLO4CHgHu7o0qjk5yd1WdOrVzbwrr08Brq+pvW6xTktSgtXMWVbW1qlZV1WrgXODmqlpZVU+tqtW99odmCIoTgBuAS6rqi23VKEkazNDcZ5FkY5LLe4vnA6cCb0/y1d7PryxieZK0rC3I16pWVQfoTNN+bN/rncDO3ut3Ae9aiNokSc2GZmQhSRpehoUkqZFhIUlqZFhIkhoZFpKkRoaFJKmRYSFJarQg91lo8T33nZ/jgYcfnVOf1ZfcMKftn3zU4XztHdM+xkvSIc6wWCYeePhRvv2eswfe/kAe1DbXcJF06HAaSpLUyLCQJDUyLCRJjQwLSVIjw0KS1MiwkCQ1MiwkSY0MC0lSI8NCktSo9bBIsiLJ7Umun9J+TZLJWfptTXJ3kr9O8rK265QkzWwhHvdxATABHL+/IckosHKmDkmeDZwLPAc4Cfh8kmdV1b6Wa5UkTaPVkUWSVcDZwLV9bSuAK4CLZ+n6m8CfVtUjVXUPcDfwwjZrlSTNrO1pqKvphsLjfW3nAzur6r5Z+j0d+G7f8vd6bZKkRdDaNFSSDcD9VbUnyViv7STgHGBsnt5jC7AFYGRkhE6nMx+7XbLm8vlMTk4e0OfpfwMthAM9PnXg2jxncQawMcnLgSPpnrO4C3gEuDsJwNFJ7q6qU6f0/T7wjL7lVb22J6iqbcA2gNHR0ZrrI7WXlRtvmNMjxw/kEeVzfQ/pQB3Q8amD0to0VFVtrapVVbWa7snqm6tqZVU9tapW99ofmiYoAHYC5yY5IskpwK8B/7etWiVJsxuaLz9KshEYraq3V9VdST4JfB14DPh9r4SSpMWzIGFRVR2gM037sX2vd9IdUexfHgfGF6A8SVKDoRlZqF3HrbmE0667ZG6drpvre0D3SmlJS41hsUz8dOI9fge3pAPms6EkSY0MC0lSI8NCktTIsJAkNTIsJEmNDAtJUiPDQpLUyPsslpE53wdx49y2f/JRh89t/5IOGYbFMjGXG/KgGyxz7SNp6XIaSpLUyLCQJDUyLCRJjQwLSVIjw0KS1MiwkCQ1MiwkSY1av88iyQpgN/D9qtqQZDswCgT4G+D1VTU5pc/hwLXA83s1fqyq3t12rctRkpnXvXf69qpqqRpJw2ohRhYXABN9y2+pqudW1a8D3wHOn6bPOcARVXUa8ALgjUlWt13oclRV0/7ccsstM66TtPy0GhZJVtH9UuZr97dV1U966wIcBUz3r08BxyQ5rLfNz4GftFmrJGlmbY8srgYuBh7vb0zyEeAHwL8APjhNv08BDwL30R19vL+qftRuqZKkmbR2ziLJBuD+qtqTZKx/XVX9Tu9cxgeB3wY+MqX7C4F9wEnASuALST5fVd+a8h5bgC0AIyMjdDqdNn6VZWlyctLPU0PL43Phpa056CTvBl4DPAYcCRwP/HlVnde3zUuAi6tqw5S+HwK+XFUf7y1/GLixqj450/uNjo7W7t275/8XWaY6nQ5jY2OLXYY0LY/P+ZNkT1WNNm3X2jRUVW2tqlVVtRo4F7gZeE2SU3sFBtgIfGOa7t8Bzuxtdwzwohm2kyQtgIW+zyLAdUn2AnuBpwGXAyTZmOTy3nYfAo5NchfwFeAjVXXHAtcqSepZkO+zqKoO0OktnjHDNjuBnb3Xk3Qvn5UkDQHv4JYkNWrtBPdCS/L3wN8tdh1LyInADxe7CGkGHp/z55lV9ZSmjZZMWGh+Jdk9yBUS0mLw+Fx4TkNJkhoZFpKkRoaFZrJtsQuQZuHxucA8ZyFJauTIQpLUyLBYwpJUkiv7li9Mclnf8muT3Jlkb5Lbk1zYa/9okv84ZV+TvT8/neQVfe1/neS/9C3/WZLfavHX0hIx2/GZ5LL9x2Pf+m8nOTHJVUn+oK/9piTX9i1fmeStvdfPSvLZJN9McluSTyYZaf2XW4IMi6XtEeC3kpw4dUWSfwf8AfDS3pdMvQh4YIB9fhE4vbePf0b3UfIv7lv/YuDWg6xby8OMx2eD/mPwl+jec/GcvvWnA7cmORK4AfgfVfVrVfV84L8DjfcU6P9nWCxtj9E9EfiWadZtBS6sqnsBquqRqvqfA+zzVnp/UXt/fgZ4SrpOAR6uqh8cfOlaBmY7PmdzK7/4H5TnAHcCP02yMskRwBrgNuBVwJeq6jP7O1ZVp6ruPOjKl6EFeTaUFtWHgDuSvG9K+1pgzyz9ruifXuqzB1ib5El0w+KvgF+l+xf0N3BUobmZ6fgEeEuS8/qWTwKoqnuTPJbkZLrH4JeAp9MNkAeAvVX18yRNx7jmwLBY4qrqJ0k+BrwZeHgOXS+qqk/tX9h/zqKqHuk9Dfj5dKeu3kc3LE6nGxZfnK/atfQ1HJ9XVdX79y8k+Xbfuv0j3NOBD9ANi9PphoXHYAuchloergY2A8f0td0FvOAA9/dF4CXAcVX1Y+DL/OIvriMLzdV0x2eT/ectTqM7DfVluiOL/mPwYI5xTWFYLAO97y//JN2/kPu9m+5U01MBkjwpyRsG3OWtwBuBr/WW76A7yjiZ7l9caWAzHJ9NbgU2AD+qqn29fZzAEy+w+BPg9CRn7++U5CW96SnNkWGxfFxJ96oRAKrqs8AfAZ/vTSvdRverbwdxK92ppy/19vUYcD+wu6oen8+itWw84fgcwN7e9l+e0vZAVf0QoKoephsob+pdOvt14PeAv5+fkpcX7+CWJDVyZCFJamRYSJIaGRaSpEaGhSSpkWEhSWpkWEiSGhkWkqRGhoUkqdH/AyzwYBlhA3zIAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import pandas as pd\n",
    "\n",
    "df = pd.DataFrame({'NCHW': nchw.all_runs, 'NHWC': nhwc.all_runs})\n",
    "ax = df.boxplot()\n",
    "ax.set_ylabel('Time [s]')\n",
    "\n",
    "means = df.mean()\n",
    "ratio = means.NHWC / means.NCHW\n",
    "f'Channels first is {100*(ratio - 1):.2f}% faster.'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-25T11:10:22.002302Z",
     "start_time": "2018-06-25T11:10:21.997857Z"
    }
   },
   "source": [
    "## Conclusion\n",
    "As we can see above, TensorFlow's documentation isn't wrong when training a VGG-esque network (_lots_ of convolution filters) on a GPU. However the difference in performance is minor so I personally wouldn't bother with it. Hopefully XLA will be turned on by default eventually and its HLO could take care of this performance gap for us. Also, keep in mind that for control-flow heavy models such as RNNs the results would probably be even blurrier.\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}