Copy of Grover large from https://colab.research.google.com/drive/1edS5Mw_3GSVOs_UfueMayMcRQAPEDmm8?offline=true&sandboxMode=true#scrollTo=Uuh_3mE-fxNZ&forceEdit=true&offline=true&sandboxMode=true

grover_large.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "grover large",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "code",
      "metadata": {
        "id": "Uuh_3mE-fxNZ",
        "colab_type": "code",
        "outputId": "01dc6a63-dea1-407c-a1e4-fb0391c6fd66",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 700
        }
      },
      "source": [
        "!cd /content && rm -rf grover && git clone https://github.com/rowanz/grover.git\n",
        "%cd /content/grover\n",
        "!pip install -r requirements-gpu.txt\n",
        "!python download_model.py large"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Cloning into 'grover'...\n",
            "remote: Enumerating objects: 67, done.\u001b[K\n",
            "remote: Counting objects: 100% (67/67), done.\u001b[K\n",
            "remote: Compressing objects: 100% (51/51), done.\u001b[K\n",
            "remote: Total 67 (delta 19), reused 59 (delta 15), pack-reused 0\u001b[K\n",
            "Unpacking objects: 100% (67/67), done.\n",
            "/content/grover\n",
            "Requirement already satisfied: pandas==0.24.2 in /usr/local/lib/python3.6/dist-packages (from -r requirements-gpu.txt (line 1)) (0.24.2)\n",
            "Requirement already satisfied: regex==2019.4.14 in /usr/local/lib/python3.6/dist-packages (from -r requirements-gpu.txt (line 2)) (2019.4.14)\n",
            "Requirement already satisfied: h5py==2.9.0 in /usr/local/lib/python3.6/dist-packages (from -r requirements-gpu.txt (line 3)) (2.9.0)\n",
            "Requirement already satisfied: numpy==1.16.2 in /usr/local/lib/python3.6/dist-packages (from -r requirements-gpu.txt (line 4)) (1.16.2)\n",
            "Requirement already satisfied: tensorboard==1.13.1 in /usr/local/lib/python3.6/dist-packages (from -r requirements-gpu.txt (line 5)) (1.13.1)\n",
            "Requirement already satisfied: tensorflow-gpu==1.13.1 in /usr/local/lib/python3.6/dist-packages (from -r requirements-gpu.txt (line 6)) (1.13.1)\n",
            "Requirement already satisfied: tqdm==4.31.1 in /usr/local/lib/python3.6/dist-packages (from -r requirements-gpu.txt (line 7)) (4.31.1)\n",
            "Requirement already satisfied: requests==2.22.0 in /usr/local/lib/python3.6/dist-packages (from -r requirements-gpu.txt (line 8)) (2.22.0)\n",
            "Requirement already satisfied: pytz>=2011k in /usr/local/lib/python3.6/dist-packages (from pandas==0.24.2->-r requirements-gpu.txt (line 1)) (2018.9)\n",
            "Requirement already satisfied: python-dateutil>=2.5.0 in /usr/local/lib/python3.6/dist-packages (from pandas==0.24.2->-r requirements-gpu.txt (line 1)) (2.5.3)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from h5py==2.9.0->-r requirements-gpu.txt (line 3)) (1.12.0)\n",
            "Requirement already satisfied: absl-py>=0.4 in /usr/local/lib/python3.6/dist-packages (from tensorboard==1.13.1->-r requirements-gpu.txt (line 5)) (0.7.1)\n",
            "Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tensorboard==1.13.1->-r requirements-gpu.txt (line 5)) (3.1.1)\n",
            "Requirement already satisfied: protobuf>=3.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorboard==1.13.1->-r requirements-gpu.txt (line 5)) (3.7.1)\n",
            "Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.6/dist-packages (from tensorboard==1.13.1->-r requirements-gpu.txt (line 5)) (0.15.4)\n",
            "Requirement already satisfied: wheel>=0.26; python_version >= \"3\" in /usr/local/lib/python3.6/dist-packages (from tensorboard==1.13.1->-r requirements-gpu.txt (line 5)) (0.33.4)\n",
            "Requirement already satisfied: grpcio>=1.6.3 in /usr/local/lib/python3.6/dist-packages (from tensorboard==1.13.1->-r requirements-gpu.txt (line 5)) (1.15.0)\n",
            "Requirement already satisfied: gast>=0.2.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.13.1->-r requirements-gpu.txt (line 6)) (0.2.2)\n",
            "Requirement already satisfied: astor>=0.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.13.1->-r requirements-gpu.txt (line 6)) (0.8.0)\n",
            "Requirement already satisfied: keras-applications>=1.0.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.13.1->-r requirements-gpu.txt (line 6)) (1.0.8)\n",
            "Requirement already satisfied: tensorflow-estimator<1.14.0rc0,>=1.13.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.13.1->-r requirements-gpu.txt (line 6)) (1.13.0)\n",
            "Requirement already satisfied: keras-preprocessing>=1.0.5 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.13.1->-r requirements-gpu.txt (line 6)) (1.1.0)\n",
            "Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.13.1->-r requirements-gpu.txt (line 6)) (1.1.0)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests==2.22.0->-r requirements-gpu.txt (line 8)) (2019.3.9)\n",
            "Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests==2.22.0->-r requirements-gpu.txt (line 8)) (3.0.4)\n",
            "Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests==2.22.0->-r requirements-gpu.txt (line 8)) (2.8)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests==2.22.0->-r requirements-gpu.txt (line 8)) (1.24.3)\n",
            "Requirement already satisfied: setuptools>=36 in /usr/local/lib/python3.6/dist-packages (from markdown>=2.6.8->tensorboard==1.13.1->-r requirements-gpu.txt (line 5)) (41.0.1)\n",
            "Requirement already satisfied: mock>=2.0.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-estimator<1.14.0rc0,>=1.13.0->tensorflow-gpu==1.13.1->-r requirements-gpu.txt (line 6)) (3.0.5)\n",
            "Just downloaded large/model.ckpt.data-00000-of-00001!\n",
            "Just downloaded large/model.ckpt.index!\n",
            "Just downloaded large/model.ckpt.meta!\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "D4pvMY8dgOLc",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import json\n",
        "from datetime import datetime\n",
        "from textwrap import wrap\n",
        "\n",
        "title = 'An abominable snowman has broken into the whitehouse' #@param {type:\"string\"}\n",
        "domain = 'nytimes.com' #@param {type:\"string\"}\n",
        "text = 'An abominable snowman has broken into the whitehouse' #@param {type:\"string\"}\n",
        "summary = 'An abominable snowman has broken into the whitehouse' #@param {type:\"string\"}\n",
        "authors = '' #@param {type:\"string\"}\n",
        "publish_date = '2019-03-14' #@param {type:\"date\"}\n",
        "\n",
        "grover_publish_date = datetime.strptime(publish_date, \"%Y-%m-%d\").strftime(\"%m-%d-%Y\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2DkwzO-pgHA4",
        "colab_type": "code",
        "outputId": "e12b4170-cba7-44f4-c58a-5e32f179a68a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 615
        }
      },
      "source": [
        "input_json = {\n",
        "  'title': title,\n",
        "  'domain': domain,\n",
        "  'text': text,\n",
        "  'summary': summary,\n",
        "  'authors': authors,\n",
        "  'publish_date': grover_publish_date,\n",
        "}\n",
        "\n",
        "with open('./input.json', 'w') as f:\n",
        "  f.write(json.dumps(input_json))\n",
        "\n",
        "!PYTHONPATH=$(pwd) python sample/contextual_generate.py -model_config_fn lm/configs/large.json \\\n",
        "  -model_ckpt models/large/model.ckpt \\\n",
        "  -metadata_fn input.json \\\n",
        "  -out_fn output.json"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\n",
            "~~\n",
            "batch size=1, max batch size=16, num chunks=1, batch size per chunk=1\n",
            "~~\n",
            "\n",
            "2019-06-26 04:09:40.664866: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA\n",
            "2019-06-26 04:09:40.830334: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:998] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2019-06-26 04:09:40.830866: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x2335ce0 executing computations on platform CUDA. Devices:\n",
            "2019-06-26 04:09:40.830899: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): Tesla T4, Compute Capability 7.5\n",
            "2019-06-26 04:09:40.832615: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2300000000 Hz\n",
            "2019-06-26 04:09:40.832856: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x2335b80 executing computations on platform Host. Devices:\n",
            "2019-06-26 04:09:40.832885: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): <undefined>, <undefined>\n",
            "2019-06-26 04:09:40.833219: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1433] Found device 0 with properties: \n",
            "name: Tesla T4 major: 7 minor: 5 memoryClockRate(GHz): 1.59\n",
            "pciBusID: 0000:00:04.0\n",
            "totalMemory: 14.73GiB freeMemory: 14.60GiB\n",
            "2019-06-26 04:09:40.833245: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1512] Adding visible gpu devices: 0\n",
            "2019-06-26 04:09:40.833739: I tensorflow/core/common_runtime/gpu/gpu_device.cc:984] Device interconnect StreamExecutor with strength 1 edge matrix:\n",
            "2019-06-26 04:09:40.833760: I tensorflow/core/common_runtime/gpu/gpu_device.cc:990]      0 \n",
            "2019-06-26 04:09:40.833770: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1003] 0:   N \n",
            "2019-06-26 04:09:40.834020: W tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:42] Overriding allow_growth setting because the TF_FORCE_GPU_ALLOW_GROWTH environment variable is set. Original config value was 0.\n",
            "2019-06-26 04:09:40.834066: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14202 MB memory) -> physical GPU (device: 0, name: Tesla T4, pci bus id: 0000:00:04.0, compute capability: 7.5)\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Colocations handled automatically by placer.\n",
            "WARNING:tensorflow:From /content/grover/lm/modeling.py:143: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use keras.layers.dense instead.\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use standard file APIs to check for files with this prefix.\n",
            "  0% 0/1 [00:00<?, ?it/s]2019-06-26 04:09:51.356404: I tensorflow/stream_executor/dso_loader.cc:152] successfully opened CUDA library libcublas.so.10.0 locally\n",
            "Written 0/1 articles\n",
            "100% 1/1 [00:12<00:00, 12.26s/it]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "B72QLfjJhRJx",
        "colab_type": "code",
        "outputId": "fe338ed2-7372-41c0-8f3f-840edb5b5a18",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 629
        }
      },
      "source": [
        "WRAP_EVERY_N_CHARACTERS = 100\n",
        "\n",
        "with open('./output.json') as f:\n",
        "  file_contents = f.read()\n",
        "  data = json.loads(file_contents)\n",
        "\n",
        "def pretty_print_output(data):\n",
        "  for field in input_json.keys():\n",
        "    print(f\"{field}: {data[field]}\")\n",
        "\n",
        "  print('-' * 100 + '\\n')\n",
        "  generated_article = data['gens_article'][0]\n",
        "  wrapped_article = '\\n'.join(wrap(generated_article, WRAP_EVERY_N_CHARACTERS))\n",
        "  print(wrapped_article)\n",
        "\n",
        "pretty_print_output(data)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "title: An abominable snowman has broken into the whitehouse\n",
            "domain: nytimes.com\n",
            "text: An abominable snowman has broken into the whitehouse\n",
            "summary: An abominable snowman has broken into the whitehouse\n",
            "authors: \n",
            "publish_date: 03-14-2019\n",
            "----------------------------------------------------------------------------------------------------\n",
            "\n",
            "When a giant, and almost lifelike, snowman was discovered in the US Capitol on Wednesday, the\n",
            "groundskeepers were not amused. The statue was broken in half, which appeared to show “two separate,\n",
            "slight changes of the face,” as Larry Koefoed, the statue’s honorary curator, pointed out in a post\n",
            "on Instagram. “Much smaller changes did not alter the statute at all,” he wrote. Koefoed (who posted\n",
            "the photograph for Instagram, without offering explanation for what he believes may have been a\n",
            "technical problem with the snowman’s headpiece) wondered if some sort of meteor had made it fall\n",
            "into the White House, or something else. “The rain seems to have filled the snow up by mouthfuls as\n",
            "we drove down Pennsylvania Avenue,” he explained. Following the incident, the White House News\n",
            "Twitter account released the following statement to reporters on Thursday: White House equipment\n",
            "manager Darren Fay said his team had searched the grounds again, and found the snowman again. He\n",
            "said the administration was “not happy with what happened.” The costume must have gotten caught in a\n",
            "grate, he said. The authenticity of the sculpture, which has been at the White House since 1952, was\n",
            "not known. Adorers of the statue have appeared at the monument with the help of flying snowmen;\n",
            "however, “yesterday was the first time,” Creighton Fischer, another White House News photographer,\n",
            "said on Facebook. “They asked to meet and have some fun with it, too, to enjoy some baked treats and\n",
            "champagne.” Fischer said the base of the sculpture was damaged from the outside. The space inside\n",
            "the White House is mostly covered, except for a table with and a few chairs. “We were all kind of\n",
            "surprised to see that it’s come out in this shape,” Fischer said. The White House press gallery\n",
            "declined to comment on the incident on Thursday. During the evening when the scene was captured,\n",
            "visitors to the White House complex could be glimpsed listening to the crackling of speakers, which\n",
            "were played out over the monument to Dr. Martin Luther King Jr. “Martin Luther King walked on land,\n",
            "not air,” Fischer said. “His speech was well received.” Related Sound of Alice walking on a frozen\n",
            "pond inspired by new book German TV uses video of Alice in Wonderland as satire Check out the\n",
            "Simpsons High School for mathletes, who get an extra lift in Kasey, the character from “Simpsons”\n",
            "Read the full story at the Washington Post. Related Photo: Nearly one-hundred pounds of marble wall\n",
            "covering is covered with layers of snow New artwork by illusionist Jessie W. Evans captures the\n",
            "wonder of “The Not-Quite Grandniece of 9/11” Share this: Facebook Twitter Google WhatsApp Pinterest\n",
            "Reddit Tumblr More Skype LinkedIn Pocket Print Telegram\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}