Skip to content

Instantly share code, notes, and snippets.

@sagorbrur

sagorbrur/bert_medium_training_in_colab.ipynb

Created May 23, 2021 06:35
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save sagorbrur/0f09110415bf3f14e1be8cffa856ec83 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save sagorbrur/0f09110415bf3f14e1be8cffa856ec83 to your computer and use it in GitHub Desktop.
Download ZIP
bert_medium_training_in_colab.ipynb
Display the source blob
Display the rendered blob
Raw
bert_medium_training_in_colab.ipynb
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "bert_medium_training_in_colab.ipynb",
"provenance": [],
"collapsed_sections": [],
"authorship_tag": "ABX9TyM415WsRdrGHyuD51M3Y7Py",
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
},
"accelerator": "GPU"
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/sagorbrur/0f09110415bf3f14e1be8cffa856ec83/bert_medium_training_in_colab.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "wPxOFkty7dDl"
},
"source": [
"# Training BERT medium in google colab"
]
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "ZvZLbrSwtXMX",
"outputId": "1ac768a5-f23e-4309-b114-eb4f0454fb8d"
},
"source": [
"!pip install tensorflow-gpu==1.15.0"
],
"execution_count": 7,
"outputs": [
{
"output_type": "stream",
"text": [
"Collecting tensorflow-gpu==1.15.0\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/bc/72/d06017379ad4760dc58781c765376ce4ba5dcf3c08d37032eeefbccf1c51/tensorflow_gpu-1.15.0-cp37-cp37m-manylinux2010_x86_64.whl (411.5MB)\n",
"\u001b[K |████████████████████████████████| 411.5MB 39kB/s \n",
"\u001b[?25hRequirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu==1.15.0) (0.36.2)\n",
"Collecting gast==0.2.2\n",
" Downloading https://files.pythonhosted.org/packages/4e/35/11749bf99b2d4e3cceb4d55ca22590b0d7c2c62b9de38ac4a4a7f4687421/gast-0.2.2.tar.gz\n",
"Collecting tensorboard<1.16.0,>=1.15.0\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/1e/e9/d3d747a97f7188f48aa5eda486907f3b345cd409f0a0850468ba867db246/tensorboard-1.15.0-py3-none-any.whl (3.8MB)\n",
"\u001b[K |████████████████████████████████| 3.8MB 33.3MB/s \n",
"\u001b[?25hRequirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu==1.15.0) (1.1.0)\n",
"Collecting tensorflow-estimator==1.15.1\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/de/62/2ee9cd74c9fa2fa450877847ba560b260f5d0fb70ee0595203082dafcc9d/tensorflow_estimator-1.15.1-py2.py3-none-any.whl (503kB)\n",
"\u001b[K |████████████████████████████████| 512kB 47.6MB/s \n",
"\u001b[?25hCollecting keras-applications>=1.0.8\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/71/e3/19762fdfc62877ae9102edf6342d71b28fbfd9dea3d2f96a882ce099b03f/Keras_Applications-1.0.8-py3-none-any.whl (50kB)\n",
"\u001b[K |████████████████████████████████| 51kB 7.8MB/s \n",
"\u001b[?25hRequirement already satisfied: numpy<2.0,>=1.16.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu==1.15.0) (1.19.5)\n",
"Requirement already satisfied: wrapt>=1.11.1 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu==1.15.0) (1.12.1)\n",
"Requirement already satisfied: six>=1.10.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu==1.15.0) (1.15.0)\n",
"Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu==1.15.0) (1.32.0)\n",
"Requirement already satisfied: keras-preprocessing>=1.0.5 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu==1.15.0) (1.1.2)\n",
"Requirement already satisfied: absl-py>=0.7.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu==1.15.0) (0.12.0)\n",
"Requirement already satisfied: astor>=0.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu==1.15.0) (0.8.1)\n",
"Requirement already satisfied: protobuf>=3.6.1 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu==1.15.0) (3.12.4)\n",
"Requirement already satisfied: google-pasta>=0.1.6 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu==1.15.0) (0.2.0)\n",
"Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu==1.15.0) (3.3.0)\n",
"Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.7/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow-gpu==1.15.0) (3.3.4)\n",
"Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow-gpu==1.15.0) (56.1.0)\n",
"Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.7/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow-gpu==1.15.0) (2.0.0)\n",
"Requirement already satisfied: h5py in /usr/local/lib/python3.7/dist-packages (from keras-applications>=1.0.8->tensorflow-gpu==1.15.0) (2.10.0)\n",
"Requirement already satisfied: importlib-metadata; python_version < \"3.8\" in /usr/local/lib/python3.7/dist-packages (from markdown>=2.6.8->tensorboard<1.16.0,>=1.15.0->tensorflow-gpu==1.15.0) (4.0.1)\n",
"Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata; python_version < \"3.8\"->markdown>=2.6.8->tensorboard<1.16.0,>=1.15.0->tensorflow-gpu==1.15.0) (3.4.1)\n",
"Requirement already satisfied: typing-extensions>=3.6.4; python_version < \"3.8\" in /usr/local/lib/python3.7/dist-packages (from importlib-metadata; python_version < \"3.8\"->markdown>=2.6.8->tensorboard<1.16.0,>=1.15.0->tensorflow-gpu==1.15.0) (3.7.4.3)\n",
"Building wheels for collected packages: gast\n",
" Building wheel for gast (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
" Created wheel for gast: filename=gast-0.2.2-cp37-none-any.whl size=7540 sha256=03bd865c198f303f3f6e79af2c9cf20cddddcea56fae3e5f82933ffc4416eca5\n",
" Stored in directory: /root/.cache/pip/wheels/5c/2e/7e/a1d4d4fcebe6c381f378ce7743a3ced3699feb89bcfbdadadd\n",
"Successfully built gast\n",
"\u001b[31mERROR: tensorflow 2.4.1 has requirement gast==0.3.3, but you'll have gast 0.2.2 which is incompatible.\u001b[0m\n",
"\u001b[31mERROR: tensorflow 2.4.1 has requirement tensorboard~=2.4, but you'll have tensorboard 1.15.0 which is incompatible.\u001b[0m\n",
"\u001b[31mERROR: tensorflow 2.4.1 has requirement tensorflow-estimator<2.5.0,>=2.4.0, but you'll have tensorflow-estimator 1.15.1 which is incompatible.\u001b[0m\n",
"\u001b[31mERROR: tensorflow-probability 0.12.1 has requirement gast>=0.3.2, but you'll have gast 0.2.2 which is incompatible.\u001b[0m\n",
"Installing collected packages: gast, tensorboard, tensorflow-estimator, keras-applications, tensorflow-gpu\n",
" Found existing installation: gast 0.3.3\n",
" Uninstalling gast-0.3.3:\n",
" Successfully uninstalled gast-0.3.3\n",
" Found existing installation: tensorboard 2.4.1\n",
" Uninstalling tensorboard-2.4.1:\n",
" Successfully uninstalled tensorboard-2.4.1\n",
" Found existing installation: tensorflow-estimator 2.4.0\n",
" Uninstalling tensorflow-estimator-2.4.0:\n",
" Successfully uninstalled tensorflow-estimator-2.4.0\n",
"Successfully installed gast-0.2.2 keras-applications-1.0.8 tensorboard-1.15.0 tensorflow-estimator-1.15.1 tensorflow-gpu-1.15.0\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "iYucJwPXsMkL",
"outputId": "1b7ba031-f926-4352-fdb3-ef28486d7349"
},
"source": [
"!git clone https://github.com/google-research/bert.git\n",
"%cd bert"
],
"execution_count": 21,
"outputs": [
{
"output_type": "stream",
"text": [
"Cloning into 'bert'...\n",
"remote: Enumerating objects: 340, done.\u001b[K\n",
"remote: Total 340 (delta 0), reused 0 (delta 0), pack-reused 340\u001b[K\n",
"Receiving objects: 100% (340/340), 328.28 KiB | 13.13 MiB/s, done.\n",
"Resolving deltas: 100% (182/182), done.\n",
"/content/bert\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "OPZoazx-7uCS"
},
"source": [
"## Download Pre-trained Model\n",
"Download bert pre-trained medium model and keep in base-model"
]
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "5zThZIb6s1bv",
"outputId": "f6ce99d9-1ab6-48a1-d951-a3a2d5064b7c"
},
"source": [
"!mkdir base-model\n",
"%cd base-model\n",
"!wget https://storage.googleapis.com/bert_models/2020_02_20/uncased_L-8_H-512_A-8.zip\n",
"!unzip uncased_L-8_H-512_A-8.zip\n",
"!rm -rf uncased_L-8_H-512_A-8.zip\n",
"%cd .."
],
"execution_count": 12,
"outputs": [
{
"output_type": "stream",
"text": [
"/content/bert/base-model\n",
"--2021-05-23 05:45:01-- https://storage.googleapis.com/bert_models/2020_02_20/uncased_L-8_H-512_A-8.zip\n",
"Resolving storage.googleapis.com (storage.googleapis.com)... 173.194.202.128, 74.125.142.128, 74.125.195.128, ...\n",
"Connecting to storage.googleapis.com (storage.googleapis.com)|173.194.202.128|:443... connected.\n",
"HTTP request sent, awaiting response... 200 OK\n",
"Length: 154608092 (147M) [application/zip]\n",
"Saving to: ‘uncased_L-8_H-512_A-8.zip’\n",
"\n",
"uncased_L-8_H-512_A 100%[===================>] 147.45M 60.0MB/s in 2.5s \n",
"\n",
"2021-05-23 05:45:03 (60.0 MB/s) - ‘uncased_L-8_H-512_A-8.zip’ saved [154608092/154608092]\n",
"\n",
"Archive: uncased_L-8_H-512_A-8.zip\n",
" inflating: bert_model.ckpt.data-00000-of-00001 \n",
" inflating: bert_config.json \n",
" inflating: vocab.txt \n",
" inflating: bert_model.ckpt.index \n",
"/content/bert\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "_nI-Ze97sW8S",
"outputId": "6fc025eb-a7c3-4e55-8b7c-aa2d48826702"
},
"source": [
"%%writefile create_pretraining_data.sh\n",
"python create_pretraining_data.py \\\n",
" --input_file=./sample_text.txt \\\n",
" --output_file=./tf_examples.tfrecord \\\n",
" --vocab_file=./base-model/vocab.txt \\\n",
" --do_lower_case=True \\\n",
" --max_seq_length=128 \\\n",
" --max_predictions_per_seq=20 \\\n",
" --masked_lm_prob=0.15 \\\n",
" --random_seed=12345 \\\n",
" --dupe_factor=5"
],
"execution_count": 4,
"outputs": [
{
"output_type": "stream",
"text": [
"Writing create_pretraining_data.sh\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "ek862pxltRcC",
"outputId": "289f8f1b-7730-4478-f6a9-c7d9c2759b1f"
},
"source": [
"!bash create_pretraining_data.sh"
],
"execution_count": 8,
"outputs": [
{
"output_type": "stream",
"text": [
"WARNING:tensorflow:From create_pretraining_data.py:469: The name tf.app.run is deprecated. Please use tf.compat.v1.app.run instead.\n",
"\n",
"WARNING:tensorflow:From create_pretraining_data.py:437: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.\n",
"\n",
"W0523 05:34:34.419707 140368108218240 module_wrapper.py:139] From create_pretraining_data.py:437: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.\n",
"\n",
"WARNING:tensorflow:From create_pretraining_data.py:437: The name tf.logging.INFO is deprecated. Please use tf.compat.v1.logging.INFO instead.\n",
"\n",
"W0523 05:34:34.419909 140368108218240 module_wrapper.py:139] From create_pretraining_data.py:437: The name tf.logging.INFO is deprecated. Please use tf.compat.v1.logging.INFO instead.\n",
"\n",
"WARNING:tensorflow:From /content/bert/tokenization.py:125: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.\n",
"\n",
"W0523 05:34:34.420102 140368108218240 module_wrapper.py:139] From /content/bert/tokenization.py:125: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.\n",
"\n",
"WARNING:tensorflow:From create_pretraining_data.py:444: The name tf.gfile.Glob is deprecated. Please use tf.io.gfile.glob instead.\n",
"\n",
"W0523 05:34:34.501689 140368108218240 module_wrapper.py:139] From create_pretraining_data.py:444: The name tf.gfile.Glob is deprecated. Please use tf.io.gfile.glob instead.\n",
"\n",
"WARNING:tensorflow:From create_pretraining_data.py:446: The name tf.logging.info is deprecated. Please use tf.compat.v1.logging.info instead.\n",
"\n",
"W0523 05:34:34.502740 140368108218240 module_wrapper.py:139] From create_pretraining_data.py:446: The name tf.logging.info is deprecated. Please use tf.compat.v1.logging.info instead.\n",
"\n",
"INFO:tensorflow:*** Reading from input files ***\n",
"I0523 05:34:34.502893 140368108218240 create_pretraining_data.py:446] *** Reading from input files ***\n",
"INFO:tensorflow: ./sample_text.txt\n",
"I0523 05:34:34.502995 140368108218240 create_pretraining_data.py:448] ./sample_text.txt\n",
"INFO:tensorflow:*** Writing to output files ***\n",
"I0523 05:34:34.541840 140368108218240 create_pretraining_data.py:457] *** Writing to output files ***\n",
"INFO:tensorflow: ./tf_examples.tfrecord\n",
"I0523 05:34:34.541998 140368108218240 create_pretraining_data.py:459] ./tf_examples.tfrecord\n",
"WARNING:tensorflow:From create_pretraining_data.py:101: The name tf.python_io.TFRecordWriter is deprecated. Please use tf.io.TFRecordWriter instead.\n",
"\n",
"W0523 05:34:34.542173 140368108218240 module_wrapper.py:139] From create_pretraining_data.py:101: The name tf.python_io.TFRecordWriter is deprecated. Please use tf.io.TFRecordWriter instead.\n",
"\n",
"INFO:tensorflow:*** Example ***\n",
"I0523 05:34:34.542732 140368108218240 create_pretraining_data.py:149] *** Example ***\n",
"INFO:tensorflow:tokens: [CLS] for more than a [MASK] up [MASK] great main street , [MASK] [MASK] [MASK] centre of the city , at right angles , [MASK] one equally magnificent , at each end ##ミ which , miles away , appeared , dim and distant over the heads of the living stream of passengers , the yellow [MASK] - hills of [MASK] [MASK] ; while at the end of the vista in front [MASK] them gleamed the blue harbour , through a network [SEP] possibly this may have been the reason why early rise ##rs in [MASK] locality , during the rainy season , adopted [MASK] [MASK] [MASK] of body , and seldom lifted [MASK] eyes [MASK] the rift [MASK] or india - ink washed skies above them . [SEP]\n",
"I0523 05:34:34.542895 140368108218240 create_pretraining_data.py:151] tokens: [CLS] for more than a [MASK] up [MASK] great main street , [MASK] [MASK] [MASK] centre of the city , at right angles , [MASK] one equally magnificent , at each end ##ミ which , miles away , appeared , dim and distant over the heads of the living stream of passengers , the yellow [MASK] - hills of [MASK] [MASK] ; while at the end of the vista in front [MASK] them gleamed the blue harbour , through a network [SEP] possibly this may have been the reason why early rise ##rs in [MASK] locality , during the rainy season , adopted [MASK] [MASK] [MASK] of body , and seldom lifted [MASK] eyes [MASK] the rift [MASK] or india - ink washed skies above them . [SEP]\n",
"INFO:tensorflow:input_ids: 101 2005 2062 2084 1037 103 2039 103 2307 2364 2395 1010 103 103 103 2803 1997 1996 2103 1010 2012 2157 12113 1010 103 2028 8053 12047 1010 2012 2169 2203 30250 2029 1010 2661 2185 1010 2596 1010 11737 1998 6802 2058 1996 4641 1997 1996 2542 5460 1997 5467 1010 1996 3756 103 1011 4564 1997 103 103 1025 2096 2012 1996 2203 1997 1996 13005 1999 2392 103 2068 25224 1996 2630 7440 1010 2083 1037 2897 102 4298 2023 2089 2031 2042 1996 3114 2339 2220 4125 2869 1999 103 10246 1010 2076 1996 16373 2161 1010 4233 103 103 103 1997 2303 1010 1998 15839 4196 103 2159 103 1996 16931 103 2030 2634 1011 10710 8871 15717 2682 2068 1012 102\n",
"I0523 05:34:34.543077 140368108218240 create_pretraining_data.py:161] input_ids: 101 2005 2062 2084 1037 103 2039 103 2307 2364 2395 1010 103 103 103 2803 1997 1996 2103 1010 2012 2157 12113 1010 103 2028 8053 12047 1010 2012 2169 2203 30250 2029 1010 2661 2185 1010 2596 1010 11737 1998 6802 2058 1996 4641 1997 1996 2542 5460 1997 5467 1010 1996 3756 103 1011 4564 1997 103 103 1025 2096 2012 1996 2203 1997 1996 13005 1999 2392 103 2068 25224 1996 2630 7440 1010 2083 1037 2897 102 4298 2023 2089 2031 2042 1996 3114 2339 2220 4125 2869 1999 103 10246 1010 2076 1996 16373 2161 1010 4233 103 103 103 1997 2303 1010 1998 15839 4196 103 2159 103 1996 16931 103 2030 2634 1011 10710 8871 15717 2682 2068 1012 102\n",
"INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"I0523 05:34:34.543220 140368108218240 create_pretraining_data.py:161] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"I0523 05:34:34.543349 140368108218240 create_pretraining_data.py:161] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"INFO:tensorflow:masked_lm_positions: 5 7 12 13 14 24 32 55 59 60 71 94 103 104 105 109 112 114 117 0\n",
"I0523 05:34:34.543435 140368108218240 create_pretraining_data.py:161] masked_lm_positions: 5 7 12 13 14 24 32 55 59 60 71 94 103 104 105 109 112 114 117 0\n",
"INFO:tensorflow:masked_lm_ids: 3542 1996 4625 1999 1996 2011 1997 5472 1996 5532 1997 2008 1037 16465 10427 1998 2037 2000 2098 0\n",
"I0523 05:34:34.543519 140368108218240 create_pretraining_data.py:161] masked_lm_ids: 3542 1996 4625 1999 1996 2011 1997 5472 1996 5532 1997 2008 1037 16465 10427 1998 2037 2000 2098 0\n",
"INFO:tensorflow:masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0\n",
"I0523 05:34:34.543606 140368108218240 create_pretraining_data.py:161] masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0\n",
"INFO:tensorflow:next_sentence_labels: 1\n",
"I0523 05:34:34.543679 140368108218240 create_pretraining_data.py:161] next_sentence_labels: 1\n",
"INFO:tensorflow:*** Example ***\n",
"I0523 05:34:34.544020 140368108218240 create_pretraining_data.py:149] *** Example ***\n",
"INFO:tensorflow:tokens: [CLS] this text is included to make sure unicode is handled bracelet : 力 加 [MASK] 北 区 ᴵ ##ᴺ ##ᵀ ##ᵃ ##ছ [MASK] ##ট ##ড ##ণ ##ত [SEP] text should be one - [MASK] - per [MASK] line , with empty lines [MASK] documents . [SEP]\n",
"I0523 05:34:34.544143 140368108218240 create_pretraining_data.py:151] tokens: [CLS] this text is included to make sure unicode is handled bracelet : 力 加 [MASK] 北 区 ᴵ ##ᴺ ##ᵀ ##ᵃ ##ছ [MASK] ##ট ##ড ##ণ ##ত [SEP] text should be one - [MASK] - per [MASK] line , with empty lines [MASK] documents . [SEP]\n",
"INFO:tensorflow:input_ids: 101 2023 3793 2003 2443 2000 2191 2469 27260 2003 8971 19688 1024 1778 1779 103 1781 1782 1493 30030 30031 30032 29893 103 29895 29896 29897 29898 102 3793 2323 2022 2028 1011 103 1011 2566 103 2240 1010 2007 4064 3210 103 5491 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.544294 140368108218240 create_pretraining_data.py:161] input_ids: 101 2023 3793 2003 2443 2000 2191 2469 27260 2003 8971 19688 1024 1778 1779 103 1781 1782 1493 30030 30031 30032 29893 103 29895 29896 29897 29898 102 3793 2323 2022 2028 1011 103 1011 2566 103 2240 1010 2007 4064 3210 103 5491 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.544424 140368108218240 create_pretraining_data.py:161] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.544554 140368108218240 create_pretraining_data.py:161] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:masked_lm_positions: 11 15 23 34 37 38 43 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.544641 140368108218240 create_pretraining_data.py:161] masked_lm_positions: 11 15 23 34 37 38 43 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:masked_lm_ids: 7919 1780 29894 6251 1011 2240 2090 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.544726 140368108218240 create_pretraining_data.py:161] masked_lm_ids: 7919 1780 29894 6251 1011 2240 2090 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0\n",
"I0523 05:34:34.544852 140368108218240 create_pretraining_data.py:161] masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0\n",
"INFO:tensorflow:next_sentence_labels: 0\n",
"I0523 05:34:34.544995 140368108218240 create_pretraining_data.py:161] next_sentence_labels: 0\n",
"INFO:tensorflow:*** Example ***\n",
"I0523 05:34:34.545458 140368108218240 create_pretraining_data.py:149] *** Example ***\n",
"INFO:tensorflow:tokens: [CLS] [MASK] from his wood pile [MASK] to [MASK] ##le a fire to dry his bed - [MASK] , and he had rec ##ours [MASK] [MASK] [MASK] more provide ##nt neighbor ' s to supply the deficiency . this was nearly opposite . [MASK] . cass [MASK] crossed the highway , and stopped suddenly . something glitter ##ed in the nearest red pool before [SEP] [MASK] [MASK] to relate , not an irregular , shape ##less fragment of crude ore , fresh from nature ' s cr ##ucible , but a [MASK] of jewel ##er ' [MASK] [MASK] ##ic ##raf ##t in the [MASK] of a [MASK] gold ring . looking at it more at ##ten ##tively , he saw that it bore the inscription , [MASK] [SEP]\n",
"I0523 05:34:34.545658 140368108218240 create_pretraining_data.py:151] tokens: [CLS] [MASK] from his wood pile [MASK] to [MASK] ##le a fire to dry his bed - [MASK] , and he had rec ##ours [MASK] [MASK] [MASK] more provide ##nt neighbor ' s to supply the deficiency . this was nearly opposite . [MASK] . cass [MASK] crossed the highway , and stopped suddenly . something glitter ##ed in the nearest red pool before [SEP] [MASK] [MASK] to relate , not an irregular , shape ##less fragment of crude ore , fresh from nature ' s cr ##ucible , but a [MASK] of jewel ##er ' [MASK] [MASK] ##ic ##raf ##t in the [MASK] of a [MASK] gold ring . looking at it more at ##ten ##tively , he saw that it bore the inscription , [MASK] [SEP]\n",
"INFO:tensorflow:input_ids: 101 103 2013 2010 3536 8632 103 2000 103 2571 1037 2543 2000 4318 2010 2793 1011 103 1010 1998 2002 2018 28667 22957 103 103 103 2062 3073 3372 11429 1005 1055 2000 4425 1996 18888 1012 2023 2001 3053 4500 1012 103 1012 16220 103 4625 1996 3307 1010 1998 3030 3402 1012 2242 27566 2098 1999 1996 7205 2417 4770 2077 102 103 103 2000 14396 1010 2025 2019 12052 1010 4338 3238 15778 1997 13587 10848 1010 4840 2013 3267 1005 1055 13675 21104 1010 2021 1037 103 1997 13713 2121 1005 103 103 2594 27528 2102 1999 1996 103 1997 1037 103 2751 3614 1012 2559 2012 2009 2062 2012 6528 25499 1010 2002 2387 2008 2009 8501 1996 9315 1010 103 102\n",
"I0523 05:34:34.545859 140368108218240 create_pretraining_data.py:161] input_ids: 101 103 2013 2010 3536 8632 103 2000 103 2571 1037 2543 2000 4318 2010 2793 1011 103 1010 1998 2002 2018 28667 22957 103 103 103 2062 3073 3372 11429 1005 1055 2000 4425 1996 18888 1012 2023 2001 3053 4500 1012 103 1012 16220 103 4625 1996 3307 1010 1998 3030 3402 1012 2242 27566 2098 1999 1996 7205 2417 4770 2077 102 103 103 2000 14396 1010 2025 2019 12052 1010 4338 3238 15778 1997 13587 10848 1010 4840 2013 3267 1005 1055 13675 21104 1010 2021 1037 103 1997 13713 2121 1005 103 103 2594 27528 2102 1999 1996 103 1997 1037 103 2751 3614 1012 2559 2012 2009 2062 2012 6528 25499 1010 2002 2387 2008 2009 8501 1996 9315 1010 103 102\n",
"INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"I0523 05:34:34.546028 140368108218240 create_pretraining_data.py:161] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"I0523 05:34:34.546194 140368108218240 create_pretraining_data.py:161] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"INFO:tensorflow:masked_lm_positions: 1 6 8 17 18 24 25 26 43 46 65 66 79 91 96 97 103 106 126 0\n",
"I0523 05:34:34.546294 140368108218240 create_pretraining_data.py:161] masked_lm_positions: 1 6 8 17 18 24 25 26 43 46 65 66 79 91 96 97 103 106 126 0\n",
"INFO:tensorflow:masked_lm_ids: 11772 4188 2785 4253 1010 2063 2000 1037 2720 4173 1010 6919 10848 2978 1055 2192 2433 5810 1000 0\n",
"I0523 05:34:34.546397 140368108218240 create_pretraining_data.py:161] masked_lm_ids: 11772 4188 2785 4253 1010 2063 2000 1037 2720 4173 1010 6919 10848 2978 1055 2192 2433 5810 1000 0\n",
"INFO:tensorflow:masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0\n",
"I0523 05:34:34.546517 140368108218240 create_pretraining_data.py:161] masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0\n",
"INFO:tensorflow:next_sentence_labels: 0\n",
"I0523 05:34:34.546621 140368108218240 create_pretraining_data.py:161] next_sentence_labels: 0\n",
"INFO:tensorflow:*** Example ***\n",
"I0523 05:34:34.547033 140368108218240 create_pretraining_data.py:149] *** Example ***\n",
"INFO:tensorflow:tokens: [CLS] there [MASK] a phil vita ##phic pleasure in [MASK] one ' [MASK] treasures to the modest [MASK] . [SEP] looking [MASK] it more at ##ten ##tively , he saw that it [MASK] the inscription , \" may to cass . \" like most of his fellow gold [MASK] seekers [MASK] cass was super ##sti ##tious . [SEP]\n",
"I0523 05:34:34.547195 140368108218240 create_pretraining_data.py:151] tokens: [CLS] there [MASK] a phil vita ##phic pleasure in [MASK] one ' [MASK] treasures to the modest [MASK] . [SEP] looking [MASK] it more at ##ten ##tively , he saw that it [MASK] the inscription , \" may to cass . \" like most of his fellow gold [MASK] seekers [MASK] cass was super ##sti ##tious . [SEP]\n",
"INFO:tensorflow:input_ids: 101 2045 103 1037 6316 19300 17926 5165 1999 103 2028 1005 103 17605 2000 1996 10754 103 1012 102 2559 103 2009 2062 2012 6528 25499 1010 2002 2387 2008 2009 103 1996 9315 1010 1000 2089 2000 16220 1012 1000 2066 2087 1997 2010 3507 2751 103 24071 103 16220 2001 3565 16643 20771 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.547368 140368108218240 create_pretraining_data.py:161] input_ids: 101 2045 103 1037 6316 19300 17926 5165 1999 103 2028 1005 103 17605 2000 1996 10754 103 1012 102 2559 103 2009 2062 2012 6528 25499 1010 2002 2387 2008 2009 103 1996 9315 1010 1000 2089 2000 16220 1012 1000 2066 2087 1997 2010 3507 2751 103 24071 103 16220 2001 3565 16643 20771 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.547533 140368108218240 create_pretraining_data.py:161] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.547695 140368108218240 create_pretraining_data.py:161] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:masked_lm_positions: 2 5 9 12 17 21 32 48 50 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.616877 140368108218240 create_pretraining_data.py:161] masked_lm_positions: 2 5 9 12 17 21 32 48 50 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:masked_lm_ids: 2003 19137 3098 1055 2402 2012 8501 1011 1010 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.617056 140368108218240 create_pretraining_data.py:161] masked_lm_ids: 2003 19137 3098 1055 2402 2012 8501 1011 1010 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0\n",
"I0523 05:34:34.617189 140368108218240 create_pretraining_data.py:161] masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0\n",
"INFO:tensorflow:next_sentence_labels: 1\n",
"I0523 05:34:34.617282 140368108218240 create_pretraining_data.py:161] next_sentence_labels: 1\n",
"INFO:tensorflow:*** Example ***\n",
"I0523 05:34:34.617748 140368108218240 create_pretraining_data.py:149] *** Example ***\n",
"INFO:tensorflow:tokens: [CLS] at last they reached the [MASK] at taxi opposite end of the street ; [SEP] text should be one - sentence - per - line , with empty lines between documents . this sample text is public domain and [MASK] randomly selected [MASK] project [MASK] ##tenberg . [SEP]\n",
"I0523 05:34:34.617940 140368108218240 create_pretraining_data.py:151] tokens: [CLS] at last they reached the [MASK] at taxi opposite end of the street ; [SEP] text should be one - sentence - per - line , with empty lines between documents . this sample text is public domain and [MASK] randomly selected [MASK] project [MASK] ##tenberg . [SEP]\n",
"INFO:tensorflow:input_ids: 101 2012 2197 2027 2584 1996 103 2012 10095 4500 2203 1997 1996 2395 1025 102 3793 2323 2022 2028 1011 6251 1011 2566 1011 2240 1010 2007 4064 3210 2090 5491 1012 2023 7099 3793 2003 2270 5884 1998 103 18154 3479 103 2622 103 21806 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.618134 140368108218240 create_pretraining_data.py:161] input_ids: 101 2012 2197 2027 2584 1996 103 2012 10095 4500 2203 1997 1996 2395 1025 102 3793 2323 2022 2028 1011 6251 1011 2566 1011 2240 1010 2007 4064 3210 2090 5491 1012 2023 7099 3793 2003 2270 5884 1998 103 18154 3479 103 2622 103 21806 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.618321 140368108218240 create_pretraining_data.py:161] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.618508 140368108218240 create_pretraining_data.py:161] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:masked_lm_positions: 6 8 13 38 40 43 45 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.618645 140368108218240 create_pretraining_data.py:161] masked_lm_positions: 6 8 13 38 40 43 45 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:masked_lm_ids: 21048 1996 2395 5884 2001 2013 9535 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.618760 140368108218240 create_pretraining_data.py:161] masked_lm_ids: 21048 1996 2395 5884 2001 2013 9535 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0\n",
"I0523 05:34:34.618887 140368108218240 create_pretraining_data.py:161] masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0\n",
"INFO:tensorflow:next_sentence_labels: 1\n",
"I0523 05:34:34.619001 140368108218240 create_pretraining_data.py:161] next_sentence_labels: 1\n",
"INFO:tensorflow:*** Example ***\n",
"I0523 05:34:34.619477 140368108218240 create_pretraining_data.py:149] *** Example ***\n",
"INFO:tensorflow:tokens: [CLS] text [MASK] ##stered one - sentence - per - [MASK] , with empty [MASK] between documents . [SEP] this sample text [MASK] public domain and was randomly selected from project gut ##tenberg . [SEP]\n",
"I0523 05:34:34.619623 140368108218240 create_pretraining_data.py:151] tokens: [CLS] text [MASK] ##stered one - sentence - per - [MASK] , with empty [MASK] between documents . [SEP] this sample text [MASK] public domain and was randomly selected from project gut ##tenberg . [SEP]\n",
"INFO:tensorflow:input_ids: 101 3793 103 24167 2028 1011 6251 1011 2566 1011 103 1010 2007 4064 103 2090 5491 1012 102 2023 7099 3793 103 2270 5884 1998 2001 18154 3479 2013 2622 9535 21806 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.619786 140368108218240 create_pretraining_data.py:161] input_ids: 101 3793 103 24167 2028 1011 6251 1011 2566 1011 103 1010 2007 4064 103 2090 5491 1012 102 2023 7099 3793 103 2270 5884 1998 2001 18154 3479 2013 2622 9535 21806 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.619928 140368108218240 create_pretraining_data.py:161] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.620057 140368108218240 create_pretraining_data.py:161] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:masked_lm_positions: 2 3 10 14 22 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.620144 140368108218240 create_pretraining_data.py:161] masked_lm_positions: 2 3 10 14 22 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:masked_lm_ids: 2323 2022 2240 3210 2003 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.620236 140368108218240 create_pretraining_data.py:161] masked_lm_ids: 2323 2022 2240 3210 2003 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0\n",
"I0523 05:34:34.620323 140368108218240 create_pretraining_data.py:161] masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0\n",
"INFO:tensorflow:next_sentence_labels: 0\n",
"I0523 05:34:34.620396 140368108218240 create_pretraining_data.py:161] next_sentence_labels: 0\n",
"INFO:tensorflow:*** Example ***\n",
"I0523 05:34:34.620737 140368108218240 create_pretraining_data.py:149] *** Example ***\n",
"INFO:tensorflow:tokens: [CLS] the rain had only ceased [MASK] the gray streaks of morning [MASK] blazing star , and the settlement [MASK] to [MASK] moral [MASK] of clean ##liness , and the finding of forgotten knives , tin [MASK] [MASK] and smaller camp ut ##ens ##ils , where [MASK] heavy showers [MASK] washed ᵘ jonah debris and dust heap ##s [MASK] the cabin [MASK] [MASK] [SEP] [MASK] had once picked up on the highway a solid chunk of gold quartz which the rain subordinate freed from its inc ##umber ##ing soil , and washed into immediate and glittering popularity . possibly this may have been the [MASK] why early rise ##rs in [MASK] locality , during the rainy season , adopted a thoughtful habit [MASK] body , and seldom [SEP]\n",
"I0523 05:34:34.620894 140368108218240 create_pretraining_data.py:151] tokens: [CLS] the rain had only ceased [MASK] the gray streaks of morning [MASK] blazing star , and the settlement [MASK] to [MASK] moral [MASK] of clean ##liness , and the finding of forgotten knives , tin [MASK] [MASK] and smaller camp ut ##ens ##ils , where [MASK] heavy showers [MASK] washed ᵘ jonah debris and dust heap ##s [MASK] the cabin [MASK] [MASK] [SEP] [MASK] had once picked up on the highway a solid chunk of gold quartz which the rain subordinate freed from its inc ##umber ##ing soil , and washed into immediate and glittering popularity . possibly this may have been the [MASK] why early rise ##rs in [MASK] locality , during the rainy season , adopted a thoughtful habit [MASK] body , and seldom [SEP]\n",
"INFO:tensorflow:input_ids: 101 1996 4542 2018 2069 7024 103 1996 3897 21295 1997 2851 103 17162 2732 1010 1998 1996 4093 103 2000 103 7191 103 1997 4550 20942 1010 1998 1996 4531 1997 6404 13227 1010 9543 103 103 1998 3760 3409 21183 6132 12146 1010 2073 103 3082 23442 103 8871 1506 15617 11385 1998 6497 16721 2015 103 1996 6644 103 103 102 103 2018 2320 3856 2039 2006 1996 3307 1037 5024 20000 1997 2751 20971 2029 1996 4542 15144 10650 2013 2049 4297 29440 2075 5800 1010 1998 8871 2046 6234 1998 20332 6217 1012 4298 2023 2089 2031 2042 1996 103 2339 2220 4125 2869 1999 103 10246 1010 2076 1996 16373 2161 1010 4233 1037 16465 10427 103 2303 1010 1998 15839 102\n",
"I0523 05:34:34.621055 140368108218240 create_pretraining_data.py:161] input_ids: 101 1996 4542 2018 2069 7024 103 1996 3897 21295 1997 2851 103 17162 2732 1010 1998 1996 4093 103 2000 103 7191 103 1997 4550 20942 1010 1998 1996 4531 1997 6404 13227 1010 9543 103 103 1998 3760 3409 21183 6132 12146 1010 2073 103 3082 23442 103 8871 1506 15617 11385 1998 6497 16721 2015 103 1996 6644 103 103 102 103 2018 2320 3856 2039 2006 1996 3307 1037 5024 20000 1997 2751 20971 2029 1996 4542 15144 10650 2013 2049 4297 29440 2075 5800 1010 1998 8871 2046 6234 1998 20332 6217 1012 4298 2023 2089 2031 2042 1996 103 2339 2220 4125 2869 1999 103 10246 1010 2076 1996 16373 2161 1010 4233 1037 16465 10427 103 2303 1010 1998 15839 102\n",
"INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"I0523 05:34:34.621202 140368108218240 create_pretraining_data.py:161] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"I0523 05:34:34.621345 140368108218240 create_pretraining_data.py:161] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"INFO:tensorflow:masked_lm_positions: 6 12 19 21 23 36 37 46 49 51 52 58 61 62 64 81 104 110 122 0\n",
"I0523 05:34:34.621439 140368108218240 create_pretraining_data.py:161] masked_lm_positions: 6 12 19 21 23 36 37 46 49 51 52 58 61 62 64 81 104 110 122 0\n",
"INFO:tensorflow:masked_lm_ids: 2007 2012 19179 1037 3168 10268 1010 1996 2018 2185 1996 2077 4303 1012 2099 2018 3114 2008 1997 0\n",
"I0523 05:34:34.621540 140368108218240 create_pretraining_data.py:161] masked_lm_ids: 2007 2012 19179 1037 3168 10268 1010 1996 2018 2185 1996 2077 4303 1012 2099 2018 3114 2008 1997 0\n",
"INFO:tensorflow:masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0\n",
"I0523 05:34:34.621637 140368108218240 create_pretraining_data.py:161] masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0\n",
"INFO:tensorflow:next_sentence_labels: 0\n",
"I0523 05:34:34.621735 140368108218240 create_pretraining_data.py:161] next_sentence_labels: 0\n",
"INFO:tensorflow:*** Example ***\n",
"I0523 05:34:34.622056 140368108218240 create_pretraining_data.py:149] *** Example ***\n",
"INFO:tensorflow:tokens: [CLS] as the ancient sage - - the name is un ##im ##port ##ant to [MASK] [MASK] - - [MASK] [MASK] nightly [MASK] he might study by day , so i , the guardian of cloak [MASK] [MASK] [MASK] ##sol ##s [MASK] at the sacred doors of her lecture - [MASK] , im ##bib ##e celestial knowledge . [MASK] [MASK] youth i felt in me a soul above the matter - en ##tangled herd . she revealed to me the glorious [SEP] possibly this may have been the [MASK] [MASK] early rise ##rs in that locality , during the rainy [MASK] , adopted a thoughtful habit of body , and [MASK] lifted their eyes to the rift ##ed or india - ink washed skies above them . [SEP]\n",
"I0523 05:34:34.622200 140368108218240 create_pretraining_data.py:151] tokens: [CLS] as the ancient sage - - the name is un ##im ##port ##ant to [MASK] [MASK] - - [MASK] [MASK] nightly [MASK] he might study by day , so i , the guardian of cloak [MASK] [MASK] [MASK] ##sol ##s [MASK] at the sacred doors of her lecture - [MASK] , im ##bib ##e celestial knowledge . [MASK] [MASK] youth i felt in me a soul above the matter - en ##tangled herd . she revealed to me the glorious [SEP] possibly this may have been the [MASK] [MASK] early rise ##rs in that locality , during the rainy [MASK] , adopted a thoughtful habit of body , and [MASK] lifted their eyes to the rift ##ed or india - ink washed skies above them . [SEP]\n",
"INFO:tensorflow:input_ids: 101 2004 1996 3418 10878 1011 1011 1996 2171 2003 4895 5714 6442 4630 2000 103 103 1011 1011 103 103 22390 103 2002 2453 2817 2011 2154 1010 2061 1045 1010 1996 6697 1997 11965 103 103 103 19454 2015 103 2012 1996 6730 4303 1997 2014 8835 1011 103 1010 10047 28065 2063 17617 3716 1012 103 103 3360 1045 2371 1999 2033 1037 3969 2682 1996 3043 1011 4372 27898 14906 1012 2016 3936 2000 2033 1996 14013 102 4298 2023 2089 2031 2042 1996 103 103 2220 4125 2869 1999 2008 10246 1010 2076 1996 16373 103 1010 4233 1037 16465 10427 1997 2303 1010 1998 103 4196 2037 2159 2000 1996 16931 2098 2030 2634 1011 10710 8871 15717 2682 2068 1012 102\n",
"I0523 05:34:34.622332 140368108218240 create_pretraining_data.py:161] input_ids: 101 2004 1996 3418 10878 1011 1011 1996 2171 2003 4895 5714 6442 4630 2000 103 103 1011 1011 103 103 22390 103 2002 2453 2817 2011 2154 1010 2061 1045 1010 1996 6697 1997 11965 103 103 103 19454 2015 103 2012 1996 6730 4303 1997 2014 8835 1011 103 1010 10047 28065 2063 17617 3716 1012 103 103 3360 1045 2371 1999 2033 1037 3969 2682 1996 3043 1011 4372 27898 14906 1012 2016 3936 2000 2033 1996 14013 102 4298 2023 2089 2031 2042 1996 103 103 2220 4125 2869 1999 2008 10246 1010 2076 1996 16373 103 1010 4233 1037 16465 10427 1997 2303 1010 1998 103 4196 2037 2159 2000 1996 16931 2098 2030 2634 1011 10710 8871 15717 2682 2068 1012 102\n",
"INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"I0523 05:34:34.622460 140368108218240 create_pretraining_data.py:161] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"I0523 05:34:34.622588 140368108218240 create_pretraining_data.py:161] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"INFO:tensorflow:masked_lm_positions: 15 16 19 20 22 24 36 37 38 41 50 58 59 74 86 88 89 100 110 0\n",
"I0523 05:34:34.622677 140368108218240 create_pretraining_data.py:161] masked_lm_positions: 15 16 19 20 22 24 36 37 38 41 50 58 59 74 86 88 89 100 110 0\n",
"INFO:tensorflow:masked_lm_ids: 1037 8284 16486 2300 2008 2453 2015 1998 11498 1010 2282 2013 2026 1012 2042 3114 2339 2161 15839 0\n",
"I0523 05:34:34.622765 140368108218240 create_pretraining_data.py:161] masked_lm_ids: 1037 8284 16486 2300 2008 2453 2015 1998 11498 1010 2282 2013 2026 1012 2042 3114 2339 2161 15839 0\n",
"INFO:tensorflow:masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0\n",
"I0523 05:34:34.622868 140368108218240 create_pretraining_data.py:161] masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0\n",
"INFO:tensorflow:next_sentence_labels: 1\n",
"I0523 05:34:34.622951 140368108218240 create_pretraining_data.py:161] next_sentence_labels: 1\n",
"INFO:tensorflow:*** Example ***\n",
"I0523 05:34:34.623253 140368108218240 create_pretraining_data.py:149] *** Example ***\n",
"INFO:tensorflow:tokens: [CLS] the rain had only ceased with the gray streaks of morning at blazing star , and the [MASK] awoke to a moral [MASK] of [MASK] ##liness , and the finding of forgotten knives [MASK] tin [MASK] , and smaller camp ut ##ens ##ils , where the [MASK] showers had washed away the debris and dust [MASK] ##s before the [MASK] doors . [SEP] this sample ##sily is public domain and was randomly selected [MASK] project gut ##tenberg . [SEP]\n",
"I0523 05:34:34.623375 140368108218240 create_pretraining_data.py:151] tokens: [CLS] the rain had only ceased with the gray streaks of morning at blazing star , and the [MASK] awoke to a moral [MASK] of [MASK] ##liness , and the finding of forgotten knives [MASK] tin [MASK] , and smaller camp ut ##ens ##ils , where the [MASK] showers had washed away the debris and dust [MASK] ##s before the [MASK] doors . [SEP] this sample ##sily is public domain and was randomly selected [MASK] project gut ##tenberg . [SEP]\n",
"INFO:tensorflow:input_ids: 101 1996 4542 2018 2069 7024 2007 1996 3897 21295 1997 2851 2012 17162 2732 1010 1998 1996 103 19179 2000 1037 7191 103 1997 103 20942 1010 1998 1996 4531 1997 6404 13227 103 9543 103 1010 1998 3760 3409 21183 6132 12146 1010 2073 1996 103 23442 2018 8871 2185 1996 11385 1998 6497 103 2015 2077 1996 103 4303 1012 102 2023 7099 26863 2003 2270 5884 1998 2001 18154 3479 103 2622 9535 21806 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.623497 140368108218240 create_pretraining_data.py:161] input_ids: 101 1996 4542 2018 2069 7024 2007 1996 3897 21295 1997 2851 2012 17162 2732 1010 1998 1996 103 19179 2000 1037 7191 103 1997 103 20942 1010 1998 1996 4531 1997 6404 13227 103 9543 103 1010 1998 3760 3409 21183 6132 12146 1010 2073 1996 103 23442 2018 8871 2185 1996 11385 1998 6497 103 2015 2077 1996 103 4303 1012 102 2023 7099 26863 2003 2270 5884 1998 2001 18154 3479 103 2622 9535 21806 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.623619 140368108218240 create_pretraining_data.py:161] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.623742 140368108218240 create_pretraining_data.py:161] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:masked_lm_positions: 8 18 23 25 34 36 47 56 60 66 74 78 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.623823 140368108218240 create_pretraining_data.py:161] masked_lm_positions: 8 18 23 25 34 36 47 56 60 66 74 78 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:masked_lm_ids: 3897 4093 3168 4550 1010 10268 3082 16721 6644 3793 2013 1012 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.623909 140368108218240 create_pretraining_data.py:161] masked_lm_ids: 3897 4093 3168 4550 1010 10268 3082 16721 6644 3793 2013 1012 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0\n",
"I0523 05:34:34.624003 140368108218240 create_pretraining_data.py:161] masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0\n",
"INFO:tensorflow:next_sentence_labels: 1\n",
"I0523 05:34:34.624076 140368108218240 create_pretraining_data.py:161] next_sentence_labels: 1\n",
"INFO:tensorflow:*** Example ***\n",
"I0523 05:34:34.624371 140368108218240 create_pretraining_data.py:149] *** Example ***\n",
"INFO:tensorflow:tokens: [CLS] this text is included to [MASK] sensual unicode is handled [MASK] : ##gp 加 勝 北 区 ᴵ ##ᴺ ##ᵀ ##ᵃ ##ছ ##জ ##ট ##ড ##ণ ##ত [SEP] roar and w ##hir ##l of the street , the perpetual stream of busy faces , the line of cu ##rri ##cles , pal ##an ##quin ##s , laden ass ##ᵐ , camel ##s , elephants [MASK] which met [MASK] passed him , and squeezed [MASK] up [MASK] and into doorway ##s , as they threaded their way through the great moon - gate [MASK] [MASK] [MASK] street beyond , drove everything from his mind [MASK] wondering curiosity , and a vague , helpless dread of that great living wilderness [MASK] more terrible than any dead wilderness of [SEP]\n",
"I0523 05:34:34.624506 140368108218240 create_pretraining_data.py:151] tokens: [CLS] this text is included to [MASK] sensual unicode is handled [MASK] : ##gp 加 勝 北 区 ᴵ ##ᴺ ##ᵀ ##ᵃ ##ছ ##জ ##ট ##ড ##ণ ##ত [SEP] roar and w ##hir ##l of the street , the perpetual stream of busy faces , the line of cu ##rri ##cles , pal ##an ##quin ##s , laden ass ##ᵐ , camel ##s , elephants [MASK] which met [MASK] passed him , and squeezed [MASK] up [MASK] and into doorway ##s , as they threaded their way through the great moon - gate [MASK] [MASK] [MASK] street beyond , drove everything from his mind [MASK] wondering curiosity , and a vague , helpless dread of that great living wilderness [MASK] more terrible than any dead wilderness of [SEP]\n",
"INFO:tensorflow:input_ids: 101 2023 3793 2003 2443 2000 103 18753 27260 2003 8971 103 1024 21600 1779 1780 1781 1782 1493 30030 30031 30032 29893 29894 29895 29896 29897 29898 102 11950 1998 1059 11961 2140 1997 1996 2395 1010 1996 18870 5460 1997 5697 5344 1010 1996 2240 1997 12731 18752 18954 1010 14412 2319 12519 2015 1010 14887 4632 30038 1010 19130 2015 1010 16825 103 2029 2777 103 2979 2032 1010 1998 7757 103 2039 103 1998 2046 7086 2015 1010 2004 2027 26583 2037 2126 2083 1996 2307 4231 1011 4796 103 103 103 2395 3458 1010 5225 2673 2013 2010 2568 103 6603 10628 1010 1998 1037 13727 1010 13346 14436 1997 2008 2307 2542 9917 103 2062 6659 2084 2151 2757 9917 1997 102\n",
"I0523 05:34:34.722806 140368108218240 create_pretraining_data.py:161] input_ids: 101 2023 3793 2003 2443 2000 103 18753 27260 2003 8971 103 1024 21600 1779 1780 1781 1782 1493 30030 30031 30032 29893 29894 29895 29896 29897 29898 102 11950 1998 1059 11961 2140 1997 1996 2395 1010 1996 18870 5460 1997 5697 5344 1010 1996 2240 1997 12731 18752 18954 1010 14412 2319 12519 2015 1010 14887 4632 30038 1010 19130 2015 1010 16825 103 2029 2777 103 2979 2032 1010 1998 7757 103 2039 103 1998 2046 7086 2015 1010 2004 2027 26583 2037 2126 2083 1996 2307 4231 1011 4796 103 103 103 2395 3458 1010 5225 2673 2013 2010 2568 103 6603 10628 1010 1998 1037 13727 1010 13346 14436 1997 2008 2307 2542 9917 103 2062 6659 2084 2151 2757 9917 1997 102\n",
"INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"I0523 05:34:34.723023 140368108218240 create_pretraining_data.py:161] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"I0523 05:34:34.723176 140368108218240 create_pretraining_data.py:161] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"INFO:tensorflow:masked_lm_positions: 4 6 7 11 12 13 19 59 65 68 71 74 76 86 93 94 95 104 119 0\n",
"I0523 05:34:34.725137 140368108218240 create_pretraining_data.py:161] masked_lm_positions: 4 6 7 11 12 13 19 59 65 68 71 74 76 86 93 94 95 104 119 0\n",
"INFO:tensorflow:masked_lm_ids: 2443 2191 2469 7919 1024 1778 30030 2229 1010 1998 1010 2032 4084 2126 2046 1996 20851 2021 1010 0\n",
"I0523 05:34:34.725321 140368108218240 create_pretraining_data.py:161] masked_lm_ids: 2443 2191 2469 7919 1024 1778 30030 2229 1010 1998 1010 2032 4084 2126 2046 1996 20851 2021 1010 0\n",
"INFO:tensorflow:masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0\n",
"I0523 05:34:34.725425 140368108218240 create_pretraining_data.py:161] masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0\n",
"INFO:tensorflow:next_sentence_labels: 1\n",
"I0523 05:34:34.725506 140368108218240 create_pretraining_data.py:161] next_sentence_labels: 1\n",
"INFO:tensorflow:*** Example ***\n",
"I0523 05:34:34.726054 140368108218240 create_pretraining_data.py:149] *** Example ***\n",
"INFO:tensorflow:tokens: [CLS] and there burst on phil ##am ##mon ' [MASK] astonished eyes a vast semi ##ci ##rcle of blue sea , ring ##ed with palaces [MASK] ##berger . [SEP] [MASK] fortunate early rise rediscovered had once picked [MASK] [MASK] the highway a solid chunk of gold quartz which the rain had freed from its inc ##umber ##ing soil [MASK] and [MASK] into immediate and [MASK] popularity . possibly [MASK] may have been the reason why early rise ##rs [MASK] that locality , [MASK] the rainy season , adopted a thoughtful habit of body [MASK] and seldom lifted their eyes to the rift ##ed [MASK] india - ink washed skies above them [MASK] \" cass \" beard had risen early that morning [MASK] but not with a view [SEP]\n",
"I0523 05:34:34.726276 140368108218240 create_pretraining_data.py:151] tokens: [CLS] and there burst on phil ##am ##mon ' [MASK] astonished eyes a vast semi ##ci ##rcle of blue sea , ring ##ed with palaces [MASK] ##berger . [SEP] [MASK] fortunate early rise rediscovered had once picked [MASK] [MASK] the highway a solid chunk of gold quartz which the rain had freed from its inc ##umber ##ing soil [MASK] and [MASK] into immediate and [MASK] popularity . possibly [MASK] may have been the reason why early rise ##rs [MASK] that locality , [MASK] the rainy season , adopted a thoughtful habit of body [MASK] and seldom lifted their eyes to the rift ##ed [MASK] india - ink washed skies above them [MASK] \" cass \" beard had risen early that morning [MASK] but not with a view [SEP]\n",
"INFO:tensorflow:input_ids: 101 1998 2045 6532 2006 6316 3286 8202 1005 103 22741 2159 1037 6565 4100 6895 21769 1997 2630 2712 1010 3614 2098 2007 22763 103 14859 1012 102 103 19590 2220 4125 26733 2018 2320 3856 103 103 1996 3307 1037 5024 20000 1997 2751 20971 2029 1996 4542 2018 10650 2013 2049 4297 29440 2075 5800 103 1998 103 2046 6234 1998 103 6217 1012 4298 103 2089 2031 2042 1996 3114 2339 2220 4125 2869 103 2008 10246 1010 103 1996 16373 2161 1010 4233 1037 16465 10427 1997 2303 103 1998 15839 4196 2037 2159 2000 1996 16931 2098 103 2634 1011 10710 8871 15717 2682 2068 103 1000 16220 1000 10154 2018 13763 2220 2008 2851 103 2021 2025 2007 1037 3193 102\n",
"I0523 05:34:34.726441 140368108218240 create_pretraining_data.py:161] input_ids: 101 1998 2045 6532 2006 6316 3286 8202 1005 103 22741 2159 1037 6565 4100 6895 21769 1997 2630 2712 1010 3614 2098 2007 22763 103 14859 1012 102 103 19590 2220 4125 26733 2018 2320 3856 103 103 1996 3307 1037 5024 20000 1997 2751 20971 2029 1996 4542 2018 10650 2013 2049 4297 29440 2075 5800 103 1998 103 2046 6234 1998 103 6217 1012 4298 103 2089 2031 2042 1996 3114 2339 2220 4125 2869 103 2008 10246 1010 103 1996 16373 2161 1010 4233 1037 16465 10427 1997 2303 103 1998 15839 4196 2037 2159 2000 1996 16931 2098 103 2634 1011 10710 8871 15717 2682 2068 103 1000 16220 1000 10154 2018 13763 2220 2008 2851 103 2021 2025 2007 1037 3193 102\n",
"INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"I0523 05:34:34.726583 140368108218240 create_pretraining_data.py:161] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"I0523 05:34:34.726738 140368108218240 create_pretraining_data.py:161] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"INFO:tensorflow:masked_lm_positions: 9 25 26 29 33 37 38 56 58 60 64 68 78 82 93 103 111 113 121 0\n",
"I0523 05:34:34.726836 140368108218240 create_pretraining_data.py:161] masked_lm_positions: 9 25 26 29 33 37 38 56 58 60 64 68 78 82 93 103 111 113 121 0\n",
"INFO:tensorflow:masked_lm_ids: 1055 1998 7626 1037 2099 2039 2006 2075 1010 8871 20332 2023 1999 2076 1010 2030 1012 16220 1010 0\n",
"I0523 05:34:34.726930 140368108218240 create_pretraining_data.py:161] masked_lm_ids: 1055 1998 7626 1037 2099 2039 2006 2075 1010 8871 20332 2023 1999 2076 1010 2030 1012 16220 1010 0\n",
"INFO:tensorflow:masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0\n",
"I0523 05:34:34.727055 140368108218240 create_pretraining_data.py:161] masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0\n",
"INFO:tensorflow:next_sentence_labels: 1\n",
"I0523 05:34:34.727200 140368108218240 create_pretraining_data.py:161] next_sentence_labels: 1\n",
"INFO:tensorflow:*** Example ***\n",
"I0523 05:34:34.727780 140368108218240 create_pretraining_data.py:149] *** Example ***\n",
"INFO:tensorflow:tokens: [CLS] like most of his fellow gold - seekers , cass was super ##sti ##tious . [SEP] at last they [MASK] the [MASK] at the opposite end of [MASK] street ; and there burst on phil ##am ##mon ' s astonished eyes a vast semi ##ci ##rcle of blue [MASK] , ring ##ed with palaces and towers . he stopped in ##vo ##lun ##tar ##ily ; and his [MASK] guide stopped also , and looked ask [MASK] at [MASK] [MASK] monk , to watch the effect [MASK] that [MASK] panorama should produce on him . [SEP]\n",
"I0523 05:34:34.727958 140368108218240 create_pretraining_data.py:151] tokens: [CLS] like most of his fellow gold - seekers , cass was super ##sti ##tious . [SEP] at last they [MASK] the [MASK] at the opposite end of [MASK] street ; and there burst on phil ##am ##mon ' s astonished eyes a vast semi ##ci ##rcle of blue [MASK] , ring ##ed with palaces and towers . he stopped in ##vo ##lun ##tar ##ily ; and his [MASK] guide stopped also , and looked ask [MASK] at [MASK] [MASK] monk , to watch the effect [MASK] that [MASK] panorama should produce on him . [SEP]\n",
"INFO:tensorflow:input_ids: 101 2066 2087 1997 2010 3507 2751 1011 24071 1010 16220 2001 3565 16643 20771 1012 102 2012 2197 2027 103 1996 103 2012 1996 4500 2203 1997 103 2395 1025 1998 2045 6532 2006 6316 3286 8202 1005 1055 22741 2159 1037 6565 4100 6895 21769 1997 2630 103 1010 3614 2098 2007 22763 1998 7626 1012 2002 3030 1999 6767 26896 7559 6588 1025 1998 2010 103 5009 3030 2036 1010 1998 2246 3198 103 2012 103 103 8284 1010 2000 3422 1996 3466 103 2008 103 23652 2323 3965 2006 2032 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.728130 140368108218240 create_pretraining_data.py:161] input_ids: 101 2066 2087 1997 2010 3507 2751 1011 24071 1010 16220 2001 3565 16643 20771 1012 102 2012 2197 2027 103 1996 103 2012 1996 4500 2203 1997 103 2395 1025 1998 2045 6532 2006 6316 3286 8202 1005 1055 22741 2159 1037 6565 4100 6895 21769 1997 2630 103 1010 3614 2098 2007 22763 1998 7626 1012 2002 3030 1999 6767 26896 7559 6588 1025 1998 2010 103 5009 3030 2036 1010 1998 2246 3198 103 2012 103 103 8284 1010 2000 3422 1996 3466 103 2008 103 23652 2323 3965 2006 2032 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.728290 140368108218240 create_pretraining_data.py:161] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.728428 140368108218240 create_pretraining_data.py:161] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:masked_lm_positions: 20 22 28 49 51 53 56 68 76 78 79 86 88 94 0 0 0 0 0 0\n",
"I0523 05:34:34.728525 140368108218240 create_pretraining_data.py:161] masked_lm_positions: 20 22 28 49 51 53 56 68 76 78 79 86 88 94 0 0 0 0 0 0\n",
"INFO:tensorflow:masked_lm_ids: 2584 21048 1996 2712 3614 2007 7626 2210 6651 1996 2402 2029 2882 1012 0 0 0 0 0 0\n",
"I0523 05:34:34.728628 140368108218240 create_pretraining_data.py:161] masked_lm_ids: 2584 21048 1996 2712 3614 2007 7626 2210 6651 1996 2402 2029 2882 1012 0 0 0 0 0 0\n",
"INFO:tensorflow:masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0\n",
"I0523 05:34:34.728741 140368108218240 create_pretraining_data.py:161] masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0\n",
"INFO:tensorflow:next_sentence_labels: 1\n",
"I0523 05:34:34.728831 140368108218240 create_pretraining_data.py:161] next_sentence_labels: 1\n",
"INFO:tensorflow:*** Example ***\n",
"I0523 05:34:34.729317 140368108218240 create_pretraining_data.py:149] *** Example ***\n",
"INFO:tensorflow:tokens: [CLS] at last they reached the quay at the opposite end of [MASK] street ; and there burst on [MASK] ua [MASK] ' s astonished eyes a vast semi ##ci [MASK] of blue sea , ring ##ed with palaces and [MASK] . [SEP] this text is included to make sure [MASK] is handled properly : [MASK] [MASK] 勝 北 区 ᴵ ##ᴺ ##ᵀ ##ᵃ [MASK] ##জ ##ট ##ড ##ণ ##ত text should [MASK] one [MASK] sentence [MASK] per - line , with empty lines between 142 . this ##sable text is public domain and was randomly selected from project gut ##tenberg . [SEP]\n",
"I0523 05:34:34.729494 140368108218240 create_pretraining_data.py:151] tokens: [CLS] at last they reached the quay at the opposite end of [MASK] street ; and there burst on [MASK] ua [MASK] ' s astonished eyes a vast semi ##ci [MASK] of blue sea , ring ##ed with palaces and [MASK] . [SEP] this text is included to make sure [MASK] is handled properly : [MASK] [MASK] 勝 北 区 ᴵ ##ᴺ ##ᵀ ##ᵃ [MASK] ##জ ##ট ##ড ##ণ ##ত text should [MASK] one [MASK] sentence [MASK] per - line , with empty lines between 142 . this ##sable text is public domain and was randomly selected from project gut ##tenberg . [SEP]\n",
"INFO:tensorflow:input_ids: 101 2012 2197 2027 2584 1996 21048 2012 1996 4500 2203 1997 103 2395 1025 1998 2045 6532 2006 103 25423 103 1005 1055 22741 2159 1037 6565 4100 6895 103 1997 2630 2712 1010 3614 2098 2007 22763 1998 103 1012 102 2023 3793 2003 2443 2000 2191 2469 103 2003 8971 7919 1024 103 103 1780 1781 1782 1493 30030 30031 30032 103 29894 29895 29896 29897 29898 3793 2323 103 2028 103 6251 103 2566 1011 2240 1010 2007 4064 3210 2090 16087 1012 2023 19150 3793 2003 2270 5884 1998 2001 18154 3479 2013 2622 9535 21806 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.729652 140368108218240 create_pretraining_data.py:161] input_ids: 101 2012 2197 2027 2584 1996 21048 2012 1996 4500 2203 1997 103 2395 1025 1998 2045 6532 2006 103 25423 103 1005 1055 22741 2159 1037 6565 4100 6895 103 1997 2630 2712 1010 3614 2098 2007 22763 1998 103 1012 102 2023 3793 2003 2443 2000 2191 2469 103 2003 8971 7919 1024 103 103 1780 1781 1782 1493 30030 30031 30032 103 29894 29895 29896 29897 29898 3793 2323 103 2028 103 6251 103 2566 1011 2240 1010 2007 4064 3210 2090 16087 1012 2023 19150 3793 2003 2270 5884 1998 2001 18154 3479 2013 2622 9535 21806 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.729803 140368108218240 create_pretraining_data.py:161] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.822823 140368108218240 create_pretraining_data.py:161] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:masked_lm_positions: 12 19 20 21 30 40 50 55 56 64 72 74 76 85 88 0 0 0 0 0\n",
"I0523 05:34:34.823260 140368108218240 create_pretraining_data.py:161] masked_lm_positions: 12 19 20 21 30 40 50 55 56 64 72 74 76 85 88 0 0 0 0 0\n",
"INFO:tensorflow:masked_lm_ids: 1996 6316 3286 8202 21769 7626 27260 1778 1779 29893 2022 1011 1011 5491 7099 0 0 0 0 0\n",
"I0523 05:34:34.823401 140368108218240 create_pretraining_data.py:161] masked_lm_ids: 1996 6316 3286 8202 21769 7626 27260 1778 1779 29893 2022 1011 1011 5491 7099 0 0 0 0 0\n",
"INFO:tensorflow:masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0\n",
"I0523 05:34:34.823719 140368108218240 create_pretraining_data.py:161] masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0\n",
"INFO:tensorflow:next_sentence_labels: 1\n",
"I0523 05:34:34.824058 140368108218240 create_pretraining_data.py:161] next_sentence_labels: 1\n",
"INFO:tensorflow:*** Example ***\n",
"I0523 05:34:34.824915 140368108218240 create_pretraining_data.py:149] *** Example ***\n",
"INFO:tensorflow:tokens: [CLS] and there burst on phil ##am ##mon [MASK] s astonished eyes a vast [MASK] ##ci ##rcle of blue sea [MASK] ring ##ed with palaces and [MASK] . [SEP] he stopped in ##vo ##lun ##tar ##ily [MASK] and his little [MASK] stopped also , [MASK] looked ask ##ance at the young [MASK] , to watch the effect which that grand panorama persuasion produce on ##書 . [SEP]\n",
"I0523 05:34:34.825074 140368108218240 create_pretraining_data.py:151] tokens: [CLS] and there burst on phil ##am ##mon [MASK] s astonished eyes a vast [MASK] ##ci ##rcle of blue sea [MASK] ring ##ed with palaces and [MASK] . [SEP] he stopped in ##vo ##lun ##tar ##ily [MASK] and his little [MASK] stopped also , [MASK] looked ask ##ance at the young [MASK] , to watch the effect which that grand panorama persuasion produce on ##書 . [SEP]\n",
"INFO:tensorflow:input_ids: 101 1998 2045 6532 2006 6316 3286 8202 103 1055 22741 2159 1037 6565 103 6895 21769 1997 2630 2712 103 3614 2098 2007 22763 1998 103 1012 102 2002 3030 1999 6767 26896 7559 6588 103 1998 2010 2210 103 3030 2036 1010 103 2246 3198 6651 2012 1996 2402 103 1010 2000 3422 1996 3466 2029 2008 2882 23652 27577 3965 2006 30397 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.825276 140368108218240 create_pretraining_data.py:161] input_ids: 101 1998 2045 6532 2006 6316 3286 8202 103 1055 22741 2159 1037 6565 103 6895 21769 1997 2630 2712 103 3614 2098 2007 22763 1998 103 1012 102 2002 3030 1999 6767 26896 7559 6588 103 1998 2010 2210 103 3030 2036 1010 103 2246 3198 6651 2012 1996 2402 103 1010 2000 3422 1996 3466 2029 2008 2882 23652 27577 3965 2006 30397 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.825414 140368108218240 create_pretraining_data.py:161] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.825540 140368108218240 create_pretraining_data.py:161] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:masked_lm_positions: 8 14 20 26 36 40 44 51 61 64 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.825623 140368108218240 create_pretraining_data.py:161] masked_lm_positions: 8 14 20 26 36 40 44 51 61 64 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:masked_lm_ids: 1005 4100 1010 7626 1025 5009 1998 8284 2323 2032 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.825707 140368108218240 create_pretraining_data.py:161] masked_lm_ids: 1005 4100 1010 7626 1025 5009 1998 8284 2323 2032 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0\n",
"I0523 05:34:34.825792 140368108218240 create_pretraining_data.py:161] masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0\n",
"INFO:tensorflow:next_sentence_labels: 0\n",
"I0523 05:34:34.825863 140368108218240 create_pretraining_data.py:161] next_sentence_labels: 0\n",
"INFO:tensorflow:*** Example ***\n",
"I0523 05:34:34.826209 140368108218240 create_pretraining_data.py:149] *** Example ***\n",
"INFO:tensorflow:tokens: [CLS] [MASK] most of his fellow gold - [MASK] , cass was super ##sti ##tious . [SEP] while at the end of the vista in front [MASK] them gleamed the [MASK] harbour [MASK] through a network of beetles mast ##s . at [MASK] they reached the quay at the opposite end of the street ; and [MASK] burst [MASK] phil ##am ##mon ' s astonished eyes a vast semi ##ci ##rcle of blue sea ##ban [MASK] ##ed with palaces and towers . he stopped [MASK] ##vo [MASK] liege ##ily ; and his little guide stopped [MASK] , and looked ask ##ance at the young monk , to [MASK] the effect which that grand panorama should produce on [MASK] whoa [SEP]\n",
"I0523 05:34:34.826432 140368108218240 create_pretraining_data.py:151] tokens: [CLS] [MASK] most of his fellow gold - [MASK] , cass was super ##sti ##tious . [SEP] while at the end of the vista in front [MASK] them gleamed the [MASK] harbour [MASK] through a network of beetles mast ##s . at [MASK] they reached the quay at the opposite end of the street ; and [MASK] burst [MASK] phil ##am ##mon ' s astonished eyes a vast semi ##ci ##rcle of blue sea ##ban [MASK] ##ed with palaces and towers . he stopped [MASK] ##vo [MASK] liege ##ily ; and his little guide stopped [MASK] , and looked ask ##ance at the young monk , to [MASK] the effect which that grand panorama should produce on [MASK] whoa [SEP]\n",
"INFO:tensorflow:input_ids: 101 103 2087 1997 2010 3507 2751 1011 103 1010 16220 2001 3565 16643 20771 1012 102 2096 2012 1996 2203 1997 1996 13005 1999 2392 103 2068 25224 1996 103 7440 103 2083 1037 2897 1997 14538 15429 2015 1012 2012 103 2027 2584 1996 21048 2012 1996 4500 2203 1997 1996 2395 1025 1998 103 6532 103 6316 3286 8202 1005 1055 22741 2159 1037 6565 4100 6895 21769 1997 2630 2712 8193 103 2098 2007 22763 1998 7626 1012 2002 3030 103 6767 103 17766 6588 1025 1998 2010 2210 5009 3030 103 1010 1998 2246 3198 6651 2012 1996 2402 8284 1010 2000 103 1996 3466 2029 2008 2882 23652 2323 3965 2006 103 23281 102 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.826583 140368108218240 create_pretraining_data.py:161] input_ids: 101 103 2087 1997 2010 3507 2751 1011 103 1010 16220 2001 3565 16643 20771 1012 102 2096 2012 1996 2203 1997 1996 13005 1999 2392 103 2068 25224 1996 103 7440 103 2083 1037 2897 1997 14538 15429 2015 1012 2012 103 2027 2584 1996 21048 2012 1996 4500 2203 1997 1996 2395 1025 1998 103 6532 103 6316 3286 8202 1005 1055 22741 2159 1037 6565 4100 6895 21769 1997 2630 2712 8193 103 2098 2007 22763 1998 7626 1012 2002 3030 103 6767 103 17766 6588 1025 1998 2010 2210 5009 3030 103 1010 1998 2246 3198 6651 2012 1996 2402 8284 1010 2000 103 1996 3466 2029 2008 2882 23652 2323 3965 2006 103 23281 102 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.826725 140368108218240 create_pretraining_data.py:161] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.826864 140368108218240 create_pretraining_data.py:161] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:masked_lm_positions: 1 8 26 30 32 37 42 56 58 74 75 84 86 87 95 107 117 118 0 0\n",
"I0523 05:34:34.826968 140368108218240 create_pretraining_data.py:161] masked_lm_positions: 1 8 26 30 32 37 42 56 58 74 75 84 86 87 95 107 117 118 0 0\n",
"INFO:tensorflow:masked_lm_ids: 2066 24071 1997 2630 1010 14518 2197 2045 2006 1010 3614 1999 26896 7559 2036 3422 2032 1012 0 0\n",
"I0523 05:34:34.827068 140368108218240 create_pretraining_data.py:161] masked_lm_ids: 2066 24071 1997 2630 1010 14518 2197 2045 2006 1010 3614 1999 26896 7559 2036 3422 2032 1012 0 0\n",
"INFO:tensorflow:masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0\n",
"I0523 05:34:34.827175 140368108218240 create_pretraining_data.py:161] masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0\n",
"INFO:tensorflow:next_sentence_labels: 1\n",
"I0523 05:34:34.827300 140368108218240 create_pretraining_data.py:161] next_sentence_labels: 1\n",
"INFO:tensorflow:*** Example ***\n",
"I0523 05:34:34.827672 140368108218240 create_pretraining_data.py:149] *** Example ***\n",
"INFO:tensorflow:tokens: [CLS] ! but , wonderful to relate , not an irregular , [MASK] ##less fragment of [MASK] [MASK] , fresh from nature ' s cr [MASK] , but a [MASK] of jewel ##er ' s hand ##ic ##raf ##t in the form of a plain gold [MASK] . looking at it [MASK] [MASK] [MASK] ##tively , he saw that [MASK] bore the inscription , [SEP] [MASK] on [MASK] more than a mile up the great main [MASK] , crossed in the centre of the city [MASK] at right angles , by one equally magnificent , at each end of which , miles away , [MASK] , dim and distant over [MASK] heads of the living stream golden passengers , the yellow [MASK] - hills of the desert [SEP]\n",
"I0523 05:34:34.827815 140368108218240 create_pretraining_data.py:151] tokens: [CLS] ! but , wonderful to relate , not an irregular , [MASK] ##less fragment of [MASK] [MASK] , fresh from nature ' s cr [MASK] , but a [MASK] of jewel ##er ' s hand ##ic ##raf ##t in the form of a plain gold [MASK] . looking at it [MASK] [MASK] [MASK] ##tively , he saw that [MASK] bore the inscription , [SEP] [MASK] on [MASK] more than a mile up the great main [MASK] , crossed in the centre of the city [MASK] at right angles , by one equally magnificent , at each end of which , miles away , [MASK] , dim and distant over [MASK] heads of the living stream golden passengers , the yellow [MASK] - hills of the desert [SEP]\n",
"INFO:tensorflow:input_ids: 101 999 2021 1010 6919 2000 14396 1010 2025 2019 12052 1010 103 3238 15778 1997 103 103 1010 4840 2013 3267 1005 1055 13675 103 1010 2021 1037 103 1997 13713 2121 1005 1055 2192 2594 27528 2102 1999 1996 2433 1997 1037 5810 2751 103 1012 2559 2012 2009 103 103 103 25499 1010 2002 2387 2008 103 8501 1996 9315 1010 102 103 2006 103 2062 2084 1037 3542 2039 1996 2307 2364 103 1010 4625 1999 1996 2803 1997 1996 2103 103 2012 2157 12113 1010 2011 2028 8053 12047 1010 2012 2169 2203 1997 2029 1010 2661 2185 1010 103 1010 11737 1998 6802 2058 103 4641 1997 1996 2542 5460 3585 5467 1010 1996 3756 103 1011 4564 1997 1996 5532 102\n",
"I0523 05:34:34.827938 140368108218240 create_pretraining_data.py:161] input_ids: 101 999 2021 1010 6919 2000 14396 1010 2025 2019 12052 1010 103 3238 15778 1997 103 103 1010 4840 2013 3267 1005 1055 13675 103 1010 2021 1037 103 1997 13713 2121 1005 1055 2192 2594 27528 2102 1999 1996 2433 1997 1037 5810 2751 103 1012 2559 2012 2009 103 103 103 25499 1010 2002 2387 2008 103 8501 1996 9315 1010 102 103 2006 103 2062 2084 1037 3542 2039 1996 2307 2364 103 1010 4625 1999 1996 2803 1997 1996 2103 103 2012 2157 12113 1010 2011 2028 8053 12047 1010 2012 2169 2203 1997 2029 1010 2661 2185 1010 103 1010 11737 1998 6802 2058 103 4641 1997 1996 2542 5460 3585 5467 1010 1996 3756 103 1011 4564 1997 1996 5532 102\n",
"INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"I0523 05:34:34.828072 140368108218240 create_pretraining_data.py:161] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"I0523 05:34:34.828197 140368108218240 create_pretraining_data.py:161] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"INFO:tensorflow:masked_lm_positions: 7 12 16 17 25 29 46 51 52 53 59 65 67 76 85 104 110 116 121 0\n",
"I0523 05:34:34.828283 140368108218240 create_pretraining_data.py:161] masked_lm_positions: 7 12 16 17 25 29 46 51 52 53 59 65 67 76 85 104 110 116 121 0\n",
"INFO:tensorflow:masked_lm_ids: 1010 4338 13587 10848 21104 2978 3614 2062 2012 6528 2009 2218 2005 2395 1010 2596 1996 1997 5472 0\n",
"I0523 05:34:34.828366 140368108218240 create_pretraining_data.py:161] masked_lm_ids: 1010 4338 13587 10848 21104 2978 3614 2062 2012 6528 2009 2218 2005 2395 1010 2596 1996 1997 5472 0\n",
"INFO:tensorflow:masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0\n",
"I0523 05:34:34.828507 140368108218240 create_pretraining_data.py:161] masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0\n",
"INFO:tensorflow:next_sentence_labels: 1\n",
"I0523 05:34:34.828590 140368108218240 create_pretraining_data.py:161] next_sentence_labels: 1\n",
"INFO:tensorflow:*** Example ***\n",
"I0523 05:34:34.828895 140368108218240 create_pretraining_data.py:149] *** Example ***\n",
"INFO:tensorflow:tokens: [CLS] there is a phil ##oso ##phic pleasure in opening one ' s treasures to [MASK] modest young . perhaps you will assist me by carrying recording basket [MASK] [MASK] ? ' [MASK] the [MASK] man jumped up , put his basket on phil ##am myths ' s head , and tr ##otted off [MASK] a neighbouring street . [SEP] his ragged little api ##sh guide ; but the novel roar [MASK] w ##hir ##l [MASK] the street , the perpetual stream of busy faces [MASK] the line of cu ##rri ##cles [MASK] pal [MASK] ##quin ##s , laden ass ##es , camel ##s [MASK] elephants , which met [MASK] passed him , and [MASK] him up steps and into [MASK] ##s , as they threaded their [SEP]\n",
"I0523 05:34:34.829036 140368108218240 create_pretraining_data.py:151] tokens: [CLS] there is a phil ##oso ##phic pleasure in opening one ' s treasures to [MASK] modest young . perhaps you will assist me by carrying recording basket [MASK] [MASK] ? ' [MASK] the [MASK] man jumped up , put his basket on phil ##am myths ' s head , and tr ##otted off [MASK] a neighbouring street . [SEP] his ragged little api ##sh guide ; but the novel roar [MASK] w ##hir ##l [MASK] the street , the perpetual stream of busy faces [MASK] the line of cu ##rri ##cles [MASK] pal [MASK] ##quin ##s , laden ass ##es , camel ##s [MASK] elephants , which met [MASK] passed him , and [MASK] him up steps and into [MASK] ##s , as they threaded their [SEP]\n",
"INFO:tensorflow:input_ids: 101 2045 2003 1037 6316 19137 17926 5165 1999 3098 2028 1005 1055 17605 2000 103 10754 2402 1012 3383 2017 2097 6509 2033 2011 4755 3405 10810 103 103 1029 1005 103 1996 103 2158 5598 2039 1010 2404 2010 10810 2006 6316 3286 17218 1005 1055 2132 1010 1998 19817 26174 2125 103 1037 9632 2395 1012 102 2010 14202 2210 17928 4095 5009 1025 2021 1996 3117 11950 103 1059 11961 2140 103 1996 2395 1010 1996 18870 5460 1997 5697 5344 103 1996 2240 1997 12731 18752 18954 103 14412 103 12519 2015 1010 14887 4632 2229 1010 19130 2015 103 16825 1010 2029 2777 103 2979 2032 1010 1998 103 2032 2039 4084 1998 2046 103 2015 1010 2004 2027 26583 2037 102\n",
"I0523 05:34:34.829163 140368108218240 create_pretraining_data.py:161] input_ids: 101 2045 2003 1037 6316 19137 17926 5165 1999 3098 2028 1005 1055 17605 2000 103 10754 2402 1012 3383 2017 2097 6509 2033 2011 4755 3405 10810 103 103 1029 1005 103 1996 103 2158 5598 2039 1010 2404 2010 10810 2006 6316 3286 17218 1005 1055 2132 1010 1998 19817 26174 2125 103 1037 9632 2395 1012 102 2010 14202 2210 17928 4095 5009 1025 2021 1996 3117 11950 103 1059 11961 2140 103 1996 2395 1010 1996 18870 5460 1997 5697 5344 103 1996 2240 1997 12731 18752 18954 103 14412 103 12519 2015 1010 14887 4632 2229 1010 19130 2015 103 16825 1010 2029 2777 103 2979 2032 1010 1998 103 2032 2039 4084 1998 2046 103 2015 1010 2004 2027 26583 2037 102\n",
"INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"I0523 05:34:34.829305 140368108218240 create_pretraining_data.py:161] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"I0523 05:34:34.829432 140368108218240 create_pretraining_data.py:161] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"INFO:tensorflow:masked_lm_positions: 15 26 28 29 32 34 45 54 71 75 85 92 94 104 109 113 114 116 120 0\n",
"I0523 05:34:34.926490 140368108218240 create_pretraining_data.py:161] masked_lm_positions: 15 26 28 29 32 34 45 54 71 75 85 92 94 104 109 113 114 116 120 0\n",
"INFO:tensorflow:masked_lm_ids: 1996 2023 1997 5909 1998 2210 8202 2039 1998 1997 1010 1010 2319 1010 1998 1998 7757 2039 7086 0\n",
"I0523 05:34:34.926674 140368108218240 create_pretraining_data.py:161] masked_lm_ids: 1996 2023 1997 5909 1998 2210 8202 2039 1998 1997 1010 1010 2319 1010 1998 1998 7757 2039 7086 0\n",
"INFO:tensorflow:masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0\n",
"I0523 05:34:34.926799 140368108218240 create_pretraining_data.py:161] masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0\n",
"INFO:tensorflow:next_sentence_labels: 0\n",
"I0523 05:34:34.926894 140368108218240 create_pretraining_data.py:161] next_sentence_labels: 0\n",
"INFO:tensorflow:*** Example ***\n",
"I0523 05:34:34.927461 140368108218240 create_pretraining_data.py:149] *** Example ***\n",
"INFO:tensorflow:tokens: [CLS] perhaps murder will assist me by carrying this [MASK] of fruit ? ' [MASK] the little man jumped up , put his basket [MASK] phil ##am ##mon ' [MASK] head , and tr ##otted off up a neighbouring street . phil ##am ##mon followed , half contempt ##uous , half wondering at what this philosophy [MASK] be , which [MASK] [MASK] the self [MASK] con ##ce ##it of anything so ab ##ject as his ragged [MASK] api ##val guide ; [SEP] text should be [MASK] [MASK] sentence - per [MASK] line [MASK] with empty lines [MASK] documents . this sample text is public domain and was randomly selected from project gut ##tenberg . [SEP]\n",
"I0523 05:34:34.927646 140368108218240 create_pretraining_data.py:151] tokens: [CLS] perhaps murder will assist me by carrying this [MASK] of fruit ? ' [MASK] the little man jumped up , put his basket [MASK] phil ##am ##mon ' [MASK] head , and tr ##otted off up a neighbouring street . phil ##am ##mon followed , half contempt ##uous , half wondering at what this philosophy [MASK] be , which [MASK] [MASK] the self [MASK] con ##ce ##it of anything so ab ##ject as his ragged [MASK] api ##val guide ; [SEP] text should be [MASK] [MASK] sentence - per [MASK] line [MASK] with empty lines [MASK] documents . this sample text is public domain and was randomly selected from project gut ##tenberg . [SEP]\n",
"INFO:tensorflow:input_ids: 101 3383 4028 2097 6509 2033 2011 4755 2023 103 1997 5909 1029 1005 103 1996 2210 2158 5598 2039 1010 2404 2010 10810 103 6316 3286 8202 1005 103 2132 1010 1998 19817 26174 2125 2039 1037 9632 2395 1012 6316 3286 8202 2628 1010 2431 17152 8918 1010 2431 6603 2012 2054 2023 4695 103 2022 1010 2029 103 103 1996 2969 103 9530 3401 4183 1997 2505 2061 11113 20614 2004 2010 14202 103 17928 10175 5009 1025 102 3793 2323 2022 103 103 6251 1011 2566 103 2240 103 2007 4064 3210 103 5491 1012 2023 7099 3793 2003 2270 5884 1998 2001 18154 3479 2013 2622 9535 21806 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.927825 140368108218240 create_pretraining_data.py:161] input_ids: 101 3383 4028 2097 6509 2033 2011 4755 2023 103 1997 5909 1029 1005 103 1996 2210 2158 5598 2039 1010 2404 2010 10810 103 6316 3286 8202 1005 103 2132 1010 1998 19817 26174 2125 2039 1037 9632 2395 1012 6316 3286 8202 2628 1010 2431 17152 8918 1010 2431 6603 2012 2054 2023 4695 103 2022 1010 2029 103 103 1996 2969 103 9530 3401 4183 1997 2505 2061 11113 20614 2004 2010 14202 103 17928 10175 5009 1025 102 3793 2323 2022 103 103 6251 1011 2566 103 2240 103 2007 4064 3210 103 5491 1012 2023 7099 3793 2003 2270 5884 1998 2001 18154 3479 2013 2622 9535 21806 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.928024 140368108218240 create_pretraining_data.py:161] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.928197 140368108218240 create_pretraining_data.py:161] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:masked_lm_positions: 2 9 14 24 29 56 60 61 63 64 76 78 85 86 90 92 96 0 0 0\n",
"I0523 05:34:34.928318 140368108218240 create_pretraining_data.py:161] masked_lm_positions: 2 9 14 24 29 56 60 61 63 64 76 78 85 86 90 92 96 0 0 0\n",
"INFO:tensorflow:masked_lm_ids: 2017 10810 1998 2006 1055 2453 2071 5438 2969 1011 2210 4095 2028 1011 1011 1010 2090 0 0 0\n",
"I0523 05:34:34.928426 140368108218240 create_pretraining_data.py:161] masked_lm_ids: 2017 10810 1998 2006 1055 2453 2071 5438 2969 1011 2210 4095 2028 1011 1011 1010 2090 0 0 0\n",
"INFO:tensorflow:masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0\n",
"I0523 05:34:34.928534 140368108218240 create_pretraining_data.py:161] masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0\n",
"INFO:tensorflow:next_sentence_labels: 1\n",
"I0523 05:34:34.928632 140368108218240 create_pretraining_data.py:161] next_sentence_labels: 1\n",
"INFO:tensorflow:*** Example ***\n",
"I0523 05:34:34.929087 140368108218240 create_pretraining_data.py:149] *** Example ***\n",
"INFO:tensorflow:tokens: [CLS] indeed , it was recorded in [MASK] star that a fortunate early [MASK] ##r had once picked up on the highway a solid chunk [MASK] gold quartz which the [MASK] had freed from its inc [MASK] ##ing soil , and washed into immediate and [MASK] popularity . [SEP] rainy season , [MASK] insult show habit of body , and seldom lifted their eyes to the rift ##ed [MASK] india - ink washed skies [MASK] them . \" cass \" beard [MASK] elliot early that morning , but not with a view to [MASK] . a leak in his [MASK] roof , - - quite [MASK] with his careless , imp ##rov ##ide ##nt habits , - - had rouse ##d him at 4 a [MASK] m [SEP]\n",
"I0523 05:34:34.929398 140368108218240 create_pretraining_data.py:151] tokens: [CLS] indeed , it was recorded in [MASK] star that a fortunate early [MASK] ##r had once picked up on the highway a solid chunk [MASK] gold quartz which the [MASK] had freed from its inc [MASK] ##ing soil , and washed into immediate and [MASK] popularity . [SEP] rainy season , [MASK] insult show habit of body , and seldom lifted their eyes to the rift ##ed [MASK] india - ink washed skies [MASK] them . \" cass \" beard [MASK] elliot early that morning , but not with a view to [MASK] . a leak in his [MASK] roof , - - quite [MASK] with his careless , imp ##rov ##ide ##nt habits , - - had rouse ##d him at 4 a [MASK] m [SEP]\n",
"INFO:tensorflow:input_ids: 101 5262 1010 2009 2001 2680 1999 103 2732 2008 1037 19590 2220 103 2099 2018 2320 3856 2039 2006 1996 3307 1037 5024 20000 103 2751 20971 2029 1996 103 2018 10650 2013 2049 4297 103 2075 5800 1010 1998 8871 2046 6234 1998 103 6217 1012 102 16373 2161 1010 103 15301 2265 10427 1997 2303 1010 1998 15839 4196 2037 2159 2000 1996 16931 2098 103 2634 1011 10710 8871 15717 103 2068 1012 1000 16220 1000 10154 103 11759 2220 2008 2851 1010 2021 2025 2007 1037 3193 2000 103 1012 1037 17271 1999 2010 103 4412 1010 1011 1011 3243 103 2007 2010 23358 1010 17727 12298 5178 3372 14243 1010 1011 1011 2018 27384 2094 2032 2012 1018 1037 103 1049 102\n",
"I0523 05:34:34.929604 140368108218240 create_pretraining_data.py:161] input_ids: 101 5262 1010 2009 2001 2680 1999 103 2732 2008 1037 19590 2220 103 2099 2018 2320 3856 2039 2006 1996 3307 1037 5024 20000 103 2751 20971 2029 1996 103 2018 10650 2013 2049 4297 103 2075 5800 1010 1998 8871 2046 6234 1998 103 6217 1012 102 16373 2161 1010 103 15301 2265 10427 1997 2303 1010 1998 15839 4196 2037 2159 2000 1996 16931 2098 103 2634 1011 10710 8871 15717 103 2068 1012 1000 16220 1000 10154 103 11759 2220 2008 2851 1010 2021 2025 2007 1037 3193 2000 103 1012 1037 17271 1999 2010 103 4412 1010 1011 1011 3243 103 2007 2010 23358 1010 17727 12298 5178 3372 14243 1010 1011 1011 2018 27384 2094 2032 2012 1018 1037 103 1049 102\n",
"INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"I0523 05:34:34.929769 140368108218240 create_pretraining_data.py:161] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"I0523 05:34:34.929909 140368108218240 create_pretraining_data.py:161] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
"INFO:tensorflow:masked_lm_positions: 7 12 13 25 30 36 45 52 53 54 68 74 81 82 93 99 103 105 125 0\n",
"I0523 05:34:34.930020 140368108218240 create_pretraining_data.py:161] masked_lm_positions: 7 12 13 25 30 36 45 52 53 54 68 74 81 82 93 99 103 105 125 0\n",
"INFO:tensorflow:masked_lm_ids: 17162 2220 4125 1997 4542 29440 20332 4233 1037 16465 2030 2682 2018 13763 5456 6644 1011 8335 1012 0\n",
"I0523 05:34:34.930125 140368108218240 create_pretraining_data.py:161] masked_lm_ids: 17162 2220 4125 1997 4542 29440 20332 4233 1037 16465 2030 2682 2018 13763 5456 6644 1011 8335 1012 0\n",
"INFO:tensorflow:masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0\n",
"I0523 05:34:34.930249 140368108218240 create_pretraining_data.py:161] masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0\n",
"INFO:tensorflow:next_sentence_labels: 0\n",
"I0523 05:34:34.930329 140368108218240 create_pretraining_data.py:161] next_sentence_labels: 0\n",
"INFO:tensorflow:*** Example ***\n",
"I0523 05:34:34.930763 140368108218240 create_pretraining_data.py:149] *** Example ***\n",
"INFO:tensorflow:tokens: [CLS] and there burst on phil ##am ##mon ' s astonished eyes a vast semi ##ci ##rcle of blue sea [MASK] ring ##ed with palaces and towers [MASK] [SEP] like most of [MASK] fellow gold - seekers , cass was super ##sti [MASK] . [SEP]\n",
"I0523 05:34:34.930938 140368108218240 create_pretraining_data.py:151] tokens: [CLS] and there burst on phil ##am ##mon ' s astonished eyes a vast semi ##ci ##rcle of blue sea [MASK] ring ##ed with palaces and towers [MASK] [SEP] like most of [MASK] fellow gold - seekers , cass was super ##sti [MASK] . [SEP]\n",
"INFO:tensorflow:input_ids: 101 1998 2045 6532 2006 6316 3286 8202 1005 1055 22741 2159 1037 6565 4100 6895 21769 1997 2630 2712 103 3614 2098 2007 22763 1998 7626 103 102 2066 2087 1997 103 3507 2751 1011 24071 1010 16220 2001 3565 16643 103 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.931124 140368108218240 create_pretraining_data.py:161] input_ids: 101 1998 2045 6532 2006 6316 3286 8202 1005 1055 22741 2159 1037 6565 4100 6895 21769 1997 2630 2712 103 3614 2098 2007 22763 1998 7626 103 102 2066 2087 1997 103 3507 2751 1011 24071 1010 16220 2001 3565 16643 103 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.931279 140368108218240 create_pretraining_data.py:161] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.931415 140368108218240 create_pretraining_data.py:161] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:masked_lm_positions: 10 20 23 27 32 39 42 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.931507 140368108218240 create_pretraining_data.py:161] masked_lm_positions: 10 20 23 27 32 39 42 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:masked_lm_ids: 22741 1010 2007 1012 2010 2001 20771 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"I0523 05:34:34.931599 140368108218240 create_pretraining_data.py:161] masked_lm_ids: 22741 1010 2007 1012 2010 2001 20771 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
"INFO:tensorflow:masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0\n",
"I0523 05:34:34.931696 140368108218240 create_pretraining_data.py:161] masked_lm_weights: 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0\n",
"INFO:tensorflow:next_sentence_labels: 1\n",
"I0523 05:34:34.931776 140368108218240 create_pretraining_data.py:161] next_sentence_labels: 1\n",
"INFO:tensorflow:Wrote 60 total instances\n",
"I0523 05:34:34.944531 140368108218240 create_pretraining_data.py:166] Wrote 60 total instances\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "qb52CZx_uk0m"
},
"source": [
"```json\n",
"# bert medium config file\n",
"{\n",
" \"hidden_size\": 512, \n",
" \"hidden_act\": \"gelu\", \n",
" \"initializer_range\": 0.02, \n",
" \"vocab_size\": 30522, \n",
" \"hidden_dropout_prob\": 0.1, \n",
" \"num_attention_heads\": 8, \n",
" \"type_vocab_size\": 2, \n",
" \"max_position_embeddings\": 512, \n",
" \"num_hidden_layers\": 8, \n",
" \"intermediate_size\": 2048, \n",
" \"attention_probs_dropout_prob\": 0.1\n",
"}\n",
"```"
]
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "SNMWppEJtUcc",
"outputId": "72acb16c-4bc5-4b46-d67a-db33f44673a9"
},
"source": [
"%%writefile pretrain.sh\n",
"python run_pretraining.py \\\n",
" --input_file=./tf_examples.tfrecord \\\n",
" --output_dir=./pretraining_output \\\n",
" --do_train=True \\\n",
" --do_eval=True \\\n",
" --bert_config_file=base-model/bert_config.json \\\n",
" --train_batch_size=32 \\\n",
" --max_seq_length=128 \\\n",
" --max_predictions_per_seq=20 \\\n",
" --num_train_steps=2000 \\\n",
" --num_warmup_steps=100 \\\n",
" --learning_rate=2e-5"
],
"execution_count": 13,
"outputs": [
{
"output_type": "stream",
"text": [
"Overwriting pretrain.sh\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "LnoQHZZuusCX",
"outputId": "ad915643-9350-44ab-dec8-aecd86f24e96"
},
"source": [
"!bash pretrain.sh"
],
"execution_count": 14,
"outputs": [
{
"output_type": "stream",
"text": [
"\u001b[1;30;43mStreaming output truncated to the last 5000 lines.\u001b[0m\n",
"INFO:tensorflow:global_step/sec: 2.8828\n",
"I0523 05:52:09.755562 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8828\n",
"INFO:tensorflow:examples/sec: 92.2495\n",
"I0523 05:52:09.755778 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2495\n",
"INFO:tensorflow:global_step/sec: 2.92069\n",
"I0523 05:52:10.097930 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.92069\n",
"INFO:tensorflow:examples/sec: 93.462\n",
"I0523 05:52:10.098340 140278000138112 tpu_estimator.py:2308] examples/sec: 93.462\n",
"INFO:tensorflow:global_step/sec: 2.90696\n",
"I0523 05:52:10.441961 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90696\n",
"INFO:tensorflow:examples/sec: 93.0227\n",
"I0523 05:52:10.442188 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0227\n",
"INFO:tensorflow:global_step/sec: 2.88043\n",
"I0523 05:52:10.789151 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88043\n",
"INFO:tensorflow:examples/sec: 92.1736\n",
"I0523 05:52:10.789379 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1736\n",
"INFO:tensorflow:global_step/sec: 2.90504\n",
"I0523 05:52:11.133340 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90504\n",
"INFO:tensorflow:examples/sec: 92.9612\n",
"I0523 05:52:11.133568 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9612\n",
"INFO:tensorflow:global_step/sec: 2.89954\n",
"I0523 05:52:11.478234 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89954\n",
"INFO:tensorflow:examples/sec: 92.7854\n",
"I0523 05:52:11.478458 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7854\n",
"INFO:tensorflow:global_step/sec: 2.87615\n",
"I0523 05:52:11.826002 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87615\n",
"INFO:tensorflow:examples/sec: 92.0367\n",
"I0523 05:52:11.826245 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0367\n",
"INFO:tensorflow:global_step/sec: 2.91442\n",
"I0523 05:52:12.169075 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91442\n",
"INFO:tensorflow:examples/sec: 93.2615\n",
"I0523 05:52:12.169335 140278000138112 tpu_estimator.py:2308] examples/sec: 93.2615\n",
"INFO:tensorflow:global_step/sec: 2.89186\n",
"I0523 05:52:12.514844 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89186\n",
"INFO:tensorflow:examples/sec: 92.5394\n",
"I0523 05:52:12.515221 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5394\n",
"INFO:tensorflow:global_step/sec: 2.88884\n",
"I0523 05:52:12.861017 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88884\n",
"INFO:tensorflow:examples/sec: 92.4428\n",
"I0523 05:52:12.861259 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4428\n",
"INFO:tensorflow:global_step/sec: 2.90451\n",
"I0523 05:52:13.205296 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90451\n",
"INFO:tensorflow:examples/sec: 92.9444\n",
"I0523 05:52:13.205528 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9444\n",
"INFO:tensorflow:global_step/sec: 2.91856\n",
"I0523 05:52:13.547933 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91856\n",
"INFO:tensorflow:examples/sec: 93.3941\n",
"I0523 05:52:13.548184 140278000138112 tpu_estimator.py:2308] examples/sec: 93.3941\n",
"INFO:tensorflow:global_step/sec: 2.87302\n",
"I0523 05:52:13.896009 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87302\n",
"INFO:tensorflow:examples/sec: 91.9365\n",
"I0523 05:52:13.896391 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9365\n",
"INFO:tensorflow:global_step/sec: 2.89658\n",
"I0523 05:52:14.241232 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89658\n",
"INFO:tensorflow:examples/sec: 92.6905\n",
"I0523 05:52:14.241457 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6905\n",
"INFO:tensorflow:global_step/sec: 2.89771\n",
"I0523 05:52:14.586385 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89771\n",
"INFO:tensorflow:examples/sec: 92.7267\n",
"I0523 05:52:14.586797 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7267\n",
"INFO:tensorflow:global_step/sec: 2.89791\n",
"I0523 05:52:14.931406 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89791\n",
"INFO:tensorflow:examples/sec: 92.733\n",
"I0523 05:52:14.931682 140278000138112 tpu_estimator.py:2308] examples/sec: 92.733\n",
"INFO:tensorflow:global_step/sec: 2.8936\n",
"I0523 05:52:15.276997 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8936\n",
"INFO:tensorflow:examples/sec: 92.5952\n",
"I0523 05:52:15.277249 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5952\n",
"INFO:tensorflow:global_step/sec: 2.88811\n",
"I0523 05:52:15.623252 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88811\n",
"INFO:tensorflow:examples/sec: 92.4195\n",
"I0523 05:52:15.623486 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4195\n",
"INFO:tensorflow:global_step/sec: 2.92756\n",
"I0523 05:52:15.964812 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.92756\n",
"INFO:tensorflow:examples/sec: 93.6819\n",
"I0523 05:52:15.965069 140278000138112 tpu_estimator.py:2308] examples/sec: 93.6819\n",
"INFO:tensorflow:global_step/sec: 2.88829\n",
"I0523 05:52:16.311057 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88829\n",
"INFO:tensorflow:examples/sec: 92.4252\n",
"I0523 05:52:16.311289 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4252\n",
"INFO:tensorflow:global_step/sec: 2.89422\n",
"I0523 05:52:16.656547 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89422\n",
"INFO:tensorflow:examples/sec: 92.615\n",
"I0523 05:52:16.656789 140278000138112 tpu_estimator.py:2308] examples/sec: 92.615\n",
"INFO:tensorflow:global_step/sec: 2.93224\n",
"I0523 05:52:16.997601 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.93224\n",
"INFO:tensorflow:examples/sec: 93.8316\n",
"I0523 05:52:16.997836 140278000138112 tpu_estimator.py:2308] examples/sec: 93.8316\n",
"INFO:tensorflow:global_step/sec: 2.87692\n",
"I0523 05:52:17.345199 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87692\n",
"INFO:tensorflow:examples/sec: 92.0614\n",
"I0523 05:52:17.345434 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0614\n",
"INFO:tensorflow:global_step/sec: 2.88107\n",
"I0523 05:52:17.692302 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88107\n",
"INFO:tensorflow:examples/sec: 92.1941\n",
"I0523 05:52:17.692554 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1941\n",
"INFO:tensorflow:global_step/sec: 2.88692\n",
"I0523 05:52:18.038681 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88692\n",
"INFO:tensorflow:examples/sec: 92.3815\n",
"I0523 05:52:18.038919 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3815\n",
"INFO:tensorflow:global_step/sec: 2.86305\n",
"I0523 05:52:18.387996 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86305\n",
"INFO:tensorflow:examples/sec: 91.6177\n",
"I0523 05:52:18.388485 140278000138112 tpu_estimator.py:2308] examples/sec: 91.6177\n",
"INFO:tensorflow:global_step/sec: 2.87882\n",
"I0523 05:52:18.735353 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87882\n",
"INFO:tensorflow:examples/sec: 92.1221\n",
"I0523 05:52:18.735592 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1221\n",
"INFO:tensorflow:global_step/sec: 2.88402\n",
"I0523 05:52:19.082088 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88402\n",
"INFO:tensorflow:examples/sec: 92.2887\n",
"I0523 05:52:19.082336 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2887\n",
"INFO:tensorflow:global_step/sec: 2.88521\n",
"I0523 05:52:19.428648 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88521\n",
"INFO:tensorflow:examples/sec: 92.3268\n",
"I0523 05:52:19.428877 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3268\n",
"INFO:tensorflow:global_step/sec: 2.89599\n",
"I0523 05:52:19.773977 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89599\n",
"INFO:tensorflow:examples/sec: 92.6718\n",
"I0523 05:52:19.774357 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6718\n",
"INFO:tensorflow:global_step/sec: 2.90478\n",
"I0523 05:52:20.118242 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90478\n",
"INFO:tensorflow:examples/sec: 92.9529\n",
"I0523 05:52:20.118478 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9529\n",
"INFO:tensorflow:global_step/sec: 2.88664\n",
"I0523 05:52:20.464641 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88664\n",
"INFO:tensorflow:examples/sec: 92.3723\n",
"I0523 05:52:20.464875 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3723\n",
"INFO:tensorflow:global_step/sec: 2.90169\n",
"I0523 05:52:20.809261 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90169\n",
"INFO:tensorflow:examples/sec: 92.8542\n",
"I0523 05:52:20.809486 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8542\n",
"INFO:tensorflow:global_step/sec: 2.88103\n",
"I0523 05:52:21.156387 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88103\n",
"INFO:tensorflow:examples/sec: 92.193\n",
"I0523 05:52:21.156632 140278000138112 tpu_estimator.py:2308] examples/sec: 92.193\n",
"INFO:tensorflow:global_step/sec: 2.87931\n",
"I0523 05:52:21.503666 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87931\n",
"INFO:tensorflow:examples/sec: 92.1378\n",
"I0523 05:52:21.503905 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1378\n",
"INFO:tensorflow:global_step/sec: 2.89632\n",
"I0523 05:52:21.848938 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89632\n",
"INFO:tensorflow:examples/sec: 92.6823\n",
"I0523 05:52:21.849359 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6823\n",
"INFO:tensorflow:global_step/sec: 2.90188\n",
"I0523 05:52:22.193547 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90188\n",
"INFO:tensorflow:examples/sec: 92.86\n",
"I0523 05:52:22.193792 140278000138112 tpu_estimator.py:2308] examples/sec: 92.86\n",
"INFO:tensorflow:global_step/sec: 2.90614\n",
"I0523 05:52:22.537628 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90614\n",
"INFO:tensorflow:examples/sec: 92.9965\n",
"I0523 05:52:22.537862 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9965\n",
"INFO:tensorflow:global_step/sec: 2.89093\n",
"I0523 05:52:22.883555 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89093\n",
"INFO:tensorflow:examples/sec: 92.5099\n",
"I0523 05:52:22.883796 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5099\n",
"INFO:tensorflow:global_step/sec: 2.89082\n",
"I0523 05:52:23.229480 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89082\n",
"INFO:tensorflow:examples/sec: 92.5062\n",
"I0523 05:52:23.229735 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5062\n",
"INFO:tensorflow:global_step/sec: 2.91078\n",
"I0523 05:52:23.573023 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91078\n",
"INFO:tensorflow:examples/sec: 93.145\n",
"I0523 05:52:23.573431 140278000138112 tpu_estimator.py:2308] examples/sec: 93.145\n",
"INFO:tensorflow:global_step/sec: 2.89024\n",
"I0523 05:52:23.918985 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89024\n",
"INFO:tensorflow:examples/sec: 92.4876\n",
"I0523 05:52:23.919291 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4876\n",
"INFO:tensorflow:global_step/sec: 2.89847\n",
"I0523 05:52:24.264039 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89847\n",
"INFO:tensorflow:examples/sec: 92.7511\n",
"I0523 05:52:24.264280 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7511\n",
"INFO:tensorflow:global_step/sec: 2.91017\n",
"I0523 05:52:24.607675 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91017\n",
"INFO:tensorflow:examples/sec: 93.1253\n",
"I0523 05:52:24.607910 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1253\n",
"INFO:tensorflow:global_step/sec: 2.8928\n",
"I0523 05:52:24.953341 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8928\n",
"INFO:tensorflow:examples/sec: 92.5696\n",
"I0523 05:52:24.953564 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5696\n",
"INFO:tensorflow:global_step/sec: 2.89873\n",
"I0523 05:52:25.298302 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89873\n",
"INFO:tensorflow:examples/sec: 92.7593\n",
"I0523 05:52:25.298518 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7593\n",
"INFO:tensorflow:global_step/sec: 2.9302\n",
"I0523 05:52:25.639586 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.9302\n",
"INFO:tensorflow:examples/sec: 93.7664\n",
"I0523 05:52:25.639820 140278000138112 tpu_estimator.py:2308] examples/sec: 93.7664\n",
"INFO:tensorflow:global_step/sec: 2.87442\n",
"I0523 05:52:25.987473 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87442\n",
"INFO:tensorflow:examples/sec: 91.9813\n",
"I0523 05:52:25.987732 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9813\n",
"INFO:tensorflow:global_step/sec: 2.8911\n",
"I0523 05:52:26.333380 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8911\n",
"INFO:tensorflow:examples/sec: 92.5153\n",
"I0523 05:52:26.333632 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5153\n",
"INFO:tensorflow:global_step/sec: 2.89554\n",
"I0523 05:52:26.678754 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89554\n",
"INFO:tensorflow:examples/sec: 92.6574\n",
"I0523 05:52:26.679023 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6574\n",
"INFO:tensorflow:global_step/sec: 2.88486\n",
"I0523 05:52:27.025371 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88486\n",
"INFO:tensorflow:examples/sec: 92.3155\n",
"I0523 05:52:27.025605 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3155\n",
"INFO:tensorflow:global_step/sec: 2.902\n",
"I0523 05:52:27.369965 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.902\n",
"INFO:tensorflow:examples/sec: 92.8639\n",
"I0523 05:52:27.370199 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8639\n",
"INFO:tensorflow:global_step/sec: 2.90225\n",
"I0523 05:52:27.714515 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90225\n",
"INFO:tensorflow:examples/sec: 92.8722\n",
"I0523 05:52:27.714895 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8722\n",
"INFO:tensorflow:global_step/sec: 2.8756\n",
"I0523 05:52:28.062335 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8756\n",
"INFO:tensorflow:examples/sec: 92.0192\n",
"I0523 05:52:28.062612 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0192\n",
"INFO:tensorflow:global_step/sec: 2.89272\n",
"I0523 05:52:28.407973 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89272\n",
"INFO:tensorflow:examples/sec: 92.5669\n",
"I0523 05:52:28.408336 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5669\n",
"INFO:tensorflow:global_step/sec: 2.89306\n",
"I0523 05:52:28.753615 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89306\n",
"INFO:tensorflow:examples/sec: 92.5781\n",
"I0523 05:52:28.753837 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5781\n",
"INFO:tensorflow:global_step/sec: 2.91178\n",
"I0523 05:52:29.097065 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91178\n",
"INFO:tensorflow:examples/sec: 93.1769\n",
"I0523 05:52:29.097279 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1769\n",
"INFO:tensorflow:global_step/sec: 2.90789\n",
"I0523 05:52:29.440959 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90789\n",
"INFO:tensorflow:examples/sec: 93.0526\n",
"I0523 05:52:29.441197 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0526\n",
"INFO:tensorflow:global_step/sec: 2.87203\n",
"I0523 05:52:29.789154 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87203\n",
"INFO:tensorflow:examples/sec: 91.9049\n",
"I0523 05:52:29.789386 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9049\n",
"INFO:tensorflow:global_step/sec: 2.90426\n",
"I0523 05:52:30.133442 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90426\n",
"INFO:tensorflow:examples/sec: 92.9365\n",
"I0523 05:52:30.133834 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9365\n",
"INFO:tensorflow:global_step/sec: 2.88054\n",
"I0523 05:52:30.480615 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88054\n",
"INFO:tensorflow:examples/sec: 92.1771\n",
"I0523 05:52:30.480882 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1771\n",
"INFO:tensorflow:global_step/sec: 2.85618\n",
"I0523 05:52:30.830741 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.85618\n",
"INFO:tensorflow:examples/sec: 91.3977\n",
"I0523 05:52:30.830979 140278000138112 tpu_estimator.py:2308] examples/sec: 91.3977\n",
"INFO:tensorflow:global_step/sec: 2.91829\n",
"I0523 05:52:31.173388 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91829\n",
"INFO:tensorflow:examples/sec: 93.3854\n",
"I0523 05:52:31.173609 140278000138112 tpu_estimator.py:2308] examples/sec: 93.3854\n",
"INFO:tensorflow:global_step/sec: 2.87946\n",
"I0523 05:52:31.520747 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87946\n",
"INFO:tensorflow:examples/sec: 92.1428\n",
"I0523 05:52:31.521049 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1428\n",
"INFO:tensorflow:global_step/sec: 2.8852\n",
"I0523 05:52:31.867288 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8852\n",
"INFO:tensorflow:examples/sec: 92.3263\n",
"I0523 05:52:31.867541 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3263\n",
"INFO:tensorflow:global_step/sec: 2.89326\n",
"I0523 05:52:32.212908 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89326\n",
"INFO:tensorflow:examples/sec: 92.5842\n",
"I0523 05:52:32.213176 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5842\n",
"INFO:tensorflow:global_step/sec: 2.89462\n",
"I0523 05:52:32.558388 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89462\n",
"INFO:tensorflow:examples/sec: 92.6279\n",
"I0523 05:52:32.558611 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6279\n",
"INFO:tensorflow:global_step/sec: 2.89653\n",
"I0523 05:52:32.903621 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89653\n",
"INFO:tensorflow:examples/sec: 92.689\n",
"I0523 05:52:32.904005 140278000138112 tpu_estimator.py:2308] examples/sec: 92.689\n",
"INFO:tensorflow:global_step/sec: 2.87581\n",
"I0523 05:52:33.251378 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87581\n",
"INFO:tensorflow:examples/sec: 92.026\n",
"I0523 05:52:33.251864 140278000138112 tpu_estimator.py:2308] examples/sec: 92.026\n",
"INFO:tensorflow:global_step/sec: 2.89223\n",
"I0523 05:52:33.597118 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89223\n",
"INFO:tensorflow:examples/sec: 92.5515\n",
"I0523 05:52:33.597511 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5515\n",
"INFO:tensorflow:global_step/sec: 2.89064\n",
"I0523 05:52:33.943073 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89064\n",
"INFO:tensorflow:examples/sec: 92.5005\n",
"I0523 05:52:33.943442 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5005\n",
"INFO:tensorflow:global_step/sec: 2.88638\n",
"I0523 05:52:34.289491 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88638\n",
"INFO:tensorflow:examples/sec: 92.3641\n",
"I0523 05:52:34.289865 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3641\n",
"INFO:tensorflow:global_step/sec: 2.92539\n",
"I0523 05:52:34.631347 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.92539\n",
"INFO:tensorflow:examples/sec: 93.6125\n",
"I0523 05:52:34.631767 140278000138112 tpu_estimator.py:2308] examples/sec: 93.6125\n",
"INFO:tensorflow:global_step/sec: 2.8921\n",
"I0523 05:52:34.977124 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8921\n",
"INFO:tensorflow:examples/sec: 92.5471\n",
"I0523 05:52:34.977350 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5471\n",
"INFO:tensorflow:global_step/sec: 2.90154\n",
"I0523 05:52:35.321736 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90154\n",
"INFO:tensorflow:examples/sec: 92.8492\n",
"I0523 05:52:35.322110 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8492\n",
"INFO:tensorflow:global_step/sec: 2.90735\n",
"I0523 05:52:35.665695 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90735\n",
"INFO:tensorflow:examples/sec: 93.0351\n",
"I0523 05:52:35.666057 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0351\n",
"INFO:tensorflow:global_step/sec: 2.90778\n",
"I0523 05:52:36.009605 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90778\n",
"INFO:tensorflow:examples/sec: 93.049\n",
"I0523 05:52:36.009968 140278000138112 tpu_estimator.py:2308] examples/sec: 93.049\n",
"INFO:tensorflow:global_step/sec: 2.88314\n",
"I0523 05:52:36.356446 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88314\n",
"INFO:tensorflow:examples/sec: 92.2604\n",
"I0523 05:52:36.356800 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2604\n",
"INFO:tensorflow:global_step/sec: 2.88638\n",
"I0523 05:52:36.702914 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88638\n",
"INFO:tensorflow:examples/sec: 92.3641\n",
"I0523 05:52:36.703176 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3641\n",
"INFO:tensorflow:global_step/sec: 2.88544\n",
"I0523 05:52:37.049465 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88544\n",
"INFO:tensorflow:examples/sec: 92.3341\n",
"I0523 05:52:37.049876 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3341\n",
"INFO:tensorflow:global_step/sec: 2.88032\n",
"I0523 05:52:37.396696 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88032\n",
"INFO:tensorflow:examples/sec: 92.1703\n",
"I0523 05:52:37.396923 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1703\n",
"INFO:tensorflow:global_step/sec: 2.9064\n",
"I0523 05:52:37.740761 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.9064\n",
"INFO:tensorflow:examples/sec: 93.0049\n",
"I0523 05:52:37.741146 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0049\n",
"INFO:tensorflow:global_step/sec: 2.88397\n",
"I0523 05:52:38.087499 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88397\n",
"INFO:tensorflow:examples/sec: 92.2872\n",
"I0523 05:52:38.087862 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2872\n",
"INFO:tensorflow:global_step/sec: 2.91375\n",
"I0523 05:52:38.430658 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91375\n",
"INFO:tensorflow:examples/sec: 93.24\n",
"I0523 05:52:38.431017 140278000138112 tpu_estimator.py:2308] examples/sec: 93.24\n",
"INFO:tensorflow:global_step/sec: 2.89176\n",
"I0523 05:52:38.776470 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89176\n",
"INFO:tensorflow:examples/sec: 92.5364\n",
"I0523 05:52:38.776690 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5364\n",
"INFO:tensorflow:global_step/sec: 2.89135\n",
"I0523 05:52:39.122330 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89135\n",
"INFO:tensorflow:examples/sec: 92.5233\n",
"I0523 05:52:39.122552 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5233\n",
"INFO:tensorflow:global_step/sec: 2.89563\n",
"I0523 05:52:39.467712 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89563\n",
"INFO:tensorflow:examples/sec: 92.6602\n",
"I0523 05:52:39.468097 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6602\n",
"INFO:tensorflow:global_step/sec: 2.89923\n",
"I0523 05:52:39.812607 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89923\n",
"INFO:tensorflow:examples/sec: 92.7754\n",
"I0523 05:52:39.813000 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7754\n",
"INFO:tensorflow:global_step/sec: 2.87645\n",
"I0523 05:52:40.160259 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87645\n",
"INFO:tensorflow:examples/sec: 92.0465\n",
"I0523 05:52:40.160478 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0465\n",
"INFO:tensorflow:global_step/sec: 2.87572\n",
"I0523 05:52:40.508068 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87572\n",
"INFO:tensorflow:examples/sec: 92.0231\n",
"I0523 05:52:40.508433 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0231\n",
"INFO:tensorflow:global_step/sec: 2.89298\n",
"I0523 05:52:40.853662 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89298\n",
"INFO:tensorflow:examples/sec: 92.5755\n",
"I0523 05:52:40.853886 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5755\n",
"INFO:tensorflow:global_step/sec: 2.8869\n",
"I0523 05:52:41.200068 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8869\n",
"INFO:tensorflow:examples/sec: 92.3809\n",
"I0523 05:52:41.200474 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3809\n",
"INFO:tensorflow:global_step/sec: 2.90631\n",
"I0523 05:52:41.544155 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90631\n",
"INFO:tensorflow:examples/sec: 93.0018\n",
"I0523 05:52:41.544553 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0018\n",
"INFO:tensorflow:global_step/sec: 2.91045\n",
"I0523 05:52:41.887753 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91045\n",
"INFO:tensorflow:examples/sec: 93.1343\n",
"I0523 05:52:41.888145 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1343\n",
"INFO:tensorflow:global_step/sec: 2.88457\n",
"I0523 05:52:42.234385 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88457\n",
"INFO:tensorflow:examples/sec: 92.3063\n",
"I0523 05:52:42.234608 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3063\n",
"INFO:tensorflow:global_step/sec: 2.90252\n",
"I0523 05:52:42.578928 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90252\n",
"INFO:tensorflow:examples/sec: 92.8808\n",
"I0523 05:52:42.579293 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8808\n",
"INFO:tensorflow:global_step/sec: 2.87627\n",
"I0523 05:52:42.926575 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87627\n",
"INFO:tensorflow:examples/sec: 92.0405\n",
"I0523 05:52:42.926917 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0405\n",
"INFO:tensorflow:global_step/sec: 2.8903\n",
"I0523 05:52:43.272581 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8903\n",
"INFO:tensorflow:examples/sec: 92.4895\n",
"I0523 05:52:43.273000 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4895\n",
"INFO:tensorflow:global_step/sec: 2.91103\n",
"I0523 05:52:43.616106 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91103\n",
"INFO:tensorflow:examples/sec: 93.153\n",
"I0523 05:52:43.616325 140278000138112 tpu_estimator.py:2308] examples/sec: 93.153\n",
"INFO:tensorflow:global_step/sec: 2.89488\n",
"I0523 05:52:43.961548 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89488\n",
"INFO:tensorflow:examples/sec: 92.636\n",
"I0523 05:52:43.961864 140278000138112 tpu_estimator.py:2308] examples/sec: 92.636\n",
"INFO:tensorflow:global_step/sec: 2.8705\n",
"I0523 05:52:44.309906 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8705\n",
"INFO:tensorflow:examples/sec: 91.8562\n",
"I0523 05:52:44.310369 140278000138112 tpu_estimator.py:2308] examples/sec: 91.8562\n",
"INFO:tensorflow:global_step/sec: 2.89839\n",
"I0523 05:52:44.654999 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89839\n",
"INFO:tensorflow:examples/sec: 92.7484\n",
"I0523 05:52:44.655299 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7484\n",
"INFO:tensorflow:global_step/sec: 2.882\n",
"I0523 05:52:45.001931 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.882\n",
"INFO:tensorflow:examples/sec: 92.2239\n",
"I0523 05:52:45.002328 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2239\n",
"INFO:tensorflow:global_step/sec: 2.90395\n",
"I0523 05:52:45.346310 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90395\n",
"INFO:tensorflow:examples/sec: 92.9263\n",
"I0523 05:52:45.346745 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9263\n",
"INFO:tensorflow:global_step/sec: 2.86969\n",
"I0523 05:52:45.694925 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86969\n",
"INFO:tensorflow:examples/sec: 91.8302\n",
"I0523 05:52:45.695346 140278000138112 tpu_estimator.py:2308] examples/sec: 91.8302\n",
"INFO:tensorflow:global_step/sec: 2.87664\n",
"I0523 05:52:46.042373 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87664\n",
"INFO:tensorflow:examples/sec: 92.0523\n",
"I0523 05:52:46.042789 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0523\n",
"INFO:tensorflow:global_step/sec: 2.91427\n",
"I0523 05:52:46.385517 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91427\n",
"INFO:tensorflow:examples/sec: 93.2566\n",
"I0523 05:52:46.385912 140278000138112 tpu_estimator.py:2308] examples/sec: 93.2566\n",
"INFO:tensorflow:global_step/sec: 2.87687\n",
"I0523 05:52:46.733132 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87687\n",
"INFO:tensorflow:examples/sec: 92.0598\n",
"I0523 05:52:46.733371 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0598\n",
"INFO:tensorflow:global_step/sec: 2.89448\n",
"I0523 05:52:47.078591 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89448\n",
"INFO:tensorflow:examples/sec: 92.6235\n",
"I0523 05:52:47.079086 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6235\n",
"INFO:tensorflow:global_step/sec: 2.89894\n",
"I0523 05:52:47.423543 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89894\n",
"INFO:tensorflow:examples/sec: 92.7659\n",
"I0523 05:52:47.423771 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7659\n",
"INFO:tensorflow:global_step/sec: 2.89572\n",
"I0523 05:52:47.768909 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89572\n",
"INFO:tensorflow:examples/sec: 92.6631\n",
"I0523 05:52:47.769199 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6631\n",
"INFO:tensorflow:global_step/sec: 2.90751\n",
"I0523 05:52:48.112834 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90751\n",
"INFO:tensorflow:examples/sec: 93.0403\n",
"I0523 05:52:48.113216 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0403\n",
"INFO:tensorflow:global_step/sec: 2.89454\n",
"I0523 05:52:48.458300 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89454\n",
"INFO:tensorflow:examples/sec: 92.6254\n",
"I0523 05:52:48.458535 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6254\n",
"INFO:tensorflow:global_step/sec: 2.89864\n",
"I0523 05:52:48.803276 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89864\n",
"INFO:tensorflow:examples/sec: 92.7564\n",
"I0523 05:52:48.803501 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7564\n",
"INFO:tensorflow:global_step/sec: 2.89247\n",
"I0523 05:52:49.149034 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89247\n",
"INFO:tensorflow:examples/sec: 92.5592\n",
"I0523 05:52:49.149395 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5592\n",
"INFO:tensorflow:global_step/sec: 2.93234\n",
"I0523 05:52:49.490045 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.93234\n",
"INFO:tensorflow:examples/sec: 93.8349\n",
"I0523 05:52:49.490268 140278000138112 tpu_estimator.py:2308] examples/sec: 93.8349\n",
"INFO:tensorflow:global_step/sec: 2.87984\n",
"I0523 05:52:49.837277 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87984\n",
"INFO:tensorflow:examples/sec: 92.1548\n",
"I0523 05:52:49.837496 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1548\n",
"INFO:tensorflow:global_step/sec: 2.89846\n",
"I0523 05:52:50.182281 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89846\n",
"INFO:tensorflow:examples/sec: 92.7508\n",
"I0523 05:52:50.182497 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7508\n",
"INFO:tensorflow:global_step/sec: 2.89738\n",
"I0523 05:52:50.527417 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89738\n",
"INFO:tensorflow:examples/sec: 92.7162\n",
"I0523 05:52:50.527643 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7162\n",
"INFO:tensorflow:global_step/sec: 2.86145\n",
"I0523 05:52:50.876929 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86145\n",
"INFO:tensorflow:examples/sec: 91.5664\n",
"I0523 05:52:50.877240 140278000138112 tpu_estimator.py:2308] examples/sec: 91.5664\n",
"INFO:tensorflow:global_step/sec: 2.91184\n",
"I0523 05:52:51.220391 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91184\n",
"INFO:tensorflow:examples/sec: 93.1789\n",
"I0523 05:52:51.220619 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1789\n",
"INFO:tensorflow:global_step/sec: 2.88965\n",
"I0523 05:52:51.566410 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88965\n",
"INFO:tensorflow:examples/sec: 92.4687\n",
"I0523 05:52:51.566656 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4687\n",
"INFO:tensorflow:global_step/sec: 2.90089\n",
"I0523 05:52:51.911259 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90089\n",
"INFO:tensorflow:examples/sec: 92.8285\n",
"I0523 05:52:51.911618 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8285\n",
"INFO:tensorflow:global_step/sec: 2.84881\n",
"I0523 05:52:52.262191 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.84881\n",
"INFO:tensorflow:examples/sec: 91.1618\n",
"I0523 05:52:52.262625 140278000138112 tpu_estimator.py:2308] examples/sec: 91.1618\n",
"INFO:tensorflow:global_step/sec: 2.86284\n",
"I0523 05:52:52.611450 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86284\n",
"INFO:tensorflow:examples/sec: 91.6109\n",
"I0523 05:52:52.611743 140278000138112 tpu_estimator.py:2308] examples/sec: 91.6109\n",
"INFO:tensorflow:global_step/sec: 2.88132\n",
"I0523 05:52:52.958522 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88132\n",
"INFO:tensorflow:examples/sec: 92.2022\n",
"I0523 05:52:52.958772 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2022\n",
"INFO:tensorflow:global_step/sec: 2.8654\n",
"I0523 05:52:53.307488 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8654\n",
"INFO:tensorflow:examples/sec: 91.6929\n",
"I0523 05:52:53.307746 140278000138112 tpu_estimator.py:2308] examples/sec: 91.6929\n",
"INFO:tensorflow:global_step/sec: 2.88515\n",
"I0523 05:52:53.654146 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88515\n",
"INFO:tensorflow:examples/sec: 92.3247\n",
"I0523 05:52:53.654404 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3247\n",
"INFO:tensorflow:global_step/sec: 2.88032\n",
"I0523 05:52:54.001284 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88032\n",
"INFO:tensorflow:examples/sec: 92.1704\n",
"I0523 05:52:54.001659 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1704\n",
"INFO:tensorflow:global_step/sec: 2.89644\n",
"I0523 05:52:54.346538 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89644\n",
"INFO:tensorflow:examples/sec: 92.6861\n",
"I0523 05:52:54.346902 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6861\n",
"INFO:tensorflow:global_step/sec: 2.88299\n",
"I0523 05:52:54.693397 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88299\n",
"INFO:tensorflow:examples/sec: 92.2558\n",
"I0523 05:52:54.693615 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2558\n",
"INFO:tensorflow:global_step/sec: 2.90069\n",
"I0523 05:52:55.038255 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90069\n",
"INFO:tensorflow:examples/sec: 92.822\n",
"I0523 05:52:55.038496 140278000138112 tpu_estimator.py:2308] examples/sec: 92.822\n",
"INFO:tensorflow:global_step/sec: 2.89022\n",
"I0523 05:52:55.384140 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89022\n",
"INFO:tensorflow:examples/sec: 92.4869\n",
"I0523 05:52:55.384379 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4869\n",
"INFO:tensorflow:global_step/sec: 2.90711\n",
"I0523 05:52:55.728134 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90711\n",
"INFO:tensorflow:examples/sec: 93.0275\n",
"I0523 05:52:55.728395 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0275\n",
"INFO:tensorflow:global_step/sec: 2.88946\n",
"I0523 05:52:56.074194 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88946\n",
"INFO:tensorflow:examples/sec: 92.4629\n",
"I0523 05:52:56.074563 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4629\n",
"INFO:tensorflow:global_step/sec: 2.89865\n",
"I0523 05:52:56.419187 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89865\n",
"INFO:tensorflow:examples/sec: 92.7569\n",
"I0523 05:52:56.419420 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7569\n",
"INFO:tensorflow:global_step/sec: 2.90131\n",
"I0523 05:52:56.763869 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90131\n",
"INFO:tensorflow:examples/sec: 92.8419\n",
"I0523 05:52:56.764098 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8419\n",
"INFO:tensorflow:global_step/sec: 2.89566\n",
"I0523 05:52:57.109233 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89566\n",
"INFO:tensorflow:examples/sec: 92.6611\n",
"I0523 05:52:57.109494 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6611\n",
"INFO:tensorflow:global_step/sec: 2.87378\n",
"I0523 05:52:57.457269 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87378\n",
"INFO:tensorflow:examples/sec: 91.961\n",
"I0523 05:52:57.457797 140278000138112 tpu_estimator.py:2308] examples/sec: 91.961\n",
"INFO:tensorflow:global_step/sec: 2.89385\n",
"I0523 05:52:57.802741 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89385\n",
"INFO:tensorflow:examples/sec: 92.6034\n",
"I0523 05:52:57.803009 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6034\n",
"INFO:tensorflow:global_step/sec: 2.84479\n",
"I0523 05:52:58.154227 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.84479\n",
"INFO:tensorflow:examples/sec: 91.0333\n",
"I0523 05:52:58.154607 140278000138112 tpu_estimator.py:2308] examples/sec: 91.0333\n",
"INFO:tensorflow:global_step/sec: 2.87401\n",
"I0523 05:52:58.502207 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87401\n",
"INFO:tensorflow:examples/sec: 91.9682\n",
"I0523 05:52:58.502443 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9682\n",
"INFO:tensorflow:global_step/sec: 2.90251\n",
"I0523 05:52:58.846772 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90251\n",
"INFO:tensorflow:examples/sec: 92.8803\n",
"I0523 05:52:58.847115 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8803\n",
"INFO:tensorflow:global_step/sec: 2.88137\n",
"I0523 05:52:59.193789 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88137\n",
"INFO:tensorflow:examples/sec: 92.2039\n",
"I0523 05:52:59.194162 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2039\n",
"INFO:tensorflow:global_step/sec: 2.90513\n",
"I0523 05:52:59.538018 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90513\n",
"INFO:tensorflow:examples/sec: 92.9643\n",
"I0523 05:52:59.538401 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9643\n",
"INFO:tensorflow:global_step/sec: 2.86493\n",
"I0523 05:52:59.887129 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86493\n",
"INFO:tensorflow:examples/sec: 91.6778\n",
"I0523 05:52:59.887420 140278000138112 tpu_estimator.py:2308] examples/sec: 91.6778\n",
"INFO:tensorflow:global_step/sec: 2.90076\n",
"I0523 05:53:00.231803 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90076\n",
"INFO:tensorflow:examples/sec: 92.8244\n",
"I0523 05:53:00.232054 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8244\n",
"INFO:tensorflow:global_step/sec: 2.8844\n",
"I0523 05:53:00.578469 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8844\n",
"INFO:tensorflow:examples/sec: 92.3007\n",
"I0523 05:53:00.578739 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3007\n",
"INFO:tensorflow:global_step/sec: 2.88906\n",
"I0523 05:53:00.924615 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88906\n",
"INFO:tensorflow:examples/sec: 92.4498\n",
"I0523 05:53:00.925024 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4498\n",
"INFO:tensorflow:Saving checkpoints for 1000 into ./pretraining_output/model.ckpt.\n",
"I0523 05:53:01.269391 140278000138112 basic_session_run_hooks.py:606] Saving checkpoints for 1000 into ./pretraining_output/model.ckpt.\n",
"INFO:tensorflow:global_step/sec: 0.415795\n",
"I0523 05:53:03.329636 140278000138112 tpu_estimator.py:2307] global_step/sec: 0.415795\n",
"INFO:tensorflow:examples/sec: 13.3054\n",
"I0523 05:53:03.329900 140278000138112 tpu_estimator.py:2308] examples/sec: 13.3054\n",
"INFO:tensorflow:global_step/sec: 2.59832\n",
"I0523 05:53:03.714528 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.59832\n",
"INFO:tensorflow:examples/sec: 83.1464\n",
"I0523 05:53:03.714772 140278000138112 tpu_estimator.py:2308] examples/sec: 83.1464\n",
"INFO:tensorflow:global_step/sec: 2.912\n",
"I0523 05:53:04.057904 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.912\n",
"INFO:tensorflow:examples/sec: 93.1841\n",
"I0523 05:53:04.058153 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1841\n",
"INFO:tensorflow:global_step/sec: 2.90216\n",
"I0523 05:53:04.402485 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90216\n",
"INFO:tensorflow:examples/sec: 92.8691\n",
"I0523 05:53:04.402871 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8691\n",
"INFO:tensorflow:global_step/sec: 2.86336\n",
"I0523 05:53:04.751737 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86336\n",
"INFO:tensorflow:examples/sec: 91.6275\n",
"I0523 05:53:04.752398 140278000138112 tpu_estimator.py:2308] examples/sec: 91.6275\n",
"INFO:tensorflow:global_step/sec: 2.92576\n",
"I0523 05:53:05.093529 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.92576\n",
"INFO:tensorflow:examples/sec: 93.6242\n",
"I0523 05:53:05.093740 140278000138112 tpu_estimator.py:2308] examples/sec: 93.6242\n",
"INFO:tensorflow:global_step/sec: 2.91071\n",
"I0523 05:53:05.437091 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91071\n",
"INFO:tensorflow:examples/sec: 93.1426\n",
"I0523 05:53:05.437331 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1426\n",
"INFO:tensorflow:global_step/sec: 2.8805\n",
"I0523 05:53:05.784225 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8805\n",
"INFO:tensorflow:examples/sec: 92.1759\n",
"I0523 05:53:05.784572 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1759\n",
"INFO:tensorflow:global_step/sec: 2.90514\n",
"I0523 05:53:06.128453 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90514\n",
"INFO:tensorflow:examples/sec: 92.9643\n",
"I0523 05:53:06.128668 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9643\n",
"INFO:tensorflow:global_step/sec: 2.87969\n",
"I0523 05:53:06.475711 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87969\n",
"INFO:tensorflow:examples/sec: 92.15\n",
"I0523 05:53:06.475933 140278000138112 tpu_estimator.py:2308] examples/sec: 92.15\n",
"INFO:tensorflow:global_step/sec: 2.88978\n",
"I0523 05:53:06.821772 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88978\n",
"INFO:tensorflow:examples/sec: 92.473\n",
"I0523 05:53:06.822007 140278000138112 tpu_estimator.py:2308] examples/sec: 92.473\n",
"INFO:tensorflow:global_step/sec: 2.89\n",
"I0523 05:53:07.167776 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89\n",
"INFO:tensorflow:examples/sec: 92.48\n",
"I0523 05:53:07.168036 140278000138112 tpu_estimator.py:2308] examples/sec: 92.48\n",
"INFO:tensorflow:global_step/sec: 2.87961\n",
"I0523 05:53:07.515077 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87961\n",
"INFO:tensorflow:examples/sec: 92.1474\n",
"I0523 05:53:07.515496 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1474\n",
"INFO:tensorflow:global_step/sec: 2.89234\n",
"I0523 05:53:07.860855 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89234\n",
"INFO:tensorflow:examples/sec: 92.5548\n",
"I0523 05:53:07.861263 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5548\n",
"INFO:tensorflow:global_step/sec: 2.90286\n",
"I0523 05:53:08.205281 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90286\n",
"INFO:tensorflow:examples/sec: 92.8915\n",
"I0523 05:53:08.205547 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8915\n",
"INFO:tensorflow:global_step/sec: 2.90142\n",
"I0523 05:53:08.549993 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90142\n",
"INFO:tensorflow:examples/sec: 92.8453\n",
"I0523 05:53:08.550450 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8453\n",
"INFO:tensorflow:global_step/sec: 2.88913\n",
"I0523 05:53:08.896093 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88913\n",
"INFO:tensorflow:examples/sec: 92.4521\n",
"I0523 05:53:08.896470 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4521\n",
"INFO:tensorflow:global_step/sec: 2.90502\n",
"I0523 05:53:09.240309 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90502\n",
"INFO:tensorflow:examples/sec: 92.9607\n",
"I0523 05:53:09.240569 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9607\n",
"INFO:tensorflow:global_step/sec: 2.89822\n",
"I0523 05:53:09.585352 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89822\n",
"INFO:tensorflow:examples/sec: 92.7432\n",
"I0523 05:53:09.585767 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7432\n",
"INFO:tensorflow:global_step/sec: 2.88018\n",
"I0523 05:53:09.932541 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88018\n",
"INFO:tensorflow:examples/sec: 92.1659\n",
"I0523 05:53:09.932800 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1659\n",
"INFO:tensorflow:global_step/sec: 2.90727\n",
"I0523 05:53:10.276495 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90727\n",
"INFO:tensorflow:examples/sec: 93.0327\n",
"I0523 05:53:10.276757 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0327\n",
"INFO:tensorflow:global_step/sec: 2.86666\n",
"I0523 05:53:10.625343 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86666\n",
"INFO:tensorflow:examples/sec: 91.7332\n",
"I0523 05:53:10.625570 140278000138112 tpu_estimator.py:2308] examples/sec: 91.7332\n",
"INFO:tensorflow:global_step/sec: 2.89877\n",
"I0523 05:53:10.970326 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89877\n",
"INFO:tensorflow:examples/sec: 92.7606\n",
"I0523 05:53:10.970563 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7606\n",
"INFO:tensorflow:global_step/sec: 2.92809\n",
"I0523 05:53:11.311883 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.92809\n",
"INFO:tensorflow:examples/sec: 93.6989\n",
"I0523 05:53:11.312116 140278000138112 tpu_estimator.py:2308] examples/sec: 93.6989\n",
"INFO:tensorflow:global_step/sec: 2.89938\n",
"I0523 05:53:11.656743 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89938\n",
"INFO:tensorflow:examples/sec: 92.7801\n",
"I0523 05:53:11.657162 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7801\n",
"INFO:tensorflow:global_step/sec: 2.8674\n",
"I0523 05:53:12.005488 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8674\n",
"INFO:tensorflow:examples/sec: 91.7569\n",
"I0523 05:53:12.005726 140278000138112 tpu_estimator.py:2308] examples/sec: 91.7569\n",
"INFO:tensorflow:global_step/sec: 2.91369\n",
"I0523 05:53:12.348698 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91369\n",
"INFO:tensorflow:examples/sec: 93.238\n",
"I0523 05:53:12.349060 140278000138112 tpu_estimator.py:2308] examples/sec: 93.238\n",
"INFO:tensorflow:global_step/sec: 2.9017\n",
"I0523 05:53:12.693306 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.9017\n",
"INFO:tensorflow:examples/sec: 92.8545\n",
"I0523 05:53:12.693665 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8545\n",
"INFO:tensorflow:global_step/sec: 2.88644\n",
"I0523 05:53:13.039771 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88644\n",
"INFO:tensorflow:examples/sec: 92.366\n",
"I0523 05:53:13.040183 140278000138112 tpu_estimator.py:2308] examples/sec: 92.366\n",
"INFO:tensorflow:global_step/sec: 2.88808\n",
"I0523 05:53:13.386031 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88808\n",
"INFO:tensorflow:examples/sec: 92.4187\n",
"I0523 05:53:13.386390 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4187\n",
"INFO:tensorflow:global_step/sec: 2.92805\n",
"I0523 05:53:13.727528 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.92805\n",
"INFO:tensorflow:examples/sec: 93.6975\n",
"I0523 05:53:13.727902 140278000138112 tpu_estimator.py:2308] examples/sec: 93.6975\n",
"INFO:tensorflow:global_step/sec: 2.86529\n",
"I0523 05:53:14.076571 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86529\n",
"INFO:tensorflow:examples/sec: 91.6893\n",
"I0523 05:53:14.076837 140278000138112 tpu_estimator.py:2308] examples/sec: 91.6893\n",
"INFO:tensorflow:global_step/sec: 2.89863\n",
"I0523 05:53:14.421531 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89863\n",
"INFO:tensorflow:examples/sec: 92.7561\n",
"I0523 05:53:14.421920 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7561\n",
"INFO:tensorflow:global_step/sec: 2.90857\n",
"I0523 05:53:14.765357 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90857\n",
"INFO:tensorflow:examples/sec: 93.0744\n",
"I0523 05:53:14.765603 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0744\n",
"INFO:tensorflow:global_step/sec: 2.89914\n",
"I0523 05:53:15.110297 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89914\n",
"INFO:tensorflow:examples/sec: 92.7725\n",
"I0523 05:53:15.110696 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7725\n",
"INFO:tensorflow:global_step/sec: 2.9072\n",
"I0523 05:53:15.454239 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.9072\n",
"INFO:tensorflow:examples/sec: 93.0303\n",
"I0523 05:53:15.454591 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0303\n",
"INFO:tensorflow:global_step/sec: 2.91361\n",
"I0523 05:53:15.797458 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91361\n",
"INFO:tensorflow:examples/sec: 93.2354\n",
"I0523 05:53:15.797822 140278000138112 tpu_estimator.py:2308] examples/sec: 93.2354\n",
"INFO:tensorflow:global_step/sec: 2.87892\n",
"I0523 05:53:16.144830 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87892\n",
"INFO:tensorflow:examples/sec: 92.1254\n",
"I0523 05:53:16.145193 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1254\n",
"INFO:tensorflow:global_step/sec: 2.90129\n",
"I0523 05:53:16.489495 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90129\n",
"INFO:tensorflow:examples/sec: 92.8411\n",
"I0523 05:53:16.489882 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8411\n",
"INFO:tensorflow:global_step/sec: 2.9057\n",
"I0523 05:53:16.833636 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.9057\n",
"INFO:tensorflow:examples/sec: 92.9825\n",
"I0523 05:53:16.833876 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9825\n",
"INFO:tensorflow:global_step/sec: 2.90136\n",
"I0523 05:53:17.178417 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90136\n",
"INFO:tensorflow:examples/sec: 92.8434\n",
"I0523 05:53:17.178735 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8434\n",
"INFO:tensorflow:global_step/sec: 2.89041\n",
"I0523 05:53:17.524302 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89041\n",
"INFO:tensorflow:examples/sec: 92.493\n",
"I0523 05:53:17.524721 140278000138112 tpu_estimator.py:2308] examples/sec: 92.493\n",
"INFO:tensorflow:global_step/sec: 2.90231\n",
"I0523 05:53:17.868832 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90231\n",
"INFO:tensorflow:examples/sec: 92.8739\n",
"I0523 05:53:17.869213 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8739\n",
"INFO:tensorflow:global_step/sec: 2.88784\n",
"I0523 05:53:18.215141 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88784\n",
"INFO:tensorflow:examples/sec: 92.4109\n",
"I0523 05:53:18.215530 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4109\n",
"INFO:tensorflow:global_step/sec: 2.89011\n",
"I0523 05:53:18.561160 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89011\n",
"INFO:tensorflow:examples/sec: 92.4836\n",
"I0523 05:53:18.561551 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4836\n",
"INFO:tensorflow:global_step/sec: 2.86961\n",
"I0523 05:53:18.909712 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86961\n",
"INFO:tensorflow:examples/sec: 91.8276\n",
"I0523 05:53:18.910338 140278000138112 tpu_estimator.py:2308] examples/sec: 91.8276\n",
"INFO:tensorflow:global_step/sec: 2.87182\n",
"I0523 05:53:19.257820 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87182\n",
"INFO:tensorflow:examples/sec: 91.8984\n",
"I0523 05:53:19.258098 140278000138112 tpu_estimator.py:2308] examples/sec: 91.8984\n",
"INFO:tensorflow:global_step/sec: 2.87106\n",
"I0523 05:53:19.606158 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87106\n",
"INFO:tensorflow:examples/sec: 91.8739\n",
"I0523 05:53:19.606570 140278000138112 tpu_estimator.py:2308] examples/sec: 91.8739\n",
"INFO:tensorflow:global_step/sec: 2.89639\n",
"I0523 05:53:19.951405 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89639\n",
"INFO:tensorflow:examples/sec: 92.6844\n",
"I0523 05:53:19.951817 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6844\n",
"INFO:tensorflow:global_step/sec: 2.88783\n",
"I0523 05:53:20.297677 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88783\n",
"INFO:tensorflow:examples/sec: 92.4107\n",
"I0523 05:53:20.298061 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4107\n",
"INFO:tensorflow:global_step/sec: 2.88074\n",
"I0523 05:53:20.644814 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88074\n",
"INFO:tensorflow:examples/sec: 92.1836\n",
"I0523 05:53:20.645207 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1836\n",
"INFO:tensorflow:global_step/sec: 2.89449\n",
"I0523 05:53:20.990273 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89449\n",
"INFO:tensorflow:examples/sec: 92.6235\n",
"I0523 05:53:20.990672 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6235\n",
"INFO:tensorflow:global_step/sec: 2.88394\n",
"I0523 05:53:21.337045 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88394\n",
"INFO:tensorflow:examples/sec: 92.286\n",
"I0523 05:53:21.337422 140278000138112 tpu_estimator.py:2308] examples/sec: 92.286\n",
"INFO:tensorflow:global_step/sec: 2.88855\n",
"I0523 05:53:21.683216 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88855\n",
"INFO:tensorflow:examples/sec: 92.4337\n",
"I0523 05:53:21.683458 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4337\n",
"INFO:tensorflow:global_step/sec: 2.90611\n",
"I0523 05:53:22.027330 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90611\n",
"INFO:tensorflow:examples/sec: 92.9954\n",
"I0523 05:53:22.027723 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9954\n",
"INFO:tensorflow:global_step/sec: 2.90143\n",
"I0523 05:53:22.372015 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90143\n",
"INFO:tensorflow:examples/sec: 92.8458\n",
"I0523 05:53:22.372242 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8458\n",
"INFO:tensorflow:global_step/sec: 2.89484\n",
"I0523 05:53:22.717431 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89484\n",
"INFO:tensorflow:examples/sec: 92.6349\n",
"I0523 05:53:22.717649 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6349\n",
"INFO:tensorflow:global_step/sec: 2.89503\n",
"I0523 05:53:23.062840 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89503\n",
"INFO:tensorflow:examples/sec: 92.641\n",
"I0523 05:53:23.063250 140278000138112 tpu_estimator.py:2308] examples/sec: 92.641\n",
"INFO:tensorflow:global_step/sec: 2.90381\n",
"I0523 05:53:23.407208 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90381\n",
"INFO:tensorflow:examples/sec: 92.922\n",
"I0523 05:53:23.407438 140278000138112 tpu_estimator.py:2308] examples/sec: 92.922\n",
"INFO:tensorflow:global_step/sec: 2.89305\n",
"I0523 05:53:23.752880 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89305\n",
"INFO:tensorflow:examples/sec: 92.5777\n",
"I0523 05:53:23.753251 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5777\n",
"INFO:tensorflow:global_step/sec: 2.89636\n",
"I0523 05:53:24.098163 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89636\n",
"INFO:tensorflow:examples/sec: 92.6834\n",
"I0523 05:53:24.098393 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6834\n",
"INFO:tensorflow:global_step/sec: 2.88758\n",
"I0523 05:53:24.444495 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88758\n",
"INFO:tensorflow:examples/sec: 92.4027\n",
"I0523 05:53:24.444719 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4027\n",
"INFO:tensorflow:global_step/sec: 2.89272\n",
"I0523 05:53:24.790153 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89272\n",
"INFO:tensorflow:examples/sec: 92.5671\n",
"I0523 05:53:24.790378 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5671\n",
"INFO:tensorflow:global_step/sec: 2.91282\n",
"I0523 05:53:25.133455 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91282\n",
"INFO:tensorflow:examples/sec: 93.2103\n",
"I0523 05:53:25.133831 140278000138112 tpu_estimator.py:2308] examples/sec: 93.2103\n",
"INFO:tensorflow:global_step/sec: 2.89807\n",
"I0523 05:53:25.478488 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89807\n",
"INFO:tensorflow:examples/sec: 92.7381\n",
"I0523 05:53:25.478702 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7381\n",
"INFO:tensorflow:global_step/sec: 2.88092\n",
"I0523 05:53:25.825611 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88092\n",
"INFO:tensorflow:examples/sec: 92.1894\n",
"I0523 05:53:25.825863 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1894\n",
"INFO:tensorflow:global_step/sec: 2.9143\n",
"I0523 05:53:26.168762 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.9143\n",
"INFO:tensorflow:examples/sec: 93.2576\n",
"I0523 05:53:26.169051 140278000138112 tpu_estimator.py:2308] examples/sec: 93.2576\n",
"INFO:tensorflow:global_step/sec: 2.88408\n",
"I0523 05:53:26.515495 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88408\n",
"INFO:tensorflow:examples/sec: 92.2905\n",
"I0523 05:53:26.515956 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2905\n",
"INFO:tensorflow:global_step/sec: 2.87049\n",
"I0523 05:53:26.863909 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87049\n",
"INFO:tensorflow:examples/sec: 91.8558\n",
"I0523 05:53:26.864331 140278000138112 tpu_estimator.py:2308] examples/sec: 91.8558\n",
"INFO:tensorflow:global_step/sec: 2.89233\n",
"I0523 05:53:27.209612 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89233\n",
"INFO:tensorflow:examples/sec: 92.5546\n",
"I0523 05:53:27.209876 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5546\n",
"INFO:tensorflow:global_step/sec: 2.88692\n",
"I0523 05:53:27.556010 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88692\n",
"INFO:tensorflow:examples/sec: 92.3814\n",
"I0523 05:53:27.556244 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3814\n",
"INFO:tensorflow:global_step/sec: 2.88011\n",
"I0523 05:53:27.903228 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88011\n",
"INFO:tensorflow:examples/sec: 92.1636\n",
"I0523 05:53:27.903468 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1636\n",
"INFO:tensorflow:global_step/sec: 2.83866\n",
"I0523 05:53:28.256388 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.83866\n",
"INFO:tensorflow:examples/sec: 90.837\n",
"I0523 05:53:28.256777 140278000138112 tpu_estimator.py:2308] examples/sec: 90.837\n",
"INFO:tensorflow:global_step/sec: 2.89621\n",
"I0523 05:53:28.600781 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89621\n",
"INFO:tensorflow:examples/sec: 92.6787\n",
"I0523 05:53:28.601235 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6787\n",
"INFO:tensorflow:global_step/sec: 2.87641\n",
"I0523 05:53:28.948413 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87641\n",
"INFO:tensorflow:examples/sec: 92.0451\n",
"I0523 05:53:28.948761 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0451\n",
"INFO:tensorflow:global_step/sec: 2.90205\n",
"I0523 05:53:29.293042 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90205\n",
"INFO:tensorflow:examples/sec: 92.8655\n",
"I0523 05:53:29.293272 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8655\n",
"INFO:tensorflow:global_step/sec: 2.89996\n",
"I0523 05:53:29.637837 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89996\n",
"INFO:tensorflow:examples/sec: 92.7988\n",
"I0523 05:53:29.638070 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7988\n",
"INFO:tensorflow:global_step/sec: 2.88854\n",
"I0523 05:53:29.984087 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88854\n",
"INFO:tensorflow:examples/sec: 92.4334\n",
"I0523 05:53:29.984331 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4334\n",
"INFO:tensorflow:global_step/sec: 2.88887\n",
"I0523 05:53:30.330181 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88887\n",
"INFO:tensorflow:examples/sec: 92.444\n",
"I0523 05:53:30.330413 140278000138112 tpu_estimator.py:2308] examples/sec: 92.444\n",
"INFO:tensorflow:global_step/sec: 2.89513\n",
"I0523 05:53:30.675580 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89513\n",
"INFO:tensorflow:examples/sec: 92.6442\n",
"I0523 05:53:30.675815 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6442\n",
"INFO:tensorflow:global_step/sec: 2.87598\n",
"I0523 05:53:31.023283 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87598\n",
"INFO:tensorflow:examples/sec: 92.0312\n",
"I0523 05:53:31.023513 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0312\n",
"INFO:tensorflow:global_step/sec: 2.8963\n",
"I0523 05:53:31.368555 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8963\n",
"INFO:tensorflow:examples/sec: 92.6816\n",
"I0523 05:53:31.368798 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6816\n",
"INFO:tensorflow:global_step/sec: 2.8894\n",
"I0523 05:53:31.714650 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8894\n",
"INFO:tensorflow:examples/sec: 92.4606\n",
"I0523 05:53:31.714891 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4606\n",
"INFO:tensorflow:global_step/sec: 2.87472\n",
"I0523 05:53:32.062521 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87472\n",
"INFO:tensorflow:examples/sec: 91.9909\n",
"I0523 05:53:32.062764 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9909\n",
"INFO:tensorflow:global_step/sec: 2.89911\n",
"I0523 05:53:32.407449 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89911\n",
"INFO:tensorflow:examples/sec: 92.7714\n",
"I0523 05:53:32.407677 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7714\n",
"INFO:tensorflow:global_step/sec: 2.89057\n",
"I0523 05:53:32.753396 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89057\n",
"INFO:tensorflow:examples/sec: 92.4982\n",
"I0523 05:53:32.753618 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4982\n",
"INFO:tensorflow:global_step/sec: 2.888\n",
"I0523 05:53:33.099657 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.888\n",
"INFO:tensorflow:examples/sec: 92.4159\n",
"I0523 05:53:33.099892 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4159\n",
"INFO:tensorflow:global_step/sec: 2.86005\n",
"I0523 05:53:33.449294 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86005\n",
"INFO:tensorflow:examples/sec: 91.5216\n",
"I0523 05:53:33.449636 140278000138112 tpu_estimator.py:2308] examples/sec: 91.5216\n",
"INFO:tensorflow:global_step/sec: 2.88556\n",
"I0523 05:53:33.795856 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88556\n",
"INFO:tensorflow:examples/sec: 92.3381\n",
"I0523 05:53:33.796227 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3381\n",
"INFO:tensorflow:global_step/sec: 2.90087\n",
"I0523 05:53:34.140597 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90087\n",
"INFO:tensorflow:examples/sec: 92.8277\n",
"I0523 05:53:34.141020 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8277\n",
"INFO:tensorflow:global_step/sec: 2.8791\n",
"I0523 05:53:34.487875 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8791\n",
"INFO:tensorflow:examples/sec: 92.1311\n",
"I0523 05:53:34.488144 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1311\n",
"INFO:tensorflow:global_step/sec: 2.87761\n",
"I0523 05:53:34.835437 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87761\n",
"INFO:tensorflow:examples/sec: 92.0835\n",
"I0523 05:53:34.835808 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0835\n",
"INFO:tensorflow:global_step/sec: 2.90706\n",
"I0523 05:53:35.179413 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90706\n",
"INFO:tensorflow:examples/sec: 93.0258\n",
"I0523 05:53:35.179647 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0258\n",
"INFO:tensorflow:global_step/sec: 2.90808\n",
"I0523 05:53:35.523303 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90808\n",
"INFO:tensorflow:examples/sec: 93.0587\n",
"I0523 05:53:35.523547 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0587\n",
"INFO:tensorflow:global_step/sec: 2.88197\n",
"I0523 05:53:35.870298 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88197\n",
"INFO:tensorflow:examples/sec: 92.2229\n",
"I0523 05:53:35.870539 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2229\n",
"INFO:tensorflow:global_step/sec: 2.89442\n",
"I0523 05:53:36.215790 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89442\n",
"INFO:tensorflow:examples/sec: 92.6214\n",
"I0523 05:53:36.216038 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6214\n",
"INFO:tensorflow:global_step/sec: 2.91043\n",
"I0523 05:53:36.559339 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91043\n",
"INFO:tensorflow:examples/sec: 93.1339\n",
"I0523 05:53:36.559707 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1339\n",
"INFO:tensorflow:global_step/sec: 2.87918\n",
"I0523 05:53:36.906662 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87918\n",
"INFO:tensorflow:examples/sec: 92.1338\n",
"I0523 05:53:36.906879 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1338\n",
"INFO:tensorflow:global_step/sec: 2.89824\n",
"I0523 05:53:37.251737 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89824\n",
"INFO:tensorflow:examples/sec: 92.7436\n",
"I0523 05:53:37.252187 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7436\n",
"INFO:tensorflow:global_step/sec: 2.9038\n",
"I0523 05:53:37.596136 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.9038\n",
"INFO:tensorflow:examples/sec: 92.9217\n",
"I0523 05:53:37.596398 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9217\n",
"INFO:tensorflow:global_step/sec: 2.87976\n",
"I0523 05:53:37.943328 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87976\n",
"INFO:tensorflow:examples/sec: 92.1523\n",
"I0523 05:53:37.943544 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1523\n",
"INFO:tensorflow:global_step/sec: 2.89759\n",
"I0523 05:53:38.288446 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89759\n",
"INFO:tensorflow:examples/sec: 92.7229\n",
"I0523 05:53:38.288679 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7229\n",
"INFO:tensorflow:global_step/sec: 2.89896\n",
"I0523 05:53:38.633397 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89896\n",
"INFO:tensorflow:examples/sec: 92.7666\n",
"I0523 05:53:38.633633 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7666\n",
"INFO:tensorflow:global_step/sec: 2.88068\n",
"I0523 05:53:38.980567 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88068\n",
"INFO:tensorflow:examples/sec: 92.1817\n",
"I0523 05:53:38.980829 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1817\n",
"INFO:tensorflow:global_step/sec: 2.88928\n",
"I0523 05:53:39.326666 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88928\n",
"INFO:tensorflow:examples/sec: 92.4568\n",
"I0523 05:53:39.327047 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4568\n",
"INFO:tensorflow:global_step/sec: 2.89847\n",
"I0523 05:53:39.671669 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89847\n",
"INFO:tensorflow:examples/sec: 92.7511\n",
"I0523 05:53:39.672119 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7511\n",
"INFO:tensorflow:global_step/sec: 2.88398\n",
"I0523 05:53:40.018404 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88398\n",
"INFO:tensorflow:examples/sec: 92.2872\n",
"I0523 05:53:40.018627 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2872\n",
"INFO:tensorflow:global_step/sec: 2.88018\n",
"I0523 05:53:40.365613 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88018\n",
"INFO:tensorflow:examples/sec: 92.1658\n",
"I0523 05:53:40.365981 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1658\n",
"INFO:tensorflow:global_step/sec: 2.88091\n",
"I0523 05:53:40.712735 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88091\n",
"INFO:tensorflow:examples/sec: 92.1891\n",
"I0523 05:53:40.713122 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1891\n",
"INFO:tensorflow:global_step/sec: 2.86859\n",
"I0523 05:53:41.061364 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86859\n",
"INFO:tensorflow:examples/sec: 91.795\n",
"I0523 05:53:41.061824 140278000138112 tpu_estimator.py:2308] examples/sec: 91.795\n",
"INFO:tensorflow:global_step/sec: 2.90042\n",
"I0523 05:53:41.406132 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90042\n",
"INFO:tensorflow:examples/sec: 92.8133\n",
"I0523 05:53:41.406532 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8133\n",
"INFO:tensorflow:global_step/sec: 2.87658\n",
"I0523 05:53:41.753760 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87658\n",
"INFO:tensorflow:examples/sec: 92.0504\n",
"I0523 05:53:41.754207 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0504\n",
"INFO:tensorflow:global_step/sec: 2.88461\n",
"I0523 05:53:42.100404 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88461\n",
"INFO:tensorflow:examples/sec: 92.3075\n",
"I0523 05:53:42.100775 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3075\n",
"INFO:tensorflow:global_step/sec: 2.87369\n",
"I0523 05:53:42.448398 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87369\n",
"INFO:tensorflow:examples/sec: 91.9582\n",
"I0523 05:53:42.448812 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9582\n",
"INFO:tensorflow:global_step/sec: 2.90684\n",
"I0523 05:53:42.792409 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90684\n",
"INFO:tensorflow:examples/sec: 93.019\n",
"I0523 05:53:42.792645 140278000138112 tpu_estimator.py:2308] examples/sec: 93.019\n",
"INFO:tensorflow:global_step/sec: 2.90393\n",
"I0523 05:53:43.136807 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90393\n",
"INFO:tensorflow:examples/sec: 92.9257\n",
"I0523 05:53:43.137195 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9257\n",
"INFO:tensorflow:global_step/sec: 2.88996\n",
"I0523 05:53:43.482833 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88996\n",
"INFO:tensorflow:examples/sec: 92.4787\n",
"I0523 05:53:43.483307 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4787\n",
"INFO:tensorflow:global_step/sec: 2.88036\n",
"I0523 05:53:43.829988 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88036\n",
"INFO:tensorflow:examples/sec: 92.1716\n",
"I0523 05:53:43.830224 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1716\n",
"INFO:tensorflow:global_step/sec: 2.90262\n",
"I0523 05:53:44.174474 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90262\n",
"INFO:tensorflow:examples/sec: 92.8837\n",
"I0523 05:53:44.174821 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8837\n",
"INFO:tensorflow:global_step/sec: 2.89709\n",
"I0523 05:53:44.519666 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89709\n",
"INFO:tensorflow:examples/sec: 92.707\n",
"I0523 05:53:44.520059 140278000138112 tpu_estimator.py:2308] examples/sec: 92.707\n",
"INFO:tensorflow:global_step/sec: 2.88913\n",
"I0523 05:53:44.865775 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88913\n",
"INFO:tensorflow:examples/sec: 92.4522\n",
"I0523 05:53:44.866188 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4522\n",
"INFO:tensorflow:global_step/sec: 2.89576\n",
"I0523 05:53:45.211155 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89576\n",
"INFO:tensorflow:examples/sec: 92.6642\n",
"I0523 05:53:45.211407 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6642\n",
"INFO:tensorflow:global_step/sec: 2.88916\n",
"I0523 05:53:45.557247 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88916\n",
"INFO:tensorflow:examples/sec: 92.4531\n",
"I0523 05:53:45.557494 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4531\n",
"INFO:tensorflow:global_step/sec: 2.89288\n",
"I0523 05:53:45.902909 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89288\n",
"INFO:tensorflow:examples/sec: 92.5723\n",
"I0523 05:53:45.903326 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5723\n",
"INFO:tensorflow:global_step/sec: 2.89654\n",
"I0523 05:53:46.248264 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89654\n",
"INFO:tensorflow:examples/sec: 92.6892\n",
"I0523 05:53:46.248506 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6892\n",
"INFO:tensorflow:global_step/sec: 2.88351\n",
"I0523 05:53:46.594970 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88351\n",
"INFO:tensorflow:examples/sec: 92.2723\n",
"I0523 05:53:46.595344 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2723\n",
"INFO:tensorflow:global_step/sec: 2.88537\n",
"I0523 05:53:46.941550 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88537\n",
"INFO:tensorflow:examples/sec: 92.3318\n",
"I0523 05:53:46.941906 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3318\n",
"INFO:tensorflow:global_step/sec: 2.89891\n",
"I0523 05:53:47.286508 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89891\n",
"INFO:tensorflow:examples/sec: 92.7651\n",
"I0523 05:53:47.286926 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7651\n",
"INFO:tensorflow:global_step/sec: 2.89912\n",
"I0523 05:53:47.631423 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89912\n",
"INFO:tensorflow:examples/sec: 92.7718\n",
"I0523 05:53:47.631663 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7718\n",
"INFO:tensorflow:global_step/sec: 2.88759\n",
"I0523 05:53:47.977748 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88759\n",
"INFO:tensorflow:examples/sec: 92.403\n",
"I0523 05:53:47.978130 140278000138112 tpu_estimator.py:2308] examples/sec: 92.403\n",
"INFO:tensorflow:global_step/sec: 2.91596\n",
"I0523 05:53:48.320704 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91596\n",
"INFO:tensorflow:examples/sec: 93.3106\n",
"I0523 05:53:48.321141 140278000138112 tpu_estimator.py:2308] examples/sec: 93.3106\n",
"INFO:tensorflow:global_step/sec: 2.8778\n",
"I0523 05:53:48.668205 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8778\n",
"INFO:tensorflow:examples/sec: 92.0897\n",
"I0523 05:53:48.668587 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0897\n",
"INFO:tensorflow:global_step/sec: 2.89535\n",
"I0523 05:53:49.013541 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89535\n",
"INFO:tensorflow:examples/sec: 92.6513\n",
"I0523 05:53:49.013909 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6513\n",
"INFO:tensorflow:global_step/sec: 2.91347\n",
"I0523 05:53:49.356799 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91347\n",
"INFO:tensorflow:examples/sec: 93.2311\n",
"I0523 05:53:49.357179 140278000138112 tpu_estimator.py:2308] examples/sec: 93.2311\n",
"INFO:tensorflow:global_step/sec: 2.88037\n",
"I0523 05:53:49.703999 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88037\n",
"INFO:tensorflow:examples/sec: 92.1719\n",
"I0523 05:53:49.704381 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1719\n",
"INFO:tensorflow:global_step/sec: 2.89718\n",
"I0523 05:53:50.049318 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89718\n",
"INFO:tensorflow:examples/sec: 92.7098\n",
"I0523 05:53:50.049581 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7098\n",
"INFO:tensorflow:global_step/sec: 2.89891\n",
"I0523 05:53:50.394100 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89891\n",
"INFO:tensorflow:examples/sec: 92.7652\n",
"I0523 05:53:50.394439 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7652\n",
"INFO:tensorflow:global_step/sec: 2.88529\n",
"I0523 05:53:50.740673 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88529\n",
"INFO:tensorflow:examples/sec: 92.3292\n",
"I0523 05:53:50.741070 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3292\n",
"INFO:tensorflow:global_step/sec: 2.87576\n",
"I0523 05:53:51.088412 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87576\n",
"INFO:tensorflow:examples/sec: 92.0242\n",
"I0523 05:53:51.088824 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0242\n",
"INFO:tensorflow:global_step/sec: 2.89431\n",
"I0523 05:53:51.433916 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89431\n",
"INFO:tensorflow:examples/sec: 92.6179\n",
"I0523 05:53:51.434173 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6179\n",
"INFO:tensorflow:global_step/sec: 2.91406\n",
"I0523 05:53:51.777106 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91406\n",
"INFO:tensorflow:examples/sec: 93.2499\n",
"I0523 05:53:51.777492 140278000138112 tpu_estimator.py:2308] examples/sec: 93.2499\n",
"INFO:tensorflow:global_step/sec: 2.92159\n",
"I0523 05:53:52.119338 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.92159\n",
"INFO:tensorflow:examples/sec: 93.4909\n",
"I0523 05:53:52.119572 140278000138112 tpu_estimator.py:2308] examples/sec: 93.4909\n",
"INFO:tensorflow:global_step/sec: 2.8965\n",
"I0523 05:53:52.464586 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8965\n",
"INFO:tensorflow:examples/sec: 92.6881\n",
"I0523 05:53:52.464967 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6881\n",
"INFO:tensorflow:global_step/sec: 2.8908\n",
"I0523 05:53:52.810518 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8908\n",
"INFO:tensorflow:examples/sec: 92.5057\n",
"I0523 05:53:52.810970 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5057\n",
"INFO:tensorflow:global_step/sec: 2.86574\n",
"I0523 05:53:53.159476 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86574\n",
"INFO:tensorflow:examples/sec: 91.7038\n",
"I0523 05:53:53.159723 140278000138112 tpu_estimator.py:2308] examples/sec: 91.7038\n",
"INFO:tensorflow:global_step/sec: 2.88489\n",
"I0523 05:53:53.506106 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88489\n",
"INFO:tensorflow:examples/sec: 92.3166\n",
"I0523 05:53:53.506501 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3166\n",
"INFO:tensorflow:global_step/sec: 2.8906\n",
"I0523 05:53:53.852047 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8906\n",
"INFO:tensorflow:examples/sec: 92.4991\n",
"I0523 05:53:53.852270 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4991\n",
"INFO:tensorflow:global_step/sec: 2.88893\n",
"I0523 05:53:54.198192 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88893\n",
"INFO:tensorflow:examples/sec: 92.4457\n",
"I0523 05:53:54.198457 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4457\n",
"INFO:tensorflow:global_step/sec: 2.92685\n",
"I0523 05:53:54.539881 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.92685\n",
"INFO:tensorflow:examples/sec: 93.6593\n",
"I0523 05:53:54.540304 140278000138112 tpu_estimator.py:2308] examples/sec: 93.6593\n",
"INFO:tensorflow:global_step/sec: 2.88835\n",
"I0523 05:53:54.886088 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88835\n",
"INFO:tensorflow:examples/sec: 92.4273\n",
"I0523 05:53:54.886450 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4273\n",
"INFO:tensorflow:global_step/sec: 2.91251\n",
"I0523 05:53:55.229407 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91251\n",
"INFO:tensorflow:examples/sec: 93.2003\n",
"I0523 05:53:55.229751 140278000138112 tpu_estimator.py:2308] examples/sec: 93.2003\n",
"INFO:tensorflow:global_step/sec: 2.88274\n",
"I0523 05:53:55.576308 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88274\n",
"INFO:tensorflow:examples/sec: 92.2477\n",
"I0523 05:53:55.576668 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2477\n",
"INFO:tensorflow:global_step/sec: 2.89779\n",
"I0523 05:53:55.921387 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89779\n",
"INFO:tensorflow:examples/sec: 92.7293\n",
"I0523 05:53:55.921611 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7293\n",
"INFO:tensorflow:global_step/sec: 2.91722\n",
"I0523 05:53:56.264183 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91722\n",
"INFO:tensorflow:examples/sec: 93.351\n",
"I0523 05:53:56.264568 140278000138112 tpu_estimator.py:2308] examples/sec: 93.351\n",
"INFO:tensorflow:global_step/sec: 2.87053\n",
"I0523 05:53:56.612601 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87053\n",
"INFO:tensorflow:examples/sec: 91.8568\n",
"I0523 05:53:56.613026 140278000138112 tpu_estimator.py:2308] examples/sec: 91.8568\n",
"INFO:tensorflow:global_step/sec: 2.89755\n",
"I0523 05:53:56.957681 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89755\n",
"INFO:tensorflow:examples/sec: 92.7215\n",
"I0523 05:53:56.958118 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7215\n",
"INFO:tensorflow:global_step/sec: 2.91175\n",
"I0523 05:53:57.301130 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91175\n",
"INFO:tensorflow:examples/sec: 93.1761\n",
"I0523 05:53:57.301514 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1761\n",
"INFO:tensorflow:global_step/sec: 2.8827\n",
"I0523 05:53:57.648045 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8827\n",
"INFO:tensorflow:examples/sec: 92.2464\n",
"I0523 05:53:57.648465 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2464\n",
"INFO:tensorflow:global_step/sec: 2.89549\n",
"I0523 05:53:57.993393 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89549\n",
"INFO:tensorflow:examples/sec: 92.6556\n",
"I0523 05:53:57.993650 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6556\n",
"INFO:tensorflow:global_step/sec: 2.88555\n",
"I0523 05:53:58.339964 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88555\n",
"INFO:tensorflow:examples/sec: 92.3377\n",
"I0523 05:53:58.340359 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3377\n",
"INFO:tensorflow:global_step/sec: 2.89972\n",
"I0523 05:53:58.684817 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89972\n",
"INFO:tensorflow:examples/sec: 92.7909\n",
"I0523 05:53:58.685212 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7909\n",
"INFO:tensorflow:global_step/sec: 2.87767\n",
"I0523 05:53:59.032347 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87767\n",
"INFO:tensorflow:examples/sec: 92.0856\n",
"I0523 05:53:59.032651 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0856\n",
"INFO:tensorflow:global_step/sec: 2.89434\n",
"I0523 05:53:59.377835 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89434\n",
"INFO:tensorflow:examples/sec: 92.619\n",
"I0523 05:53:59.378317 140278000138112 tpu_estimator.py:2308] examples/sec: 92.619\n",
"INFO:tensorflow:global_step/sec: 2.86693\n",
"I0523 05:53:59.726608 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86693\n",
"INFO:tensorflow:examples/sec: 91.7418\n",
"I0523 05:53:59.726996 140278000138112 tpu_estimator.py:2308] examples/sec: 91.7418\n",
"INFO:tensorflow:global_step/sec: 2.89158\n",
"I0523 05:54:00.072434 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89158\n",
"INFO:tensorflow:examples/sec: 92.5307\n",
"I0523 05:54:00.072824 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5307\n",
"INFO:tensorflow:global_step/sec: 2.8899\n",
"I0523 05:54:00.418479 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8899\n",
"INFO:tensorflow:examples/sec: 92.4768\n",
"I0523 05:54:00.418718 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4768\n",
"INFO:tensorflow:global_step/sec: 2.91957\n",
"I0523 05:54:00.760997 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91957\n",
"INFO:tensorflow:examples/sec: 93.4262\n",
"I0523 05:54:00.761238 140278000138112 tpu_estimator.py:2308] examples/sec: 93.4262\n",
"INFO:tensorflow:global_step/sec: 2.8925\n",
"I0523 05:54:01.106707 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8925\n",
"INFO:tensorflow:examples/sec: 92.5601\n",
"I0523 05:54:01.107083 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5601\n",
"INFO:tensorflow:global_step/sec: 2.90964\n",
"I0523 05:54:01.450380 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90964\n",
"INFO:tensorflow:examples/sec: 93.1085\n",
"I0523 05:54:01.450605 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1085\n",
"INFO:tensorflow:global_step/sec: 2.90045\n",
"I0523 05:54:01.795177 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90045\n",
"INFO:tensorflow:examples/sec: 92.8145\n",
"I0523 05:54:01.795563 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8145\n",
"INFO:tensorflow:global_step/sec: 2.88787\n",
"I0523 05:54:02.141451 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88787\n",
"INFO:tensorflow:examples/sec: 92.4119\n",
"I0523 05:54:02.141893 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4119\n",
"INFO:tensorflow:global_step/sec: 2.90511\n",
"I0523 05:54:02.485642 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90511\n",
"INFO:tensorflow:examples/sec: 92.9636\n",
"I0523 05:54:02.486011 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9636\n",
"INFO:tensorflow:global_step/sec: 2.90186\n",
"I0523 05:54:02.830251 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90186\n",
"INFO:tensorflow:examples/sec: 92.8594\n",
"I0523 05:54:02.830622 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8594\n",
"INFO:tensorflow:global_step/sec: 2.89743\n",
"I0523 05:54:03.175384 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89743\n",
"INFO:tensorflow:examples/sec: 92.7177\n",
"I0523 05:54:03.175599 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7177\n",
"INFO:tensorflow:global_step/sec: 2.89551\n",
"I0523 05:54:03.520814 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89551\n",
"INFO:tensorflow:examples/sec: 92.6564\n",
"I0523 05:54:03.521218 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6564\n",
"INFO:tensorflow:global_step/sec: 2.8936\n",
"I0523 05:54:03.866368 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8936\n",
"INFO:tensorflow:examples/sec: 92.5952\n",
"I0523 05:54:03.866727 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5952\n",
"INFO:tensorflow:global_step/sec: 2.91833\n",
"I0523 05:54:04.209064 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91833\n",
"INFO:tensorflow:examples/sec: 93.3865\n",
"I0523 05:54:04.209471 140278000138112 tpu_estimator.py:2308] examples/sec: 93.3865\n",
"INFO:tensorflow:global_step/sec: 2.89639\n",
"I0523 05:54:04.554266 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89639\n",
"INFO:tensorflow:examples/sec: 92.6845\n",
"I0523 05:54:04.554487 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6845\n",
"INFO:tensorflow:global_step/sec: 2.89805\n",
"I0523 05:54:04.899339 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89805\n",
"INFO:tensorflow:examples/sec: 92.7376\n",
"I0523 05:54:04.899720 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7376\n",
"INFO:tensorflow:global_step/sec: 2.8899\n",
"I0523 05:54:05.245356 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8899\n",
"INFO:tensorflow:examples/sec: 92.4769\n",
"I0523 05:54:05.245697 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4769\n",
"INFO:tensorflow:global_step/sec: 2.89854\n",
"I0523 05:54:05.590358 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89854\n",
"INFO:tensorflow:examples/sec: 92.7534\n",
"I0523 05:54:05.590684 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7534\n",
"INFO:tensorflow:global_step/sec: 2.91218\n",
"I0523 05:54:05.933753 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91218\n",
"INFO:tensorflow:examples/sec: 93.1896\n",
"I0523 05:54:05.934175 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1896\n",
"INFO:tensorflow:global_step/sec: 2.90283\n",
"I0523 05:54:06.278241 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90283\n",
"INFO:tensorflow:examples/sec: 92.8905\n",
"I0523 05:54:06.278631 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8905\n",
"INFO:tensorflow:global_step/sec: 2.8801\n",
"I0523 05:54:06.625436 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8801\n",
"INFO:tensorflow:examples/sec: 92.1631\n",
"I0523 05:54:06.625788 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1631\n",
"INFO:tensorflow:global_step/sec: 2.89693\n",
"I0523 05:54:06.970638 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89693\n",
"INFO:tensorflow:examples/sec: 92.7019\n",
"I0523 05:54:06.971041 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7019\n",
"INFO:tensorflow:global_step/sec: 2.91929\n",
"I0523 05:54:07.313211 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91929\n",
"INFO:tensorflow:examples/sec: 93.4173\n",
"I0523 05:54:07.313441 140278000138112 tpu_estimator.py:2308] examples/sec: 93.4173\n",
"INFO:tensorflow:global_step/sec: 2.89446\n",
"I0523 05:54:07.658676 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89446\n",
"INFO:tensorflow:examples/sec: 92.6228\n",
"I0523 05:54:07.659047 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6228\n",
"INFO:tensorflow:global_step/sec: 2.88364\n",
"I0523 05:54:08.005467 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88364\n",
"INFO:tensorflow:examples/sec: 92.2764\n",
"I0523 05:54:08.005714 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2764\n",
"INFO:tensorflow:global_step/sec: 2.90276\n",
"I0523 05:54:08.350004 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90276\n",
"INFO:tensorflow:examples/sec: 92.8883\n",
"I0523 05:54:08.350408 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8883\n",
"INFO:tensorflow:global_step/sec: 2.88438\n",
"I0523 05:54:08.696610 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88438\n",
"INFO:tensorflow:examples/sec: 92.3001\n",
"I0523 05:54:08.696836 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3001\n",
"INFO:tensorflow:global_step/sec: 2.90316\n",
"I0523 05:54:09.041172 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90316\n",
"INFO:tensorflow:examples/sec: 92.901\n",
"I0523 05:54:09.041414 140278000138112 tpu_estimator.py:2308] examples/sec: 92.901\n",
"INFO:tensorflow:global_step/sec: 2.91123\n",
"I0523 05:54:09.384604 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91123\n",
"INFO:tensorflow:examples/sec: 93.1592\n",
"I0523 05:54:09.385001 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1592\n",
"INFO:tensorflow:global_step/sec: 2.89371\n",
"I0523 05:54:09.730184 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89371\n",
"INFO:tensorflow:examples/sec: 92.5988\n",
"I0523 05:54:09.730585 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5988\n",
"INFO:tensorflow:global_step/sec: 2.89353\n",
"I0523 05:54:10.075833 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89353\n",
"INFO:tensorflow:examples/sec: 92.5931\n",
"I0523 05:54:10.076196 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5931\n",
"INFO:tensorflow:global_step/sec: 2.90884\n",
"I0523 05:54:10.419564 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90884\n",
"INFO:tensorflow:examples/sec: 93.083\n",
"I0523 05:54:10.419978 140278000138112 tpu_estimator.py:2308] examples/sec: 93.083\n",
"INFO:tensorflow:global_step/sec: 2.90935\n",
"I0523 05:54:10.763293 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90935\n",
"INFO:tensorflow:examples/sec: 93.0993\n",
"I0523 05:54:10.763696 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0993\n",
"INFO:tensorflow:global_step/sec: 2.88428\n",
"I0523 05:54:11.110035 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88428\n",
"INFO:tensorflow:examples/sec: 92.2969\n",
"I0523 05:54:11.110468 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2969\n",
"INFO:tensorflow:global_step/sec: 2.88289\n",
"I0523 05:54:11.456887 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88289\n",
"INFO:tensorflow:examples/sec: 92.2525\n",
"I0523 05:54:11.457322 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2525\n",
"INFO:tensorflow:global_step/sec: 2.87738\n",
"I0523 05:54:11.804442 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87738\n",
"INFO:tensorflow:examples/sec: 92.0761\n",
"I0523 05:54:11.804860 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0761\n",
"INFO:tensorflow:global_step/sec: 2.90832\n",
"I0523 05:54:12.148260 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90832\n",
"INFO:tensorflow:examples/sec: 93.0661\n",
"I0523 05:54:12.148639 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0661\n",
"INFO:tensorflow:global_step/sec: 2.88335\n",
"I0523 05:54:12.495146 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88335\n",
"INFO:tensorflow:examples/sec: 92.267\n",
"I0523 05:54:12.495553 140278000138112 tpu_estimator.py:2308] examples/sec: 92.267\n",
"INFO:tensorflow:global_step/sec: 2.8758\n",
"I0523 05:54:12.842839 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8758\n",
"INFO:tensorflow:examples/sec: 92.0256\n",
"I0523 05:54:12.843206 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0256\n",
"INFO:tensorflow:global_step/sec: 2.88612\n",
"I0523 05:54:13.189290 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88612\n",
"INFO:tensorflow:examples/sec: 92.3559\n",
"I0523 05:54:13.189667 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3559\n",
"INFO:tensorflow:global_step/sec: 2.8684\n",
"I0523 05:54:13.537906 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8684\n",
"INFO:tensorflow:examples/sec: 91.7888\n",
"I0523 05:54:13.538272 140278000138112 tpu_estimator.py:2308] examples/sec: 91.7888\n",
"INFO:tensorflow:global_step/sec: 2.90014\n",
"I0523 05:54:13.882735 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90014\n",
"INFO:tensorflow:examples/sec: 92.8043\n",
"I0523 05:54:13.883138 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8043\n",
"INFO:tensorflow:global_step/sec: 2.9028\n",
"I0523 05:54:14.227235 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.9028\n",
"INFO:tensorflow:examples/sec: 92.8894\n",
"I0523 05:54:14.227488 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8894\n",
"INFO:tensorflow:global_step/sec: 2.91367\n",
"I0523 05:54:14.570413 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91367\n",
"INFO:tensorflow:examples/sec: 93.2374\n",
"I0523 05:54:14.570771 140278000138112 tpu_estimator.py:2308] examples/sec: 93.2374\n",
"INFO:tensorflow:global_step/sec: 2.87692\n",
"I0523 05:54:14.918057 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87692\n",
"INFO:tensorflow:examples/sec: 92.0613\n",
"I0523 05:54:14.918296 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0613\n",
"INFO:tensorflow:global_step/sec: 2.8853\n",
"I0523 05:54:15.264593 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8853\n",
"INFO:tensorflow:examples/sec: 92.3295\n",
"I0523 05:54:15.264963 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3295\n",
"INFO:tensorflow:global_step/sec: 2.9134\n",
"I0523 05:54:15.607853 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.9134\n",
"INFO:tensorflow:examples/sec: 93.2288\n",
"I0523 05:54:15.608099 140278000138112 tpu_estimator.py:2308] examples/sec: 93.2288\n",
"INFO:tensorflow:global_step/sec: 2.89116\n",
"I0523 05:54:15.953743 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89116\n",
"INFO:tensorflow:examples/sec: 92.5172\n",
"I0523 05:54:15.954144 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5172\n",
"INFO:tensorflow:global_step/sec: 2.89732\n",
"I0523 05:54:16.298874 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89732\n",
"INFO:tensorflow:examples/sec: 92.7141\n",
"I0523 05:54:16.299165 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7141\n",
"INFO:tensorflow:global_step/sec: 2.90342\n",
"I0523 05:54:16.643329 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90342\n",
"INFO:tensorflow:examples/sec: 92.9096\n",
"I0523 05:54:16.643606 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9096\n",
"INFO:tensorflow:global_step/sec: 2.8921\n",
"I0523 05:54:16.989096 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8921\n",
"INFO:tensorflow:examples/sec: 92.5472\n",
"I0523 05:54:16.989460 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5472\n",
"INFO:tensorflow:global_step/sec: 2.85466\n",
"I0523 05:54:17.339360 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.85466\n",
"INFO:tensorflow:examples/sec: 91.3493\n",
"I0523 05:54:17.339592 140278000138112 tpu_estimator.py:2308] examples/sec: 91.3493\n",
"INFO:tensorflow:global_step/sec: 2.90375\n",
"I0523 05:54:17.683754 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90375\n",
"INFO:tensorflow:examples/sec: 92.9201\n",
"I0523 05:54:17.684122 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9201\n",
"INFO:tensorflow:global_step/sec: 2.90312\n",
"I0523 05:54:18.028258 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90312\n",
"INFO:tensorflow:examples/sec: 92.8999\n",
"I0523 05:54:18.028504 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8999\n",
"INFO:tensorflow:global_step/sec: 2.88615\n",
"I0523 05:54:18.374671 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88615\n",
"INFO:tensorflow:examples/sec: 92.3569\n",
"I0523 05:54:18.374889 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3569\n",
"INFO:tensorflow:global_step/sec: 2.91833\n",
"I0523 05:54:18.717351 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91833\n",
"INFO:tensorflow:examples/sec: 93.3865\n",
"I0523 05:54:18.717580 140278000138112 tpu_estimator.py:2308] examples/sec: 93.3865\n",
"INFO:tensorflow:global_step/sec: 2.89581\n",
"I0523 05:54:19.062678 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89581\n",
"INFO:tensorflow:examples/sec: 92.666\n",
"I0523 05:54:19.063060 140278000138112 tpu_estimator.py:2308] examples/sec: 92.666\n",
"INFO:tensorflow:global_step/sec: 2.91034\n",
"I0523 05:54:19.406286 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91034\n",
"INFO:tensorflow:examples/sec: 93.1308\n",
"I0523 05:54:19.406518 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1308\n",
"INFO:tensorflow:global_step/sec: 2.87952\n",
"I0523 05:54:19.753561 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87952\n",
"INFO:tensorflow:examples/sec: 92.1448\n",
"I0523 05:54:19.753785 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1448\n",
"INFO:tensorflow:global_step/sec: 2.89293\n",
"I0523 05:54:20.099223 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89293\n",
"INFO:tensorflow:examples/sec: 92.5739\n",
"I0523 05:54:20.099450 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5739\n",
"INFO:tensorflow:global_step/sec: 2.89273\n",
"I0523 05:54:20.444943 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89273\n",
"INFO:tensorflow:examples/sec: 92.5674\n",
"I0523 05:54:20.445197 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5674\n",
"INFO:tensorflow:global_step/sec: 2.88523\n",
"I0523 05:54:20.791513 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88523\n",
"INFO:tensorflow:examples/sec: 92.3273\n",
"I0523 05:54:20.791745 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3273\n",
"INFO:tensorflow:global_step/sec: 2.89545\n",
"I0523 05:54:21.136920 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89545\n",
"INFO:tensorflow:examples/sec: 92.6544\n",
"I0523 05:54:21.137326 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6544\n",
"INFO:tensorflow:global_step/sec: 2.9069\n",
"I0523 05:54:21.480916 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.9069\n",
"INFO:tensorflow:examples/sec: 93.0206\n",
"I0523 05:54:21.481181 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0206\n",
"INFO:tensorflow:global_step/sec: 2.89538\n",
"I0523 05:54:21.826276 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89538\n",
"INFO:tensorflow:examples/sec: 92.652\n",
"I0523 05:54:21.826514 140278000138112 tpu_estimator.py:2308] examples/sec: 92.652\n",
"INFO:tensorflow:global_step/sec: 2.88587\n",
"I0523 05:54:22.172796 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88587\n",
"INFO:tensorflow:examples/sec: 92.348\n",
"I0523 05:54:22.173059 140278000138112 tpu_estimator.py:2308] examples/sec: 92.348\n",
"INFO:tensorflow:global_step/sec: 2.91847\n",
"I0523 05:54:22.515475 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91847\n",
"INFO:tensorflow:examples/sec: 93.3911\n",
"I0523 05:54:22.515956 140278000138112 tpu_estimator.py:2308] examples/sec: 93.3911\n",
"INFO:tensorflow:global_step/sec: 2.88892\n",
"I0523 05:54:22.861635 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88892\n",
"INFO:tensorflow:examples/sec: 92.4454\n",
"I0523 05:54:22.862123 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4454\n",
"INFO:tensorflow:global_step/sec: 2.86694\n",
"I0523 05:54:23.210393 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86694\n",
"INFO:tensorflow:examples/sec: 91.7421\n",
"I0523 05:54:23.210775 140278000138112 tpu_estimator.py:2308] examples/sec: 91.7421\n",
"INFO:tensorflow:global_step/sec: 2.89084\n",
"I0523 05:54:23.556317 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89084\n",
"INFO:tensorflow:examples/sec: 92.507\n",
"I0523 05:54:23.556585 140278000138112 tpu_estimator.py:2308] examples/sec: 92.507\n",
"INFO:tensorflow:global_step/sec: 2.89309\n",
"I0523 05:54:23.901984 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89309\n",
"INFO:tensorflow:examples/sec: 92.5788\n",
"I0523 05:54:23.902477 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5788\n",
"INFO:tensorflow:global_step/sec: 2.90225\n",
"I0523 05:54:24.246543 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90225\n",
"INFO:tensorflow:examples/sec: 92.8719\n",
"I0523 05:54:24.246800 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8719\n",
"INFO:tensorflow:global_step/sec: 2.90423\n",
"I0523 05:54:24.590835 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90423\n",
"INFO:tensorflow:examples/sec: 92.9354\n",
"I0523 05:54:24.591210 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9354\n",
"INFO:tensorflow:global_step/sec: 2.90319\n",
"I0523 05:54:24.935292 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90319\n",
"INFO:tensorflow:examples/sec: 92.9022\n",
"I0523 05:54:24.935504 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9022\n",
"INFO:tensorflow:global_step/sec: 2.90351\n",
"I0523 05:54:25.279727 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90351\n",
"INFO:tensorflow:examples/sec: 92.9123\n",
"I0523 05:54:25.279988 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9123\n",
"INFO:tensorflow:global_step/sec: 2.88589\n",
"I0523 05:54:25.626211 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88589\n",
"INFO:tensorflow:examples/sec: 92.3485\n",
"I0523 05:54:25.626461 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3485\n",
"INFO:tensorflow:global_step/sec: 2.91887\n",
"I0523 05:54:25.968814 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91887\n",
"INFO:tensorflow:examples/sec: 93.4039\n",
"I0523 05:54:25.969052 140278000138112 tpu_estimator.py:2308] examples/sec: 93.4039\n",
"INFO:tensorflow:global_step/sec: 2.8937\n",
"I0523 05:54:26.314394 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8937\n",
"INFO:tensorflow:examples/sec: 92.5982\n",
"I0523 05:54:26.314768 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5982\n",
"INFO:tensorflow:global_step/sec: 2.88706\n",
"I0523 05:54:26.660924 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88706\n",
"INFO:tensorflow:examples/sec: 92.3861\n",
"I0523 05:54:26.661187 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3861\n",
"INFO:tensorflow:global_step/sec: 2.89497\n",
"I0523 05:54:27.006194 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89497\n",
"INFO:tensorflow:examples/sec: 92.639\n",
"I0523 05:54:27.006425 140278000138112 tpu_estimator.py:2308] examples/sec: 92.639\n",
"INFO:tensorflow:global_step/sec: 2.88505\n",
"I0523 05:54:27.352880 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88505\n",
"INFO:tensorflow:examples/sec: 92.3217\n",
"I0523 05:54:27.353148 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3217\n",
"INFO:tensorflow:global_step/sec: 2.90493\n",
"I0523 05:54:27.697118 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90493\n",
"INFO:tensorflow:examples/sec: 92.9578\n",
"I0523 05:54:27.697603 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9578\n",
"INFO:tensorflow:global_step/sec: 2.89873\n",
"I0523 05:54:28.042079 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89873\n",
"INFO:tensorflow:examples/sec: 92.7593\n",
"I0523 05:54:28.042315 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7593\n",
"INFO:tensorflow:global_step/sec: 2.86785\n",
"I0523 05:54:28.390717 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86785\n",
"INFO:tensorflow:examples/sec: 91.7712\n",
"I0523 05:54:28.391000 140278000138112 tpu_estimator.py:2308] examples/sec: 91.7712\n",
"INFO:tensorflow:global_step/sec: 2.88935\n",
"I0523 05:54:28.736890 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88935\n",
"INFO:tensorflow:examples/sec: 92.4593\n",
"I0523 05:54:28.737303 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4593\n",
"INFO:tensorflow:global_step/sec: 2.91973\n",
"I0523 05:54:29.079327 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91973\n",
"INFO:tensorflow:examples/sec: 93.4313\n",
"I0523 05:54:29.079548 140278000138112 tpu_estimator.py:2308] examples/sec: 93.4313\n",
"INFO:tensorflow:global_step/sec: 2.89939\n",
"I0523 05:54:29.424226 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89939\n",
"INFO:tensorflow:examples/sec: 92.7804\n",
"I0523 05:54:29.424584 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7804\n",
"INFO:tensorflow:global_step/sec: 2.92478\n",
"I0523 05:54:29.766189 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.92478\n",
"INFO:tensorflow:examples/sec: 93.593\n",
"I0523 05:54:29.766419 140278000138112 tpu_estimator.py:2308] examples/sec: 93.593\n",
"INFO:tensorflow:global_step/sec: 2.89896\n",
"I0523 05:54:30.111077 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89896\n",
"INFO:tensorflow:examples/sec: 92.7666\n",
"I0523 05:54:30.111437 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7666\n",
"INFO:tensorflow:global_step/sec: 2.88885\n",
"I0523 05:54:30.457249 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88885\n",
"INFO:tensorflow:examples/sec: 92.4432\n",
"I0523 05:54:30.457595 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4432\n",
"INFO:tensorflow:global_step/sec: 2.89345\n",
"I0523 05:54:30.802868 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89345\n",
"INFO:tensorflow:examples/sec: 92.5905\n",
"I0523 05:54:30.803118 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5905\n",
"INFO:tensorflow:global_step/sec: 2.91388\n",
"I0523 05:54:31.146042 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91388\n",
"INFO:tensorflow:examples/sec: 93.2441\n",
"I0523 05:54:31.146317 140278000138112 tpu_estimator.py:2308] examples/sec: 93.2441\n",
"INFO:tensorflow:global_step/sec: 2.88829\n",
"I0523 05:54:31.492253 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88829\n",
"INFO:tensorflow:examples/sec: 92.4254\n",
"I0523 05:54:31.492503 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4254\n",
"INFO:tensorflow:global_step/sec: 2.89809\n",
"I0523 05:54:31.837620 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89809\n",
"INFO:tensorflow:examples/sec: 92.7389\n",
"I0523 05:54:31.837899 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7389\n",
"INFO:tensorflow:global_step/sec: 2.88795\n",
"I0523 05:54:32.183584 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88795\n",
"INFO:tensorflow:examples/sec: 92.4144\n",
"I0523 05:54:32.183826 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4144\n",
"INFO:tensorflow:global_step/sec: 2.90026\n",
"I0523 05:54:32.528378 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90026\n",
"INFO:tensorflow:examples/sec: 92.8083\n",
"I0523 05:54:32.528617 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8083\n",
"INFO:tensorflow:global_step/sec: 2.93072\n",
"I0523 05:54:32.869585 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.93072\n",
"INFO:tensorflow:examples/sec: 93.783\n",
"I0523 05:54:32.869824 140278000138112 tpu_estimator.py:2308] examples/sec: 93.783\n",
"INFO:tensorflow:global_step/sec: 2.89652\n",
"I0523 05:54:33.214818 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89652\n",
"INFO:tensorflow:examples/sec: 92.6888\n",
"I0523 05:54:33.215052 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6888\n",
"INFO:tensorflow:global_step/sec: 2.8753\n",
"I0523 05:54:33.562626 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8753\n",
"INFO:tensorflow:examples/sec: 92.0095\n",
"I0523 05:54:33.562863 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0095\n",
"INFO:tensorflow:global_step/sec: 2.90056\n",
"I0523 05:54:33.907379 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90056\n",
"INFO:tensorflow:examples/sec: 92.8178\n",
"I0523 05:54:33.907620 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8178\n",
"INFO:tensorflow:global_step/sec: 2.88576\n",
"I0523 05:54:34.253892 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88576\n",
"INFO:tensorflow:examples/sec: 92.3442\n",
"I0523 05:54:34.254132 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3442\n",
"INFO:tensorflow:global_step/sec: 2.90017\n",
"I0523 05:54:34.598798 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90017\n",
"INFO:tensorflow:examples/sec: 92.8056\n",
"I0523 05:54:34.599371 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8056\n",
"INFO:tensorflow:global_step/sec: 2.90298\n",
"I0523 05:54:34.943183 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90298\n",
"INFO:tensorflow:examples/sec: 92.8953\n",
"I0523 05:54:34.943420 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8953\n",
"INFO:tensorflow:global_step/sec: 2.90951\n",
"I0523 05:54:35.286887 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90951\n",
"INFO:tensorflow:examples/sec: 93.1044\n",
"I0523 05:54:35.287159 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1044\n",
"INFO:tensorflow:global_step/sec: 2.88327\n",
"I0523 05:54:35.633718 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88327\n",
"INFO:tensorflow:examples/sec: 92.2646\n",
"I0523 05:54:35.634130 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2646\n",
"INFO:tensorflow:global_step/sec: 2.89779\n",
"I0523 05:54:35.978807 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89779\n",
"INFO:tensorflow:examples/sec: 92.7293\n",
"I0523 05:54:35.979201 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7293\n",
"INFO:tensorflow:global_step/sec: 2.93864\n",
"I0523 05:54:36.319123 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.93864\n",
"INFO:tensorflow:examples/sec: 94.0366\n",
"I0523 05:54:36.319561 140278000138112 tpu_estimator.py:2308] examples/sec: 94.0366\n",
"INFO:tensorflow:global_step/sec: 2.90843\n",
"I0523 05:54:36.662929 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90843\n",
"INFO:tensorflow:examples/sec: 93.0696\n",
"I0523 05:54:36.663195 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0696\n",
"INFO:tensorflow:global_step/sec: 2.85315\n",
"I0523 05:54:37.013404 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.85315\n",
"INFO:tensorflow:examples/sec: 91.3007\n",
"I0523 05:54:37.013627 140278000138112 tpu_estimator.py:2308] examples/sec: 91.3007\n",
"INFO:tensorflow:global_step/sec: 2.92206\n",
"I0523 05:54:37.355641 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.92206\n",
"INFO:tensorflow:examples/sec: 93.5058\n",
"I0523 05:54:37.355873 140278000138112 tpu_estimator.py:2308] examples/sec: 93.5058\n",
"INFO:tensorflow:global_step/sec: 2.88977\n",
"I0523 05:54:37.701716 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88977\n",
"INFO:tensorflow:examples/sec: 92.4728\n",
"I0523 05:54:37.701973 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4728\n",
"INFO:tensorflow:global_step/sec: 2.89295\n",
"I0523 05:54:38.047367 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89295\n",
"INFO:tensorflow:examples/sec: 92.5745\n",
"I0523 05:54:38.047605 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5745\n",
"INFO:tensorflow:global_step/sec: 2.91891\n",
"I0523 05:54:38.389938 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91891\n",
"INFO:tensorflow:examples/sec: 93.4052\n",
"I0523 05:54:38.390193 140278000138112 tpu_estimator.py:2308] examples/sec: 93.4052\n",
"INFO:tensorflow:global_step/sec: 2.89634\n",
"I0523 05:54:38.735228 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89634\n",
"INFO:tensorflow:examples/sec: 92.683\n",
"I0523 05:54:38.735459 140278000138112 tpu_estimator.py:2308] examples/sec: 92.683\n",
"INFO:tensorflow:global_step/sec: 2.87686\n",
"I0523 05:54:39.082827 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87686\n",
"INFO:tensorflow:examples/sec: 92.0595\n",
"I0523 05:54:39.083076 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0595\n",
"INFO:tensorflow:global_step/sec: 2.92451\n",
"I0523 05:54:39.424748 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.92451\n",
"INFO:tensorflow:examples/sec: 93.5842\n",
"I0523 05:54:39.425105 140278000138112 tpu_estimator.py:2308] examples/sec: 93.5842\n",
"INFO:tensorflow:global_step/sec: 2.91445\n",
"I0523 05:54:39.767888 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91445\n",
"INFO:tensorflow:examples/sec: 93.2625\n",
"I0523 05:54:39.768153 140278000138112 tpu_estimator.py:2308] examples/sec: 93.2625\n",
"INFO:tensorflow:global_step/sec: 2.86473\n",
"I0523 05:54:40.116959 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86473\n",
"INFO:tensorflow:examples/sec: 91.6715\n",
"I0523 05:54:40.117196 140278000138112 tpu_estimator.py:2308] examples/sec: 91.6715\n",
"INFO:tensorflow:global_step/sec: 2.89491\n",
"I0523 05:54:40.462371 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89491\n",
"INFO:tensorflow:examples/sec: 92.6371\n",
"I0523 05:54:40.462593 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6371\n",
"INFO:tensorflow:global_step/sec: 2.90655\n",
"I0523 05:54:40.806448 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90655\n",
"INFO:tensorflow:examples/sec: 93.0098\n",
"I0523 05:54:40.806826 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0098\n",
"INFO:tensorflow:global_step/sec: 2.87331\n",
"I0523 05:54:41.154448 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87331\n",
"INFO:tensorflow:examples/sec: 91.9459\n",
"I0523 05:54:41.154709 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9459\n",
"INFO:tensorflow:global_step/sec: 2.87153\n",
"I0523 05:54:41.502718 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87153\n",
"INFO:tensorflow:examples/sec: 91.8891\n",
"I0523 05:54:41.502972 140278000138112 tpu_estimator.py:2308] examples/sec: 91.8891\n",
"INFO:tensorflow:global_step/sec: 2.89539\n",
"I0523 05:54:41.848105 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89539\n",
"INFO:tensorflow:examples/sec: 92.6523\n",
"I0523 05:54:41.848340 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6523\n",
"INFO:tensorflow:global_step/sec: 2.8702\n",
"I0523 05:54:42.196508 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8702\n",
"INFO:tensorflow:examples/sec: 91.8463\n",
"I0523 05:54:42.196959 140278000138112 tpu_estimator.py:2308] examples/sec: 91.8463\n",
"INFO:tensorflow:global_step/sec: 2.88924\n",
"I0523 05:54:42.542602 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88924\n",
"INFO:tensorflow:examples/sec: 92.4557\n",
"I0523 05:54:42.543005 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4557\n",
"INFO:tensorflow:global_step/sec: 2.8893\n",
"I0523 05:54:42.888655 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8893\n",
"INFO:tensorflow:examples/sec: 92.4575\n",
"I0523 05:54:42.888919 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4575\n",
"INFO:tensorflow:global_step/sec: 2.88658\n",
"I0523 05:54:43.235154 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88658\n",
"INFO:tensorflow:examples/sec: 92.3705\n",
"I0523 05:54:43.235539 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3705\n",
"INFO:tensorflow:global_step/sec: 2.89902\n",
"I0523 05:54:43.580086 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89902\n",
"INFO:tensorflow:examples/sec: 92.7687\n",
"I0523 05:54:43.580312 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7687\n",
"INFO:tensorflow:global_step/sec: 2.90457\n",
"I0523 05:54:43.924350 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90457\n",
"INFO:tensorflow:examples/sec: 92.9461\n",
"I0523 05:54:43.924570 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9461\n",
"INFO:tensorflow:global_step/sec: 2.88307\n",
"I0523 05:54:44.271241 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88307\n",
"INFO:tensorflow:examples/sec: 92.2584\n",
"I0523 05:54:44.271466 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2584\n",
"INFO:tensorflow:global_step/sec: 2.8917\n",
"I0523 05:54:44.617058 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8917\n",
"INFO:tensorflow:examples/sec: 92.5343\n",
"I0523 05:54:44.617469 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5343\n",
"INFO:tensorflow:global_step/sec: 2.9094\n",
"I0523 05:54:44.960983 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.9094\n",
"INFO:tensorflow:examples/sec: 93.1009\n",
"I0523 05:54:44.961382 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1009\n",
"INFO:tensorflow:global_step/sec: 2.87997\n",
"I0523 05:54:45.307994 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87997\n",
"INFO:tensorflow:examples/sec: 92.1591\n",
"I0523 05:54:45.308414 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1591\n",
"INFO:tensorflow:global_step/sec: 2.89518\n",
"I0523 05:54:45.653384 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89518\n",
"INFO:tensorflow:examples/sec: 92.6456\n",
"I0523 05:54:45.653608 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6456\n",
"INFO:tensorflow:global_step/sec: 2.89929\n",
"I0523 05:54:45.998326 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89929\n",
"INFO:tensorflow:examples/sec: 92.7771\n",
"I0523 05:54:45.998584 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7771\n",
"INFO:tensorflow:global_step/sec: 2.88766\n",
"I0523 05:54:46.344585 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88766\n",
"INFO:tensorflow:examples/sec: 92.4052\n",
"I0523 05:54:46.344805 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4052\n",
"INFO:tensorflow:global_step/sec: 2.86625\n",
"I0523 05:54:46.693470 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86625\n",
"INFO:tensorflow:examples/sec: 91.72\n",
"I0523 05:54:46.693747 140278000138112 tpu_estimator.py:2308] examples/sec: 91.72\n",
"INFO:tensorflow:global_step/sec: 2.90446\n",
"I0523 05:54:47.037781 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90446\n",
"INFO:tensorflow:examples/sec: 92.9428\n",
"I0523 05:54:47.038030 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9428\n",
"INFO:tensorflow:global_step/sec: 2.88572\n",
"I0523 05:54:47.384305 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88572\n",
"INFO:tensorflow:examples/sec: 92.343\n",
"I0523 05:54:47.384538 140278000138112 tpu_estimator.py:2308] examples/sec: 92.343\n",
"INFO:tensorflow:global_step/sec: 2.89821\n",
"I0523 05:54:47.729379 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89821\n",
"INFO:tensorflow:examples/sec: 92.7427\n",
"I0523 05:54:47.729595 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7427\n",
"INFO:tensorflow:global_step/sec: 2.88311\n",
"I0523 05:54:48.076195 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88311\n",
"INFO:tensorflow:examples/sec: 92.2596\n",
"I0523 05:54:48.076419 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2596\n",
"INFO:tensorflow:global_step/sec: 2.90678\n",
"I0523 05:54:48.420234 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90678\n",
"INFO:tensorflow:examples/sec: 93.0171\n",
"I0523 05:54:48.420455 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0171\n",
"INFO:tensorflow:global_step/sec: 2.917\n",
"I0523 05:54:48.763050 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.917\n",
"INFO:tensorflow:examples/sec: 93.3441\n",
"I0523 05:54:48.763428 140278000138112 tpu_estimator.py:2308] examples/sec: 93.3441\n",
"INFO:tensorflow:global_step/sec: 2.88715\n",
"I0523 05:54:49.109388 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88715\n",
"INFO:tensorflow:examples/sec: 92.3887\n",
"I0523 05:54:49.109610 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3887\n",
"INFO:tensorflow:global_step/sec: 2.89458\n",
"I0523 05:54:49.454873 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89458\n",
"INFO:tensorflow:examples/sec: 92.6265\n",
"I0523 05:54:49.455267 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6265\n",
"INFO:tensorflow:global_step/sec: 2.88204\n",
"I0523 05:54:49.801858 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88204\n",
"INFO:tensorflow:examples/sec: 92.2252\n",
"I0523 05:54:49.802091 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2252\n",
"INFO:tensorflow:global_step/sec: 2.8884\n",
"I0523 05:54:50.148089 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8884\n",
"INFO:tensorflow:examples/sec: 92.4289\n",
"I0523 05:54:50.148340 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4289\n",
"INFO:tensorflow:global_step/sec: 2.88141\n",
"I0523 05:54:50.495123 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88141\n",
"INFO:tensorflow:examples/sec: 92.2052\n",
"I0523 05:54:50.495397 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2052\n",
"INFO:tensorflow:global_step/sec: 2.89563\n",
"I0523 05:54:50.840459 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89563\n",
"INFO:tensorflow:examples/sec: 92.6603\n",
"I0523 05:54:50.840856 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6603\n",
"INFO:tensorflow:global_step/sec: 2.87379\n",
"I0523 05:54:51.188434 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87379\n",
"INFO:tensorflow:examples/sec: 91.9612\n",
"I0523 05:54:51.188664 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9612\n",
"INFO:tensorflow:global_step/sec: 2.89845\n",
"I0523 05:54:51.533476 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89845\n",
"INFO:tensorflow:examples/sec: 92.7504\n",
"I0523 05:54:51.533744 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7504\n",
"INFO:tensorflow:global_step/sec: 2.8866\n",
"I0523 05:54:51.879901 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8866\n",
"INFO:tensorflow:examples/sec: 92.3711\n",
"I0523 05:54:51.880159 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3711\n",
"INFO:tensorflow:global_step/sec: 2.89713\n",
"I0523 05:54:52.225086 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89713\n",
"INFO:tensorflow:examples/sec: 92.7081\n",
"I0523 05:54:52.225331 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7081\n",
"INFO:tensorflow:global_step/sec: 2.86951\n",
"I0523 05:54:52.573533 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86951\n",
"INFO:tensorflow:examples/sec: 91.8243\n",
"I0523 05:54:52.573774 140278000138112 tpu_estimator.py:2308] examples/sec: 91.8243\n",
"INFO:tensorflow:global_step/sec: 2.89578\n",
"I0523 05:54:52.918882 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89578\n",
"INFO:tensorflow:examples/sec: 92.6651\n",
"I0523 05:54:52.919172 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6651\n",
"INFO:tensorflow:global_step/sec: 2.90449\n",
"I0523 05:54:53.263172 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90449\n",
"INFO:tensorflow:examples/sec: 92.9437\n",
"I0523 05:54:53.263795 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9437\n",
"INFO:tensorflow:global_step/sec: 2.90219\n",
"I0523 05:54:53.607717 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90219\n",
"INFO:tensorflow:examples/sec: 92.87\n",
"I0523 05:54:53.607935 140278000138112 tpu_estimator.py:2308] examples/sec: 92.87\n",
"INFO:tensorflow:global_step/sec: 2.88827\n",
"I0523 05:54:53.954030 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88827\n",
"INFO:tensorflow:examples/sec: 92.4245\n",
"I0523 05:54:53.954269 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4245\n",
"INFO:tensorflow:global_step/sec: 2.89433\n",
"I0523 05:54:54.299480 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89433\n",
"INFO:tensorflow:examples/sec: 92.6185\n",
"I0523 05:54:54.299863 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6185\n",
"INFO:tensorflow:global_step/sec: 2.85743\n",
"I0523 05:54:54.649441 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.85743\n",
"INFO:tensorflow:examples/sec: 91.4377\n",
"I0523 05:54:54.649693 140278000138112 tpu_estimator.py:2308] examples/sec: 91.4377\n",
"INFO:tensorflow:global_step/sec: 2.85993\n",
"I0523 05:54:54.999127 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.85993\n",
"INFO:tensorflow:examples/sec: 91.5178\n",
"I0523 05:54:54.999391 140278000138112 tpu_estimator.py:2308] examples/sec: 91.5178\n",
"INFO:tensorflow:global_step/sec: 2.91402\n",
"I0523 05:54:55.342256 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91402\n",
"INFO:tensorflow:examples/sec: 93.2485\n",
"I0523 05:54:55.342517 140278000138112 tpu_estimator.py:2308] examples/sec: 93.2485\n",
"INFO:tensorflow:global_step/sec: 2.85933\n",
"I0523 05:54:55.692063 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.85933\n",
"INFO:tensorflow:examples/sec: 91.4984\n",
"I0523 05:54:55.692350 140278000138112 tpu_estimator.py:2308] examples/sec: 91.4984\n",
"INFO:tensorflow:global_step/sec: 2.89847\n",
"I0523 05:54:56.037014 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89847\n",
"INFO:tensorflow:examples/sec: 92.7512\n",
"I0523 05:54:56.037255 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7512\n",
"INFO:tensorflow:global_step/sec: 2.89316\n",
"I0523 05:54:56.382648 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89316\n",
"INFO:tensorflow:examples/sec: 92.5813\n",
"I0523 05:54:56.383129 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5813\n",
"INFO:tensorflow:global_step/sec: 2.87722\n",
"I0523 05:54:56.730198 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87722\n",
"INFO:tensorflow:examples/sec: 92.0711\n",
"I0523 05:54:56.730422 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0711\n",
"INFO:tensorflow:global_step/sec: 2.91322\n",
"I0523 05:54:57.073465 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91322\n",
"INFO:tensorflow:examples/sec: 93.2231\n",
"I0523 05:54:57.073820 140278000138112 tpu_estimator.py:2308] examples/sec: 93.2231\n",
"INFO:tensorflow:global_step/sec: 2.90362\n",
"I0523 05:54:57.417854 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90362\n",
"INFO:tensorflow:examples/sec: 92.9158\n",
"I0523 05:54:57.418256 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9158\n",
"INFO:tensorflow:global_step/sec: 2.89713\n",
"I0523 05:54:57.763024 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89713\n",
"INFO:tensorflow:examples/sec: 92.7082\n",
"I0523 05:54:57.763242 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7082\n",
"INFO:tensorflow:global_step/sec: 2.90697\n",
"I0523 05:54:58.107052 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90697\n",
"INFO:tensorflow:examples/sec: 93.0232\n",
"I0523 05:54:58.107708 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0232\n",
"INFO:tensorflow:global_step/sec: 2.89519\n",
"I0523 05:54:58.452447 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89519\n",
"INFO:tensorflow:examples/sec: 92.6461\n",
"I0523 05:54:58.452834 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6461\n",
"INFO:tensorflow:global_step/sec: 2.88448\n",
"I0523 05:54:58.799216 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88448\n",
"INFO:tensorflow:examples/sec: 92.3033\n",
"I0523 05:54:58.799468 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3033\n",
"INFO:tensorflow:global_step/sec: 2.89536\n",
"I0523 05:54:59.144505 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89536\n",
"INFO:tensorflow:examples/sec: 92.6514\n",
"I0523 05:54:59.144727 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6514\n",
"INFO:tensorflow:global_step/sec: 2.89338\n",
"I0523 05:54:59.490152 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89338\n",
"INFO:tensorflow:examples/sec: 92.5882\n",
"I0523 05:54:59.490591 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5882\n",
"INFO:tensorflow:global_step/sec: 2.89096\n",
"I0523 05:54:59.836038 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89096\n",
"INFO:tensorflow:examples/sec: 92.5108\n",
"I0523 05:54:59.836283 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5108\n",
"INFO:tensorflow:global_step/sec: 2.88254\n",
"I0523 05:55:00.182936 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88254\n",
"INFO:tensorflow:examples/sec: 92.2413\n",
"I0523 05:55:00.183205 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2413\n",
"INFO:tensorflow:global_step/sec: 2.89107\n",
"I0523 05:55:00.528830 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89107\n",
"INFO:tensorflow:examples/sec: 92.5142\n",
"I0523 05:55:00.529224 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5142\n",
"INFO:tensorflow:global_step/sec: 2.9053\n",
"I0523 05:55:00.873085 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.9053\n",
"INFO:tensorflow:examples/sec: 92.9697\n",
"I0523 05:55:00.873331 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9697\n",
"INFO:tensorflow:global_step/sec: 2.88007\n",
"I0523 05:55:01.220221 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88007\n",
"INFO:tensorflow:examples/sec: 92.1622\n",
"I0523 05:55:01.220618 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1622\n",
"INFO:tensorflow:global_step/sec: 2.9067\n",
"I0523 05:55:01.564269 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.9067\n",
"INFO:tensorflow:examples/sec: 93.0143\n",
"I0523 05:55:01.564976 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0143\n",
"INFO:tensorflow:global_step/sec: 2.89291\n",
"I0523 05:55:01.909991 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89291\n",
"INFO:tensorflow:examples/sec: 92.5732\n",
"I0523 05:55:01.910530 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5732\n",
"INFO:tensorflow:global_step/sec: 2.89064\n",
"I0523 05:55:02.255881 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89064\n",
"INFO:tensorflow:examples/sec: 92.5003\n",
"I0523 05:55:02.256267 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5003\n",
"INFO:tensorflow:global_step/sec: 2.88288\n",
"I0523 05:55:02.602800 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88288\n",
"INFO:tensorflow:examples/sec: 92.2522\n",
"I0523 05:55:02.603066 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2522\n",
"INFO:tensorflow:global_step/sec: 2.89877\n",
"I0523 05:55:02.947725 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89877\n",
"INFO:tensorflow:examples/sec: 92.7607\n",
"I0523 05:55:02.947968 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7607\n",
"INFO:tensorflow:global_step/sec: 2.88998\n",
"I0523 05:55:03.293776 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88998\n",
"INFO:tensorflow:examples/sec: 92.4793\n",
"I0523 05:55:03.294160 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4793\n",
"INFO:tensorflow:global_step/sec: 2.88134\n",
"I0523 05:55:03.640848 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88134\n",
"INFO:tensorflow:examples/sec: 92.2028\n",
"I0523 05:55:03.641359 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2028\n",
"INFO:tensorflow:global_step/sec: 2.89547\n",
"I0523 05:55:03.986196 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89547\n",
"INFO:tensorflow:examples/sec: 92.6551\n",
"I0523 05:55:03.986465 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6551\n",
"INFO:tensorflow:global_step/sec: 2.90739\n",
"I0523 05:55:04.330159 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90739\n",
"INFO:tensorflow:examples/sec: 93.0366\n",
"I0523 05:55:04.330519 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0366\n",
"INFO:tensorflow:global_step/sec: 2.8888\n",
"I0523 05:55:04.676311 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8888\n",
"INFO:tensorflow:examples/sec: 92.4415\n",
"I0523 05:55:04.676721 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4415\n",
"INFO:tensorflow:global_step/sec: 2.9038\n",
"I0523 05:55:05.020687 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.9038\n",
"INFO:tensorflow:examples/sec: 92.9215\n",
"I0523 05:55:05.021053 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9215\n",
"INFO:tensorflow:global_step/sec: 2.87949\n",
"I0523 05:55:05.367970 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87949\n",
"INFO:tensorflow:examples/sec: 92.1436\n",
"I0523 05:55:05.368317 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1436\n",
"INFO:tensorflow:global_step/sec: 2.88651\n",
"I0523 05:55:05.714389 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88651\n",
"INFO:tensorflow:examples/sec: 92.3683\n",
"I0523 05:55:05.714766 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3683\n",
"INFO:tensorflow:global_step/sec: 2.8827\n",
"I0523 05:55:06.061306 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8827\n",
"INFO:tensorflow:examples/sec: 92.2463\n",
"I0523 05:55:06.061685 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2463\n",
"INFO:tensorflow:global_step/sec: 2.90508\n",
"I0523 05:55:06.405531 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90508\n",
"INFO:tensorflow:examples/sec: 92.9625\n",
"I0523 05:55:06.405876 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9625\n",
"INFO:tensorflow:global_step/sec: 2.89455\n",
"I0523 05:55:06.751379 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89455\n",
"INFO:tensorflow:examples/sec: 92.6257\n",
"I0523 05:55:06.751624 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6257\n",
"INFO:tensorflow:global_step/sec: 2.86131\n",
"I0523 05:55:07.100475 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86131\n",
"INFO:tensorflow:examples/sec: 91.562\n",
"I0523 05:55:07.100836 140278000138112 tpu_estimator.py:2308] examples/sec: 91.562\n",
"INFO:tensorflow:global_step/sec: 2.89762\n",
"I0523 05:55:07.445625 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89762\n",
"INFO:tensorflow:examples/sec: 92.7238\n",
"I0523 05:55:07.445852 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7238\n",
"INFO:tensorflow:global_step/sec: 2.87615\n",
"I0523 05:55:07.793278 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87615\n",
"INFO:tensorflow:examples/sec: 92.0368\n",
"I0523 05:55:07.793625 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0368\n",
"INFO:tensorflow:global_step/sec: 2.90497\n",
"I0523 05:55:08.137514 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90497\n",
"INFO:tensorflow:examples/sec: 92.9591\n",
"I0523 05:55:08.137871 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9591\n",
"INFO:tensorflow:global_step/sec: 2.89311\n",
"I0523 05:55:08.483190 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89311\n",
"INFO:tensorflow:examples/sec: 92.5795\n",
"I0523 05:55:08.483427 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5795\n",
"INFO:tensorflow:global_step/sec: 2.88\n",
"I0523 05:55:08.830419 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88\n",
"INFO:tensorflow:examples/sec: 92.1599\n",
"I0523 05:55:08.830643 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1599\n",
"INFO:tensorflow:global_step/sec: 2.90848\n",
"I0523 05:55:09.174231 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90848\n",
"INFO:tensorflow:examples/sec: 93.0714\n",
"I0523 05:55:09.174462 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0714\n",
"INFO:tensorflow:global_step/sec: 2.89272\n",
"I0523 05:55:09.519902 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89272\n",
"INFO:tensorflow:examples/sec: 92.5671\n",
"I0523 05:55:09.520242 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5671\n",
"INFO:tensorflow:global_step/sec: 2.89174\n",
"I0523 05:55:09.865723 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89174\n",
"INFO:tensorflow:examples/sec: 92.5358\n",
"I0523 05:55:09.866087 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5358\n",
"INFO:tensorflow:global_step/sec: 2.89461\n",
"I0523 05:55:10.211192 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89461\n",
"INFO:tensorflow:examples/sec: 92.6276\n",
"I0523 05:55:10.211419 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6276\n",
"INFO:tensorflow:global_step/sec: 2.89012\n",
"I0523 05:55:10.557217 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89012\n",
"INFO:tensorflow:examples/sec: 92.4839\n",
"I0523 05:55:10.557438 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4839\n",
"INFO:tensorflow:global_step/sec: 2.89486\n",
"I0523 05:55:10.902641 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89486\n",
"INFO:tensorflow:examples/sec: 92.6355\n",
"I0523 05:55:10.903022 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6355\n",
"INFO:tensorflow:global_step/sec: 2.91131\n",
"I0523 05:55:11.246144 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91131\n",
"INFO:tensorflow:examples/sec: 93.1618\n",
"I0523 05:55:11.246519 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1618\n",
"INFO:tensorflow:global_step/sec: 2.89984\n",
"I0523 05:55:11.590983 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89984\n",
"INFO:tensorflow:examples/sec: 92.7949\n",
"I0523 05:55:11.591351 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7949\n",
"INFO:tensorflow:global_step/sec: 2.89051\n",
"I0523 05:55:11.936973 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89051\n",
"INFO:tensorflow:examples/sec: 92.4963\n",
"I0523 05:55:11.937202 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4963\n",
"INFO:tensorflow:global_step/sec: 2.90402\n",
"I0523 05:55:12.281287 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90402\n",
"INFO:tensorflow:examples/sec: 92.9285\n",
"I0523 05:55:12.281519 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9285\n",
"INFO:tensorflow:global_step/sec: 2.90046\n",
"I0523 05:55:12.626083 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90046\n",
"INFO:tensorflow:examples/sec: 92.8148\n",
"I0523 05:55:12.626465 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8148\n",
"INFO:tensorflow:global_step/sec: 2.90787\n",
"I0523 05:55:12.969962 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90787\n",
"INFO:tensorflow:examples/sec: 93.052\n",
"I0523 05:55:12.970352 140278000138112 tpu_estimator.py:2308] examples/sec: 93.052\n",
"INFO:tensorflow:global_step/sec: 2.88186\n",
"I0523 05:55:13.317009 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88186\n",
"INFO:tensorflow:examples/sec: 92.2196\n",
"I0523 05:55:13.317401 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2196\n",
"INFO:tensorflow:global_step/sec: 2.88571\n",
"I0523 05:55:13.663527 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88571\n",
"INFO:tensorflow:examples/sec: 92.3426\n",
"I0523 05:55:13.663774 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3426\n",
"INFO:tensorflow:global_step/sec: 2.89581\n",
"I0523 05:55:14.008837 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89581\n",
"INFO:tensorflow:examples/sec: 92.666\n",
"I0523 05:55:14.009270 140278000138112 tpu_estimator.py:2308] examples/sec: 92.666\n",
"INFO:tensorflow:global_step/sec: 2.92123\n",
"I0523 05:55:14.351144 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.92123\n",
"INFO:tensorflow:examples/sec: 93.4793\n",
"I0523 05:55:14.351513 140278000138112 tpu_estimator.py:2308] examples/sec: 93.4793\n",
"INFO:tensorflow:global_step/sec: 2.87977\n",
"I0523 05:55:14.698396 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87977\n",
"INFO:tensorflow:examples/sec: 92.1528\n",
"I0523 05:55:14.698628 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1528\n",
"INFO:tensorflow:global_step/sec: 2.87645\n",
"I0523 05:55:15.046096 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87645\n",
"INFO:tensorflow:examples/sec: 92.0463\n",
"I0523 05:55:15.046478 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0463\n",
"INFO:tensorflow:global_step/sec: 2.89841\n",
"I0523 05:55:15.391078 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89841\n",
"INFO:tensorflow:examples/sec: 92.7493\n",
"I0523 05:55:15.391440 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7493\n",
"INFO:tensorflow:global_step/sec: 2.87984\n",
"I0523 05:55:15.738309 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87984\n",
"INFO:tensorflow:examples/sec: 92.1548\n",
"I0523 05:55:15.738699 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1548\n",
"INFO:tensorflow:global_step/sec: 2.87649\n",
"I0523 05:55:16.085938 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87649\n",
"INFO:tensorflow:examples/sec: 92.0478\n",
"I0523 05:55:16.086188 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0478\n",
"INFO:tensorflow:global_step/sec: 2.88033\n",
"I0523 05:55:16.433214 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88033\n",
"INFO:tensorflow:examples/sec: 92.1705\n",
"I0523 05:55:16.433716 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1705\n",
"INFO:tensorflow:global_step/sec: 2.8604\n",
"I0523 05:55:16.782730 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8604\n",
"INFO:tensorflow:examples/sec: 91.5329\n",
"I0523 05:55:16.783124 140278000138112 tpu_estimator.py:2308] examples/sec: 91.5329\n",
"INFO:tensorflow:global_step/sec: 2.90186\n",
"I0523 05:55:17.127277 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90186\n",
"INFO:tensorflow:examples/sec: 92.8595\n",
"I0523 05:55:17.127515 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8595\n",
"INFO:tensorflow:global_step/sec: 2.89932\n",
"I0523 05:55:17.472233 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89932\n",
"INFO:tensorflow:examples/sec: 92.7781\n",
"I0523 05:55:17.472602 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7781\n",
"INFO:tensorflow:global_step/sec: 2.89311\n",
"I0523 05:55:17.817882 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89311\n",
"INFO:tensorflow:examples/sec: 92.5795\n",
"I0523 05:55:17.818273 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5795\n",
"INFO:tensorflow:global_step/sec: 2.89127\n",
"I0523 05:55:18.163794 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89127\n",
"INFO:tensorflow:examples/sec: 92.5207\n",
"I0523 05:55:18.164177 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5207\n",
"INFO:tensorflow:global_step/sec: 2.90386\n",
"I0523 05:55:18.508152 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90386\n",
"INFO:tensorflow:examples/sec: 92.9235\n",
"I0523 05:55:18.508526 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9235\n",
"INFO:tensorflow:global_step/sec: 2.89379\n",
"I0523 05:55:18.853682 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89379\n",
"INFO:tensorflow:examples/sec: 92.6014\n",
"I0523 05:55:18.854058 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6014\n",
"INFO:tensorflow:global_step/sec: 2.89133\n",
"I0523 05:55:19.199533 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89133\n",
"INFO:tensorflow:examples/sec: 92.5225\n",
"I0523 05:55:19.199896 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5225\n",
"INFO:tensorflow:global_step/sec: 2.8779\n",
"I0523 05:55:19.547035 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8779\n",
"INFO:tensorflow:examples/sec: 92.0929\n",
"I0523 05:55:19.547403 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0929\n",
"INFO:tensorflow:global_step/sec: 2.89106\n",
"I0523 05:55:19.892925 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89106\n",
"INFO:tensorflow:examples/sec: 92.514\n",
"I0523 05:55:19.893306 140278000138112 tpu_estimator.py:2308] examples/sec: 92.514\n",
"INFO:tensorflow:global_step/sec: 2.88101\n",
"I0523 05:55:20.240045 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88101\n",
"INFO:tensorflow:examples/sec: 92.1923\n",
"I0523 05:55:20.240457 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1923\n",
"INFO:tensorflow:global_step/sec: 2.91016\n",
"I0523 05:55:20.583662 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91016\n",
"INFO:tensorflow:examples/sec: 93.1251\n",
"I0523 05:55:20.584068 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1251\n",
"INFO:tensorflow:global_step/sec: 2.87704\n",
"I0523 05:55:20.931226 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87704\n",
"INFO:tensorflow:examples/sec: 92.0651\n",
"I0523 05:55:20.931473 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0651\n",
"INFO:tensorflow:global_step/sec: 2.8936\n",
"I0523 05:55:21.276799 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8936\n",
"INFO:tensorflow:examples/sec: 92.595\n",
"I0523 05:55:21.277045 140278000138112 tpu_estimator.py:2308] examples/sec: 92.595\n",
"INFO:tensorflow:global_step/sec: 2.89876\n",
"I0523 05:55:21.621799 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89876\n",
"INFO:tensorflow:examples/sec: 92.7603\n",
"I0523 05:55:21.622170 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7603\n",
"INFO:tensorflow:global_step/sec: 2.88208\n",
"I0523 05:55:21.968779 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88208\n",
"INFO:tensorflow:examples/sec: 92.2266\n",
"I0523 05:55:21.969238 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2266\n",
"INFO:tensorflow:global_step/sec: 2.8738\n",
"I0523 05:55:22.316721 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8738\n",
"INFO:tensorflow:examples/sec: 91.9616\n",
"I0523 05:55:22.317102 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9616\n",
"INFO:tensorflow:global_step/sec: 2.89391\n",
"I0523 05:55:22.662303 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89391\n",
"INFO:tensorflow:examples/sec: 92.6051\n",
"I0523 05:55:22.662667 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6051\n",
"INFO:tensorflow:global_step/sec: 2.88575\n",
"I0523 05:55:23.008820 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88575\n",
"INFO:tensorflow:examples/sec: 92.344\n",
"I0523 05:55:23.009188 140278000138112 tpu_estimator.py:2308] examples/sec: 92.344\n",
"INFO:tensorflow:global_step/sec: 2.87265\n",
"I0523 05:55:23.356971 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87265\n",
"INFO:tensorflow:examples/sec: 91.9249\n",
"I0523 05:55:23.357386 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9249\n",
"INFO:tensorflow:global_step/sec: 2.86082\n",
"I0523 05:55:23.706478 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86082\n",
"INFO:tensorflow:examples/sec: 91.5461\n",
"I0523 05:55:23.706701 140278000138112 tpu_estimator.py:2308] examples/sec: 91.5461\n",
"INFO:tensorflow:global_step/sec: 2.912\n",
"I0523 05:55:24.049916 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.912\n",
"INFO:tensorflow:examples/sec: 93.1838\n",
"I0523 05:55:24.050295 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1838\n",
"INFO:tensorflow:global_step/sec: 2.88805\n",
"I0523 05:55:24.396165 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88805\n",
"INFO:tensorflow:examples/sec: 92.4175\n",
"I0523 05:55:24.396414 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4175\n",
"INFO:tensorflow:global_step/sec: 2.87238\n",
"I0523 05:55:24.744286 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87238\n",
"INFO:tensorflow:examples/sec: 91.9162\n",
"I0523 05:55:24.744640 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9162\n",
"INFO:tensorflow:global_step/sec: 2.91169\n",
"I0523 05:55:25.087713 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91169\n",
"INFO:tensorflow:examples/sec: 93.1742\n",
"I0523 05:55:25.088073 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1742\n",
"INFO:tensorflow:global_step/sec: 2.88513\n",
"I0523 05:55:25.434317 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88513\n",
"INFO:tensorflow:examples/sec: 92.3243\n",
"I0523 05:55:25.434532 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3243\n",
"INFO:tensorflow:global_step/sec: 2.88086\n",
"I0523 05:55:25.781457 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88086\n",
"INFO:tensorflow:examples/sec: 92.1876\n",
"I0523 05:55:25.781685 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1876\n",
"INFO:tensorflow:global_step/sec: 2.9039\n",
"I0523 05:55:26.125810 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.9039\n",
"INFO:tensorflow:examples/sec: 92.9249\n",
"I0523 05:55:26.126220 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9249\n",
"INFO:tensorflow:global_step/sec: 2.88903\n",
"I0523 05:55:26.471977 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88903\n",
"INFO:tensorflow:examples/sec: 92.449\n",
"I0523 05:55:26.472226 140278000138112 tpu_estimator.py:2308] examples/sec: 92.449\n",
"INFO:tensorflow:global_step/sec: 2.86135\n",
"I0523 05:55:26.821456 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86135\n",
"INFO:tensorflow:examples/sec: 91.5632\n",
"I0523 05:55:26.822082 140278000138112 tpu_estimator.py:2308] examples/sec: 91.5632\n",
"INFO:tensorflow:global_step/sec: 2.89105\n",
"I0523 05:55:27.167594 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89105\n",
"INFO:tensorflow:examples/sec: 92.5136\n",
"I0523 05:55:27.168015 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5136\n",
"INFO:tensorflow:global_step/sec: 2.8798\n",
"I0523 05:55:27.514590 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8798\n",
"INFO:tensorflow:examples/sec: 92.1535\n",
"I0523 05:55:27.514993 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1535\n",
"INFO:tensorflow:global_step/sec: 2.88814\n",
"I0523 05:55:27.860876 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88814\n",
"INFO:tensorflow:examples/sec: 92.4205\n",
"I0523 05:55:27.861157 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4205\n",
"INFO:tensorflow:global_step/sec: 2.8847\n",
"I0523 05:55:28.207482 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8847\n",
"INFO:tensorflow:examples/sec: 92.3104\n",
"I0523 05:55:28.207867 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3104\n",
"INFO:tensorflow:global_step/sec: 2.89574\n",
"I0523 05:55:28.552892 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89574\n",
"INFO:tensorflow:examples/sec: 92.6636\n",
"I0523 05:55:28.553271 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6636\n",
"INFO:tensorflow:global_step/sec: 2.905\n",
"I0523 05:55:28.897060 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.905\n",
"INFO:tensorflow:examples/sec: 92.9602\n",
"I0523 05:55:28.897426 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9602\n",
"INFO:tensorflow:global_step/sec: 2.90139\n",
"I0523 05:55:29.241732 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90139\n",
"INFO:tensorflow:examples/sec: 92.8445\n",
"I0523 05:55:29.242171 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8445\n",
"INFO:tensorflow:global_step/sec: 2.91392\n",
"I0523 05:55:29.584899 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91392\n",
"INFO:tensorflow:examples/sec: 93.2455\n",
"I0523 05:55:29.585282 140278000138112 tpu_estimator.py:2308] examples/sec: 93.2455\n",
"INFO:tensorflow:global_step/sec: 2.88612\n",
"I0523 05:55:29.931372 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88612\n",
"INFO:tensorflow:examples/sec: 92.3557\n",
"I0523 05:55:29.931613 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3557\n",
"INFO:tensorflow:global_step/sec: 2.85341\n",
"I0523 05:55:30.281862 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.85341\n",
"INFO:tensorflow:examples/sec: 91.3092\n",
"I0523 05:55:30.282267 140278000138112 tpu_estimator.py:2308] examples/sec: 91.3092\n",
"INFO:tensorflow:global_step/sec: 2.8997\n",
"I0523 05:55:30.626702 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8997\n",
"INFO:tensorflow:examples/sec: 92.7904\n",
"I0523 05:55:30.627138 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7904\n",
"INFO:tensorflow:global_step/sec: 2.91233\n",
"I0523 05:55:30.970099 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91233\n",
"INFO:tensorflow:examples/sec: 93.1947\n",
"I0523 05:55:30.970522 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1947\n",
"INFO:tensorflow:global_step/sec: 2.8999\n",
"I0523 05:55:31.314909 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8999\n",
"INFO:tensorflow:examples/sec: 92.7966\n",
"I0523 05:55:31.315316 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7966\n",
"INFO:tensorflow:global_step/sec: 2.88764\n",
"I0523 05:55:31.661254 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88764\n",
"INFO:tensorflow:examples/sec: 92.4046\n",
"I0523 05:55:31.661720 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4046\n",
"INFO:tensorflow:global_step/sec: 2.86139\n",
"I0523 05:55:32.010694 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86139\n",
"INFO:tensorflow:examples/sec: 91.5646\n",
"I0523 05:55:32.011089 140278000138112 tpu_estimator.py:2308] examples/sec: 91.5646\n",
"INFO:tensorflow:global_step/sec: 2.89896\n",
"I0523 05:55:32.355663 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89896\n",
"INFO:tensorflow:examples/sec: 92.7666\n",
"I0523 05:55:32.356092 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7666\n",
"INFO:tensorflow:global_step/sec: 2.87145\n",
"I0523 05:55:32.703922 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87145\n",
"INFO:tensorflow:examples/sec: 91.8865\n",
"I0523 05:55:32.704332 140278000138112 tpu_estimator.py:2308] examples/sec: 91.8865\n",
"INFO:tensorflow:global_step/sec: 2.88909\n",
"I0523 05:55:33.050048 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88909\n",
"INFO:tensorflow:examples/sec: 92.451\n",
"I0523 05:55:33.050424 140278000138112 tpu_estimator.py:2308] examples/sec: 92.451\n",
"INFO:tensorflow:global_step/sec: 2.8613\n",
"I0523 05:55:33.399538 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8613\n",
"INFO:tensorflow:examples/sec: 91.5615\n",
"I0523 05:55:33.399774 140278000138112 tpu_estimator.py:2308] examples/sec: 91.5615\n",
"INFO:tensorflow:global_step/sec: 2.8797\n",
"I0523 05:55:33.746773 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8797\n",
"INFO:tensorflow:examples/sec: 92.1505\n",
"I0523 05:55:33.747195 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1505\n",
"INFO:tensorflow:global_step/sec: 2.86806\n",
"I0523 05:55:34.095432 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86806\n",
"INFO:tensorflow:examples/sec: 91.778\n",
"I0523 05:55:34.095777 140278000138112 tpu_estimator.py:2308] examples/sec: 91.778\n",
"INFO:tensorflow:global_step/sec: 2.91076\n",
"I0523 05:55:34.438995 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91076\n",
"INFO:tensorflow:examples/sec: 93.1443\n",
"I0523 05:55:34.439347 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1443\n",
"INFO:tensorflow:global_step/sec: 2.91299\n",
"I0523 05:55:34.782293 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91299\n",
"INFO:tensorflow:examples/sec: 93.2158\n",
"I0523 05:55:34.782536 140278000138112 tpu_estimator.py:2308] examples/sec: 93.2158\n",
"INFO:tensorflow:global_step/sec: 2.89227\n",
"I0523 05:55:35.128044 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89227\n",
"INFO:tensorflow:examples/sec: 92.5528\n",
"I0523 05:55:35.128278 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5528\n",
"INFO:tensorflow:global_step/sec: 2.88882\n",
"I0523 05:55:35.474193 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88882\n",
"INFO:tensorflow:examples/sec: 92.4422\n",
"I0523 05:55:35.474418 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4422\n",
"INFO:tensorflow:global_step/sec: 2.90349\n",
"I0523 05:55:35.818659 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90349\n",
"INFO:tensorflow:examples/sec: 92.9118\n",
"I0523 05:55:35.818905 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9118\n",
"INFO:tensorflow:global_step/sec: 2.88926\n",
"I0523 05:55:36.164732 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88926\n",
"INFO:tensorflow:examples/sec: 92.4564\n",
"I0523 05:55:36.164983 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4564\n",
"INFO:tensorflow:global_step/sec: 2.90158\n",
"I0523 05:55:36.509371 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90158\n",
"INFO:tensorflow:examples/sec: 92.8504\n",
"I0523 05:55:36.509608 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8504\n",
"INFO:tensorflow:global_step/sec: 2.90889\n",
"I0523 05:55:36.853140 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90889\n",
"INFO:tensorflow:examples/sec: 93.0846\n",
"I0523 05:55:36.853365 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0846\n",
"INFO:tensorflow:global_step/sec: 2.89926\n",
"I0523 05:55:37.198087 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89926\n",
"INFO:tensorflow:examples/sec: 92.7762\n",
"I0523 05:55:37.198343 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7762\n",
"INFO:tensorflow:global_step/sec: 2.89396\n",
"I0523 05:55:37.543638 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89396\n",
"INFO:tensorflow:examples/sec: 92.6069\n",
"I0523 05:55:37.543874 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6069\n",
"INFO:tensorflow:global_step/sec: 2.90611\n",
"I0523 05:55:37.887706 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90611\n",
"INFO:tensorflow:examples/sec: 92.9954\n",
"I0523 05:55:37.888146 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9954\n",
"INFO:tensorflow:global_step/sec: 2.88729\n",
"I0523 05:55:38.234046 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88729\n",
"INFO:tensorflow:examples/sec: 92.3934\n",
"I0523 05:55:38.234415 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3934\n",
"INFO:tensorflow:global_step/sec: 2.87854\n",
"I0523 05:55:38.581449 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87854\n",
"INFO:tensorflow:examples/sec: 92.1132\n",
"I0523 05:55:38.581676 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1132\n",
"INFO:tensorflow:global_step/sec: 2.89934\n",
"I0523 05:55:38.926358 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89934\n",
"INFO:tensorflow:examples/sec: 92.779\n",
"I0523 05:55:38.926590 140278000138112 tpu_estimator.py:2308] examples/sec: 92.779\n",
"INFO:tensorflow:global_step/sec: 2.89078\n",
"I0523 05:55:39.272291 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89078\n",
"INFO:tensorflow:examples/sec: 92.5049\n",
"I0523 05:55:39.272598 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5049\n",
"INFO:tensorflow:global_step/sec: 2.89126\n",
"I0523 05:55:39.618186 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89126\n",
"INFO:tensorflow:examples/sec: 92.5202\n",
"I0523 05:55:39.618593 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5202\n",
"INFO:tensorflow:global_step/sec: 2.89321\n",
"I0523 05:55:39.963771 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89321\n",
"INFO:tensorflow:examples/sec: 92.5826\n",
"I0523 05:55:39.964052 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5826\n",
"INFO:tensorflow:global_step/sec: 2.8763\n",
"I0523 05:55:40.311450 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8763\n",
"INFO:tensorflow:examples/sec: 92.0415\n",
"I0523 05:55:40.311690 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0415\n",
"INFO:tensorflow:global_step/sec: 2.88576\n",
"I0523 05:55:40.657981 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88576\n",
"INFO:tensorflow:examples/sec: 92.3444\n",
"I0523 05:55:40.658387 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3444\n",
"INFO:tensorflow:global_step/sec: 2.87662\n",
"I0523 05:55:41.005609 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87662\n",
"INFO:tensorflow:examples/sec: 92.0517\n",
"I0523 05:55:41.005856 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0517\n",
"INFO:tensorflow:global_step/sec: 2.89001\n",
"I0523 05:55:41.351618 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89001\n",
"INFO:tensorflow:examples/sec: 92.4803\n",
"I0523 05:55:41.352143 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4803\n",
"INFO:tensorflow:global_step/sec: 2.86503\n",
"I0523 05:55:41.700688 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86503\n",
"INFO:tensorflow:examples/sec: 91.6808\n",
"I0523 05:55:41.700923 140278000138112 tpu_estimator.py:2308] examples/sec: 91.6808\n",
"INFO:tensorflow:global_step/sec: 2.88497\n",
"I0523 05:55:42.047323 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88497\n",
"INFO:tensorflow:examples/sec: 92.319\n",
"I0523 05:55:42.047735 140278000138112 tpu_estimator.py:2308] examples/sec: 92.319\n",
"INFO:tensorflow:global_step/sec: 2.88143\n",
"I0523 05:55:42.394338 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88143\n",
"INFO:tensorflow:examples/sec: 92.2056\n",
"I0523 05:55:42.394566 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2056\n",
"INFO:tensorflow:global_step/sec: 2.88948\n",
"I0523 05:55:42.740412 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88948\n",
"INFO:tensorflow:examples/sec: 92.4634\n",
"I0523 05:55:42.740668 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4634\n",
"INFO:tensorflow:global_step/sec: 2.88914\n",
"I0523 05:55:43.086540 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88914\n",
"INFO:tensorflow:examples/sec: 92.4526\n",
"I0523 05:55:43.086768 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4526\n",
"INFO:tensorflow:global_step/sec: 2.88187\n",
"I0523 05:55:43.433552 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88187\n",
"INFO:tensorflow:examples/sec: 92.2197\n",
"I0523 05:55:43.433770 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2197\n",
"INFO:tensorflow:global_step/sec: 2.91791\n",
"I0523 05:55:43.776249 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91791\n",
"INFO:tensorflow:examples/sec: 93.3732\n",
"I0523 05:55:43.776474 140278000138112 tpu_estimator.py:2308] examples/sec: 93.3732\n",
"INFO:tensorflow:global_step/sec: 2.90246\n",
"I0523 05:55:44.120806 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90246\n",
"INFO:tensorflow:examples/sec: 92.8788\n",
"I0523 05:55:44.121269 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8788\n",
"INFO:tensorflow:global_step/sec: 2.89376\n",
"I0523 05:55:44.466408 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89376\n",
"INFO:tensorflow:examples/sec: 92.6005\n",
"I0523 05:55:44.466640 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6005\n",
"INFO:tensorflow:global_step/sec: 2.89776\n",
"I0523 05:55:44.811449 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89776\n",
"INFO:tensorflow:examples/sec: 92.7284\n",
"I0523 05:55:44.811667 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7284\n",
"INFO:tensorflow:global_step/sec: 2.9041\n",
"I0523 05:55:45.155779 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.9041\n",
"INFO:tensorflow:examples/sec: 92.9313\n",
"I0523 05:55:45.156016 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9313\n",
"INFO:tensorflow:global_step/sec: 2.89745\n",
"I0523 05:55:45.501009 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89745\n",
"INFO:tensorflow:examples/sec: 92.7185\n",
"I0523 05:55:45.501286 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7185\n",
"INFO:tensorflow:global_step/sec: 2.87943\n",
"I0523 05:55:45.848271 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87943\n",
"INFO:tensorflow:examples/sec: 92.1418\n",
"I0523 05:55:45.848807 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1418\n",
"INFO:tensorflow:global_step/sec: 2.89522\n",
"I0523 05:55:46.193634 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89522\n",
"INFO:tensorflow:examples/sec: 92.6471\n",
"I0523 05:55:46.193884 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6471\n",
"INFO:tensorflow:global_step/sec: 2.88201\n",
"I0523 05:55:46.540675 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88201\n",
"INFO:tensorflow:examples/sec: 92.2244\n",
"I0523 05:55:46.540960 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2244\n",
"INFO:tensorflow:global_step/sec: 2.8686\n",
"I0523 05:55:46.889185 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8686\n",
"INFO:tensorflow:examples/sec: 91.7951\n",
"I0523 05:55:46.889421 140278000138112 tpu_estimator.py:2308] examples/sec: 91.7951\n",
"INFO:tensorflow:global_step/sec: 2.89679\n",
"I0523 05:55:47.234473 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89679\n",
"INFO:tensorflow:examples/sec: 92.6972\n",
"I0523 05:55:47.234729 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6972\n",
"INFO:tensorflow:global_step/sec: 2.87141\n",
"I0523 05:55:47.582673 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87141\n",
"INFO:tensorflow:examples/sec: 91.8852\n",
"I0523 05:55:47.582913 140278000138112 tpu_estimator.py:2308] examples/sec: 91.8852\n",
"INFO:tensorflow:global_step/sec: 2.90027\n",
"I0523 05:55:47.927684 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90027\n",
"INFO:tensorflow:examples/sec: 92.8085\n",
"I0523 05:55:47.927915 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8085\n",
"INFO:tensorflow:global_step/sec: 2.88985\n",
"I0523 05:55:48.273490 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88985\n",
"INFO:tensorflow:examples/sec: 92.4753\n",
"I0523 05:55:48.273738 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4753\n",
"INFO:tensorflow:global_step/sec: 2.88218\n",
"I0523 05:55:48.620464 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88218\n",
"INFO:tensorflow:examples/sec: 92.2298\n",
"I0523 05:55:48.620725 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2298\n",
"INFO:tensorflow:global_step/sec: 2.87763\n",
"I0523 05:55:48.967974 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87763\n",
"INFO:tensorflow:examples/sec: 92.0842\n",
"I0523 05:55:48.968211 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0842\n",
"INFO:tensorflow:global_step/sec: 2.87648\n",
"I0523 05:55:49.315619 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87648\n",
"INFO:tensorflow:examples/sec: 92.0473\n",
"I0523 05:55:49.315867 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0473\n",
"INFO:tensorflow:global_step/sec: 2.91034\n",
"I0523 05:55:49.659241 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91034\n",
"INFO:tensorflow:examples/sec: 93.1309\n",
"I0523 05:55:49.659487 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1309\n",
"INFO:tensorflow:global_step/sec: 2.87358\n",
"I0523 05:55:50.007199 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87358\n",
"INFO:tensorflow:examples/sec: 91.9546\n",
"I0523 05:55:50.007456 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9546\n",
"INFO:tensorflow:global_step/sec: 2.90117\n",
"I0523 05:55:50.351928 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90117\n",
"INFO:tensorflow:examples/sec: 92.8376\n",
"I0523 05:55:50.352341 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8376\n",
"INFO:tensorflow:global_step/sec: 2.89055\n",
"I0523 05:55:50.697855 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89055\n",
"INFO:tensorflow:examples/sec: 92.4975\n",
"I0523 05:55:50.698100 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4975\n",
"INFO:tensorflow:global_step/sec: 2.89137\n",
"I0523 05:55:51.043753 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89137\n",
"INFO:tensorflow:examples/sec: 92.5238\n",
"I0523 05:55:51.044201 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5238\n",
"INFO:tensorflow:global_step/sec: 2.88546\n",
"I0523 05:55:51.390224 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88546\n",
"INFO:tensorflow:examples/sec: 92.3346\n",
"I0523 05:55:51.390486 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3346\n",
"INFO:tensorflow:global_step/sec: 2.89297\n",
"I0523 05:55:51.736002 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89297\n",
"INFO:tensorflow:examples/sec: 92.575\n",
"I0523 05:55:51.736233 140278000138112 tpu_estimator.py:2308] examples/sec: 92.575\n",
"INFO:tensorflow:global_step/sec: 2.86952\n",
"I0523 05:55:52.084452 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86952\n",
"INFO:tensorflow:examples/sec: 91.8247\n",
"I0523 05:55:52.084825 140278000138112 tpu_estimator.py:2308] examples/sec: 91.8247\n",
"INFO:tensorflow:global_step/sec: 2.88118\n",
"I0523 05:55:52.431514 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88118\n",
"INFO:tensorflow:examples/sec: 92.1977\n",
"I0523 05:55:52.431907 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1977\n",
"INFO:tensorflow:global_step/sec: 2.89213\n",
"I0523 05:55:52.777316 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89213\n",
"INFO:tensorflow:examples/sec: 92.5481\n",
"I0523 05:55:52.777572 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5481\n",
"INFO:tensorflow:global_step/sec: 2.87384\n",
"I0523 05:55:53.125241 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87384\n",
"INFO:tensorflow:examples/sec: 91.9628\n",
"I0523 05:55:53.125479 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9628\n",
"INFO:tensorflow:global_step/sec: 2.90283\n",
"I0523 05:55:53.469730 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90283\n",
"INFO:tensorflow:examples/sec: 92.8905\n",
"I0523 05:55:53.469966 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8905\n",
"INFO:tensorflow:global_step/sec: 2.88327\n",
"I0523 05:55:53.816560 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88327\n",
"INFO:tensorflow:examples/sec: 92.2646\n",
"I0523 05:55:53.816784 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2646\n",
"INFO:tensorflow:global_step/sec: 2.87671\n",
"I0523 05:55:54.164212 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87671\n",
"INFO:tensorflow:examples/sec: 92.0548\n",
"I0523 05:55:54.164616 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0548\n",
"INFO:tensorflow:global_step/sec: 2.90224\n",
"I0523 05:55:54.508749 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90224\n",
"INFO:tensorflow:examples/sec: 92.8717\n",
"I0523 05:55:54.509074 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8717\n",
"INFO:tensorflow:global_step/sec: 2.8742\n",
"I0523 05:55:54.856661 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8742\n",
"INFO:tensorflow:examples/sec: 91.9746\n",
"I0523 05:55:54.856902 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9746\n",
"INFO:tensorflow:global_step/sec: 2.88155\n",
"I0523 05:55:55.203721 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88155\n",
"INFO:tensorflow:examples/sec: 92.2095\n",
"I0523 05:55:55.203986 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2095\n",
"INFO:tensorflow:global_step/sec: 2.89922\n",
"I0523 05:55:55.548614 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89922\n",
"INFO:tensorflow:examples/sec: 92.7751\n",
"I0523 05:55:55.548856 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7751\n",
"INFO:tensorflow:global_step/sec: 2.89375\n",
"I0523 05:55:55.894233 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89375\n",
"INFO:tensorflow:examples/sec: 92.6\n",
"I0523 05:55:55.894707 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6\n",
"INFO:tensorflow:global_step/sec: 2.89883\n",
"I0523 05:55:56.239171 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89883\n",
"INFO:tensorflow:examples/sec: 92.7625\n",
"I0523 05:55:56.239401 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7625\n",
"INFO:tensorflow:global_step/sec: 2.87644\n",
"I0523 05:55:56.586819 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87644\n",
"INFO:tensorflow:examples/sec: 92.0462\n",
"I0523 05:55:56.587208 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0462\n",
"INFO:tensorflow:global_step/sec: 2.87967\n",
"I0523 05:55:56.934096 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87967\n",
"INFO:tensorflow:examples/sec: 92.1493\n",
"I0523 05:55:56.934493 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1493\n",
"INFO:tensorflow:global_step/sec: 2.90249\n",
"I0523 05:55:57.278609 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90249\n",
"INFO:tensorflow:examples/sec: 92.8797\n",
"I0523 05:55:57.278889 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8797\n",
"INFO:tensorflow:global_step/sec: 2.89445\n",
"I0523 05:55:57.624112 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89445\n",
"INFO:tensorflow:examples/sec: 92.6223\n",
"I0523 05:55:57.624587 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6223\n",
"INFO:tensorflow:global_step/sec: 2.88038\n",
"I0523 05:55:57.971272 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88038\n",
"INFO:tensorflow:examples/sec: 92.1721\n",
"I0523 05:55:57.971495 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1721\n",
"INFO:tensorflow:global_step/sec: 2.89042\n",
"I0523 05:55:58.317337 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89042\n",
"INFO:tensorflow:examples/sec: 92.4935\n",
"I0523 05:55:58.317579 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4935\n",
"INFO:tensorflow:global_step/sec: 2.88816\n",
"I0523 05:55:58.663497 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88816\n",
"INFO:tensorflow:examples/sec: 92.4212\n",
"I0523 05:55:58.663745 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4212\n",
"INFO:tensorflow:global_step/sec: 2.87634\n",
"I0523 05:55:59.011172 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87634\n",
"INFO:tensorflow:examples/sec: 92.0427\n",
"I0523 05:55:59.011431 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0427\n",
"INFO:tensorflow:global_step/sec: 2.88404\n",
"I0523 05:55:59.357911 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88404\n",
"INFO:tensorflow:examples/sec: 92.2892\n",
"I0523 05:55:59.358315 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2892\n",
"INFO:tensorflow:global_step/sec: 2.88067\n",
"I0523 05:55:59.705060 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88067\n",
"INFO:tensorflow:examples/sec: 92.1814\n",
"I0523 05:55:59.705289 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1814\n",
"INFO:tensorflow:global_step/sec: 2.87327\n",
"I0523 05:56:00.053076 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87327\n",
"INFO:tensorflow:examples/sec: 91.9445\n",
"I0523 05:56:00.053330 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9445\n",
"INFO:tensorflow:global_step/sec: 2.89004\n",
"I0523 05:56:00.399122 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89004\n",
"INFO:tensorflow:examples/sec: 92.4814\n",
"I0523 05:56:00.399389 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4814\n",
"INFO:tensorflow:global_step/sec: 2.90772\n",
"I0523 05:56:00.743017 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90772\n",
"INFO:tensorflow:examples/sec: 93.0471\n",
"I0523 05:56:00.743250 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0471\n",
"INFO:tensorflow:global_step/sec: 2.88439\n",
"I0523 05:56:01.089684 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88439\n",
"INFO:tensorflow:examples/sec: 92.3005\n",
"I0523 05:56:01.090031 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3005\n",
"INFO:tensorflow:global_step/sec: 2.89227\n",
"I0523 05:56:01.435451 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89227\n",
"INFO:tensorflow:examples/sec: 92.5528\n",
"I0523 05:56:01.435696 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5528\n",
"INFO:tensorflow:global_step/sec: 2.91547\n",
"I0523 05:56:01.778436 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91547\n",
"INFO:tensorflow:examples/sec: 93.295\n",
"I0523 05:56:01.778686 140278000138112 tpu_estimator.py:2308] examples/sec: 93.295\n",
"INFO:tensorflow:global_step/sec: 2.86729\n",
"I0523 05:56:02.127198 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86729\n",
"INFO:tensorflow:examples/sec: 91.7532\n",
"I0523 05:56:02.127430 140278000138112 tpu_estimator.py:2308] examples/sec: 91.7532\n",
"INFO:tensorflow:global_step/sec: 2.88249\n",
"I0523 05:56:02.474149 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88249\n",
"INFO:tensorflow:examples/sec: 92.2398\n",
"I0523 05:56:02.474390 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2398\n",
"INFO:tensorflow:global_step/sec: 2.88997\n",
"I0523 05:56:02.820158 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88997\n",
"INFO:tensorflow:examples/sec: 92.4792\n",
"I0523 05:56:02.820390 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4792\n",
"INFO:tensorflow:global_step/sec: 2.87506\n",
"I0523 05:56:03.168043 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87506\n",
"INFO:tensorflow:examples/sec: 92.0018\n",
"I0523 05:56:03.168300 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0018\n",
"INFO:tensorflow:global_step/sec: 2.89353\n",
"I0523 05:56:03.513566 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89353\n",
"INFO:tensorflow:examples/sec: 92.5929\n",
"I0523 05:56:03.513784 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5929\n",
"INFO:tensorflow:global_step/sec: 2.90249\n",
"I0523 05:56:03.858093 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90249\n",
"INFO:tensorflow:examples/sec: 92.8797\n",
"I0523 05:56:03.858342 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8797\n",
"INFO:tensorflow:global_step/sec: 2.88495\n",
"I0523 05:56:04.204715 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88495\n",
"INFO:tensorflow:examples/sec: 92.3183\n",
"I0523 05:56:04.204958 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3183\n",
"INFO:tensorflow:global_step/sec: 2.89052\n",
"I0523 05:56:04.550683 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89052\n",
"INFO:tensorflow:examples/sec: 92.4967\n",
"I0523 05:56:04.550931 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4967\n",
"INFO:tensorflow:global_step/sec: 2.90124\n",
"I0523 05:56:04.895371 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90124\n",
"INFO:tensorflow:examples/sec: 92.8398\n",
"I0523 05:56:04.895631 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8398\n",
"INFO:tensorflow:global_step/sec: 2.87207\n",
"I0523 05:56:05.243566 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87207\n",
"INFO:tensorflow:examples/sec: 91.9062\n",
"I0523 05:56:05.243845 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9062\n",
"INFO:tensorflow:global_step/sec: 2.9015\n",
"I0523 05:56:05.588184 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.9015\n",
"INFO:tensorflow:examples/sec: 92.8481\n",
"I0523 05:56:05.588621 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8481\n",
"INFO:tensorflow:global_step/sec: 2.88958\n",
"I0523 05:56:05.934289 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88958\n",
"INFO:tensorflow:examples/sec: 92.4666\n",
"I0523 05:56:05.934534 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4666\n",
"INFO:tensorflow:global_step/sec: 2.88554\n",
"I0523 05:56:06.280836 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88554\n",
"INFO:tensorflow:examples/sec: 92.3374\n",
"I0523 05:56:06.281218 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3374\n",
"INFO:tensorflow:global_step/sec: 2.89714\n",
"I0523 05:56:06.626077 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89714\n",
"INFO:tensorflow:examples/sec: 92.7086\n",
"I0523 05:56:06.626647 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7086\n",
"INFO:tensorflow:global_step/sec: 2.8692\n",
"I0523 05:56:06.974524 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8692\n",
"INFO:tensorflow:examples/sec: 91.8146\n",
"I0523 05:56:06.974781 140278000138112 tpu_estimator.py:2308] examples/sec: 91.8146\n",
"INFO:tensorflow:global_step/sec: 2.89917\n",
"I0523 05:56:07.319456 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89917\n",
"INFO:tensorflow:examples/sec: 92.7733\n",
"I0523 05:56:07.319693 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7733\n",
"INFO:tensorflow:global_step/sec: 2.89429\n",
"I0523 05:56:07.664983 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89429\n",
"INFO:tensorflow:examples/sec: 92.6173\n",
"I0523 05:56:07.665429 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6173\n",
"INFO:tensorflow:global_step/sec: 2.89171\n",
"I0523 05:56:08.010779 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89171\n",
"INFO:tensorflow:examples/sec: 92.5347\n",
"I0523 05:56:08.011164 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5347\n",
"INFO:tensorflow:global_step/sec: 2.89273\n",
"I0523 05:56:08.356445 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89273\n",
"INFO:tensorflow:examples/sec: 92.5675\n",
"I0523 05:56:08.356673 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5675\n",
"INFO:tensorflow:global_step/sec: 2.90193\n",
"I0523 05:56:08.701088 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90193\n",
"INFO:tensorflow:examples/sec: 92.8619\n",
"I0523 05:56:08.701318 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8619\n",
"INFO:tensorflow:global_step/sec: 2.89639\n",
"I0523 05:56:09.046306 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89639\n",
"INFO:tensorflow:examples/sec: 92.6846\n",
"I0523 05:56:09.046543 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6846\n",
"INFO:tensorflow:global_step/sec: 2.87515\n",
"I0523 05:56:09.394132 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87515\n",
"INFO:tensorflow:examples/sec: 92.0048\n",
"I0523 05:56:09.394374 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0048\n",
"INFO:tensorflow:global_step/sec: 2.86605\n",
"I0523 05:56:09.743051 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86605\n",
"INFO:tensorflow:examples/sec: 91.7135\n",
"I0523 05:56:09.743294 140278000138112 tpu_estimator.py:2308] examples/sec: 91.7135\n",
"INFO:tensorflow:global_step/sec: 2.88012\n",
"I0523 05:56:10.090267 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88012\n",
"INFO:tensorflow:examples/sec: 92.1637\n",
"I0523 05:56:10.090496 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1637\n",
"INFO:tensorflow:global_step/sec: 2.89187\n",
"I0523 05:56:10.436056 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89187\n",
"INFO:tensorflow:examples/sec: 92.5398\n",
"I0523 05:56:10.436334 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5398\n",
"INFO:tensorflow:global_step/sec: 2.87151\n",
"I0523 05:56:10.784290 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87151\n",
"INFO:tensorflow:examples/sec: 91.8882\n",
"I0523 05:56:10.784514 140278000138112 tpu_estimator.py:2308] examples/sec: 91.8882\n",
"INFO:tensorflow:global_step/sec: 2.88568\n",
"I0523 05:56:11.130827 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88568\n",
"INFO:tensorflow:examples/sec: 92.3419\n",
"I0523 05:56:11.131062 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3419\n",
"INFO:tensorflow:global_step/sec: 2.88966\n",
"I0523 05:56:11.476894 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88966\n",
"INFO:tensorflow:examples/sec: 92.469\n",
"I0523 05:56:11.477314 140278000138112 tpu_estimator.py:2308] examples/sec: 92.469\n",
"INFO:tensorflow:global_step/sec: 2.89836\n",
"I0523 05:56:11.821934 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89836\n",
"INFO:tensorflow:examples/sec: 92.7475\n",
"I0523 05:56:11.822402 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7475\n",
"INFO:tensorflow:global_step/sec: 2.89761\n",
"I0523 05:56:12.167045 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89761\n",
"INFO:tensorflow:examples/sec: 92.7234\n",
"I0523 05:56:12.167303 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7234\n",
"INFO:tensorflow:global_step/sec: 2.89444\n",
"I0523 05:56:12.512507 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89444\n",
"INFO:tensorflow:examples/sec: 92.6221\n",
"I0523 05:56:12.512734 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6221\n",
"INFO:tensorflow:global_step/sec: 2.89222\n",
"I0523 05:56:12.858263 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89222\n",
"INFO:tensorflow:examples/sec: 92.5509\n",
"I0523 05:56:12.858485 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5509\n",
"INFO:tensorflow:global_step/sec: 2.8905\n",
"I0523 05:56:13.204228 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8905\n",
"INFO:tensorflow:examples/sec: 92.496\n",
"I0523 05:56:13.204466 140278000138112 tpu_estimator.py:2308] examples/sec: 92.496\n",
"INFO:tensorflow:global_step/sec: 2.88082\n",
"I0523 05:56:13.551363 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88082\n",
"INFO:tensorflow:examples/sec: 92.1864\n",
"I0523 05:56:13.551724 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1864\n",
"INFO:tensorflow:global_step/sec: 2.88984\n",
"I0523 05:56:13.897408 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88984\n",
"INFO:tensorflow:examples/sec: 92.475\n",
"I0523 05:56:13.897655 140278000138112 tpu_estimator.py:2308] examples/sec: 92.475\n",
"INFO:tensorflow:global_step/sec: 2.90524\n",
"I0523 05:56:14.241601 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90524\n",
"INFO:tensorflow:examples/sec: 92.9676\n",
"I0523 05:56:14.241846 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9676\n",
"INFO:tensorflow:global_step/sec: 2.89118\n",
"I0523 05:56:14.587476 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89118\n",
"INFO:tensorflow:examples/sec: 92.5179\n",
"I0523 05:56:14.587857 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5179\n",
"INFO:tensorflow:global_step/sec: 2.89888\n",
"I0523 05:56:14.932438 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89888\n",
"INFO:tensorflow:examples/sec: 92.7642\n",
"I0523 05:56:14.932824 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7642\n",
"INFO:tensorflow:global_step/sec: 2.90246\n",
"I0523 05:56:15.276980 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90246\n",
"INFO:tensorflow:examples/sec: 92.8786\n",
"I0523 05:56:15.277339 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8786\n",
"INFO:tensorflow:global_step/sec: 2.89474\n",
"I0523 05:56:15.622432 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89474\n",
"INFO:tensorflow:examples/sec: 92.6316\n",
"I0523 05:56:15.622678 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6316\n",
"INFO:tensorflow:global_step/sec: 2.88262\n",
"I0523 05:56:15.969521 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88262\n",
"INFO:tensorflow:examples/sec: 92.2437\n",
"I0523 05:56:15.969865 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2437\n",
"INFO:tensorflow:global_step/sec: 2.89706\n",
"I0523 05:56:16.314520 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89706\n",
"INFO:tensorflow:examples/sec: 92.706\n",
"I0523 05:56:16.314760 140278000138112 tpu_estimator.py:2308] examples/sec: 92.706\n",
"INFO:tensorflow:global_step/sec: 2.87381\n",
"I0523 05:56:16.662492 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87381\n",
"INFO:tensorflow:examples/sec: 91.9619\n",
"I0523 05:56:16.662747 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9619\n",
"INFO:tensorflow:global_step/sec: 2.89182\n",
"I0523 05:56:17.008312 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89182\n",
"INFO:tensorflow:examples/sec: 92.5382\n",
"I0523 05:56:17.008697 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5382\n",
"INFO:tensorflow:global_step/sec: 2.91001\n",
"I0523 05:56:17.351943 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91001\n",
"INFO:tensorflow:examples/sec: 93.1202\n",
"I0523 05:56:17.352201 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1202\n",
"INFO:tensorflow:global_step/sec: 2.89619\n",
"I0523 05:56:17.697214 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89619\n",
"INFO:tensorflow:examples/sec: 92.6782\n",
"I0523 05:56:17.697473 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6782\n",
"INFO:tensorflow:global_step/sec: 2.89169\n",
"I0523 05:56:18.043060 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89169\n",
"INFO:tensorflow:examples/sec: 92.534\n",
"I0523 05:56:18.043440 140278000138112 tpu_estimator.py:2308] examples/sec: 92.534\n",
"INFO:tensorflow:global_step/sec: 2.89547\n",
"I0523 05:56:18.388392 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89547\n",
"INFO:tensorflow:examples/sec: 92.6551\n",
"I0523 05:56:18.388807 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6551\n",
"INFO:tensorflow:global_step/sec: 2.90671\n",
"I0523 05:56:18.732427 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90671\n",
"INFO:tensorflow:examples/sec: 93.0148\n",
"I0523 05:56:18.732647 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0148\n",
"INFO:tensorflow:global_step/sec: 2.89134\n",
"I0523 05:56:19.078274 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89134\n",
"INFO:tensorflow:examples/sec: 92.5229\n",
"I0523 05:56:19.078509 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5229\n",
"INFO:tensorflow:global_step/sec: 2.89779\n",
"I0523 05:56:19.423409 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89779\n",
"INFO:tensorflow:examples/sec: 92.7294\n",
"I0523 05:56:19.423820 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7294\n",
"INFO:tensorflow:global_step/sec: 2.89685\n",
"I0523 05:56:19.768587 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89685\n",
"INFO:tensorflow:examples/sec: 92.6991\n",
"I0523 05:56:19.768807 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6991\n",
"INFO:tensorflow:global_step/sec: 2.87738\n",
"I0523 05:56:20.116131 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87738\n",
"INFO:tensorflow:examples/sec: 92.0762\n",
"I0523 05:56:20.116537 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0762\n",
"INFO:tensorflow:global_step/sec: 2.8987\n",
"I0523 05:56:20.461114 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8987\n",
"INFO:tensorflow:examples/sec: 92.7584\n",
"I0523 05:56:20.461362 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7584\n",
"INFO:tensorflow:global_step/sec: 2.88229\n",
"I0523 05:56:20.808066 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88229\n",
"INFO:tensorflow:examples/sec: 92.2332\n",
"I0523 05:56:20.808319 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2332\n",
"INFO:tensorflow:global_step/sec: 2.88127\n",
"I0523 05:56:21.155135 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88127\n",
"INFO:tensorflow:examples/sec: 92.2006\n",
"I0523 05:56:21.155391 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2006\n",
"INFO:tensorflow:global_step/sec: 2.87559\n",
"I0523 05:56:21.502873 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87559\n",
"INFO:tensorflow:examples/sec: 92.019\n",
"I0523 05:56:21.503596 140278000138112 tpu_estimator.py:2308] examples/sec: 92.019\n",
"INFO:tensorflow:global_step/sec: 2.86661\n",
"I0523 05:56:21.851708 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86661\n",
"INFO:tensorflow:examples/sec: 91.7316\n",
"I0523 05:56:21.851938 140278000138112 tpu_estimator.py:2308] examples/sec: 91.7316\n",
"INFO:tensorflow:global_step/sec: 2.90134\n",
"I0523 05:56:22.196380 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90134\n",
"INFO:tensorflow:examples/sec: 92.8427\n",
"I0523 05:56:22.196615 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8427\n",
"INFO:tensorflow:global_step/sec: 2.88784\n",
"I0523 05:56:22.542685 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88784\n",
"INFO:tensorflow:examples/sec: 92.4109\n",
"I0523 05:56:22.543187 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4109\n",
"INFO:tensorflow:global_step/sec: 2.87342\n",
"I0523 05:56:22.890792 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87342\n",
"INFO:tensorflow:examples/sec: 91.9495\n",
"I0523 05:56:22.891434 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9495\n",
"INFO:tensorflow:global_step/sec: 2.88776\n",
"I0523 05:56:23.236983 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88776\n",
"INFO:tensorflow:examples/sec: 92.4084\n",
"I0523 05:56:23.237354 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4084\n",
"INFO:tensorflow:global_step/sec: 2.91361\n",
"I0523 05:56:23.580203 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91361\n",
"INFO:tensorflow:examples/sec: 93.2354\n",
"I0523 05:56:23.580450 140278000138112 tpu_estimator.py:2308] examples/sec: 93.2354\n",
"INFO:tensorflow:global_step/sec: 2.8675\n",
"I0523 05:56:23.928938 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8675\n",
"INFO:tensorflow:examples/sec: 91.7599\n",
"I0523 05:56:23.929338 140278000138112 tpu_estimator.py:2308] examples/sec: 91.7599\n",
"INFO:tensorflow:global_step/sec: 2.91546\n",
"I0523 05:56:24.271906 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91546\n",
"INFO:tensorflow:examples/sec: 93.2947\n",
"I0523 05:56:24.272155 140278000138112 tpu_estimator.py:2308] examples/sec: 93.2947\n",
"INFO:tensorflow:global_step/sec: 2.9143\n",
"I0523 05:56:24.615093 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.9143\n",
"INFO:tensorflow:examples/sec: 93.2577\n",
"I0523 05:56:24.615465 140278000138112 tpu_estimator.py:2308] examples/sec: 93.2577\n",
"INFO:tensorflow:global_step/sec: 2.87037\n",
"I0523 05:56:24.963437 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87037\n",
"INFO:tensorflow:examples/sec: 91.8518\n",
"I0523 05:56:24.963669 140278000138112 tpu_estimator.py:2308] examples/sec: 91.8518\n",
"INFO:tensorflow:global_step/sec: 2.89417\n",
"I0523 05:56:25.308978 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89417\n",
"INFO:tensorflow:examples/sec: 92.6135\n",
"I0523 05:56:25.309195 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6135\n",
"INFO:tensorflow:global_step/sec: 2.90195\n",
"I0523 05:56:25.653506 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90195\n",
"INFO:tensorflow:examples/sec: 92.8623\n",
"I0523 05:56:25.653737 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8623\n",
"INFO:tensorflow:global_step/sec: 2.88807\n",
"I0523 05:56:25.999793 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88807\n",
"INFO:tensorflow:examples/sec: 92.4184\n",
"I0523 05:56:26.000037 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4184\n",
"INFO:tensorflow:global_step/sec: 2.85928\n",
"I0523 05:56:26.349539 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.85928\n",
"INFO:tensorflow:examples/sec: 91.497\n",
"I0523 05:56:26.349758 140278000138112 tpu_estimator.py:2308] examples/sec: 91.497\n",
"INFO:tensorflow:global_step/sec: 2.90478\n",
"I0523 05:56:26.693805 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90478\n",
"INFO:tensorflow:examples/sec: 92.953\n",
"I0523 05:56:26.694080 140278000138112 tpu_estimator.py:2308] examples/sec: 92.953\n",
"INFO:tensorflow:global_step/sec: 2.90372\n",
"I0523 05:56:27.038252 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90372\n",
"INFO:tensorflow:examples/sec: 92.919\n",
"I0523 05:56:27.038492 140278000138112 tpu_estimator.py:2308] examples/sec: 92.919\n",
"INFO:tensorflow:global_step/sec: 2.87189\n",
"I0523 05:56:27.386417 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87189\n",
"INFO:tensorflow:examples/sec: 91.9004\n",
"I0523 05:56:27.386652 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9004\n",
"INFO:tensorflow:global_step/sec: 2.91293\n",
"I0523 05:56:27.729700 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91293\n",
"INFO:tensorflow:examples/sec: 93.2137\n",
"I0523 05:56:27.730080 140278000138112 tpu_estimator.py:2308] examples/sec: 93.2137\n",
"INFO:tensorflow:global_step/sec: 2.89889\n",
"I0523 05:56:28.074649 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89889\n",
"INFO:tensorflow:examples/sec: 92.7645\n",
"I0523 05:56:28.074903 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7645\n",
"INFO:tensorflow:global_step/sec: 2.86558\n",
"I0523 05:56:28.423607 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86558\n",
"INFO:tensorflow:examples/sec: 91.6984\n",
"I0523 05:56:28.423826 140278000138112 tpu_estimator.py:2308] examples/sec: 91.6984\n",
"INFO:tensorflow:global_step/sec: 2.88606\n",
"I0523 05:56:28.770115 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88606\n",
"INFO:tensorflow:examples/sec: 92.354\n",
"I0523 05:56:28.770470 140278000138112 tpu_estimator.py:2308] examples/sec: 92.354\n",
"INFO:tensorflow:global_step/sec: 2.8837\n",
"I0523 05:56:29.116909 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8837\n",
"INFO:tensorflow:examples/sec: 92.2783\n",
"I0523 05:56:29.117178 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2783\n",
"INFO:tensorflow:global_step/sec: 2.87579\n",
"I0523 05:56:29.464609 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87579\n",
"INFO:tensorflow:examples/sec: 92.0254\n",
"I0523 05:56:29.464826 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0254\n",
"INFO:tensorflow:global_step/sec: 2.90504\n",
"I0523 05:56:29.808855 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90504\n",
"INFO:tensorflow:examples/sec: 92.9614\n",
"I0523 05:56:29.809085 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9614\n",
"INFO:tensorflow:global_step/sec: 2.89505\n",
"I0523 05:56:30.154277 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89505\n",
"INFO:tensorflow:examples/sec: 92.6414\n",
"I0523 05:56:30.154493 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6414\n",
"INFO:tensorflow:global_step/sec: 2.89562\n",
"I0523 05:56:30.499607 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89562\n",
"INFO:tensorflow:examples/sec: 92.6598\n",
"I0523 05:56:30.499826 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6598\n",
"INFO:tensorflow:global_step/sec: 2.901\n",
"I0523 05:56:30.844312 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.901\n",
"INFO:tensorflow:examples/sec: 92.8321\n",
"I0523 05:56:30.844538 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8321\n",
"INFO:tensorflow:global_step/sec: 2.89462\n",
"I0523 05:56:31.189806 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89462\n",
"INFO:tensorflow:examples/sec: 92.6278\n",
"I0523 05:56:31.190068 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6278\n",
"INFO:tensorflow:global_step/sec: 2.89959\n",
"I0523 05:56:31.534683 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89959\n",
"INFO:tensorflow:examples/sec: 92.7868\n",
"I0523 05:56:31.534958 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7868\n",
"INFO:tensorflow:global_step/sec: 2.90612\n",
"I0523 05:56:31.878773 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90612\n",
"INFO:tensorflow:examples/sec: 92.996\n",
"I0523 05:56:31.879034 140278000138112 tpu_estimator.py:2308] examples/sec: 92.996\n",
"INFO:tensorflow:global_step/sec: 2.88961\n",
"I0523 05:56:32.224842 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88961\n",
"INFO:tensorflow:examples/sec: 92.4675\n",
"I0523 05:56:32.225225 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4675\n",
"INFO:tensorflow:global_step/sec: 2.89364\n",
"I0523 05:56:32.570431 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89364\n",
"INFO:tensorflow:examples/sec: 92.5965\n",
"I0523 05:56:32.570666 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5965\n",
"INFO:tensorflow:global_step/sec: 2.90395\n",
"I0523 05:56:32.914777 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90395\n",
"INFO:tensorflow:examples/sec: 92.9264\n",
"I0523 05:56:32.915033 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9264\n",
"INFO:tensorflow:global_step/sec: 2.89377\n",
"I0523 05:56:33.260373 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89377\n",
"INFO:tensorflow:examples/sec: 92.6005\n",
"I0523 05:56:33.260606 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6005\n",
"INFO:tensorflow:global_step/sec: 2.89583\n",
"I0523 05:56:33.605667 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89583\n",
"INFO:tensorflow:examples/sec: 92.6665\n",
"I0523 05:56:33.605892 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6665\n",
"INFO:tensorflow:global_step/sec: 2.90509\n",
"I0523 05:56:33.949895 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90509\n",
"INFO:tensorflow:examples/sec: 92.9629\n",
"I0523 05:56:33.950150 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9629\n",
"INFO:tensorflow:global_step/sec: 2.89823\n",
"I0523 05:56:34.294955 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89823\n",
"INFO:tensorflow:examples/sec: 92.7433\n",
"I0523 05:56:34.295169 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7433\n",
"INFO:tensorflow:global_step/sec: 2.87363\n",
"I0523 05:56:34.642938 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87363\n",
"INFO:tensorflow:examples/sec: 91.9563\n",
"I0523 05:56:34.643166 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9563\n",
"INFO:tensorflow:global_step/sec: 2.9149\n",
"I0523 05:56:34.986007 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.9149\n",
"INFO:tensorflow:examples/sec: 93.2769\n",
"I0523 05:56:34.986249 140278000138112 tpu_estimator.py:2308] examples/sec: 93.2769\n",
"INFO:tensorflow:global_step/sec: 2.86323\n",
"I0523 05:56:35.335240 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86323\n",
"INFO:tensorflow:examples/sec: 91.6235\n",
"I0523 05:56:35.335457 140278000138112 tpu_estimator.py:2308] examples/sec: 91.6235\n",
"INFO:tensorflow:global_step/sec: 2.91144\n",
"I0523 05:56:35.678715 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91144\n",
"INFO:tensorflow:examples/sec: 93.1659\n",
"I0523 05:56:35.678965 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1659\n",
"INFO:tensorflow:global_step/sec: 2.88904\n",
"I0523 05:56:36.024861 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88904\n",
"INFO:tensorflow:examples/sec: 92.4494\n",
"I0523 05:56:36.025119 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4494\n",
"INFO:tensorflow:global_step/sec: 2.89742\n",
"I0523 05:56:36.370002 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89742\n",
"INFO:tensorflow:examples/sec: 92.7173\n",
"I0523 05:56:36.370224 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7173\n",
"INFO:tensorflow:global_step/sec: 2.90349\n",
"I0523 05:56:36.714407 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90349\n",
"INFO:tensorflow:examples/sec: 92.9116\n",
"I0523 05:56:36.714650 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9116\n",
"INFO:tensorflow:global_step/sec: 2.89649\n",
"I0523 05:56:37.059664 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89649\n",
"INFO:tensorflow:examples/sec: 92.6876\n",
"I0523 05:56:37.059910 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6876\n",
"INFO:tensorflow:global_step/sec: 2.91399\n",
"I0523 05:56:37.402847 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91399\n",
"INFO:tensorflow:examples/sec: 93.2476\n",
"I0523 05:56:37.403088 140278000138112 tpu_estimator.py:2308] examples/sec: 93.2476\n",
"INFO:tensorflow:global_step/sec: 2.87618\n",
"I0523 05:56:37.750518 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87618\n",
"INFO:tensorflow:examples/sec: 92.0378\n",
"I0523 05:56:37.750807 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0378\n",
"INFO:tensorflow:global_step/sec: 2.90569\n",
"I0523 05:56:38.094670 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90569\n",
"INFO:tensorflow:examples/sec: 92.9822\n",
"I0523 05:56:38.094926 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9822\n",
"INFO:tensorflow:global_step/sec: 2.91742\n",
"I0523 05:56:38.437431 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91742\n",
"INFO:tensorflow:examples/sec: 93.3574\n",
"I0523 05:56:38.437666 140278000138112 tpu_estimator.py:2308] examples/sec: 93.3574\n",
"INFO:tensorflow:global_step/sec: 2.89521\n",
"I0523 05:56:38.782841 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89521\n",
"INFO:tensorflow:examples/sec: 92.6468\n",
"I0523 05:56:38.783220 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6468\n",
"INFO:tensorflow:global_step/sec: 2.8992\n",
"I0523 05:56:39.127761 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8992\n",
"INFO:tensorflow:examples/sec: 92.7745\n",
"I0523 05:56:39.128023 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7745\n",
"INFO:tensorflow:global_step/sec: 2.89026\n",
"I0523 05:56:39.473748 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89026\n",
"INFO:tensorflow:examples/sec: 92.4884\n",
"I0523 05:56:39.474132 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4884\n",
"INFO:tensorflow:global_step/sec: 2.88918\n",
"I0523 05:56:39.819871 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88918\n",
"INFO:tensorflow:examples/sec: 92.4538\n",
"I0523 05:56:39.820123 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4538\n",
"INFO:tensorflow:global_step/sec: 2.891\n",
"I0523 05:56:40.165862 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.891\n",
"INFO:tensorflow:examples/sec: 92.512\n",
"I0523 05:56:40.166249 140278000138112 tpu_estimator.py:2308] examples/sec: 92.512\n",
"INFO:tensorflow:global_step/sec: 2.90001\n",
"I0523 05:56:40.510630 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90001\n",
"INFO:tensorflow:examples/sec: 92.8002\n",
"I0523 05:56:40.510912 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8002\n",
"INFO:tensorflow:global_step/sec: 2.87097\n",
"I0523 05:56:40.858912 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87097\n",
"INFO:tensorflow:examples/sec: 91.8711\n",
"I0523 05:56:40.859159 140278000138112 tpu_estimator.py:2308] examples/sec: 91.8711\n",
"INFO:tensorflow:global_step/sec: 2.89203\n",
"I0523 05:56:41.204689 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89203\n",
"INFO:tensorflow:examples/sec: 92.5449\n",
"I0523 05:56:41.204969 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5449\n",
"INFO:tensorflow:global_step/sec: 2.90385\n",
"I0523 05:56:41.549081 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90385\n",
"INFO:tensorflow:examples/sec: 92.9233\n",
"I0523 05:56:41.549496 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9233\n",
"INFO:tensorflow:global_step/sec: 2.87637\n",
"I0523 05:56:41.896725 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87637\n",
"INFO:tensorflow:examples/sec: 92.0439\n",
"I0523 05:56:41.896978 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0439\n",
"INFO:tensorflow:global_step/sec: 2.87795\n",
"I0523 05:56:42.244181 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87795\n",
"INFO:tensorflow:examples/sec: 92.0943\n",
"I0523 05:56:42.244404 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0943\n",
"INFO:tensorflow:global_step/sec: 2.88644\n",
"I0523 05:56:42.590617 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88644\n",
"INFO:tensorflow:examples/sec: 92.366\n",
"I0523 05:56:42.590841 140278000138112 tpu_estimator.py:2308] examples/sec: 92.366\n",
"INFO:tensorflow:global_step/sec: 2.8722\n",
"I0523 05:56:42.938821 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8722\n",
"INFO:tensorflow:examples/sec: 91.9105\n",
"I0523 05:56:42.939064 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9105\n",
"INFO:tensorflow:global_step/sec: 2.88861\n",
"I0523 05:56:43.285018 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88861\n",
"INFO:tensorflow:examples/sec: 92.4354\n",
"I0523 05:56:43.285275 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4354\n",
"INFO:tensorflow:global_step/sec: 2.90181\n",
"I0523 05:56:43.629595 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90181\n",
"INFO:tensorflow:examples/sec: 92.8579\n",
"I0523 05:56:43.629858 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8579\n",
"INFO:tensorflow:global_step/sec: 2.88476\n",
"I0523 05:56:43.976281 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88476\n",
"INFO:tensorflow:examples/sec: 92.3124\n",
"I0523 05:56:43.976541 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3124\n",
"INFO:tensorflow:global_step/sec: 2.866\n",
"I0523 05:56:44.325231 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.866\n",
"INFO:tensorflow:examples/sec: 91.7121\n",
"I0523 05:56:44.325709 140278000138112 tpu_estimator.py:2308] examples/sec: 91.7121\n",
"INFO:tensorflow:global_step/sec: 2.87732\n",
"I0523 05:56:44.672708 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87732\n",
"INFO:tensorflow:examples/sec: 92.0742\n",
"I0523 05:56:44.673085 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0742\n",
"INFO:tensorflow:global_step/sec: 2.90767\n",
"I0523 05:56:45.016668 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90767\n",
"INFO:tensorflow:examples/sec: 93.0453\n",
"I0523 05:56:45.016903 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0453\n",
"INFO:tensorflow:global_step/sec: 2.87619\n",
"I0523 05:56:45.364305 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87619\n",
"INFO:tensorflow:examples/sec: 92.0381\n",
"I0523 05:56:45.364557 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0381\n",
"INFO:tensorflow:global_step/sec: 2.88249\n",
"I0523 05:56:45.711223 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88249\n",
"INFO:tensorflow:examples/sec: 92.2398\n",
"I0523 05:56:45.711578 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2398\n",
"INFO:tensorflow:global_step/sec: 2.89227\n",
"I0523 05:56:46.056979 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89227\n",
"INFO:tensorflow:examples/sec: 92.5526\n",
"I0523 05:56:46.057204 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5526\n",
"INFO:tensorflow:global_step/sec: 2.89121\n",
"I0523 05:56:46.402852 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89121\n",
"INFO:tensorflow:examples/sec: 92.5187\n",
"I0523 05:56:46.403195 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5187\n",
"INFO:tensorflow:global_step/sec: 2.88013\n",
"I0523 05:56:46.750076 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88013\n",
"INFO:tensorflow:examples/sec: 92.1642\n",
"I0523 05:56:46.750450 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1642\n",
"INFO:tensorflow:global_step/sec: 2.8876\n",
"I0523 05:56:47.096369 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8876\n",
"INFO:tensorflow:examples/sec: 92.4032\n",
"I0523 05:56:47.096743 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4032\n",
"INFO:tensorflow:global_step/sec: 2.88229\n",
"I0523 05:56:47.443315 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88229\n",
"INFO:tensorflow:examples/sec: 92.2332\n",
"I0523 05:56:47.443592 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2332\n",
"INFO:tensorflow:global_step/sec: 2.87868\n",
"I0523 05:56:47.790720 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87868\n",
"INFO:tensorflow:examples/sec: 92.1178\n",
"I0523 05:56:47.790991 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1178\n",
"INFO:tensorflow:global_step/sec: 2.91713\n",
"I0523 05:56:48.133511 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91713\n",
"INFO:tensorflow:examples/sec: 93.348\n",
"I0523 05:56:48.134275 140278000138112 tpu_estimator.py:2308] examples/sec: 93.348\n",
"INFO:tensorflow:global_step/sec: 2.87371\n",
"I0523 05:56:48.481495 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87371\n",
"INFO:tensorflow:examples/sec: 91.9587\n",
"I0523 05:56:48.481911 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9587\n",
"INFO:tensorflow:global_step/sec: 2.90763\n",
"I0523 05:56:48.825398 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90763\n",
"INFO:tensorflow:examples/sec: 93.0441\n",
"I0523 05:56:48.825622 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0441\n",
"INFO:tensorflow:global_step/sec: 2.89087\n",
"I0523 05:56:49.171344 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89087\n",
"INFO:tensorflow:examples/sec: 92.5077\n",
"I0523 05:56:49.171730 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5077\n",
"INFO:tensorflow:global_step/sec: 2.89277\n",
"I0523 05:56:49.517032 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89277\n",
"INFO:tensorflow:examples/sec: 92.5685\n",
"I0523 05:56:49.517399 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5685\n",
"INFO:tensorflow:global_step/sec: 2.8991\n",
"I0523 05:56:49.861976 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8991\n",
"INFO:tensorflow:examples/sec: 92.7712\n",
"I0523 05:56:49.862200 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7712\n",
"INFO:tensorflow:global_step/sec: 2.89453\n",
"I0523 05:56:50.207435 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89453\n",
"INFO:tensorflow:examples/sec: 92.625\n",
"I0523 05:56:50.207873 140278000138112 tpu_estimator.py:2308] examples/sec: 92.625\n",
"INFO:tensorflow:global_step/sec: 2.87636\n",
"I0523 05:56:50.555117 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87636\n",
"INFO:tensorflow:examples/sec: 92.0434\n",
"I0523 05:56:50.555499 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0434\n",
"INFO:tensorflow:global_step/sec: 2.89248\n",
"I0523 05:56:50.900883 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89248\n",
"INFO:tensorflow:examples/sec: 92.5595\n",
"I0523 05:56:50.901141 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5595\n",
"INFO:tensorflow:global_step/sec: 2.89601\n",
"I0523 05:56:51.246144 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89601\n",
"INFO:tensorflow:examples/sec: 92.6723\n",
"I0523 05:56:51.246404 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6723\n",
"INFO:tensorflow:global_step/sec: 2.88633\n",
"I0523 05:56:51.592615 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88633\n",
"INFO:tensorflow:examples/sec: 92.3624\n",
"I0523 05:56:51.593108 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3624\n",
"INFO:tensorflow:global_step/sec: 2.88812\n",
"I0523 05:56:51.938856 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88812\n",
"INFO:tensorflow:examples/sec: 92.4199\n",
"I0523 05:56:51.939404 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4199\n",
"INFO:tensorflow:global_step/sec: 2.9418\n",
"I0523 05:56:52.278799 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.9418\n",
"INFO:tensorflow:examples/sec: 94.1375\n",
"I0523 05:56:52.279065 140278000138112 tpu_estimator.py:2308] examples/sec: 94.1375\n",
"INFO:tensorflow:global_step/sec: 2.89577\n",
"I0523 05:56:52.624118 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89577\n",
"INFO:tensorflow:examples/sec: 92.6647\n",
"I0523 05:56:52.624488 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6647\n",
"INFO:tensorflow:global_step/sec: 2.87468\n",
"I0523 05:56:52.972137 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87468\n",
"INFO:tensorflow:examples/sec: 91.9896\n",
"I0523 05:56:52.972379 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9896\n",
"INFO:tensorflow:global_step/sec: 2.89132\n",
"I0523 05:56:53.317816 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89132\n",
"INFO:tensorflow:examples/sec: 92.5222\n",
"I0523 05:56:53.318233 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5222\n",
"INFO:tensorflow:global_step/sec: 2.91058\n",
"I0523 05:56:53.661382 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91058\n",
"INFO:tensorflow:examples/sec: 93.1386\n",
"I0523 05:56:53.661621 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1386\n",
"INFO:tensorflow:global_step/sec: 2.88041\n",
"I0523 05:56:54.008558 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88041\n",
"INFO:tensorflow:examples/sec: 92.1732\n",
"I0523 05:56:54.008802 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1732\n",
"INFO:tensorflow:global_step/sec: 2.90235\n",
"I0523 05:56:54.353114 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90235\n",
"INFO:tensorflow:examples/sec: 92.8753\n",
"I0523 05:56:54.353495 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8753\n",
"INFO:tensorflow:global_step/sec: 2.89661\n",
"I0523 05:56:54.698352 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89661\n",
"INFO:tensorflow:examples/sec: 92.6915\n",
"I0523 05:56:54.698596 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6915\n",
"INFO:tensorflow:global_step/sec: 2.87605\n",
"I0523 05:56:55.046077 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87605\n",
"INFO:tensorflow:examples/sec: 92.0335\n",
"I0523 05:56:55.046328 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0335\n",
"INFO:tensorflow:global_step/sec: 2.88479\n",
"I0523 05:56:55.392688 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88479\n",
"INFO:tensorflow:examples/sec: 92.3133\n",
"I0523 05:56:55.392913 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3133\n",
"INFO:tensorflow:global_step/sec: 2.90389\n",
"I0523 05:56:55.737084 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90389\n",
"INFO:tensorflow:examples/sec: 92.9245\n",
"I0523 05:56:55.737336 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9245\n",
"INFO:tensorflow:global_step/sec: 2.8666\n",
"I0523 05:56:56.086045 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8666\n",
"INFO:tensorflow:examples/sec: 91.7311\n",
"I0523 05:56:56.086274 140278000138112 tpu_estimator.py:2308] examples/sec: 91.7311\n",
"INFO:tensorflow:global_step/sec: 2.88657\n",
"I0523 05:56:56.432315 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88657\n",
"INFO:tensorflow:examples/sec: 92.3704\n",
"I0523 05:56:56.432536 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3704\n",
"INFO:tensorflow:global_step/sec: 2.90581\n",
"I0523 05:56:56.776464 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90581\n",
"INFO:tensorflow:examples/sec: 92.986\n",
"I0523 05:56:56.776706 140278000138112 tpu_estimator.py:2308] examples/sec: 92.986\n",
"INFO:tensorflow:global_step/sec: 2.86778\n",
"I0523 05:56:57.125157 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86778\n",
"INFO:tensorflow:examples/sec: 91.7689\n",
"I0523 05:56:57.125393 140278000138112 tpu_estimator.py:2308] examples/sec: 91.7689\n",
"INFO:tensorflow:global_step/sec: 2.87894\n",
"I0523 05:56:57.472508 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87894\n",
"INFO:tensorflow:examples/sec: 92.1262\n",
"I0523 05:56:57.472734 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1262\n",
"INFO:tensorflow:global_step/sec: 2.90304\n",
"I0523 05:56:57.817007 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90304\n",
"INFO:tensorflow:examples/sec: 92.8972\n",
"I0523 05:56:57.817233 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8972\n",
"INFO:tensorflow:global_step/sec: 2.887\n",
"I0523 05:56:58.163372 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.887\n",
"INFO:tensorflow:examples/sec: 92.3842\n",
"I0523 05:56:58.163599 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3842\n",
"INFO:tensorflow:global_step/sec: 2.92059\n",
"I0523 05:56:58.505755 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.92059\n",
"INFO:tensorflow:examples/sec: 93.4589\n",
"I0523 05:56:58.505999 140278000138112 tpu_estimator.py:2308] examples/sec: 93.4589\n",
"INFO:tensorflow:global_step/sec: 2.91859\n",
"I0523 05:56:58.848386 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91859\n",
"INFO:tensorflow:examples/sec: 93.3948\n",
"I0523 05:56:58.848633 140278000138112 tpu_estimator.py:2308] examples/sec: 93.3948\n",
"INFO:tensorflow:global_step/sec: 2.86606\n",
"I0523 05:56:59.197306 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86606\n",
"INFO:tensorflow:examples/sec: 91.714\n",
"I0523 05:56:59.197704 140278000138112 tpu_estimator.py:2308] examples/sec: 91.714\n",
"INFO:tensorflow:global_step/sec: 2.88874\n",
"I0523 05:56:59.543459 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88874\n",
"INFO:tensorflow:examples/sec: 92.4396\n",
"I0523 05:56:59.543812 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4396\n",
"INFO:tensorflow:global_step/sec: 2.89969\n",
"I0523 05:56:59.888289 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89969\n",
"INFO:tensorflow:examples/sec: 92.7902\n",
"I0523 05:56:59.888505 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7902\n",
"INFO:tensorflow:global_step/sec: 2.90439\n",
"I0523 05:57:00.232629 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90439\n",
"INFO:tensorflow:examples/sec: 92.9405\n",
"I0523 05:57:00.232849 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9405\n",
"INFO:tensorflow:global_step/sec: 2.88943\n",
"I0523 05:57:00.578739 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88943\n",
"INFO:tensorflow:examples/sec: 92.4616\n",
"I0523 05:57:00.578991 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4616\n",
"INFO:tensorflow:global_step/sec: 2.87292\n",
"I0523 05:57:00.926800 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87292\n",
"INFO:tensorflow:examples/sec: 91.9334\n",
"I0523 05:57:00.927189 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9334\n",
"INFO:tensorflow:global_step/sec: 2.91227\n",
"I0523 05:57:01.270172 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91227\n",
"INFO:tensorflow:examples/sec: 93.1925\n",
"I0523 05:57:01.270531 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1925\n",
"INFO:tensorflow:global_step/sec: 2.89384\n",
"I0523 05:57:01.615739 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89384\n",
"INFO:tensorflow:examples/sec: 92.6028\n",
"I0523 05:57:01.616115 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6028\n",
"INFO:tensorflow:global_step/sec: 2.89391\n",
"I0523 05:57:01.961346 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89391\n",
"INFO:tensorflow:examples/sec: 92.6052\n",
"I0523 05:57:01.961638 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6052\n",
"INFO:tensorflow:global_step/sec: 2.88349\n",
"I0523 05:57:02.308130 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88349\n",
"INFO:tensorflow:examples/sec: 92.2717\n",
"I0523 05:57:02.308541 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2717\n",
"INFO:tensorflow:global_step/sec: 2.90692\n",
"I0523 05:57:02.652139 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90692\n",
"INFO:tensorflow:examples/sec: 93.0215\n",
"I0523 05:57:02.652520 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0215\n",
"INFO:tensorflow:global_step/sec: 2.88256\n",
"I0523 05:57:02.999094 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88256\n",
"INFO:tensorflow:examples/sec: 92.2421\n",
"I0523 05:57:02.999487 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2421\n",
"INFO:tensorflow:global_step/sec: 2.89248\n",
"I0523 05:57:03.344744 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89248\n",
"INFO:tensorflow:examples/sec: 92.5594\n",
"I0523 05:57:03.345144 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5594\n",
"INFO:tensorflow:global_step/sec: 2.89526\n",
"I0523 05:57:03.690244 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89526\n",
"INFO:tensorflow:examples/sec: 92.6482\n",
"I0523 05:57:03.690930 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6482\n",
"INFO:tensorflow:global_step/sec: 2.89101\n",
"I0523 05:57:04.036087 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89101\n",
"INFO:tensorflow:examples/sec: 92.5125\n",
"I0523 05:57:04.036571 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5125\n",
"INFO:tensorflow:global_step/sec: 2.88688\n",
"I0523 05:57:04.382455 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88688\n",
"INFO:tensorflow:examples/sec: 92.38\n",
"I0523 05:57:04.382840 140278000138112 tpu_estimator.py:2308] examples/sec: 92.38\n",
"INFO:tensorflow:global_step/sec: 2.88501\n",
"I0523 05:57:04.729085 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88501\n",
"INFO:tensorflow:examples/sec: 92.3202\n",
"I0523 05:57:04.729612 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3202\n",
"INFO:tensorflow:global_step/sec: 2.8773\n",
"I0523 05:57:05.076593 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8773\n",
"INFO:tensorflow:examples/sec: 92.0736\n",
"I0523 05:57:05.076963 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0736\n",
"INFO:tensorflow:global_step/sec: 2.87314\n",
"I0523 05:57:05.424641 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87314\n",
"INFO:tensorflow:examples/sec: 91.9406\n",
"I0523 05:57:05.425045 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9406\n",
"INFO:tensorflow:global_step/sec: 2.88636\n",
"I0523 05:57:05.771126 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88636\n",
"INFO:tensorflow:examples/sec: 92.3636\n",
"I0523 05:57:05.771495 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3636\n",
"INFO:tensorflow:global_step/sec: 2.88289\n",
"I0523 05:57:06.117990 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88289\n",
"INFO:tensorflow:examples/sec: 92.2525\n",
"I0523 05:57:06.118355 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2525\n",
"INFO:tensorflow:global_step/sec: 2.87369\n",
"I0523 05:57:06.465977 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87369\n",
"INFO:tensorflow:examples/sec: 91.958\n",
"I0523 05:57:06.466293 140278000138112 tpu_estimator.py:2308] examples/sec: 91.958\n",
"INFO:tensorflow:global_step/sec: 2.92006\n",
"I0523 05:57:06.808411 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.92006\n",
"INFO:tensorflow:examples/sec: 93.442\n",
"I0523 05:57:06.808782 140278000138112 tpu_estimator.py:2308] examples/sec: 93.442\n",
"INFO:tensorflow:global_step/sec: 2.90121\n",
"I0523 05:57:07.153141 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90121\n",
"INFO:tensorflow:examples/sec: 92.8387\n",
"I0523 05:57:07.153529 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8387\n",
"INFO:tensorflow:global_step/sec: 2.89685\n",
"I0523 05:57:07.498295 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89685\n",
"INFO:tensorflow:examples/sec: 92.6993\n",
"I0523 05:57:07.498507 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6993\n",
"INFO:tensorflow:global_step/sec: 2.88444\n",
"I0523 05:57:07.845047 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88444\n",
"INFO:tensorflow:examples/sec: 92.3022\n",
"I0523 05:57:07.845414 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3022\n",
"INFO:tensorflow:global_step/sec: 2.87268\n",
"I0523 05:57:08.193169 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87268\n",
"INFO:tensorflow:examples/sec: 91.9257\n",
"I0523 05:57:08.193552 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9257\n",
"INFO:tensorflow:global_step/sec: 2.88425\n",
"I0523 05:57:08.539816 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88425\n",
"INFO:tensorflow:examples/sec: 92.2961\n",
"I0523 05:57:08.540218 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2961\n",
"INFO:tensorflow:global_step/sec: 2.89922\n",
"I0523 05:57:08.884736 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89922\n",
"INFO:tensorflow:examples/sec: 92.775\n",
"I0523 05:57:08.885133 140278000138112 tpu_estimator.py:2308] examples/sec: 92.775\n",
"INFO:tensorflow:global_step/sec: 2.8957\n",
"I0523 05:57:09.230110 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8957\n",
"INFO:tensorflow:examples/sec: 92.6623\n",
"I0523 05:57:09.230533 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6623\n",
"INFO:tensorflow:global_step/sec: 2.9021\n",
"I0523 05:57:09.574643 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.9021\n",
"INFO:tensorflow:examples/sec: 92.8671\n",
"I0523 05:57:09.574879 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8671\n",
"INFO:tensorflow:global_step/sec: 2.88907\n",
"I0523 05:57:09.920808 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88907\n",
"INFO:tensorflow:examples/sec: 92.4501\n",
"I0523 05:57:09.921046 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4501\n",
"INFO:tensorflow:global_step/sec: 2.89013\n",
"I0523 05:57:10.266782 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89013\n",
"INFO:tensorflow:examples/sec: 92.4842\n",
"I0523 05:57:10.267181 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4842\n",
"INFO:tensorflow:global_step/sec: 2.87048\n",
"I0523 05:57:10.615173 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87048\n",
"INFO:tensorflow:examples/sec: 91.8554\n",
"I0523 05:57:10.615421 140278000138112 tpu_estimator.py:2308] examples/sec: 91.8554\n",
"INFO:tensorflow:global_step/sec: 2.91285\n",
"I0523 05:57:10.958460 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91285\n",
"INFO:tensorflow:examples/sec: 93.2112\n",
"I0523 05:57:10.958689 140278000138112 tpu_estimator.py:2308] examples/sec: 93.2112\n",
"INFO:tensorflow:global_step/sec: 2.89128\n",
"I0523 05:57:11.304332 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89128\n",
"INFO:tensorflow:examples/sec: 92.5209\n",
"I0523 05:57:11.304559 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5209\n",
"INFO:tensorflow:global_step/sec: 2.86438\n",
"I0523 05:57:11.653467 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86438\n",
"INFO:tensorflow:examples/sec: 91.6603\n",
"I0523 05:57:11.653703 140278000138112 tpu_estimator.py:2308] examples/sec: 91.6603\n",
"INFO:tensorflow:global_step/sec: 2.91713\n",
"I0523 05:57:11.996281 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91713\n",
"INFO:tensorflow:examples/sec: 93.3483\n",
"I0523 05:57:11.996516 140278000138112 tpu_estimator.py:2308] examples/sec: 93.3483\n",
"INFO:tensorflow:global_step/sec: 2.90116\n",
"I0523 05:57:12.340966 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90116\n",
"INFO:tensorflow:examples/sec: 92.837\n",
"I0523 05:57:12.341346 140278000138112 tpu_estimator.py:2308] examples/sec: 92.837\n",
"INFO:tensorflow:global_step/sec: 2.88838\n",
"I0523 05:57:12.687153 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88838\n",
"INFO:tensorflow:examples/sec: 92.4282\n",
"I0523 05:57:12.687382 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4282\n",
"INFO:tensorflow:global_step/sec: 2.90438\n",
"I0523 05:57:13.031472 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90438\n",
"INFO:tensorflow:examples/sec: 92.9401\n",
"I0523 05:57:13.031723 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9401\n",
"INFO:tensorflow:global_step/sec: 2.89299\n",
"I0523 05:57:13.377160 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89299\n",
"INFO:tensorflow:examples/sec: 92.5758\n",
"I0523 05:57:13.377568 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5758\n",
"INFO:tensorflow:global_step/sec: 2.90312\n",
"I0523 05:57:13.721660 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90312\n",
"INFO:tensorflow:examples/sec: 92.8998\n",
"I0523 05:57:13.721981 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8998\n",
"INFO:tensorflow:global_step/sec: 2.88879\n",
"I0523 05:57:14.067815 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88879\n",
"INFO:tensorflow:examples/sec: 92.4414\n",
"I0523 05:57:14.068081 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4414\n",
"INFO:tensorflow:global_step/sec: 2.89795\n",
"I0523 05:57:14.412827 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89795\n",
"INFO:tensorflow:examples/sec: 92.7345\n",
"I0523 05:57:14.413280 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7345\n",
"INFO:tensorflow:global_step/sec: 2.90056\n",
"I0523 05:57:14.757585 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90056\n",
"INFO:tensorflow:examples/sec: 92.818\n",
"I0523 05:57:14.757813 140278000138112 tpu_estimator.py:2308] examples/sec: 92.818\n",
"INFO:tensorflow:global_step/sec: 2.88451\n",
"I0523 05:57:15.104292 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88451\n",
"INFO:tensorflow:examples/sec: 92.3045\n",
"I0523 05:57:15.104536 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3045\n",
"INFO:tensorflow:global_step/sec: 2.90322\n",
"I0523 05:57:15.448734 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90322\n",
"INFO:tensorflow:examples/sec: 92.903\n",
"I0523 05:57:15.448979 140278000138112 tpu_estimator.py:2308] examples/sec: 92.903\n",
"INFO:tensorflow:global_step/sec: 2.88787\n",
"I0523 05:57:15.794988 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88787\n",
"INFO:tensorflow:examples/sec: 92.4119\n",
"I0523 05:57:15.795207 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4119\n",
"INFO:tensorflow:global_step/sec: 2.88491\n",
"I0523 05:57:16.141708 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88491\n",
"INFO:tensorflow:examples/sec: 92.3172\n",
"I0523 05:57:16.142043 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3172\n",
"INFO:tensorflow:global_step/sec: 2.87661\n",
"I0523 05:57:16.489249 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87661\n",
"INFO:tensorflow:examples/sec: 92.0514\n",
"I0523 05:57:16.489472 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0514\n",
"INFO:tensorflow:global_step/sec: 2.91407\n",
"I0523 05:57:16.832414 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91407\n",
"INFO:tensorflow:examples/sec: 93.2503\n",
"I0523 05:57:16.832658 140278000138112 tpu_estimator.py:2308] examples/sec: 93.2503\n",
"INFO:tensorflow:global_step/sec: 2.88606\n",
"I0523 05:57:17.178897 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88606\n",
"INFO:tensorflow:examples/sec: 92.3538\n",
"I0523 05:57:17.179137 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3538\n",
"INFO:tensorflow:global_step/sec: 2.9001\n",
"I0523 05:57:17.523728 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.9001\n",
"INFO:tensorflow:examples/sec: 92.8032\n",
"I0523 05:57:17.524152 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8032\n",
"INFO:tensorflow:global_step/sec: 2.88772\n",
"I0523 05:57:17.870052 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88772\n",
"INFO:tensorflow:examples/sec: 92.407\n",
"I0523 05:57:17.870301 140278000138112 tpu_estimator.py:2308] examples/sec: 92.407\n",
"INFO:tensorflow:global_step/sec: 2.86785\n",
"I0523 05:57:18.218709 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86785\n",
"INFO:tensorflow:examples/sec: 91.7711\n",
"I0523 05:57:18.218953 140278000138112 tpu_estimator.py:2308] examples/sec: 91.7711\n",
"INFO:tensorflow:global_step/sec: 2.88547\n",
"I0523 05:57:18.565262 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88547\n",
"INFO:tensorflow:examples/sec: 92.335\n",
"I0523 05:57:18.565486 140278000138112 tpu_estimator.py:2308] examples/sec: 92.335\n",
"INFO:tensorflow:global_step/sec: 2.90209\n",
"I0523 05:57:18.909837 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90209\n",
"INFO:tensorflow:examples/sec: 92.8668\n",
"I0523 05:57:18.910060 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8668\n",
"INFO:tensorflow:global_step/sec: 2.88071\n",
"I0523 05:57:19.257014 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88071\n",
"INFO:tensorflow:examples/sec: 92.1826\n",
"I0523 05:57:19.257237 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1826\n",
"INFO:tensorflow:global_step/sec: 2.90777\n",
"I0523 05:57:19.600892 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90777\n",
"INFO:tensorflow:examples/sec: 93.0486\n",
"I0523 05:57:19.601309 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0486\n",
"INFO:tensorflow:global_step/sec: 2.87721\n",
"I0523 05:57:19.948442 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87721\n",
"INFO:tensorflow:examples/sec: 92.0708\n",
"I0523 05:57:19.948810 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0708\n",
"INFO:tensorflow:global_step/sec: 2.88943\n",
"I0523 05:57:20.294552 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88943\n",
"INFO:tensorflow:examples/sec: 92.4617\n",
"I0523 05:57:20.294779 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4617\n",
"INFO:tensorflow:global_step/sec: 2.8892\n",
"I0523 05:57:20.640652 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8892\n",
"INFO:tensorflow:examples/sec: 92.4543\n",
"I0523 05:57:20.640927 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4543\n",
"INFO:tensorflow:global_step/sec: 2.88818\n",
"I0523 05:57:20.986919 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88818\n",
"INFO:tensorflow:examples/sec: 92.4216\n",
"I0523 05:57:20.987303 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4216\n",
"INFO:tensorflow:global_step/sec: 2.8971\n",
"I0523 05:57:21.332088 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8971\n",
"INFO:tensorflow:examples/sec: 92.7073\n",
"I0523 05:57:21.332346 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7073\n",
"INFO:tensorflow:global_step/sec: 2.89709\n",
"I0523 05:57:21.677255 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89709\n",
"INFO:tensorflow:examples/sec: 92.7069\n",
"I0523 05:57:21.677490 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7069\n",
"INFO:tensorflow:global_step/sec: 2.89758\n",
"I0523 05:57:22.022381 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89758\n",
"INFO:tensorflow:examples/sec: 92.7224\n",
"I0523 05:57:22.022827 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7224\n",
"INFO:tensorflow:global_step/sec: 2.90079\n",
"I0523 05:57:22.367117 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90079\n",
"INFO:tensorflow:examples/sec: 92.8253\n",
"I0523 05:57:22.367495 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8253\n",
"INFO:tensorflow:global_step/sec: 2.88714\n",
"I0523 05:57:22.713468 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88714\n",
"INFO:tensorflow:examples/sec: 92.3885\n",
"I0523 05:57:22.713703 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3885\n",
"INFO:tensorflow:global_step/sec: 2.86861\n",
"I0523 05:57:23.062088 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86861\n",
"INFO:tensorflow:examples/sec: 91.7954\n",
"I0523 05:57:23.062457 140278000138112 tpu_estimator.py:2308] examples/sec: 91.7954\n",
"INFO:tensorflow:global_step/sec: 2.91022\n",
"I0523 05:57:23.405672 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91022\n",
"INFO:tensorflow:examples/sec: 93.1271\n",
"I0523 05:57:23.406116 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1271\n",
"INFO:tensorflow:global_step/sec: 2.91213\n",
"I0523 05:57:23.749090 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91213\n",
"INFO:tensorflow:examples/sec: 93.1882\n",
"I0523 05:57:23.749476 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1882\n",
"INFO:tensorflow:global_step/sec: 2.89504\n",
"I0523 05:57:24.094485 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89504\n",
"INFO:tensorflow:examples/sec: 92.6414\n",
"I0523 05:57:24.094712 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6414\n",
"INFO:tensorflow:global_step/sec: 2.89604\n",
"I0523 05:57:24.439794 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89604\n",
"INFO:tensorflow:examples/sec: 92.6732\n",
"I0523 05:57:24.440197 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6732\n",
"INFO:tensorflow:global_step/sec: 2.88117\n",
"I0523 05:57:24.786867 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88117\n",
"INFO:tensorflow:examples/sec: 92.1973\n",
"I0523 05:57:24.787201 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1973\n",
"INFO:tensorflow:global_step/sec: 2.88359\n",
"I0523 05:57:25.133640 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88359\n",
"INFO:tensorflow:examples/sec: 92.2748\n",
"I0523 05:57:25.133867 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2748\n",
"INFO:tensorflow:global_step/sec: 2.91139\n",
"I0523 05:57:25.477145 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91139\n",
"INFO:tensorflow:examples/sec: 93.1646\n",
"I0523 05:57:25.477542 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1646\n",
"INFO:tensorflow:global_step/sec: 2.89957\n",
"I0523 05:57:25.822022 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89957\n",
"INFO:tensorflow:examples/sec: 92.7864\n",
"I0523 05:57:25.822383 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7864\n",
"INFO:tensorflow:global_step/sec: 2.88783\n",
"I0523 05:57:26.168305 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88783\n",
"INFO:tensorflow:examples/sec: 92.4105\n",
"I0523 05:57:26.168665 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4105\n",
"INFO:tensorflow:global_step/sec: 2.88376\n",
"I0523 05:57:26.515106 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88376\n",
"INFO:tensorflow:examples/sec: 92.2803\n",
"I0523 05:57:26.515491 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2803\n",
"INFO:tensorflow:global_step/sec: 2.89142\n",
"I0523 05:57:26.860909 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89142\n",
"INFO:tensorflow:examples/sec: 92.5253\n",
"I0523 05:57:26.861270 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5253\n",
"INFO:tensorflow:global_step/sec: 2.86943\n",
"I0523 05:57:27.209416 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86943\n",
"INFO:tensorflow:examples/sec: 91.8217\n",
"I0523 05:57:27.209818 140278000138112 tpu_estimator.py:2308] examples/sec: 91.8217\n",
"INFO:tensorflow:global_step/sec: 2.87352\n",
"I0523 05:57:27.557413 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87352\n",
"INFO:tensorflow:examples/sec: 91.9526\n",
"I0523 05:57:27.557629 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9526\n",
"INFO:tensorflow:global_step/sec: 2.89181\n",
"I0523 05:57:27.903214 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89181\n",
"INFO:tensorflow:examples/sec: 92.538\n",
"I0523 05:57:27.903431 140278000138112 tpu_estimator.py:2308] examples/sec: 92.538\n",
"INFO:tensorflow:global_step/sec: 2.88692\n",
"I0523 05:57:28.249607 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88692\n",
"INFO:tensorflow:examples/sec: 92.3814\n",
"I0523 05:57:28.250002 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3814\n",
"INFO:tensorflow:global_step/sec: 2.87838\n",
"I0523 05:57:28.597054 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87838\n",
"INFO:tensorflow:examples/sec: 92.1083\n",
"I0523 05:57:28.597446 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1083\n",
"INFO:tensorflow:global_step/sec: 2.89526\n",
"I0523 05:57:28.942506 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89526\n",
"INFO:tensorflow:examples/sec: 92.6483\n",
"I0523 05:57:28.942858 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6483\n",
"INFO:tensorflow:global_step/sec: 2.87753\n",
"I0523 05:57:29.290003 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87753\n",
"INFO:tensorflow:examples/sec: 92.0808\n",
"I0523 05:57:29.290246 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0808\n",
"INFO:tensorflow:global_step/sec: 2.8674\n",
"I0523 05:57:29.638680 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8674\n",
"INFO:tensorflow:examples/sec: 91.7569\n",
"I0523 05:57:29.638919 140278000138112 tpu_estimator.py:2308] examples/sec: 91.7569\n",
"INFO:tensorflow:global_step/sec: 2.88086\n",
"I0523 05:57:29.985805 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88086\n",
"INFO:tensorflow:examples/sec: 92.1874\n",
"I0523 05:57:29.986038 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1874\n",
"INFO:tensorflow:global_step/sec: 2.88026\n",
"I0523 05:57:30.333006 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88026\n",
"INFO:tensorflow:examples/sec: 92.1682\n",
"I0523 05:57:30.333229 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1682\n",
"INFO:tensorflow:global_step/sec: 2.87857\n",
"I0523 05:57:30.680388 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87857\n",
"INFO:tensorflow:examples/sec: 92.1141\n",
"I0523 05:57:30.680609 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1141\n",
"INFO:tensorflow:global_step/sec: 2.89926\n",
"I0523 05:57:31.025306 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89926\n",
"INFO:tensorflow:examples/sec: 92.7764\n",
"I0523 05:57:31.025529 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7764\n",
"INFO:tensorflow:global_step/sec: 2.88248\n",
"I0523 05:57:31.372247 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88248\n",
"INFO:tensorflow:examples/sec: 92.2394\n",
"I0523 05:57:31.372486 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2394\n",
"INFO:tensorflow:global_step/sec: 2.87388\n",
"I0523 05:57:31.720212 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87388\n",
"INFO:tensorflow:examples/sec: 91.9642\n",
"I0523 05:57:31.720448 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9642\n",
"INFO:tensorflow:global_step/sec: 2.86555\n",
"I0523 05:57:32.069181 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86555\n",
"INFO:tensorflow:examples/sec: 91.6976\n",
"I0523 05:57:32.069418 140278000138112 tpu_estimator.py:2308] examples/sec: 91.6976\n",
"INFO:tensorflow:global_step/sec: 2.91887\n",
"I0523 05:57:32.411785 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91887\n",
"INFO:tensorflow:examples/sec: 93.4039\n",
"I0523 05:57:32.412026 140278000138112 tpu_estimator.py:2308] examples/sec: 93.4039\n",
"INFO:tensorflow:global_step/sec: 2.87619\n",
"I0523 05:57:32.759465 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87619\n",
"INFO:tensorflow:examples/sec: 92.0381\n",
"I0523 05:57:32.759701 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0381\n",
"INFO:tensorflow:global_step/sec: 2.888\n",
"I0523 05:57:33.105701 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.888\n",
"INFO:tensorflow:examples/sec: 92.4161\n",
"I0523 05:57:33.105921 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4161\n",
"INFO:tensorflow:global_step/sec: 2.90977\n",
"I0523 05:57:33.449383 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90977\n",
"INFO:tensorflow:examples/sec: 93.1127\n",
"I0523 05:57:33.449602 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1127\n",
"INFO:tensorflow:global_step/sec: 2.8781\n",
"I0523 05:57:33.796820 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8781\n",
"INFO:tensorflow:examples/sec: 92.0993\n",
"I0523 05:57:33.797217 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0993\n",
"INFO:tensorflow:global_step/sec: 2.87685\n",
"I0523 05:57:34.144379 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87685\n",
"INFO:tensorflow:examples/sec: 92.0591\n",
"I0523 05:57:34.144594 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0591\n",
"INFO:tensorflow:global_step/sec: 2.88177\n",
"I0523 05:57:34.491453 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88177\n",
"INFO:tensorflow:examples/sec: 92.2166\n",
"I0523 05:57:34.491701 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2166\n",
"INFO:tensorflow:global_step/sec: 2.8739\n",
"I0523 05:57:34.839408 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8739\n",
"INFO:tensorflow:examples/sec: 91.9647\n",
"I0523 05:57:34.839652 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9647\n",
"INFO:tensorflow:global_step/sec: 2.88062\n",
"I0523 05:57:35.186564 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88062\n",
"INFO:tensorflow:examples/sec: 92.1798\n",
"I0523 05:57:35.186839 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1798\n",
"INFO:tensorflow:global_step/sec: 2.8872\n",
"I0523 05:57:35.532925 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8872\n",
"INFO:tensorflow:examples/sec: 92.3903\n",
"I0523 05:57:35.533162 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3903\n",
"INFO:tensorflow:global_step/sec: 2.91278\n",
"I0523 05:57:35.876300 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91278\n",
"INFO:tensorflow:examples/sec: 93.209\n",
"I0523 05:57:35.876758 140278000138112 tpu_estimator.py:2308] examples/sec: 93.209\n",
"INFO:tensorflow:global_step/sec: 2.8723\n",
"I0523 05:57:36.224385 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8723\n",
"INFO:tensorflow:examples/sec: 91.9137\n",
"I0523 05:57:36.224665 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9137\n",
"INFO:tensorflow:global_step/sec: 2.87922\n",
"I0523 05:57:36.571748 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87922\n",
"INFO:tensorflow:examples/sec: 92.135\n",
"I0523 05:57:36.572041 140278000138112 tpu_estimator.py:2308] examples/sec: 92.135\n",
"INFO:tensorflow:global_step/sec: 2.91095\n",
"I0523 05:57:36.915216 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91095\n",
"INFO:tensorflow:examples/sec: 93.1505\n",
"I0523 05:57:36.915442 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1505\n",
"INFO:tensorflow:global_step/sec: 2.89115\n",
"I0523 05:57:37.261139 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89115\n",
"INFO:tensorflow:examples/sec: 92.5168\n",
"I0523 05:57:37.261384 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5168\n",
"INFO:tensorflow:global_step/sec: 2.89619\n",
"I0523 05:57:37.606379 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89619\n",
"INFO:tensorflow:examples/sec: 92.6782\n",
"I0523 05:57:37.606605 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6782\n",
"INFO:tensorflow:global_step/sec: 2.88884\n",
"I0523 05:57:37.952542 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88884\n",
"INFO:tensorflow:examples/sec: 92.4429\n",
"I0523 05:57:37.952773 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4429\n",
"INFO:tensorflow:global_step/sec: 2.90269\n",
"I0523 05:57:38.297068 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90269\n",
"INFO:tensorflow:examples/sec: 92.886\n",
"I0523 05:57:38.297333 140278000138112 tpu_estimator.py:2308] examples/sec: 92.886\n",
"INFO:tensorflow:global_step/sec: 2.90452\n",
"I0523 05:57:38.641335 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90452\n",
"INFO:tensorflow:examples/sec: 92.9446\n",
"I0523 05:57:38.641562 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9446\n",
"INFO:tensorflow:global_step/sec: 2.87365\n",
"I0523 05:57:38.989348 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87365\n",
"INFO:tensorflow:examples/sec: 91.9568\n",
"I0523 05:57:38.989570 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9568\n",
"INFO:tensorflow:global_step/sec: 2.89915\n",
"I0523 05:57:39.334254 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89915\n",
"INFO:tensorflow:examples/sec: 92.7728\n",
"I0523 05:57:39.334474 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7728\n",
"INFO:tensorflow:global_step/sec: 2.90348\n",
"I0523 05:57:39.678662 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90348\n",
"INFO:tensorflow:examples/sec: 92.9112\n",
"I0523 05:57:39.678907 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9112\n",
"INFO:tensorflow:global_step/sec: 2.91075\n",
"I0523 05:57:40.022228 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91075\n",
"INFO:tensorflow:examples/sec: 93.1441\n",
"I0523 05:57:40.022620 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1441\n",
"INFO:tensorflow:global_step/sec: 2.90888\n",
"I0523 05:57:40.366040 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90888\n",
"INFO:tensorflow:examples/sec: 93.084\n",
"I0523 05:57:40.366297 140278000138112 tpu_estimator.py:2308] examples/sec: 93.084\n",
"INFO:tensorflow:global_step/sec: 2.88219\n",
"I0523 05:57:40.712980 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88219\n",
"INFO:tensorflow:examples/sec: 92.2301\n",
"I0523 05:57:40.713213 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2301\n",
"INFO:tensorflow:global_step/sec: 2.91248\n",
"I0523 05:57:41.056319 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91248\n",
"INFO:tensorflow:examples/sec: 93.1994\n",
"I0523 05:57:41.056580 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1994\n",
"INFO:tensorflow:global_step/sec: 2.90762\n",
"I0523 05:57:41.400252 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90762\n",
"INFO:tensorflow:examples/sec: 93.044\n",
"I0523 05:57:41.400487 140278000138112 tpu_estimator.py:2308] examples/sec: 93.044\n",
"INFO:tensorflow:global_step/sec: 2.87775\n",
"I0523 05:57:41.747773 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87775\n",
"INFO:tensorflow:examples/sec: 92.088\n",
"I0523 05:57:41.748026 140278000138112 tpu_estimator.py:2308] examples/sec: 92.088\n",
"INFO:tensorflow:global_step/sec: 2.91653\n",
"I0523 05:57:42.090602 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91653\n",
"INFO:tensorflow:examples/sec: 93.329\n",
"I0523 05:57:42.090826 140278000138112 tpu_estimator.py:2308] examples/sec: 93.329\n",
"INFO:tensorflow:global_step/sec: 2.89236\n",
"I0523 05:57:42.436353 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89236\n",
"INFO:tensorflow:examples/sec: 92.5557\n",
"I0523 05:57:42.436578 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5557\n",
"INFO:tensorflow:global_step/sec: 2.88508\n",
"I0523 05:57:42.782972 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88508\n",
"INFO:tensorflow:examples/sec: 92.3226\n",
"I0523 05:57:42.783208 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3226\n",
"INFO:tensorflow:global_step/sec: 2.92698\n",
"I0523 05:57:43.124618 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.92698\n",
"INFO:tensorflow:examples/sec: 93.6634\n",
"I0523 05:57:43.124840 140278000138112 tpu_estimator.py:2308] examples/sec: 93.6634\n",
"INFO:tensorflow:global_step/sec: 2.88573\n",
"I0523 05:57:43.471145 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88573\n",
"INFO:tensorflow:examples/sec: 92.3435\n",
"I0523 05:57:43.471378 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3435\n",
"INFO:tensorflow:global_step/sec: 2.89272\n",
"I0523 05:57:43.816830 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89272\n",
"INFO:tensorflow:examples/sec: 92.5671\n",
"I0523 05:57:43.817071 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5671\n",
"INFO:tensorflow:global_step/sec: 2.87963\n",
"I0523 05:57:44.164095 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87963\n",
"INFO:tensorflow:examples/sec: 92.1481\n",
"I0523 05:57:44.164503 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1481\n",
"INFO:tensorflow:global_step/sec: 2.91807\n",
"I0523 05:57:44.506784 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91807\n",
"INFO:tensorflow:examples/sec: 93.3782\n",
"I0523 05:57:44.507159 140278000138112 tpu_estimator.py:2308] examples/sec: 93.3782\n",
"INFO:tensorflow:global_step/sec: 2.88072\n",
"I0523 05:57:44.853931 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88072\n",
"INFO:tensorflow:examples/sec: 92.1829\n",
"I0523 05:57:44.854159 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1829\n",
"INFO:tensorflow:global_step/sec: 2.90303\n",
"I0523 05:57:45.198376 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90303\n",
"INFO:tensorflow:examples/sec: 92.8968\n",
"I0523 05:57:45.198593 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8968\n",
"INFO:tensorflow:global_step/sec: 2.90627\n",
"I0523 05:57:45.542494 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90627\n",
"INFO:tensorflow:examples/sec: 93.0005\n",
"I0523 05:57:45.542720 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0005\n",
"INFO:tensorflow:global_step/sec: 2.87152\n",
"I0523 05:57:45.890738 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87152\n",
"INFO:tensorflow:examples/sec: 91.8886\n",
"I0523 05:57:45.891004 140278000138112 tpu_estimator.py:2308] examples/sec: 91.8886\n",
"INFO:tensorflow:global_step/sec: 2.90346\n",
"I0523 05:57:46.235148 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90346\n",
"INFO:tensorflow:examples/sec: 92.9108\n",
"I0523 05:57:46.235378 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9108\n",
"INFO:tensorflow:global_step/sec: 2.90065\n",
"I0523 05:57:46.579896 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90065\n",
"INFO:tensorflow:examples/sec: 92.8208\n",
"I0523 05:57:46.580145 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8208\n",
"INFO:tensorflow:global_step/sec: 2.90107\n",
"I0523 05:57:46.924608 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90107\n",
"INFO:tensorflow:examples/sec: 92.8343\n",
"I0523 05:57:46.924847 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8343\n",
"INFO:tensorflow:global_step/sec: 2.91206\n",
"I0523 05:57:47.268002 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91206\n",
"INFO:tensorflow:examples/sec: 93.186\n",
"I0523 05:57:47.268243 140278000138112 tpu_estimator.py:2308] examples/sec: 93.186\n",
"INFO:tensorflow:global_step/sec: 2.87992\n",
"I0523 05:57:47.615223 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87992\n",
"INFO:tensorflow:examples/sec: 92.1574\n",
"I0523 05:57:47.615590 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1574\n",
"INFO:tensorflow:global_step/sec: 2.9112\n",
"I0523 05:57:47.958711 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.9112\n",
"INFO:tensorflow:examples/sec: 93.1586\n",
"I0523 05:57:47.958924 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1586\n",
"INFO:tensorflow:global_step/sec: 2.90469\n",
"I0523 05:57:48.303008 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90469\n",
"INFO:tensorflow:examples/sec: 92.9502\n",
"I0523 05:57:48.303254 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9502\n",
"INFO:tensorflow:global_step/sec: 2.87036\n",
"I0523 05:57:48.651378 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87036\n",
"INFO:tensorflow:examples/sec: 91.8514\n",
"I0523 05:57:48.651603 140278000138112 tpu_estimator.py:2308] examples/sec: 91.8514\n",
"INFO:tensorflow:global_step/sec: 2.88788\n",
"I0523 05:57:48.997660 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88788\n",
"INFO:tensorflow:examples/sec: 92.4121\n",
"I0523 05:57:48.997889 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4121\n",
"INFO:tensorflow:global_step/sec: 2.91296\n",
"I0523 05:57:49.340998 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91296\n",
"INFO:tensorflow:examples/sec: 93.2147\n",
"I0523 05:57:49.341250 140278000138112 tpu_estimator.py:2308] examples/sec: 93.2147\n",
"INFO:tensorflow:global_step/sec: 2.88066\n",
"I0523 05:57:49.688152 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88066\n",
"INFO:tensorflow:examples/sec: 92.1812\n",
"I0523 05:57:49.688596 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1812\n",
"INFO:tensorflow:global_step/sec: 2.87756\n",
"I0523 05:57:50.035619 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87756\n",
"INFO:tensorflow:examples/sec: 92.082\n",
"I0523 05:57:50.036075 140278000138112 tpu_estimator.py:2308] examples/sec: 92.082\n",
"INFO:tensorflow:global_step/sec: 2.91242\n",
"I0523 05:57:50.379010 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91242\n",
"INFO:tensorflow:examples/sec: 93.1973\n",
"I0523 05:57:50.379248 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1973\n",
"INFO:tensorflow:global_step/sec: 2.89946\n",
"I0523 05:57:50.723866 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89946\n",
"INFO:tensorflow:examples/sec: 92.7828\n",
"I0523 05:57:50.724124 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7828\n",
"INFO:tensorflow:global_step/sec: 2.89646\n",
"I0523 05:57:51.069133 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89646\n",
"INFO:tensorflow:examples/sec: 92.6867\n",
"I0523 05:57:51.069366 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6867\n",
"INFO:tensorflow:global_step/sec: 2.92713\n",
"I0523 05:57:51.410746 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.92713\n",
"INFO:tensorflow:examples/sec: 93.6682\n",
"I0523 05:57:51.410985 140278000138112 tpu_estimator.py:2308] examples/sec: 93.6682\n",
"INFO:tensorflow:global_step/sec: 2.89164\n",
"I0523 05:57:51.756574 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89164\n",
"INFO:tensorflow:examples/sec: 92.5324\n",
"I0523 05:57:51.756822 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5324\n",
"INFO:tensorflow:global_step/sec: 2.89861\n",
"I0523 05:57:52.101585 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89861\n",
"INFO:tensorflow:examples/sec: 92.7556\n",
"I0523 05:57:52.102029 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7556\n",
"INFO:tensorflow:global_step/sec: 2.88937\n",
"I0523 05:57:52.447649 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88937\n",
"INFO:tensorflow:examples/sec: 92.4598\n",
"I0523 05:57:52.447873 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4598\n",
"INFO:tensorflow:global_step/sec: 2.89623\n",
"I0523 05:57:52.792931 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89623\n",
"INFO:tensorflow:examples/sec: 92.6794\n",
"I0523 05:57:52.793187 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6794\n",
"INFO:tensorflow:global_step/sec: 2.8904\n",
"I0523 05:57:53.138885 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8904\n",
"INFO:tensorflow:examples/sec: 92.493\n",
"I0523 05:57:53.139130 140278000138112 tpu_estimator.py:2308] examples/sec: 92.493\n",
"INFO:tensorflow:global_step/sec: 2.90506\n",
"I0523 05:57:53.483188 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90506\n",
"INFO:tensorflow:examples/sec: 92.9619\n",
"I0523 05:57:53.483439 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9619\n",
"INFO:tensorflow:global_step/sec: 2.8822\n",
"I0523 05:57:53.830158 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8822\n",
"INFO:tensorflow:examples/sec: 92.2305\n",
"I0523 05:57:53.830409 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2305\n",
"INFO:tensorflow:global_step/sec: 2.90184\n",
"I0523 05:57:54.174701 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90184\n",
"INFO:tensorflow:examples/sec: 92.859\n",
"I0523 05:57:54.175113 140278000138112 tpu_estimator.py:2308] examples/sec: 92.859\n",
"INFO:tensorflow:global_step/sec: 2.91646\n",
"I0523 05:57:54.517586 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91646\n",
"INFO:tensorflow:examples/sec: 93.3268\n",
"I0523 05:57:54.517812 140278000138112 tpu_estimator.py:2308] examples/sec: 93.3268\n",
"INFO:tensorflow:global_step/sec: 2.9013\n",
"I0523 05:57:54.862246 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.9013\n",
"INFO:tensorflow:examples/sec: 92.8415\n",
"I0523 05:57:54.862590 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8415\n",
"INFO:tensorflow:global_step/sec: 2.87646\n",
"I0523 05:57:55.209890 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87646\n",
"INFO:tensorflow:examples/sec: 92.0466\n",
"I0523 05:57:55.210291 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0466\n",
"INFO:tensorflow:global_step/sec: 2.90067\n",
"I0523 05:57:55.554661 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90067\n",
"INFO:tensorflow:examples/sec: 92.8214\n",
"I0523 05:57:55.555070 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8214\n",
"INFO:tensorflow:global_step/sec: 2.88621\n",
"I0523 05:57:55.901142 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88621\n",
"INFO:tensorflow:examples/sec: 92.3586\n",
"I0523 05:57:55.901505 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3586\n",
"INFO:tensorflow:global_step/sec: 2.90925\n",
"I0523 05:57:56.244850 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90925\n",
"INFO:tensorflow:examples/sec: 93.096\n",
"I0523 05:57:56.245208 140278000138112 tpu_estimator.py:2308] examples/sec: 93.096\n",
"INFO:tensorflow:global_step/sec: 2.89719\n",
"I0523 05:57:56.590048 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89719\n",
"INFO:tensorflow:examples/sec: 92.71\n",
"I0523 05:57:56.590287 140278000138112 tpu_estimator.py:2308] examples/sec: 92.71\n",
"INFO:tensorflow:global_step/sec: 2.89189\n",
"I0523 05:57:56.935814 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89189\n",
"INFO:tensorflow:examples/sec: 92.5403\n",
"I0523 05:57:56.936239 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5403\n",
"INFO:tensorflow:global_step/sec: 2.89429\n",
"I0523 05:57:57.281338 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89429\n",
"INFO:tensorflow:examples/sec: 92.6172\n",
"I0523 05:57:57.281565 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6172\n",
"INFO:tensorflow:global_step/sec: 2.90049\n",
"I0523 05:57:57.626109 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90049\n",
"INFO:tensorflow:examples/sec: 92.8156\n",
"I0523 05:57:57.626341 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8156\n",
"INFO:tensorflow:global_step/sec: 2.89905\n",
"I0523 05:57:57.971120 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89905\n",
"INFO:tensorflow:examples/sec: 92.7695\n",
"I0523 05:57:57.971619 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7695\n",
"INFO:tensorflow:global_step/sec: 2.90634\n",
"I0523 05:57:58.315137 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90634\n",
"INFO:tensorflow:examples/sec: 93.0027\n",
"I0523 05:57:58.315548 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0027\n",
"INFO:tensorflow:global_step/sec: 2.89102\n",
"I0523 05:57:58.661026 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89102\n",
"INFO:tensorflow:examples/sec: 92.5127\n",
"I0523 05:57:58.661266 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5127\n",
"INFO:tensorflow:global_step/sec: 2.85641\n",
"I0523 05:57:59.011122 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.85641\n",
"INFO:tensorflow:examples/sec: 91.4051\n",
"I0523 05:57:59.011370 140278000138112 tpu_estimator.py:2308] examples/sec: 91.4051\n",
"INFO:tensorflow:global_step/sec: 2.90322\n",
"I0523 05:57:59.355541 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90322\n",
"INFO:tensorflow:examples/sec: 92.9031\n",
"I0523 05:57:59.355779 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9031\n",
"INFO:tensorflow:global_step/sec: 2.88537\n",
"I0523 05:57:59.702156 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88537\n",
"INFO:tensorflow:examples/sec: 92.3318\n",
"I0523 05:57:59.702427 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3318\n",
"INFO:tensorflow:global_step/sec: 2.86397\n",
"I0523 05:58:00.051281 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86397\n",
"INFO:tensorflow:examples/sec: 91.6471\n",
"I0523 05:58:00.051649 140278000138112 tpu_estimator.py:2308] examples/sec: 91.6471\n",
"INFO:tensorflow:global_step/sec: 2.88437\n",
"I0523 05:58:00.398012 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88437\n",
"INFO:tensorflow:examples/sec: 92.2999\n",
"I0523 05:58:00.398983 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2999\n",
"INFO:tensorflow:global_step/sec: 2.88166\n",
"I0523 05:58:00.745013 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88166\n",
"INFO:tensorflow:examples/sec: 92.2131\n",
"I0523 05:58:00.745249 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2131\n",
"INFO:tensorflow:global_step/sec: 2.86918\n",
"I0523 05:58:01.093533 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86918\n",
"INFO:tensorflow:examples/sec: 91.8139\n",
"I0523 05:58:01.093797 140278000138112 tpu_estimator.py:2308] examples/sec: 91.8139\n",
"INFO:tensorflow:global_step/sec: 2.89306\n",
"I0523 05:58:01.439219 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89306\n",
"INFO:tensorflow:examples/sec: 92.5779\n",
"I0523 05:58:01.439467 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5779\n",
"INFO:tensorflow:global_step/sec: 2.92647\n",
"I0523 05:58:01.780915 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.92647\n",
"INFO:tensorflow:examples/sec: 93.6471\n",
"I0523 05:58:01.781178 140278000138112 tpu_estimator.py:2308] examples/sec: 93.6471\n",
"INFO:tensorflow:global_step/sec: 2.87509\n",
"I0523 05:58:02.128720 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87509\n",
"INFO:tensorflow:examples/sec: 92.003\n",
"I0523 05:58:02.128969 140278000138112 tpu_estimator.py:2308] examples/sec: 92.003\n",
"INFO:tensorflow:global_step/sec: 2.89957\n",
"I0523 05:58:02.473624 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89957\n",
"INFO:tensorflow:examples/sec: 92.7863\n",
"I0523 05:58:02.473872 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7863\n",
"INFO:tensorflow:global_step/sec: 2.90247\n",
"I0523 05:58:02.818178 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90247\n",
"INFO:tensorflow:examples/sec: 92.879\n",
"I0523 05:58:02.818448 140278000138112 tpu_estimator.py:2308] examples/sec: 92.879\n",
"INFO:tensorflow:global_step/sec: 2.89225\n",
"I0523 05:58:03.163868 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89225\n",
"INFO:tensorflow:examples/sec: 92.552\n",
"I0523 05:58:03.164112 140278000138112 tpu_estimator.py:2308] examples/sec: 92.552\n",
"INFO:tensorflow:global_step/sec: 2.89671\n",
"I0523 05:58:03.509107 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89671\n",
"INFO:tensorflow:examples/sec: 92.6947\n",
"I0523 05:58:03.509340 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6947\n",
"INFO:tensorflow:global_step/sec: 2.91311\n",
"I0523 05:58:03.852375 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91311\n",
"INFO:tensorflow:examples/sec: 93.2195\n",
"I0523 05:58:03.852610 140278000138112 tpu_estimator.py:2308] examples/sec: 93.2195\n",
"INFO:tensorflow:global_step/sec: 2.84957\n",
"I0523 05:58:04.203290 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.84957\n",
"INFO:tensorflow:examples/sec: 91.1862\n",
"I0523 05:58:04.203645 140278000138112 tpu_estimator.py:2308] examples/sec: 91.1862\n",
"INFO:tensorflow:global_step/sec: 2.91995\n",
"I0523 05:58:04.545801 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91995\n",
"INFO:tensorflow:examples/sec: 93.4383\n",
"I0523 05:58:04.546180 140278000138112 tpu_estimator.py:2308] examples/sec: 93.4383\n",
"INFO:tensorflow:global_step/sec: 2.89091\n",
"I0523 05:58:04.891697 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89091\n",
"INFO:tensorflow:examples/sec: 92.5093\n",
"I0523 05:58:04.891931 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5093\n",
"INFO:tensorflow:global_step/sec: 2.88387\n",
"I0523 05:58:05.238435 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88387\n",
"INFO:tensorflow:examples/sec: 92.2837\n",
"I0523 05:58:05.238780 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2837\n",
"INFO:tensorflow:global_step/sec: 2.91108\n",
"I0523 05:58:05.581992 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91108\n",
"INFO:tensorflow:examples/sec: 93.1547\n",
"I0523 05:58:05.582224 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1547\n",
"INFO:tensorflow:global_step/sec: 2.90351\n",
"I0523 05:58:05.926367 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90351\n",
"INFO:tensorflow:examples/sec: 92.9124\n",
"I0523 05:58:05.926730 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9124\n",
"INFO:tensorflow:global_step/sec: 2.88918\n",
"I0523 05:58:06.272478 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88918\n",
"INFO:tensorflow:examples/sec: 92.4537\n",
"I0523 05:58:06.272708 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4537\n",
"INFO:tensorflow:global_step/sec: 2.88645\n",
"I0523 05:58:06.618937 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88645\n",
"INFO:tensorflow:examples/sec: 92.3665\n",
"I0523 05:58:06.619183 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3665\n",
"INFO:tensorflow:global_step/sec: 2.88473\n",
"I0523 05:58:06.965590 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88473\n",
"INFO:tensorflow:examples/sec: 92.3115\n",
"I0523 05:58:06.965845 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3115\n",
"INFO:tensorflow:global_step/sec: 2.894\n",
"I0523 05:58:07.311172 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.894\n",
"INFO:tensorflow:examples/sec: 92.608\n",
"I0523 05:58:07.311425 140278000138112 tpu_estimator.py:2308] examples/sec: 92.608\n",
"INFO:tensorflow:global_step/sec: 2.87456\n",
"I0523 05:58:07.659035 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87456\n",
"INFO:tensorflow:examples/sec: 91.986\n",
"I0523 05:58:07.659271 140278000138112 tpu_estimator.py:2308] examples/sec: 91.986\n",
"INFO:tensorflow:global_step/sec: 2.88196\n",
"I0523 05:58:08.006029 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88196\n",
"INFO:tensorflow:examples/sec: 92.2228\n",
"I0523 05:58:08.006256 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2228\n",
"INFO:tensorflow:global_step/sec: 2.8879\n",
"I0523 05:58:08.352214 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8879\n",
"INFO:tensorflow:examples/sec: 92.4128\n",
"I0523 05:58:08.352538 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4128\n",
"INFO:tensorflow:global_step/sec: 2.87248\n",
"I0523 05:58:08.700401 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87248\n",
"INFO:tensorflow:examples/sec: 91.9194\n",
"I0523 05:58:08.700621 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9194\n",
"INFO:tensorflow:global_step/sec: 2.91099\n",
"I0523 05:58:09.043909 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91099\n",
"INFO:tensorflow:examples/sec: 93.1518\n",
"I0523 05:58:09.044161 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1518\n",
"INFO:tensorflow:global_step/sec: 2.89795\n",
"I0523 05:58:09.389016 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89795\n",
"INFO:tensorflow:examples/sec: 92.7345\n",
"I0523 05:58:09.389250 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7345\n",
"INFO:tensorflow:global_step/sec: 2.88433\n",
"I0523 05:58:09.735686 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88433\n",
"INFO:tensorflow:examples/sec: 92.2987\n",
"I0523 05:58:09.735911 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2987\n",
"INFO:tensorflow:global_step/sec: 2.89077\n",
"I0523 05:58:10.081634 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89077\n",
"INFO:tensorflow:examples/sec: 92.5047\n",
"I0523 05:58:10.081882 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5047\n",
"INFO:tensorflow:global_step/sec: 2.90419\n",
"I0523 05:58:10.426051 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90419\n",
"INFO:tensorflow:examples/sec: 92.934\n",
"I0523 05:58:10.426297 140278000138112 tpu_estimator.py:2308] examples/sec: 92.934\n",
"INFO:tensorflow:global_step/sec: 2.89439\n",
"I0523 05:58:10.771500 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89439\n",
"INFO:tensorflow:examples/sec: 92.6206\n",
"I0523 05:58:10.772071 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6206\n",
"INFO:tensorflow:global_step/sec: 2.88297\n",
"I0523 05:58:11.118331 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88297\n",
"INFO:tensorflow:examples/sec: 92.2551\n",
"I0523 05:58:11.118710 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2551\n",
"INFO:tensorflow:global_step/sec: 2.89428\n",
"I0523 05:58:11.463852 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89428\n",
"INFO:tensorflow:examples/sec: 92.617\n",
"I0523 05:58:11.464238 140278000138112 tpu_estimator.py:2308] examples/sec: 92.617\n",
"INFO:tensorflow:global_step/sec: 2.88035\n",
"I0523 05:58:11.811028 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88035\n",
"INFO:tensorflow:examples/sec: 92.1712\n",
"I0523 05:58:11.811272 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1712\n",
"INFO:tensorflow:global_step/sec: 2.90388\n",
"I0523 05:58:12.155373 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90388\n",
"INFO:tensorflow:examples/sec: 92.924\n",
"I0523 05:58:12.155641 140278000138112 tpu_estimator.py:2308] examples/sec: 92.924\n",
"INFO:tensorflow:global_step/sec: 2.88418\n",
"I0523 05:58:12.502103 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88418\n",
"INFO:tensorflow:examples/sec: 92.2938\n",
"I0523 05:58:12.502475 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2938\n",
"INFO:tensorflow:global_step/sec: 2.8888\n",
"I0523 05:58:12.848272 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8888\n",
"INFO:tensorflow:examples/sec: 92.4415\n",
"I0523 05:58:12.848523 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4415\n",
"INFO:tensorflow:global_step/sec: 2.91392\n",
"I0523 05:58:13.191429 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91392\n",
"INFO:tensorflow:examples/sec: 93.2455\n",
"I0523 05:58:13.191648 140278000138112 tpu_estimator.py:2308] examples/sec: 93.2455\n",
"INFO:tensorflow:global_step/sec: 2.88342\n",
"I0523 05:58:13.538241 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88342\n",
"INFO:tensorflow:examples/sec: 92.2696\n",
"I0523 05:58:13.538476 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2696\n",
"INFO:tensorflow:global_step/sec: 2.87633\n",
"I0523 05:58:13.885908 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87633\n",
"INFO:tensorflow:examples/sec: 92.0426\n",
"I0523 05:58:13.886324 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0426\n",
"INFO:tensorflow:global_step/sec: 2.89818\n",
"I0523 05:58:14.230970 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89818\n",
"INFO:tensorflow:examples/sec: 92.7417\n",
"I0523 05:58:14.231205 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7417\n",
"INFO:tensorflow:global_step/sec: 2.90088\n",
"I0523 05:58:14.575684 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90088\n",
"INFO:tensorflow:examples/sec: 92.8283\n",
"I0523 05:58:14.575974 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8283\n",
"INFO:tensorflow:global_step/sec: 2.89406\n",
"I0523 05:58:14.921235 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89406\n",
"INFO:tensorflow:examples/sec: 92.61\n",
"I0523 05:58:14.921504 140278000138112 tpu_estimator.py:2308] examples/sec: 92.61\n",
"INFO:tensorflow:global_step/sec: 2.90176\n",
"I0523 05:58:15.265824 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90176\n",
"INFO:tensorflow:examples/sec: 92.8562\n",
"I0523 05:58:15.266081 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8562\n",
"INFO:tensorflow:global_step/sec: 2.87441\n",
"I0523 05:58:15.613720 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87441\n",
"INFO:tensorflow:examples/sec: 91.9812\n",
"I0523 05:58:15.614091 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9812\n",
"INFO:tensorflow:global_step/sec: 2.89265\n",
"I0523 05:58:15.959449 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89265\n",
"INFO:tensorflow:examples/sec: 92.5648\n",
"I0523 05:58:15.959907 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5648\n",
"INFO:tensorflow:global_step/sec: 2.89484\n",
"I0523 05:58:16.304890 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89484\n",
"INFO:tensorflow:examples/sec: 92.6349\n",
"I0523 05:58:16.305300 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6349\n",
"INFO:tensorflow:global_step/sec: 2.88688\n",
"I0523 05:58:16.651282 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88688\n",
"INFO:tensorflow:examples/sec: 92.3801\n",
"I0523 05:58:16.651542 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3801\n",
"INFO:tensorflow:global_step/sec: 2.90247\n",
"I0523 05:58:16.995857 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90247\n",
"INFO:tensorflow:examples/sec: 92.879\n",
"I0523 05:58:16.996261 140278000138112 tpu_estimator.py:2308] examples/sec: 92.879\n",
"INFO:tensorflow:global_step/sec: 2.8997\n",
"I0523 05:58:17.340656 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8997\n",
"INFO:tensorflow:examples/sec: 92.7904\n",
"I0523 05:58:17.340912 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7904\n",
"INFO:tensorflow:global_step/sec: 2.89072\n",
"I0523 05:58:17.686594 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89072\n",
"INFO:tensorflow:examples/sec: 92.5029\n",
"I0523 05:58:17.686836 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5029\n",
"INFO:tensorflow:global_step/sec: 2.89162\n",
"I0523 05:58:18.032430 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89162\n",
"INFO:tensorflow:examples/sec: 92.5317\n",
"I0523 05:58:18.032661 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5317\n",
"INFO:tensorflow:global_step/sec: 2.90091\n",
"I0523 05:58:18.377156 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90091\n",
"INFO:tensorflow:examples/sec: 92.8292\n",
"I0523 05:58:18.377648 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8292\n",
"INFO:tensorflow:global_step/sec: 2.89097\n",
"I0523 05:58:18.723077 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89097\n",
"INFO:tensorflow:examples/sec: 92.5112\n",
"I0523 05:58:18.723568 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5112\n",
"INFO:tensorflow:global_step/sec: 2.88713\n",
"I0523 05:58:19.069401 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88713\n",
"INFO:tensorflow:examples/sec: 92.3882\n",
"I0523 05:58:19.069763 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3882\n",
"INFO:tensorflow:global_step/sec: 2.91083\n",
"I0523 05:58:19.412993 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91083\n",
"INFO:tensorflow:examples/sec: 93.1464\n",
"I0523 05:58:19.413368 140278000138112 tpu_estimator.py:2308] examples/sec: 93.1464\n",
"INFO:tensorflow:global_step/sec: 2.90184\n",
"I0523 05:58:19.757544 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90184\n",
"INFO:tensorflow:examples/sec: 92.8587\n",
"I0523 05:58:19.757762 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8587\n",
"INFO:tensorflow:global_step/sec: 2.8908\n",
"I0523 05:58:20.103562 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8908\n",
"INFO:tensorflow:examples/sec: 92.5056\n",
"I0523 05:58:20.103832 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5056\n",
"INFO:tensorflow:global_step/sec: 2.88787\n",
"I0523 05:58:20.449817 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88787\n",
"INFO:tensorflow:examples/sec: 92.4119\n",
"I0523 05:58:20.450274 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4119\n",
"INFO:tensorflow:global_step/sec: 2.88487\n",
"I0523 05:58:20.796413 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88487\n",
"INFO:tensorflow:examples/sec: 92.3157\n",
"I0523 05:58:20.796653 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3157\n",
"INFO:tensorflow:global_step/sec: 2.88595\n",
"I0523 05:58:21.142922 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88595\n",
"INFO:tensorflow:examples/sec: 92.3505\n",
"I0523 05:58:21.143318 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3505\n",
"INFO:tensorflow:global_step/sec: 2.8943\n",
"I0523 05:58:21.488432 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8943\n",
"INFO:tensorflow:examples/sec: 92.6175\n",
"I0523 05:58:21.488701 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6175\n",
"INFO:tensorflow:global_step/sec: 2.88614\n",
"I0523 05:58:21.834913 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88614\n",
"INFO:tensorflow:examples/sec: 92.3566\n",
"I0523 05:58:21.835330 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3566\n",
"INFO:tensorflow:global_step/sec: 2.90548\n",
"I0523 05:58:22.179097 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90548\n",
"INFO:tensorflow:examples/sec: 92.9755\n",
"I0523 05:58:22.179473 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9755\n",
"INFO:tensorflow:global_step/sec: 2.88356\n",
"I0523 05:58:22.525884 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88356\n",
"INFO:tensorflow:examples/sec: 92.274\n",
"I0523 05:58:22.526151 140278000138112 tpu_estimator.py:2308] examples/sec: 92.274\n",
"INFO:tensorflow:global_step/sec: 2.88561\n",
"I0523 05:58:22.872407 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88561\n",
"INFO:tensorflow:examples/sec: 92.3395\n",
"I0523 05:58:22.872783 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3395\n",
"INFO:tensorflow:global_step/sec: 2.88434\n",
"I0523 05:58:23.219113 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88434\n",
"INFO:tensorflow:examples/sec: 92.2988\n",
"I0523 05:58:23.219500 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2988\n",
"INFO:tensorflow:global_step/sec: 2.89779\n",
"I0523 05:58:23.564211 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89779\n",
"INFO:tensorflow:examples/sec: 92.7293\n",
"I0523 05:58:23.564613 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7293\n",
"INFO:tensorflow:global_step/sec: 2.86654\n",
"I0523 05:58:23.913096 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86654\n",
"INFO:tensorflow:examples/sec: 91.7292\n",
"I0523 05:58:23.913335 140278000138112 tpu_estimator.py:2308] examples/sec: 91.7292\n",
"INFO:tensorflow:global_step/sec: 2.88541\n",
"I0523 05:58:24.259671 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88541\n",
"INFO:tensorflow:examples/sec: 92.3332\n",
"I0523 05:58:24.259895 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3332\n",
"INFO:tensorflow:global_step/sec: 2.89111\n",
"I0523 05:58:24.605521 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89111\n",
"INFO:tensorflow:examples/sec: 92.5155\n",
"I0523 05:58:24.605935 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5155\n",
"INFO:tensorflow:global_step/sec: 2.86719\n",
"I0523 05:58:24.954290 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86719\n",
"INFO:tensorflow:examples/sec: 91.75\n",
"I0523 05:58:24.954640 140278000138112 tpu_estimator.py:2308] examples/sec: 91.75\n",
"INFO:tensorflow:global_step/sec: 2.90158\n",
"I0523 05:58:25.298923 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90158\n",
"INFO:tensorflow:examples/sec: 92.8504\n",
"I0523 05:58:25.299292 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8504\n",
"INFO:tensorflow:global_step/sec: 2.88866\n",
"I0523 05:58:25.645198 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88866\n",
"INFO:tensorflow:examples/sec: 92.437\n",
"I0523 05:58:25.645421 140278000138112 tpu_estimator.py:2308] examples/sec: 92.437\n",
"INFO:tensorflow:global_step/sec: 2.88184\n",
"I0523 05:58:25.992113 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88184\n",
"INFO:tensorflow:examples/sec: 92.2188\n",
"I0523 05:58:25.992457 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2188\n",
"INFO:tensorflow:global_step/sec: 2.88478\n",
"I0523 05:58:26.338774 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88478\n",
"INFO:tensorflow:examples/sec: 92.3131\n",
"I0523 05:58:26.339202 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3131\n",
"INFO:tensorflow:global_step/sec: 2.87981\n",
"I0523 05:58:26.686031 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87981\n",
"INFO:tensorflow:examples/sec: 92.154\n",
"I0523 05:58:26.686412 140278000138112 tpu_estimator.py:2308] examples/sec: 92.154\n",
"INFO:tensorflow:global_step/sec: 2.89243\n",
"I0523 05:58:27.031726 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89243\n",
"INFO:tensorflow:examples/sec: 92.5578\n",
"I0523 05:58:27.031973 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5578\n",
"INFO:tensorflow:global_step/sec: 2.89122\n",
"I0523 05:58:27.377644 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89122\n",
"INFO:tensorflow:examples/sec: 92.519\n",
"I0523 05:58:27.378016 140278000138112 tpu_estimator.py:2308] examples/sec: 92.519\n",
"INFO:tensorflow:global_step/sec: 2.90292\n",
"I0523 05:58:27.722106 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90292\n",
"INFO:tensorflow:examples/sec: 92.8934\n",
"I0523 05:58:27.722332 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8934\n",
"INFO:tensorflow:global_step/sec: 2.8573\n",
"I0523 05:58:28.072098 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8573\n",
"INFO:tensorflow:examples/sec: 91.4336\n",
"I0523 05:58:28.072497 140278000138112 tpu_estimator.py:2308] examples/sec: 91.4336\n",
"INFO:tensorflow:global_step/sec: 2.91166\n",
"I0523 05:58:28.415516 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91166\n",
"INFO:tensorflow:examples/sec: 93.173\n",
"I0523 05:58:28.415755 140278000138112 tpu_estimator.py:2308] examples/sec: 93.173\n",
"INFO:tensorflow:global_step/sec: 2.89615\n",
"I0523 05:58:28.760834 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89615\n",
"INFO:tensorflow:examples/sec: 92.6767\n",
"I0523 05:58:28.761221 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6767\n",
"INFO:tensorflow:global_step/sec: 2.81995\n",
"I0523 05:58:29.115461 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.81995\n",
"INFO:tensorflow:examples/sec: 90.2384\n",
"I0523 05:58:29.117223 140278000138112 tpu_estimator.py:2308] examples/sec: 90.2384\n",
"INFO:tensorflow:global_step/sec: 2.84871\n",
"I0523 05:58:29.466475 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.84871\n",
"INFO:tensorflow:examples/sec: 91.1587\n",
"I0523 05:58:29.466721 140278000138112 tpu_estimator.py:2308] examples/sec: 91.1587\n",
"INFO:tensorflow:global_step/sec: 2.90502\n",
"I0523 05:58:29.810714 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90502\n",
"INFO:tensorflow:examples/sec: 92.9608\n",
"I0523 05:58:29.811297 140278000138112 tpu_estimator.py:2308] examples/sec: 92.9608\n",
"INFO:tensorflow:global_step/sec: 2.84001\n",
"I0523 05:58:30.162791 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.84001\n",
"INFO:tensorflow:examples/sec: 90.8804\n",
"I0523 05:58:30.163199 140278000138112 tpu_estimator.py:2308] examples/sec: 90.8804\n",
"INFO:tensorflow:global_step/sec: 2.89411\n",
"I0523 05:58:30.508311 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89411\n",
"INFO:tensorflow:examples/sec: 92.6115\n",
"I0523 05:58:30.508684 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6115\n",
"INFO:tensorflow:global_step/sec: 2.89272\n",
"I0523 05:58:30.854027 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89272\n",
"INFO:tensorflow:examples/sec: 92.567\n",
"I0523 05:58:30.854380 140278000138112 tpu_estimator.py:2308] examples/sec: 92.567\n",
"INFO:tensorflow:global_step/sec: 2.88262\n",
"I0523 05:58:31.200924 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88262\n",
"INFO:tensorflow:examples/sec: 92.244\n",
"I0523 05:58:31.201303 140278000138112 tpu_estimator.py:2308] examples/sec: 92.244\n",
"INFO:tensorflow:global_step/sec: 2.88906\n",
"I0523 05:58:31.547087 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88906\n",
"INFO:tensorflow:examples/sec: 92.45\n",
"I0523 05:58:31.547483 140278000138112 tpu_estimator.py:2308] examples/sec: 92.45\n",
"INFO:tensorflow:global_step/sec: 2.92207\n",
"I0523 05:58:31.889307 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.92207\n",
"INFO:tensorflow:examples/sec: 93.5062\n",
"I0523 05:58:31.889731 140278000138112 tpu_estimator.py:2308] examples/sec: 93.5062\n",
"INFO:tensorflow:global_step/sec: 2.87379\n",
"I0523 05:58:32.237278 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87379\n",
"INFO:tensorflow:examples/sec: 91.9611\n",
"I0523 05:58:32.237681 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9611\n",
"INFO:tensorflow:global_step/sec: 2.87974\n",
"I0523 05:58:32.584507 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87974\n",
"INFO:tensorflow:examples/sec: 92.1517\n",
"I0523 05:58:32.584872 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1517\n",
"INFO:tensorflow:global_step/sec: 2.8995\n",
"I0523 05:58:32.929406 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8995\n",
"INFO:tensorflow:examples/sec: 92.7839\n",
"I0523 05:58:32.929757 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7839\n",
"INFO:tensorflow:global_step/sec: 2.87574\n",
"I0523 05:58:33.277147 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87574\n",
"INFO:tensorflow:examples/sec: 92.0237\n",
"I0523 05:58:33.277526 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0237\n",
"INFO:tensorflow:global_step/sec: 2.87943\n",
"I0523 05:58:33.624409 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87943\n",
"INFO:tensorflow:examples/sec: 92.1418\n",
"I0523 05:58:33.624750 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1418\n",
"INFO:tensorflow:global_step/sec: 2.89596\n",
"I0523 05:58:33.969748 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89596\n",
"INFO:tensorflow:examples/sec: 92.6707\n",
"I0523 05:58:33.970174 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6707\n",
"INFO:tensorflow:global_step/sec: 2.86647\n",
"I0523 05:58:34.318613 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86647\n",
"INFO:tensorflow:examples/sec: 91.7271\n",
"I0523 05:58:34.318992 140278000138112 tpu_estimator.py:2308] examples/sec: 91.7271\n",
"INFO:tensorflow:global_step/sec: 2.89335\n",
"I0523 05:58:34.664206 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89335\n",
"INFO:tensorflow:examples/sec: 92.5873\n",
"I0523 05:58:34.664435 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5873\n",
"INFO:tensorflow:global_step/sec: 2.8931\n",
"I0523 05:58:35.009899 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8931\n",
"INFO:tensorflow:examples/sec: 92.5791\n",
"I0523 05:58:35.010312 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5791\n",
"INFO:tensorflow:global_step/sec: 2.87683\n",
"I0523 05:58:35.357470 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87683\n",
"INFO:tensorflow:examples/sec: 92.0587\n",
"I0523 05:58:35.357834 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0587\n",
"INFO:tensorflow:global_step/sec: 2.89065\n",
"I0523 05:58:35.703399 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89065\n",
"INFO:tensorflow:examples/sec: 92.5008\n",
"I0523 05:58:35.703736 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5008\n",
"INFO:tensorflow:global_step/sec: 2.90642\n",
"I0523 05:58:36.047558 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90642\n",
"INFO:tensorflow:examples/sec: 93.0054\n",
"I0523 05:58:36.047789 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0054\n",
"INFO:tensorflow:global_step/sec: 2.8918\n",
"I0523 05:58:36.393274 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8918\n",
"INFO:tensorflow:examples/sec: 92.5375\n",
"I0523 05:58:36.393499 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5375\n",
"INFO:tensorflow:global_step/sec: 2.89072\n",
"I0523 05:58:36.739237 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89072\n",
"INFO:tensorflow:examples/sec: 92.5029\n",
"I0523 05:58:36.739619 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5029\n",
"INFO:tensorflow:global_step/sec: 2.917\n",
"I0523 05:58:37.082051 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.917\n",
"INFO:tensorflow:examples/sec: 93.3439\n",
"I0523 05:58:37.082437 140278000138112 tpu_estimator.py:2308] examples/sec: 93.3439\n",
"INFO:tensorflow:global_step/sec: 2.88249\n",
"I0523 05:58:37.428978 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88249\n",
"INFO:tensorflow:examples/sec: 92.2396\n",
"I0523 05:58:37.429290 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2396\n",
"INFO:tensorflow:global_step/sec: 2.88715\n",
"I0523 05:58:37.775328 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88715\n",
"INFO:tensorflow:examples/sec: 92.3887\n",
"I0523 05:58:37.775770 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3887\n",
"INFO:tensorflow:global_step/sec: 2.89909\n",
"I0523 05:58:38.120273 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89909\n",
"INFO:tensorflow:examples/sec: 92.7708\n",
"I0523 05:58:38.120710 140278000138112 tpu_estimator.py:2308] examples/sec: 92.7708\n",
"INFO:tensorflow:global_step/sec: 2.91693\n",
"I0523 05:58:38.463097 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.91693\n",
"INFO:tensorflow:examples/sec: 93.3418\n",
"I0523 05:58:38.463504 140278000138112 tpu_estimator.py:2308] examples/sec: 93.3418\n",
"INFO:tensorflow:global_step/sec: 2.87252\n",
"I0523 05:58:38.811230 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87252\n",
"INFO:tensorflow:examples/sec: 91.9206\n",
"I0523 05:58:38.811501 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9206\n",
"INFO:tensorflow:global_step/sec: 2.86316\n",
"I0523 05:58:39.160502 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86316\n",
"INFO:tensorflow:examples/sec: 91.621\n",
"I0523 05:58:39.160918 140278000138112 tpu_estimator.py:2308] examples/sec: 91.621\n",
"INFO:tensorflow:global_step/sec: 2.90716\n",
"I0523 05:58:39.504451 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90716\n",
"INFO:tensorflow:examples/sec: 93.0292\n",
"I0523 05:58:39.504680 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0292\n",
"INFO:tensorflow:global_step/sec: 2.88369\n",
"I0523 05:58:39.851221 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88369\n",
"INFO:tensorflow:examples/sec: 92.2781\n",
"I0523 05:58:39.851595 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2781\n",
"INFO:tensorflow:global_step/sec: 2.88514\n",
"I0523 05:58:40.197860 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88514\n",
"INFO:tensorflow:examples/sec: 92.3245\n",
"I0523 05:58:40.198247 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3245\n",
"INFO:tensorflow:global_step/sec: 2.88255\n",
"I0523 05:58:40.544739 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88255\n",
"INFO:tensorflow:examples/sec: 92.2417\n",
"I0523 05:58:40.545102 140278000138112 tpu_estimator.py:2308] examples/sec: 92.2417\n",
"INFO:tensorflow:global_step/sec: 2.88564\n",
"I0523 05:58:40.891339 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88564\n",
"INFO:tensorflow:examples/sec: 92.3404\n",
"I0523 05:58:40.891646 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3404\n",
"INFO:tensorflow:global_step/sec: 2.86818\n",
"I0523 05:58:41.239981 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86818\n",
"INFO:tensorflow:examples/sec: 91.7817\n",
"I0523 05:58:41.240355 140278000138112 tpu_estimator.py:2308] examples/sec: 91.7817\n",
"INFO:tensorflow:global_step/sec: 2.90784\n",
"I0523 05:58:41.583890 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90784\n",
"INFO:tensorflow:examples/sec: 93.0509\n",
"I0523 05:58:41.585281 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0509\n",
"INFO:tensorflow:global_step/sec: 2.88902\n",
"I0523 05:58:41.930019 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88902\n",
"INFO:tensorflow:examples/sec: 92.4486\n",
"I0523 05:58:41.930410 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4486\n",
"INFO:tensorflow:global_step/sec: 2.87842\n",
"I0523 05:58:42.277640 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87842\n",
"INFO:tensorflow:examples/sec: 92.1095\n",
"I0523 05:58:42.278029 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1095\n",
"INFO:tensorflow:global_step/sec: 2.88617\n",
"I0523 05:58:42.623897 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88617\n",
"INFO:tensorflow:examples/sec: 92.3575\n",
"I0523 05:58:42.624249 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3575\n",
"INFO:tensorflow:global_step/sec: 2.90234\n",
"I0523 05:58:42.968434 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90234\n",
"INFO:tensorflow:examples/sec: 92.8748\n",
"I0523 05:58:42.968782 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8748\n",
"INFO:tensorflow:global_step/sec: 2.85659\n",
"I0523 05:58:43.318545 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.85659\n",
"INFO:tensorflow:examples/sec: 91.411\n",
"I0523 05:58:43.318984 140278000138112 tpu_estimator.py:2308] examples/sec: 91.411\n",
"INFO:tensorflow:global_step/sec: 2.88497\n",
"I0523 05:58:43.665170 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88497\n",
"INFO:tensorflow:examples/sec: 92.3189\n",
"I0523 05:58:43.665704 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3189\n",
"INFO:tensorflow:global_step/sec: 2.89497\n",
"I0523 05:58:44.010535 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89497\n",
"INFO:tensorflow:examples/sec: 92.6391\n",
"I0523 05:58:44.010955 140278000138112 tpu_estimator.py:2308] examples/sec: 92.6391\n",
"INFO:tensorflow:global_step/sec: 2.88755\n",
"I0523 05:58:44.356867 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88755\n",
"INFO:tensorflow:examples/sec: 92.4015\n",
"I0523 05:58:44.357244 140278000138112 tpu_estimator.py:2308] examples/sec: 92.4015\n",
"INFO:tensorflow:global_step/sec: 2.88054\n",
"I0523 05:58:44.704029 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88054\n",
"INFO:tensorflow:examples/sec: 92.1773\n",
"I0523 05:58:44.704259 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1773\n",
"INFO:tensorflow:global_step/sec: 2.87817\n",
"I0523 05:58:45.051456 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87817\n",
"INFO:tensorflow:examples/sec: 92.1013\n",
"I0523 05:58:45.051817 140278000138112 tpu_estimator.py:2308] examples/sec: 92.1013\n",
"INFO:tensorflow:global_step/sec: 2.8746\n",
"I0523 05:58:45.399402 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.8746\n",
"INFO:tensorflow:examples/sec: 91.9872\n",
"I0523 05:58:45.399870 140278000138112 tpu_estimator.py:2308] examples/sec: 91.9872\n",
"INFO:tensorflow:global_step/sec: 2.9016\n",
"I0523 05:58:45.743983 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.9016\n",
"INFO:tensorflow:examples/sec: 92.8512\n",
"I0523 05:58:45.744735 140278000138112 tpu_estimator.py:2308] examples/sec: 92.8512\n",
"INFO:tensorflow:global_step/sec: 2.86879\n",
"I0523 05:58:46.092580 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.86879\n",
"INFO:tensorflow:examples/sec: 91.8013\n",
"I0523 05:58:46.093008 140278000138112 tpu_estimator.py:2308] examples/sec: 91.8013\n",
"INFO:tensorflow:global_step/sec: 2.90028\n",
"I0523 05:58:46.437361 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90028\n",
"INFO:tensorflow:examples/sec: 92.809\n",
"I0523 05:58:46.437769 140278000138112 tpu_estimator.py:2308] examples/sec: 92.809\n",
"INFO:tensorflow:global_step/sec: 2.88704\n",
"I0523 05:58:46.783712 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88704\n",
"INFO:tensorflow:examples/sec: 92.3852\n",
"I0523 05:58:46.784073 140278000138112 tpu_estimator.py:2308] examples/sec: 92.3852\n",
"INFO:tensorflow:global_step/sec: 2.87623\n",
"I0523 05:58:47.131437 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.87623\n",
"INFO:tensorflow:examples/sec: 92.0395\n",
"I0523 05:58:47.131938 140278000138112 tpu_estimator.py:2308] examples/sec: 92.0395\n",
"INFO:tensorflow:global_step/sec: 2.89094\n",
"I0523 05:58:47.477347 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89094\n",
"INFO:tensorflow:examples/sec: 92.5102\n",
"I0523 05:58:47.477750 140278000138112 tpu_estimator.py:2308] examples/sec: 92.5102\n",
"INFO:tensorflow:global_step/sec: 2.90643\n",
"I0523 05:58:47.821373 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90643\n",
"INFO:tensorflow:examples/sec: 93.0056\n",
"I0523 05:58:47.821610 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0056\n",
"INFO:tensorflow:global_step/sec: 2.89085\n",
"I0523 05:58:48.167312 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.89085\n",
"INFO:tensorflow:examples/sec: 92.507\n",
"I0523 05:58:48.167546 140278000138112 tpu_estimator.py:2308] examples/sec: 92.507\n",
"INFO:tensorflow:global_step/sec: 2.90806\n",
"I0523 05:58:48.511165 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.90806\n",
"INFO:tensorflow:examples/sec: 93.0579\n",
"I0523 05:58:48.511512 140278000138112 tpu_estimator.py:2308] examples/sec: 93.0579\n",
"INFO:tensorflow:global_step/sec: 2.88753\n",
"I0523 05:58:48.857486 140278000138112 tpu_estimator.py:2307] global_step/sec: 2.88753\n",
"INFO:tensorflow:examples/sec: 92.401\n",
"I0523 05:58:48.857712 140278000138112 tpu_estimator.py:2308] examples/sec: 92.401\n",
"INFO:tensorflow:Saving checkpoints for 2000 into ./pretraining_output/model.ckpt.\n",
"I0523 05:58:49.199793 140278000138112 basic_session_run_hooks.py:606] Saving checkpoints for 2000 into ./pretraining_output/model.ckpt.\n",
"INFO:tensorflow:global_step/sec: 0.421307\n",
"I0523 05:58:51.231056 140278000138112 tpu_estimator.py:2307] global_step/sec: 0.421307\n",
"INFO:tensorflow:examples/sec: 13.4818\n",
"I0523 05:58:51.231326 140278000138112 tpu_estimator.py:2308] examples/sec: 13.4818\n",
"INFO:tensorflow:Loss for final step: 0.050185096.\n",
"I0523 05:58:51.382073 140278000138112 estimator.py:371] Loss for final step: 0.050185096.\n",
"INFO:tensorflow:training_loop marked as finished\n",
"I0523 05:58:51.382603 140278000138112 error_handling.py:101] training_loop marked as finished\n",
"INFO:tensorflow:***** Running evaluation *****\n",
"I0523 05:58:51.382758 140278000138112 run_pretraining.py:469] ***** Running evaluation *****\n",
"INFO:tensorflow: Batch size = 8\n",
"I0523 05:58:51.382849 140278000138112 run_pretraining.py:470] Batch size = 8\n",
"INFO:tensorflow:Calling model_fn.\n",
"I0523 05:58:51.426540 140278000138112 estimator.py:1148] Calling model_fn.\n",
"INFO:tensorflow:Running eval on CPU\n",
"I0523 05:58:51.426895 140278000138112 tpu_estimator.py:3124] Running eval on CPU\n",
"INFO:tensorflow:*** Features ***\n",
"I0523 05:58:51.427314 140278000138112 run_pretraining.py:117] *** Features ***\n",
"INFO:tensorflow: name = input_ids, shape = (8, 128)\n",
"I0523 05:58:51.427501 140278000138112 run_pretraining.py:119] name = input_ids, shape = (8, 128)\n",
"INFO:tensorflow: name = input_mask, shape = (8, 128)\n",
"I0523 05:58:51.427630 140278000138112 run_pretraining.py:119] name = input_mask, shape = (8, 128)\n",
"INFO:tensorflow: name = masked_lm_ids, shape = (8, 20)\n",
"I0523 05:58:51.427742 140278000138112 run_pretraining.py:119] name = masked_lm_ids, shape = (8, 20)\n",
"INFO:tensorflow: name = masked_lm_positions, shape = (8, 20)\n",
"I0523 05:58:51.427861 140278000138112 run_pretraining.py:119] name = masked_lm_positions, shape = (8, 20)\n",
"INFO:tensorflow: name = masked_lm_weights, shape = (8, 20)\n",
"I0523 05:58:51.427992 140278000138112 run_pretraining.py:119] name = masked_lm_weights, shape = (8, 20)\n",
"INFO:tensorflow: name = next_sentence_labels, shape = (8, 1)\n",
"I0523 05:58:51.428103 140278000138112 run_pretraining.py:119] name = next_sentence_labels, shape = (8, 1)\n",
"INFO:tensorflow: name = segment_ids, shape = (8, 128)\n",
"I0523 05:58:51.428216 140278000138112 run_pretraining.py:119] name = segment_ids, shape = (8, 128)\n",
"INFO:tensorflow:**** Trainable Variables ****\n",
"I0523 05:58:52.504786 140278000138112 run_pretraining.py:167] **** Trainable Variables ****\n",
"INFO:tensorflow: name = bert/embeddings/word_embeddings:0, shape = (30522, 512)\n",
"I0523 05:58:52.505040 140278000138112 run_pretraining.py:173] name = bert/embeddings/word_embeddings:0, shape = (30522, 512)\n",
"INFO:tensorflow: name = bert/embeddings/token_type_embeddings:0, shape = (2, 512)\n",
"I0523 05:58:52.505197 140278000138112 run_pretraining.py:173] name = bert/embeddings/token_type_embeddings:0, shape = (2, 512)\n",
"INFO:tensorflow: name = bert/embeddings/position_embeddings:0, shape = (512, 512)\n",
"I0523 05:58:52.505326 140278000138112 run_pretraining.py:173] name = bert/embeddings/position_embeddings:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/embeddings/LayerNorm/beta:0, shape = (512,)\n",
"I0523 05:58:52.505449 140278000138112 run_pretraining.py:173] name = bert/embeddings/LayerNorm/beta:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/embeddings/LayerNorm/gamma:0, shape = (512,)\n",
"I0523 05:58:52.505565 140278000138112 run_pretraining.py:173] name = bert/embeddings/LayerNorm/gamma:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_0/attention/self/query/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.505682 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_0/attention/self/query/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_0/attention/self/query/bias:0, shape = (512,)\n",
"I0523 05:58:52.505829 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_0/attention/self/query/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_0/attention/self/key/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.505972 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_0/attention/self/key/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_0/attention/self/key/bias:0, shape = (512,)\n",
"I0523 05:58:52.506098 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_0/attention/self/key/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_0/attention/self/value/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.506219 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_0/attention/self/value/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_0/attention/self/value/bias:0, shape = (512,)\n",
"I0523 05:58:52.506347 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_0/attention/self/value/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_0/attention/output/dense/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.506467 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_0/attention/output/dense/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_0/attention/output/dense/bias:0, shape = (512,)\n",
"I0523 05:58:52.506589 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_0/attention/output/dense/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_0/attention/output/LayerNorm/beta:0, shape = (512,)\n",
"I0523 05:58:52.506707 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_0/attention/output/LayerNorm/beta:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_0/attention/output/LayerNorm/gamma:0, shape = (512,)\n",
"I0523 05:58:52.506837 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_0/attention/output/LayerNorm/gamma:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_0/intermediate/dense/kernel:0, shape = (512, 2048)\n",
"I0523 05:58:52.506987 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_0/intermediate/dense/kernel:0, shape = (512, 2048)\n",
"INFO:tensorflow: name = bert/encoder/layer_0/intermediate/dense/bias:0, shape = (2048,)\n",
"I0523 05:58:52.507098 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_0/intermediate/dense/bias:0, shape = (2048,)\n",
"INFO:tensorflow: name = bert/encoder/layer_0/output/dense/kernel:0, shape = (2048, 512)\n",
"I0523 05:58:52.507212 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_0/output/dense/kernel:0, shape = (2048, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_0/output/dense/bias:0, shape = (512,)\n",
"I0523 05:58:52.507327 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_0/output/dense/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_0/output/LayerNorm/beta:0, shape = (512,)\n",
"I0523 05:58:52.507442 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_0/output/LayerNorm/beta:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_0/output/LayerNorm/gamma:0, shape = (512,)\n",
"I0523 05:58:52.507551 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_0/output/LayerNorm/gamma:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_1/attention/self/query/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.507681 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_1/attention/self/query/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_1/attention/self/query/bias:0, shape = (512,)\n",
"I0523 05:58:52.507810 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_1/attention/self/query/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_1/attention/self/key/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.507930 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_1/attention/self/key/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_1/attention/self/key/bias:0, shape = (512,)\n",
"I0523 05:58:52.508063 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_1/attention/self/key/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_1/attention/self/value/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.508182 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_1/attention/self/value/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_1/attention/self/value/bias:0, shape = (512,)\n",
"I0523 05:58:52.508302 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_1/attention/self/value/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_1/attention/output/dense/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.508420 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_1/attention/output/dense/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_1/attention/output/dense/bias:0, shape = (512,)\n",
"I0523 05:58:52.508549 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_1/attention/output/dense/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_1/attention/output/LayerNorm/beta:0, shape = (512,)\n",
"I0523 05:58:52.508668 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_1/attention/output/LayerNorm/beta:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_1/attention/output/LayerNorm/gamma:0, shape = (512,)\n",
"I0523 05:58:52.508785 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_1/attention/output/LayerNorm/gamma:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_1/intermediate/dense/kernel:0, shape = (512, 2048)\n",
"I0523 05:58:52.508898 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_1/intermediate/dense/kernel:0, shape = (512, 2048)\n",
"INFO:tensorflow: name = bert/encoder/layer_1/intermediate/dense/bias:0, shape = (2048,)\n",
"I0523 05:58:52.509025 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_1/intermediate/dense/bias:0, shape = (2048,)\n",
"INFO:tensorflow: name = bert/encoder/layer_1/output/dense/kernel:0, shape = (2048, 512)\n",
"I0523 05:58:52.509148 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_1/output/dense/kernel:0, shape = (2048, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_1/output/dense/bias:0, shape = (512,)\n",
"I0523 05:58:52.509259 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_1/output/dense/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_1/output/LayerNorm/beta:0, shape = (512,)\n",
"I0523 05:58:52.509364 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_1/output/LayerNorm/beta:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_1/output/LayerNorm/gamma:0, shape = (512,)\n",
"I0523 05:58:52.509466 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_1/output/LayerNorm/gamma:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_2/attention/self/query/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.509571 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_2/attention/self/query/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_2/attention/self/query/bias:0, shape = (512,)\n",
"I0523 05:58:52.509680 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_2/attention/self/query/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_2/attention/self/key/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.509812 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_2/attention/self/key/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_2/attention/self/key/bias:0, shape = (512,)\n",
"I0523 05:58:52.509926 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_2/attention/self/key/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_2/attention/self/value/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.510048 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_2/attention/self/value/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_2/attention/self/value/bias:0, shape = (512,)\n",
"I0523 05:58:52.510164 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_2/attention/self/value/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_2/attention/output/dense/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.510282 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_2/attention/output/dense/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_2/attention/output/dense/bias:0, shape = (512,)\n",
"I0523 05:58:52.510396 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_2/attention/output/dense/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_2/attention/output/LayerNorm/beta:0, shape = (512,)\n",
"I0523 05:58:52.510505 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_2/attention/output/LayerNorm/beta:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_2/attention/output/LayerNorm/gamma:0, shape = (512,)\n",
"I0523 05:58:52.510613 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_2/attention/output/LayerNorm/gamma:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_2/intermediate/dense/kernel:0, shape = (512, 2048)\n",
"I0523 05:58:52.510730 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_2/intermediate/dense/kernel:0, shape = (512, 2048)\n",
"INFO:tensorflow: name = bert/encoder/layer_2/intermediate/dense/bias:0, shape = (2048,)\n",
"I0523 05:58:52.510897 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_2/intermediate/dense/bias:0, shape = (2048,)\n",
"INFO:tensorflow: name = bert/encoder/layer_2/output/dense/kernel:0, shape = (2048, 512)\n",
"I0523 05:58:52.511032 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_2/output/dense/kernel:0, shape = (2048, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_2/output/dense/bias:0, shape = (512,)\n",
"I0523 05:58:52.511149 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_2/output/dense/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_2/output/LayerNorm/beta:0, shape = (512,)\n",
"I0523 05:58:52.511270 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_2/output/LayerNorm/beta:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_2/output/LayerNorm/gamma:0, shape = (512,)\n",
"I0523 05:58:52.511374 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_2/output/LayerNorm/gamma:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_3/attention/self/query/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.511480 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_3/attention/self/query/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_3/attention/self/query/bias:0, shape = (512,)\n",
"I0523 05:58:52.511610 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_3/attention/self/query/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_3/attention/self/key/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.511729 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_3/attention/self/key/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_3/attention/self/key/bias:0, shape = (512,)\n",
"I0523 05:58:52.511853 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_3/attention/self/key/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_3/attention/self/value/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.511977 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_3/attention/self/value/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_3/attention/self/value/bias:0, shape = (512,)\n",
"I0523 05:58:52.512090 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_3/attention/self/value/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_3/attention/output/dense/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.512201 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_3/attention/output/dense/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_3/attention/output/dense/bias:0, shape = (512,)\n",
"I0523 05:58:52.512317 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_3/attention/output/dense/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_3/attention/output/LayerNorm/beta:0, shape = (512,)\n",
"I0523 05:58:52.512426 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_3/attention/output/LayerNorm/beta:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_3/attention/output/LayerNorm/gamma:0, shape = (512,)\n",
"I0523 05:58:52.512536 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_3/attention/output/LayerNorm/gamma:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_3/intermediate/dense/kernel:0, shape = (512, 2048)\n",
"I0523 05:58:52.512665 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_3/intermediate/dense/kernel:0, shape = (512, 2048)\n",
"INFO:tensorflow: name = bert/encoder/layer_3/intermediate/dense/bias:0, shape = (2048,)\n",
"I0523 05:58:52.512796 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_3/intermediate/dense/bias:0, shape = (2048,)\n",
"INFO:tensorflow: name = bert/encoder/layer_3/output/dense/kernel:0, shape = (2048, 512)\n",
"I0523 05:58:52.512915 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_3/output/dense/kernel:0, shape = (2048, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_3/output/dense/bias:0, shape = (512,)\n",
"I0523 05:58:52.513049 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_3/output/dense/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_3/output/LayerNorm/beta:0, shape = (512,)\n",
"I0523 05:58:52.513170 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_3/output/LayerNorm/beta:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_3/output/LayerNorm/gamma:0, shape = (512,)\n",
"I0523 05:58:52.513289 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_3/output/LayerNorm/gamma:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_4/attention/self/query/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.513407 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_4/attention/self/query/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_4/attention/self/query/bias:0, shape = (512,)\n",
"I0523 05:58:52.513528 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_4/attention/self/query/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_4/attention/self/key/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.513653 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_4/attention/self/key/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_4/attention/self/key/bias:0, shape = (512,)\n",
"I0523 05:58:52.513806 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_4/attention/self/key/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_4/attention/self/value/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.513911 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_4/attention/self/value/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_4/attention/self/value/bias:0, shape = (512,)\n",
"I0523 05:58:52.514036 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_4/attention/self/value/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_4/attention/output/dense/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.514154 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_4/attention/output/dense/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_4/attention/output/dense/bias:0, shape = (512,)\n",
"I0523 05:58:52.514284 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_4/attention/output/dense/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_4/attention/output/LayerNorm/beta:0, shape = (512,)\n",
"I0523 05:58:52.514393 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_4/attention/output/LayerNorm/beta:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_4/attention/output/LayerNorm/gamma:0, shape = (512,)\n",
"I0523 05:58:52.514507 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_4/attention/output/LayerNorm/gamma:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_4/intermediate/dense/kernel:0, shape = (512, 2048)\n",
"I0523 05:58:52.514617 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_4/intermediate/dense/kernel:0, shape = (512, 2048)\n",
"INFO:tensorflow: name = bert/encoder/layer_4/intermediate/dense/bias:0, shape = (2048,)\n",
"I0523 05:58:52.514735 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_4/intermediate/dense/bias:0, shape = (2048,)\n",
"INFO:tensorflow: name = bert/encoder/layer_4/output/dense/kernel:0, shape = (2048, 512)\n",
"I0523 05:58:52.514903 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_4/output/dense/kernel:0, shape = (2048, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_4/output/dense/bias:0, shape = (512,)\n",
"I0523 05:58:52.515052 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_4/output/dense/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_4/output/LayerNorm/beta:0, shape = (512,)\n",
"I0523 05:58:52.515168 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_4/output/LayerNorm/beta:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_4/output/LayerNorm/gamma:0, shape = (512,)\n",
"I0523 05:58:52.515298 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_4/output/LayerNorm/gamma:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_5/attention/self/query/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.515440 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_5/attention/self/query/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_5/attention/self/query/bias:0, shape = (512,)\n",
"I0523 05:58:52.515553 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_5/attention/self/query/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_5/attention/self/key/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.515692 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_5/attention/self/key/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_5/attention/self/key/bias:0, shape = (512,)\n",
"I0523 05:58:52.515861 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_5/attention/self/key/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_5/attention/self/value/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.516030 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_5/attention/self/value/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_5/attention/self/value/bias:0, shape = (512,)\n",
"I0523 05:58:52.516184 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_5/attention/self/value/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_5/attention/output/dense/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.516325 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_5/attention/output/dense/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_5/attention/output/dense/bias:0, shape = (512,)\n",
"I0523 05:58:52.516471 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_5/attention/output/dense/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_5/attention/output/LayerNorm/beta:0, shape = (512,)\n",
"I0523 05:58:52.516625 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_5/attention/output/LayerNorm/beta:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_5/attention/output/LayerNorm/gamma:0, shape = (512,)\n",
"I0523 05:58:52.516778 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_5/attention/output/LayerNorm/gamma:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_5/intermediate/dense/kernel:0, shape = (512, 2048)\n",
"I0523 05:58:52.516927 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_5/intermediate/dense/kernel:0, shape = (512, 2048)\n",
"INFO:tensorflow: name = bert/encoder/layer_5/intermediate/dense/bias:0, shape = (2048,)\n",
"I0523 05:58:52.517121 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_5/intermediate/dense/bias:0, shape = (2048,)\n",
"INFO:tensorflow: name = bert/encoder/layer_5/output/dense/kernel:0, shape = (2048, 512)\n",
"I0523 05:58:52.517273 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_5/output/dense/kernel:0, shape = (2048, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_5/output/dense/bias:0, shape = (512,)\n",
"I0523 05:58:52.517435 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_5/output/dense/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_5/output/LayerNorm/beta:0, shape = (512,)\n",
"I0523 05:58:52.517593 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_5/output/LayerNorm/beta:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_5/output/LayerNorm/gamma:0, shape = (512,)\n",
"I0523 05:58:52.517748 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_5/output/LayerNorm/gamma:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_6/attention/self/query/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.517901 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_6/attention/self/query/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_6/attention/self/query/bias:0, shape = (512,)\n",
"I0523 05:58:52.518102 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_6/attention/self/query/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_6/attention/self/key/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.518268 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_6/attention/self/key/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_6/attention/self/key/bias:0, shape = (512,)\n",
"I0523 05:58:52.518436 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_6/attention/self/key/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_6/attention/self/value/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.518593 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_6/attention/self/value/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_6/attention/self/value/bias:0, shape = (512,)\n",
"I0523 05:58:52.518768 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_6/attention/self/value/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_6/attention/output/dense/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.518919 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_6/attention/output/dense/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_6/attention/output/dense/bias:0, shape = (512,)\n",
"I0523 05:58:52.519117 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_6/attention/output/dense/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_6/attention/output/LayerNorm/beta:0, shape = (512,)\n",
"I0523 05:58:52.519360 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_6/attention/output/LayerNorm/beta:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_6/attention/output/LayerNorm/gamma:0, shape = (512,)\n",
"I0523 05:58:52.519520 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_6/attention/output/LayerNorm/gamma:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_6/intermediate/dense/kernel:0, shape = (512, 2048)\n",
"I0523 05:58:52.519665 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_6/intermediate/dense/kernel:0, shape = (512, 2048)\n",
"INFO:tensorflow: name = bert/encoder/layer_6/intermediate/dense/bias:0, shape = (2048,)\n",
"I0523 05:58:52.519863 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_6/intermediate/dense/bias:0, shape = (2048,)\n",
"INFO:tensorflow: name = bert/encoder/layer_6/output/dense/kernel:0, shape = (2048, 512)\n",
"I0523 05:58:52.520041 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_6/output/dense/kernel:0, shape = (2048, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_6/output/dense/bias:0, shape = (512,)\n",
"I0523 05:58:52.520205 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_6/output/dense/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_6/output/LayerNorm/beta:0, shape = (512,)\n",
"I0523 05:58:52.520353 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_6/output/LayerNorm/beta:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_6/output/LayerNorm/gamma:0, shape = (512,)\n",
"I0523 05:58:52.520496 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_6/output/LayerNorm/gamma:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_7/attention/self/query/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.520650 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_7/attention/self/query/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_7/attention/self/query/bias:0, shape = (512,)\n",
"I0523 05:58:52.520856 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_7/attention/self/query/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_7/attention/self/key/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.521028 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_7/attention/self/key/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_7/attention/self/key/bias:0, shape = (512,)\n",
"I0523 05:58:52.521215 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_7/attention/self/key/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_7/attention/self/value/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.521364 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_7/attention/self/value/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_7/attention/self/value/bias:0, shape = (512,)\n",
"I0523 05:58:52.521590 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_7/attention/self/value/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_7/attention/output/dense/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.521786 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_7/attention/output/dense/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_7/attention/output/dense/bias:0, shape = (512,)\n",
"I0523 05:58:52.521940 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_7/attention/output/dense/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_7/attention/output/LayerNorm/beta:0, shape = (512,)\n",
"I0523 05:58:52.522103 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_7/attention/output/LayerNorm/beta:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_7/attention/output/LayerNorm/gamma:0, shape = (512,)\n",
"I0523 05:58:52.522244 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_7/attention/output/LayerNorm/gamma:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_7/intermediate/dense/kernel:0, shape = (512, 2048)\n",
"I0523 05:58:52.522388 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_7/intermediate/dense/kernel:0, shape = (512, 2048)\n",
"INFO:tensorflow: name = bert/encoder/layer_7/intermediate/dense/bias:0, shape = (2048,)\n",
"I0523 05:58:52.522536 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_7/intermediate/dense/bias:0, shape = (2048,)\n",
"INFO:tensorflow: name = bert/encoder/layer_7/output/dense/kernel:0, shape = (2048, 512)\n",
"I0523 05:58:52.522677 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_7/output/dense/kernel:0, shape = (2048, 512)\n",
"INFO:tensorflow: name = bert/encoder/layer_7/output/dense/bias:0, shape = (512,)\n",
"I0523 05:58:52.522838 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_7/output/dense/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_7/output/LayerNorm/beta:0, shape = (512,)\n",
"I0523 05:58:52.522999 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_7/output/LayerNorm/beta:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/encoder/layer_7/output/LayerNorm/gamma:0, shape = (512,)\n",
"I0523 05:58:52.523281 140278000138112 run_pretraining.py:173] name = bert/encoder/layer_7/output/LayerNorm/gamma:0, shape = (512,)\n",
"INFO:tensorflow: name = bert/pooler/dense/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.523464 140278000138112 run_pretraining.py:173] name = bert/pooler/dense/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = bert/pooler/dense/bias:0, shape = (512,)\n",
"I0523 05:58:52.523625 140278000138112 run_pretraining.py:173] name = bert/pooler/dense/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = cls/predictions/transform/dense/kernel:0, shape = (512, 512)\n",
"I0523 05:58:52.523799 140278000138112 run_pretraining.py:173] name = cls/predictions/transform/dense/kernel:0, shape = (512, 512)\n",
"INFO:tensorflow: name = cls/predictions/transform/dense/bias:0, shape = (512,)\n",
"I0523 05:58:52.524036 140278000138112 run_pretraining.py:173] name = cls/predictions/transform/dense/bias:0, shape = (512,)\n",
"INFO:tensorflow: name = cls/predictions/transform/LayerNorm/beta:0, shape = (512,)\n",
"I0523 05:58:52.524207 140278000138112 run_pretraining.py:173] name = cls/predictions/transform/LayerNorm/beta:0, shape = (512,)\n",
"INFO:tensorflow: name = cls/predictions/transform/LayerNorm/gamma:0, shape = (512,)\n",
"I0523 05:58:52.524457 140278000138112 run_pretraining.py:173] name = cls/predictions/transform/LayerNorm/gamma:0, shape = (512,)\n",
"INFO:tensorflow: name = cls/predictions/output_bias:0, shape = (30522,)\n",
"I0523 05:58:52.524618 140278000138112 run_pretraining.py:173] name = cls/predictions/output_bias:0, shape = (30522,)\n",
"INFO:tensorflow: name = cls/seq_relationship/output_weights:0, shape = (2, 512)\n",
"I0523 05:58:52.524814 140278000138112 run_pretraining.py:173] name = cls/seq_relationship/output_weights:0, shape = (2, 512)\n",
"INFO:tensorflow: name = cls/seq_relationship/output_bias:0, shape = (2,)\n",
"I0523 05:58:52.525023 140278000138112 run_pretraining.py:173] name = cls/seq_relationship/output_bias:0, shape = (2,)\n",
"WARNING:tensorflow:From run_pretraining.py:198: The name tf.metrics.accuracy is deprecated. Please use tf.compat.v1.metrics.accuracy instead.\n",
"\n",
"W0523 05:58:52.531900 140278000138112 module_wrapper.py:139] From run_pretraining.py:198: The name tf.metrics.accuracy is deprecated. Please use tf.compat.v1.metrics.accuracy instead.\n",
"\n",
"WARNING:tensorflow:From run_pretraining.py:202: The name tf.metrics.mean is deprecated. Please use tf.compat.v1.metrics.mean instead.\n",
"\n",
"W0523 05:58:52.544584 140278000138112 module_wrapper.py:139] From run_pretraining.py:202: The name tf.metrics.mean is deprecated. Please use tf.compat.v1.metrics.mean instead.\n",
"\n",
"INFO:tensorflow:Done calling model_fn.\n",
"I0523 05:58:52.582737 140278000138112 estimator.py:1150] Done calling model_fn.\n",
"INFO:tensorflow:Starting evaluation at 2021-05-23T05:58:52Z\n",
"I0523 05:58:52.598697 140278000138112 evaluation.py:255] Starting evaluation at 2021-05-23T05:58:52Z\n",
"INFO:tensorflow:Graph was finalized.\n",
"I0523 05:58:52.915798 140278000138112 monitored_session.py:240] Graph was finalized.\n",
"2021-05-23 05:58:52.916509: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
"2021-05-23 05:58:52.916987: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: \n",
"name: Tesla T4 major: 7 minor: 5 memoryClockRate(GHz): 1.59\n",
"pciBusID: 0000:00:04.0\n",
"2021-05-23 05:58:52.917110: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0\n",
"2021-05-23 05:58:52.917136: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0\n",
"2021-05-23 05:58:52.917158: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0\n",
"2021-05-23 05:58:52.917178: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0\n",
"2021-05-23 05:58:52.917198: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0\n",
"2021-05-23 05:58:52.917218: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0\n",
"2021-05-23 05:58:52.917239: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7\n",
"2021-05-23 05:58:52.917450: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
"2021-05-23 05:58:52.917955: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
"2021-05-23 05:58:52.918331: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0\n",
"2021-05-23 05:58:52.918416: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:\n",
"2021-05-23 05:58:52.918432: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165] 0 \n",
"2021-05-23 05:58:52.918442: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0: N \n",
"2021-05-23 05:58:52.918549: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
"2021-05-23 05:58:52.918989: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
"2021-05-23 05:58:52.919365: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14257 MB memory) -> physical GPU (device: 0, name: Tesla T4, pci bus id: 0000:00:04.0, compute capability: 7.5)\n",
"INFO:tensorflow:Restoring parameters from ./pretraining_output/model.ckpt-2000\n",
"I0523 05:58:52.920472 140278000138112 saver.py:1284] Restoring parameters from ./pretraining_output/model.ckpt-2000\n",
"INFO:tensorflow:Running local_init_op.\n",
"I0523 05:58:53.340634 140278000138112 session_manager.py:500] Running local_init_op.\n",
"INFO:tensorflow:Done running local_init_op.\n",
"I0523 05:58:53.379347 140278000138112 session_manager.py:502] Done running local_init_op.\n",
"INFO:tensorflow:Evaluation [10/100]\n",
"I0523 05:58:54.207664 140278000138112 evaluation.py:167] Evaluation [10/100]\n",
"INFO:tensorflow:Evaluation [20/100]\n",
"I0523 05:58:54.533823 140278000138112 evaluation.py:167] Evaluation [20/100]\n",
"INFO:tensorflow:Evaluation [30/100]\n",
"I0523 05:58:54.856418 140278000138112 evaluation.py:167] Evaluation [30/100]\n",
"INFO:tensorflow:Evaluation [40/100]\n",
"I0523 05:58:55.186884 140278000138112 evaluation.py:167] Evaluation [40/100]\n",
"INFO:tensorflow:Evaluation [50/100]\n",
"I0523 05:58:55.510119 140278000138112 evaluation.py:167] Evaluation [50/100]\n",
"INFO:tensorflow:Evaluation [60/100]\n",
"I0523 05:58:55.834377 140278000138112 evaluation.py:167] Evaluation [60/100]\n",
"INFO:tensorflow:Evaluation [70/100]\n",
"I0523 05:58:56.157641 140278000138112 evaluation.py:167] Evaluation [70/100]\n",
"INFO:tensorflow:Evaluation [80/100]\n",
"I0523 05:58:56.480048 140278000138112 evaluation.py:167] Evaluation [80/100]\n",
"INFO:tensorflow:Evaluation [90/100]\n",
"I0523 05:58:56.805264 140278000138112 evaluation.py:167] Evaluation [90/100]\n",
"INFO:tensorflow:Evaluation [100/100]\n",
"I0523 05:58:57.132653 140278000138112 evaluation.py:167] Evaluation [100/100]\n",
"INFO:tensorflow:Finished evaluation at 2021-05-23-05:58:57\n",
"I0523 05:58:57.200767 140278000138112 evaluation.py:275] Finished evaluation at 2021-05-23-05:58:57\n",
"INFO:tensorflow:Saving dict for global step 2000: global_step = 2000, loss = 0.026402967, masked_lm_accuracy = 1.0, masked_lm_loss = 0.026246134, next_sentence_accuracy = 1.0, next_sentence_loss = 8.266584e-05\n",
"I0523 05:58:57.201126 140278000138112 estimator.py:2049] Saving dict for global step 2000: global_step = 2000, loss = 0.026402967, masked_lm_accuracy = 1.0, masked_lm_loss = 0.026246134, next_sentence_accuracy = 1.0, next_sentence_loss = 8.266584e-05\n",
"INFO:tensorflow:Saving 'checkpoint_path' summary for global step 2000: ./pretraining_output/model.ckpt-2000\n",
"I0523 05:58:57.432043 140278000138112 estimator.py:2109] Saving 'checkpoint_path' summary for global step 2000: ./pretraining_output/model.ckpt-2000\n",
"INFO:tensorflow:evaluation_loop marked as finished\n",
"I0523 05:58:57.432663 140278000138112 error_handling.py:101] evaluation_loop marked as finished\n",
"INFO:tensorflow:***** Eval results *****\n",
"I0523 05:58:57.432916 140278000138112 run_pretraining.py:483] ***** Eval results *****\n",
"INFO:tensorflow: global_step = 2000\n",
"I0523 05:58:57.433038 140278000138112 run_pretraining.py:485] global_step = 2000\n",
"INFO:tensorflow: loss = 0.026402967\n",
"I0523 05:58:57.433249 140278000138112 run_pretraining.py:485] loss = 0.026402967\n",
"INFO:tensorflow: masked_lm_accuracy = 1.0\n",
"I0523 05:58:57.433364 140278000138112 run_pretraining.py:485] masked_lm_accuracy = 1.0\n",
"INFO:tensorflow: masked_lm_loss = 0.026246134\n",
"I0523 05:58:57.433449 140278000138112 run_pretraining.py:485] masked_lm_loss = 0.026246134\n",
"INFO:tensorflow: next_sentence_accuracy = 1.0\n",
"I0523 05:58:57.433530 140278000138112 run_pretraining.py:485] next_sentence_accuracy = 1.0\n",
"INFO:tensorflow: next_sentence_loss = 8.266584e-05\n",
"I0523 05:58:57.433609 140278000138112 run_pretraining.py:485] next_sentence_loss = 8.266584e-05\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "hO11uGkW0SUB"
},
"source": [
""
],
"execution_count": null,
"outputs": []
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment