Skip to content

Instantly share code, notes, and snippets.

@NHDaly

NHDaly/openai-vnc-logs-deluge-example.ipynb

Last active May 4, 2017 06:07
Show Gist options
  • Save NHDaly/346315a83d3a386de7c6ed6249c82b57 to your computer and use it in GitHub Desktop.
Save NHDaly/346315a83d3a386de7c6ed6249c82b57 to your computer and use it in GitHub Desktop.
Download ZIP
OpenAi VNCDriver environments cause too much output; makes iPython unusable.
Display the source blob
Display the rendered blob
Raw
openai-vnc-logs-deluge-example.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Example of VNC log output flooding iPython."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"import universe, gym"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Load the environment"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": false,
"scrolled": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"[2017-05-04 01:03:13,116] Making new env: internet.SlitherIO-v0\n"
]
}
],
"source": [
"env = gym.make('internet.SlitherIO-v0')"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# My attempt at silencing logs. Doesn't work. See next cell.\n",
"universe.wrappers.logger.logger.setLevel('ERROR')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that the above line doesn't seem to do anything, since the logs still contain a bunch of lines like the following:\n",
"```\n",
"universe-Au0Qha-0 | [2017-05-04 05:57:26,886] [INFO:universe.wrappers.logger] Stats for the past 5.00s: vnc_updates_ps=4.4 n=1 reaction_time=None observation_lag=None action_lag=None reward_ps=0.0 reward_total=0.0 vnc_bytes_ps[total]=543820.8 vnc_pixels_ps[total]=515305.1 reward_lag=None rewarder_message_lag=None fps=31.79\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": false,
"scrolled": true
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"[2017-05-04 01:03:18,811] Called reset on <Timer<Render<ThrottleVNCEnv<internet.SlitherIO-v0>>>> before configuring. Configuring automatically with default arguments\n",
"[2017-05-04 01:03:18,813] Writing logs to file: /tmp/universe-37421.log\n",
"[2017-05-04 01:03:19,850] Ports used: dict_keys([])\n",
"[2017-05-04 01:03:19,851] [0] Creating container: image=quay.io/openai/universe.flashgames:0.20.26. Run the same thing by hand as: docker run -p 5900:5900 -p 15900:15900 --cap-add SYS_ADMIN --privileged --ipc host quay.io/openai/universe.flashgames:0.20.26\n",
"[2017-05-04 01:03:20,402] Remote closed: address=localhost:15900\n",
"[2017-05-04 01:03:20,405] Remote closed: address=localhost:5900\n",
"[2017-05-04 01:03:20,407] At least one sockets was closed by the remote. Sleeping 1s...\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[36muniverse-196mHq-0 |\u001b[0m Setting VNC and rewarder password: openai\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [Thu May 4 06:03:20 UTC 2017] Waiting for /tmp/.X11-unix/X0 to be created (try 1/10)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [Thu May 4 06:03:20 UTC 2017] [/usr/local/bin/sudoable-env-setup] Disabling outbound network traffic for none\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [init] [2017-05-04 06:03:20,600] Launching system_diagnostics_logger.py, recorder_logdir=/tmp/demo\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [init] [2017-05-04 06:03:20,602] Launching reward_recorder.py, recorder_logdir=/tmp/demo\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [init] [2017-05-04 06:03:20,605] Launching vnc_recorder.py, recorder_logdir=/tmp/demo\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [init] [2017-05-04 06:03:20,609] PID 52 launched with command ['sudo', '-H', '-u', 'nobody', 'DISPLAY=:0', 'DBUS_SESSION_BUS_ADDRESS=/dev/null', '/app/universe-envs/controlplane/bin/controlplane.py', '--rewarder-port=15901']\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] \n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Xvnc TigerVNC 1.7.0 - built Sep 8 2016 10:39:22\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Copyright (C) 1999-2016 TigerVNC Team and many others (see README.txt)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] See http://www.tigervnc.org for information on TigerVNC.\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Underlying X server release 11400000, The X.Org Foundation\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] \n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Initializing built-in extension VNC-EXTENSION\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Initializing built-in extension Generic Event Extension\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Initializing built-in extension SHAPE\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Initializing built-in extension MIT-SHM\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Initializing built-in extension XInputExtension\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Initializing built-in extension XTEST\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Initializing built-in extension BIG-REQUESTS\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Initializing built-in extension SYNC\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Initializing built-in extension XKEYBOARD\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Initializing built-in extension XC-MISC\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Initializing built-in extension XINERAMA\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Initializing built-in extension XFIXES\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Initializing built-in extension RENDER\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Initializing built-in extension RANDR\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Initializing built-in extension COMPOSITE\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Initializing built-in extension DAMAGE\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Initializing built-in extension MIT-SCREEN-SAVER\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Initializing built-in extension DOUBLE-BUFFER\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Initializing built-in extension RECORD\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Initializing built-in extension DPMS\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Initializing built-in extension X-Resource\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Initializing built-in extension XVideo\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Initializing built-in extension XVideo-MotionCompensation\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Initializing built-in extension GLX\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] \n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Thu May 4 06:03:20 2017\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] vncext: VNC extension running!\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] vncext: Listening for VNC connections on all interface(s), port 5900\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] vncext: created VNC server for screen 0\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] [dix] Could not init font path element /usr/share/fonts/X11/Type1/, removing from list!\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] [dix] Could not init font path element /usr/share/fonts/X11/75dpi/, removing from list!\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] [dix] Could not init font path element /usr/share/fonts/X11/100dpi/, removing from list!\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [init] [2017-05-04 06:03:20,834] init detected end of child process 56 with exit code 0, not killed by signal\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m WebSocket server settings:\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m - Listen on :5898\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m - Flash security policy server\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m - No SSL/TLS support (no cert file)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m - proxying from :5898 to localhost:5900\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[2017-05-04 01:03:21,410] Remote closed: address=localhost:5900\n",
"[2017-05-04 01:03:21,411] Remote closed: address=localhost:15900\n",
"[2017-05-04 01:03:21,413] At least one sockets was closed by the remote. Sleeping 1s...\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] \n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Thu May 4 06:03:21 2017\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Connections: accepted: 172.17.0.1::52128\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [nginx] 2017/05/04 06:03:21 [error] 65#65: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.17.0.1, server: , request: \"GET / HTTP/1.1\", upstream: \"http://127.0.0.1:15901/\", host: \"127.0.0.1:10003\"\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [nginx] 172.17.0.1 - openai [04/May/2017:06:03:21 +0000] \"GET / HTTP/1.1\" 502 182 \"-\" \"-\"\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [reward_recorder] [2017-05-04 06:03:21,458] Listening on 0.0.0.0:15898\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [vnc_recorder] [2017-05-04 06:03:21,459] Listening on 0.0.0.0:5899\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [init] [2017-05-04 06:03:21,598] init detected end of child process 17 with exit code 0, not killed by signal\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:21,633] [INFO:root] Starting play_controlplane.py with the following: command=['/app/universe-envs/controlplane/bin/controlplane.py', '--rewarder-port=15901'] args=Namespace(bot_demonstration=False, demonstration=False, env_id=None, idle_timeout=None, integrator_mode=False, no_env=False, no_rewarder=False, no_scorer=False, no_vexpect=False, remotes='vnc://127.0.0.1:5900', rewarder_fps=60, rewarder_port=15901, verbosity=0) env=environ({'SUDO_UID': '0', 'SUDO_GID': '0', 'USER': 'nobody', 'TERM': 'xterm', 'LOGNAME': 'nobody', 'SUDO_USER': 'root', 'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin', 'DBUS_SESSION_BUS_ADDRESS': '/dev/null', 'USERNAME': 'nobody', 'HOSTNAME': '248e8c97ebbf', 'HOME': '/nonexistent', 'SHELL': '/usr/sbin/nologin', 'DISPLAY': ':0', 'SUDO_COMMAND': '/app/universe-envs/controlplane/bin/controlplane.py --rewarder-port=15901', 'MAIL': '/var/mail/nobody'})\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:21,633] [INFO:root] [EnvStatus] Changing env_state: None (env_id=None) -> None (env_id=None) (episode_id: 0->0, fps=60)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:21,633] [INFO:universe.rewarder.remote] Starting Rewarder on port=15901\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:21,636] [INFO:universe.extra.universe.wrappers.logger] Running VNC environments with Logger set to print_frequency=5. To change this, pass \"print_frequency=k\" or \"print_frequency=None\" to \"env.configure\".\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:21,636] [INFO:universe.remotes.hardcoded_addresses] No rewarder addresses were provided, so this env cannot connect to the remote's rewarder channel, and cannot send control messages (e.g. reset)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:21,637] [INFO:universe.envs.vnc_env] Using the golang VNC implementation\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:21,637] [INFO:universe.envs.vnc_env] Using VNCSession arguments: {'start_timeout': 7, 'fine_quality_level': 50, 'subsample_level': 2, 'encoding': 'zrle', 'compress_level': 9}. (Customize by running \"env.configure(vnc_kwargs={...})\"\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:21,637] [INFO:universe.envs.vnc_env] Printed stats will ignore clock skew. (This usually makes sense only when the environment and agent are on the same machine.)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:21,643] [INFO:universe.envs.vnc_env] [0] Connecting to environment: vnc://127.0.0.1:5900 password=openai. If desired, you can manually connect a VNC viewer, such as TurboVNC. Most environments provide a convenient in-browser VNC client: http://None/viewer/?password=openai\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:21,643] [INFO:universe.extra.universe.envs.vnc_env] [0] Connecting to environment details: vnc_address=127.0.0.1:5900 vnc_password=openai rewarder_address=None rewarder_password=openai\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:21,644] [INFO:root] [EnvStatus] Changing env_state: None (env_id=None) -> resetting (env_id=None) (episode_id: 0->1, fps=60)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:21,644] [INFO:root] [MainThread] Env state: env_id=None episode_id=1\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m 2017/05/04 06:03:21 I0504 06:03:21.644221 55 gymvnc.go:417] [0:127.0.0.1:5900] opening connection to VNC server\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Connections: accepted: 127.0.0.1::35482\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] SConnection: Client needs protocol version 3.8\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] SConnection: Client requests security type VncAuth(2)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] VNCSConnST: Server default pixel format depth 24 (32bpp) little-endian rgb888\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] VNCSConnST: Client pixel format depth 24 (32bpp) little-endian bgr888\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m 2017/05/04 06:03:21 I0504 06:03:21.652357 55 gymvnc.go:550] [0:127.0.0.1:5900] connection established\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [Thu May 4 06:03:21 UTC 2017] [/usr/local/bin/sudoable-env-setup] Disabling outbound network traffic for none\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:21,666] [INFO:gym_flashgames.launcher] [MainThread] Launching new Chrome process (attempt 0/10)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:21,667] [INFO:root] Replacing selenium_wrapper_server since we currently do it at every episode boundary\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:21,777] [selenium_wrapper_server] Calling webdriver.Chrome()\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[2017-05-04 01:03:22,414] Using the golang VNC implementation\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[36muniverse-196mHq-0 |\u001b[0m [nginx] 2017/05/04 06:03:22 [info] 65#65: *1 client 172.17.0.1 closed keepalive connection\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[2017-05-04 01:03:22,416] Using VNCSession arguments: {'encoding': 'tight', 'subsample_level': 2, 'start_timeout': 7, 'fine_quality_level': 50}. (Customize by running \"env.configure(vnc_kwargs={...})\"\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] \n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Thu May 4 06:03:22 2017\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Connections: closed: 172.17.0.1::52128 (Clean disconnection)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] EncodeManager: Framebuffer updates: 0\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] EncodeManager: Total: 0 rects, 0 pixels\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] EncodeManager: 0 B (1:-nan ratio)\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[2017-05-04 01:03:22,427] [0] Connecting to environment: vnc://localhost:5900 password=openai. If desired, you can manually connect a VNC viewer, such as TurboVNC. Most environments provide a convenient in-browser VNC client: http://localhost:15900/viewer/?password=openai\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Connections: accepted: 172.17.0.1::52140\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] SConnection: Client needs protocol version 3.8\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] SConnection: Client requests security type VncAuth(2)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] VNCSConnST: Server default pixel format depth 24 (32bpp) little-endian rgb888\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] VNCSConnST: Client pixel format depth 24 (32bpp) little-endian bgr888\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:22,453] [INFO:universe.rewarder.remote] Client connecting: peer=tcp4:127.0.0.1:49726 observer=False\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:22,453] [INFO:universe.rewarder.remote] WebSocket connection established\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [nginx] 2017/05/04 06:03:23 [info] 65#65: *5 client sent invalid request while reading client request line, client: 127.0.0.1, server: , request: \"CONNECT www.google.com:443 HTTP/1.1\"\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [nginx] 2017/05/04 06:03:23 [info] 65#65: *6 client sent invalid request while reading client request line, client: 127.0.0.1, server: , request: \"CONNECT www.google.com:443 HTTP/1.1\"\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:23,831] [selenium_wrapper_server] Call to webdriver.Chrome() completed: 2.05s\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:23,831] [INFO:gym_flashgames.launcher] [MainThread] Navigating browser to url=http://localhost\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [nginx] 2017/05/04 06:03:23 [info] 65#65: *7 client sent invalid request while reading client request line, client: 127.0.0.1, server: , request: \"CONNECT www.google.com:443 HTTP/1.1\"\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:24,078] [INFO:root] [EnvStatus] Changing env_state: resetting (env_id=None) -> running (env_id=None) (episode_id: 1->1, fps=60)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m Manhole[1493877804.0861]: Patched <built-in function fork> and <built-in function fork>.\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m Manhole[1493877804.0863]: Manhole UDS path: /tmp/manhole-55\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m Manhole[1493877804.0865]: Waiting for new connection (in pid:55) ...\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:26,652] [INFO:universe.wrappers.logger] Stats for the past 5.01s: vnc_updates_ps=2.8 n=1 reaction_time=None observation_lag=None action_lag=None reward_ps=0.0 reward_total=0.0 vnc_bytes_ps[total]=356572.4 vnc_pixels_ps[total]=531254.8 reward_lag=None rewarder_message_lag=None fps=30.95\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:29,083] [INFO:universe.pyprofile] [pyprofile] period=5.00s timers={\"rewarder.frame\": {\"std\": \"988.28us\", \"mean\": \"17.88ms\", \"calls\": 300}, \"rewarder.compute_reward\": {\"std\": \"161.68us\", \"mean\": \"381.78us\", \"calls\": 301}, \"rewarder.sleep\": {\"std\": \"1.05ms\", \"mean\": \"14.99ms\", \"calls\": 300}, \"vnc_env.VNCEnv.vnc_session.step\": {\"std\": \"98.88us\", \"mean\": \"177.43us\", \"calls\": 301}} counters={\"reward.vnc.updates.n\": {\"calls\": 301, \"mean\": 0.046511627906976785, \"std\": 0.7008300726265069}} gauges={} (export_time=114.92us)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:29,084] [INFO:universe.rewarder.remote] [Rewarder] Over past 5.00s, sent 1 reward messages to agent: reward=0 reward_min=0 reward_max=0 done=False info={'rewarder.vnc.updates.bytes': 0, 'rewarder.vnc.updates.pixels': 0, 'rewarder.vnc.updates.n': 0, 'rewarder.profile': '<761 bytes>'}\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:31,666] [INFO:universe.wrappers.logger] Stats for the past 5.01s: vnc_updates_ps=0.0 n=1 reaction_time=None observation_lag=None action_lag=None reward_ps=0.0 reward_total=0.0 vnc_bytes_ps[total]=0.0 vnc_pixels_ps[total]=0.0 reward_lag=None rewarder_message_lag=None fps=60.05\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:34,100] [INFO:universe.pyprofile] [pyprofile] period=5.02s timers={\"rewarder.frame\": {\"std\": \"1.10ms\", \"mean\": \"18.01ms\", \"calls\": 301}, \"rewarder.compute_reward\": {\"std\": \"141.20us\", \"mean\": \"375.52us\", \"calls\": 301}, \"rewarder.sleep\": {\"std\": \"1.16ms\", \"mean\": \"14.84ms\", \"calls\": 301}, \"vnc_env.VNCEnv.vnc_session.step\": {\"std\": \"88.07us\", \"mean\": \"167.89us\", \"calls\": 301}} counters={\"agent_conn.reward\": {\"calls\": 1, \"mean\": 0.0, \"std\": 0}, \"reward.vnc.updates.n\": {\"calls\": 301, \"mean\": 0.0, \"std\": 0.0}} gauges={} (export_time=80.35us)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:34,100] [INFO:universe.rewarder.remote] [Rewarder] Over past 5.02s, sent 1 reward messages to agent: reward=0 reward_min=0 reward_max=0 done=False info={'rewarder.vnc.updates.bytes': 0, 'rewarder.vnc.updates.pixels': 0, 'rewarder.vnc.updates.n': 0, 'rewarder.profile': '<802 bytes>'}\n"
]
}
],
"source": [
"# After starting the VNC environment, every cell is flooded\n",
"# with the output from that process.\n",
"s = env.reset()"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": false,
"scrolled": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Doing other things is hard!\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:36,683] [INFO:universe.wrappers.logger] Stats for the past 5.02s: vnc_updates_ps=0.0 n=1 reaction_time=None observation_lag=None action_lag=None reward_ps=0.0 reward_total=0.0 vnc_bytes_ps[total]=0.0 vnc_pixels_ps[total]=0.0 reward_lag=None rewarder_message_lag=None fps=60.01\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [nginx] 2017/05/04 06:03:36 [info] 65#65: *10 client closed connection while waiting for request, client: 172.17.0.1, server: 0.0.0.0:15900\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [nginx] 2017/05/04 06:03:36 [info] 65#65: *11 client closed connection while waiting for request, client: 172.17.0.1, server: 0.0.0.0:15900\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [nginx] 2017/05/04 06:03:36 [info] 65#65: *12 client closed connection while waiting for request, client: 172.17.0.1, server: 0.0.0.0:15900\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [nginx] 2017/05/04 06:03:36 [info] 65#65: *13 client closed connection while waiting for request, client: 172.17.0.1, server: 0.0.0.0:15900\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [nginx] 2017/05/04 06:03:36 [info] 65#65: *14 client closed connection while waiting for request, client: 172.17.0.1, server: 0.0.0.0:15900\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [nginx] 2017/05/04 06:03:36 [info] 65#65: *15 client closed connection while waiting for request, client: 172.17.0.1, server: 0.0.0.0:15900\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [nginx] 2017/05/04 06:03:36 [info] 65#65: *16 client closed connection while waiting for request, client: 172.17.0.1, server: 0.0.0.0:15900\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [nginx] 2017/05/04 06:03:36 [info] 65#65: *17 client closed connection while waiting for request, client: 172.17.0.1, server: 0.0.0.0:15900\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [nginx] 2017/05/04 06:03:36 [info] 65#65: *18 client closed connection while waiting for request, client: 172.17.0.1, server: 0.0.0.0:15900\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [nginx] 2017/05/04 06:03:36 [info] 65#65: *19 client closed connection while waiting for request, client: 172.17.0.1, server: 0.0.0.0:15900\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:36,799] [INFO:universe.rewarder.remote] CONNECTION STATUS: Marking connection as active: observer=False peer=tcp4:127.0.0.1:49726 total_conns=1\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[2017-05-04 01:03:36,812] [0:localhost:5900] Sending reset for env_id=internet.SlitherIO-v0 fps=60 episode_id=0\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:36,814] [INFO:universe.rewarder.remote] Received reset message: {'body': {'env_id': 'internet.SlitherIO-v0', 'seed': None, 'fps': 60}, 'headers': {'message_id': 10, 'sent_at': 1493877816.813549, 'episode_id': '0'}, 'method': 'v0.env.reset'}\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:36,824] [INFO:root] [EnvStatus] Changing env_state: running (env_id=None) -> resetting (env_id=internet.SlitherIO-v0) (episode_id: 1->2, fps=60)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:36,824] [ERROR:root] Closing server (via subprocess.close()) and all chromes (via pkill chromedriver || :; pkill chrome || :)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:36,831] [INFO:root] [Rewarder] Blocking until env finishes resetting\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [init] [2017-05-04 06:03:36,836] init detected end of child process 111 with exit code 0, killed by SIGTERM: 15\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:36,839] [INFO:root] [EnvController] RESET CAUSE: changing out environments due to v0.env.reset (with episode_id=0): internet.SlitherIO-v0 -> internet.SlitherIO-v0 (new episode_id=2 fps=60)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:36,839] [INFO:root] [EnvController] Env state: env_id=internet.SlitherIO-v0 episode_id=2\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [init] [2017-05-04 06:03:36,843] init detected end of child process 126 with exit code 0, not killed by signal\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [init] [2017-05-04 06:03:36,848] init detected end of child process 332 with exit code 0, killed by SIGTERM: 15\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [init] [2017-05-04 06:03:36,851] init detected end of child process 349 with exit code 0, killed by SIGTERM: 15\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [Thu May 4 06:03:36 UTC 2017] [/usr/local/bin/sudoable-env-setup] Allowing outbound network traffic to non-private IPs for git-lfs. (Going to fetch files via git lfs.)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [nginx] 2017/05/04 06:03:36 [info] 65#65: *9 client closed connection while waiting for request, client: 127.0.0.1, server: 0.0.0.0:80\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [nginx] 2017/05/04 06:03:36 [info] 65#65: *8 client 127.0.0.1 closed keepalive connection\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [init] [2017-05-04 06:03:36,930] init detected end of child process 125 with exit code 0, killed by SIGTERM: 15\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [init] [2017-05-04 06:03:36,935] init detected end of child process 114 with exit code 0, not killed by signal\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [init] [2017-05-04 06:03:36,936] init detected end of child process 122 with exit code 0, not killed by signal\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [init] [2017-05-04 06:03:36,936] init detected end of child process 123 with exit code 0, not killed by signal\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [unpack-lfs] [2017-05-04 06:03:36,978] Fetching files: git lfs pull -I git-lfs/internet.SlitherIO-v0.tar.gz\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [unpack-lfs] [2017-05-04 06:03:36,979] If this hangs, your docker container may not be able to communicate with Github\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [unpack-lfs] [2017-05-04 06:03:40,921] Finished running git lfs pull\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [unpack-lfs] [2017-05-04 06:03:40,921] git-lfs fetch succeeded\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [unpack-lfs] [2017-05-04 06:03:40,921] Unpacking files for internet.SlitherIO-v0\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [unpack-lfs] [2017-05-04 06:03:40,944] Merged 4 files from /tmp/internet.SlitherIO-v0/public -> /app/universe-envs/flashgames/build/public/internet.SlitherIO-v0\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [unpack-lfs] [2017-05-04 06:03:40,945] Merged 6 files from /tmp/internet.SlitherIO-v0/private -> /app/universe-envs/flashgames/build/private/internet.SlitherIO-v0\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [unpack-lfs] [2017-05-04 06:03:40,946] Completed unpack for internet.SlitherIO-v0 in 3.971s\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [Thu May 4 06:03:41 UTC 2017] [/usr/local/bin/sudoable-env-setup] [debug] unpack-lfs completed with status code: 0. Created completion file: /usr/local/openai/git-lfs/internet.SlitherIO-v0\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [Thu May 4 06:03:41 UTC 2017] [/usr/local/bin/sudoable-env-setup] Allowing outbound network traffic to non-private IPs for internet.SlitherIO-v0\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:41,061] [INFO:gym_flashgames.launcher] [EnvController] Launching new Chrome process (attempt 0/10)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:41,061] [INFO:root] Replacing selenium_wrapper_server since we currently do it at every episode boundary\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:41,151] [selenium_wrapper_server] Calling webdriver.Chrome()\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:42,441] [selenium_wrapper_server] Call to webdriver.Chrome() completed: 1.29s\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:42,442] [INFO:gym_flashgames.launcher] [EnvController] Navigating browser to url=http://localhost/internet.SlitherIO-v0\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m 2017/05/04 06:03:44 I0504 06:03:44.658788 55 gymvnc.go:374] [0:127.0.0.1:5900] update queue max of 60 reached; pausing further updates\n"
]
}
],
"source": [
"# When I execute this line, the output starts coming from here.\n",
"print(\"Doing other things is hard!\")"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": false,
"scrolled": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:47,096] [INFO:root] [EnvController] Running command: /app/universe-envs/controlplane/bin/play_vexpect -e internet.SlitherIO-v0 -r vnc://127.0.0.1:5900 -d\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:47,623] [play_vexpect] No rewarder addresses were provided, so this env cannot connect to the remote's rewarder channel, and cannot send control messages (e.g. reset)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:47,623] [play_vexpect] Using the golang VNC implementation\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:47,624] [play_vexpect] Using VNCSession arguments: {'encoding': 'zrle', 'subsample_level': 2, 'start_timeout': 7, 'compress_level': 0, 'fine_quality_level': 50}. (Customize by running \"env.configure(vnc_kwargs={...})\"\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:47,624] [play_vexpect] Printed stats will ignore clock skew. (This usually makes sense only when the environment and agent are on the same machine.)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:47,629] [play_vexpect] [0] Connecting to environment: vnc://127.0.0.1:5900 password=openai. If desired, you can manually connect a VNC viewer, such as TurboVNC. Most environments provide a convenient in-browser VNC client: http://None/viewer/?password=openai\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:47,629] [play_vexpect] [0] Connecting to environment details: vnc_address=127.0.0.1:5900 vnc_password=openai rewarder_address=None rewarder_password=openai\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m 2017/05/04 06:03:47 I0504 06:03:47.629544 729 gymvnc.go:417] [0:127.0.0.1:5900] opening connection to VNC server\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] \n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Thu May 4 06:03:47 2017\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Connections: accepted: 127.0.0.1::35748\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] SConnection: Client needs protocol version 3.8\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] SConnection: Client requests security type VncAuth(2)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] VNCSConnST: Server default pixel format depth 24 (32bpp) little-endian rgb888\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] VNCSConnST: Client pixel format depth 24 (32bpp) little-endian bgr888\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m 2017/05/04 06:03:47 I0504 06:03:47.632811 729 gymvnc.go:550] [0:127.0.0.1:5900] connection established\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:47,815] [play_vexpect] Waiting for any of [ready0, ready1, ready2] to activate\n"
]
}
],
"source": [
"import universe.spaces.vnc_action_space as vnc_a"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": false,
"scrolled": false
},
"outputs": [
{
"data": {
"text/plain": [
"[KeyEvent<key=n (0x6e) direction=up>]"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:50,982] [play_vexpect] Applying transition: ClickTransition<ready1->[] x=0 y=0 buttonmask=0> for active state ready1. (Summary: plausible_states=[ready0, ready1, ready2] distance_m=[0.0049178139, 0.0, 0.0] match_time_m=['198us', '89us', '78us'])\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:50,982] [play_vexpect] Reaching start state: ready1\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:50,983] [play_vexpect] vexpect macro complete in 3.318742s\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] \n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Thu May 4 06:03:51 2017\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Connections: closed: 127.0.0.1::35748 (Clean disconnection)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] EncodeManager: Framebuffer updates: 9\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] EncodeManager: ZRLE:\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] EncodeManager: Solid: 5 rects, 508 pixels\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] EncodeManager: 150 B (1:13.9467 ratio)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] EncodeManager: Bitmap RLE: 1 rects, 11.448 kpixels\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] EncodeManager: 1.50195 KiB (1:29.7815 ratio)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] EncodeManager: Indexed RLE: 7 rects, 68.018 kpixels\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] EncodeManager: 18.0312 KiB (1:14.7398 ratio)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] EncodeManager: Full Colour: 27 rects, 1.01581 Mpixels\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] EncodeManager: 2.90756 MiB (1:1.33284 ratio)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] EncodeManager: Total: 40 rects, 1.09578 Mpixels\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] EncodeManager: 2.92678 MiB (1:1.42837 ratio)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:51,261] [INFO:root] [EnvStatus] Changing env_state: resetting (env_id=internet.SlitherIO-v0) -> running (env_id=internet.SlitherIO-v0) (episode_id: 2->2, fps=60)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:51,262] [INFO:root] [Rewarder] Unblocking since env reset finished\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:51,262] [INFO:root] [Rewarder] Changing reward_parsers: None -> internet.SlitherIO-v0\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:51,272] [INFO:universe.rewarder.remote] Sending rewarder message: {'body': {}, 'headers': {'message_id': 16, 'parent_message_id': 10, 'parent_runtime': 14.45747995376587, 'sent_at': 1493877831.272119, 'episode_id': '2'}, 'method': 'v0.reply.env.reset'}\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[2017-05-04 01:03:51,274] [0:localhost:5900] Initial reset complete: episode_id=2\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:52,745] [INFO:gym_controlplane.registration] Loaded scorer: <gym_controlplane.reward.score.OCRScorerV0 object at 0x7f2ebc1d1eb8>\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:52,854] [INFO:gym_controlplane.registration] Created reward parser for internet.SlitherIO-v0: Reward<scorer=<gym_controlplane.reward.score.OCRScorerV0 object at 0x7f2ebc1d1eb8> vexpect=VExpect<{'ready0': <gym_controlplane.integration.transition.ClickTransition object at 0x7f2e887c16a0>, 'ready2': <gym_controlplane.integration.transition.ClickTransition object at 0x7f2e887c16d8>, 'ready1': <gym_controlplane.integration.transition.ClickTransition object at 0x7f2e887c1978>}>>\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:52,855] [INFO:root] Using metadata_encoding={'width': 100, 'height': 100, 'y': 658, 'x': 914, 'type': 'qrcode'} probe_key=96 subscription=[(79, 25, 482, 25), (47, 25, 481, 25), (96, 25, 480, 25), (62, 100, 730, 18), (914, 100, 658, 100)]\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:52,855] [INFO:universe.rewarder.remote] [Rewarder] Over past 18.75s, sent 0 reward messages to agent: reward=0 reward_min=(empty) reward_max=(empty) done=False info={}\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:52,855] [INFO:universe.rewarder.remote] [Rewarder] Ending previous episode: episode_reward=0 episode_count=2 episode_duration=28.77\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:52,856] [INFO:universe.wrappers.logger] Stats for the past 16.17s: vnc_updates_ps=0.0 n=1 reaction_time=None observation_lag=None action_lag=None reward_ps=0.0 reward_total=0.0 vnc_bytes_ps[total]=0.0 vnc_pixels_ps[total]=0.0 reward_lag=None rewarder_message_lag=None fps=0.56\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m 2017/05/04 06:03:52 I0504 06:03:52.857798 55 gymvnc.go:278] [0:127.0.0.1:5900] resuming updates\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:52,906] [INFO:universe.pyprofile] [pyprofile] period=18.81s timers={\"score.crop_cache.get.MatchImage\": {\"std\": \"0.50us\", \"mean\": \"3.02us\", \"calls\": 3}, \"reward.parsing.gameover\": {\"std\": \"0.00us\", \"mean\": \"255.35us\", \"calls\": 1}, \"vnc_env.VNCEnv.vnc_session.step\": {\"std\": \"90.46us\", \"mean\": \"166.83us\", \"calls\": 164}, \"reward.parsing.score\": {\"std\": \"0.00us\", \"mean\": \"49.01ms\", \"calls\": 1}, \"score.crop_cache.readthrough.MatchImage\": {\"std\": \"25.72us\", \"mean\": \"40.69us\", \"calls\": 3}, \"rewarder.frame\": {\"std\": \"989.35us\", \"mean\": \"18.09ms\", \"calls\": 164}, \"score.crop_cache.readthrough.OCRScorerV0\": {\"std\": \"0.00us\", \"mean\": \"48.84ms\", \"calls\": 1}, \"rewarder_protocol.latency.rtt.skew_unadjusted\": {\"std\": \"850.84us\", \"mean\": \"1.68ms\", \"calls\": 11}, \"rewarder.compute_reward\": {\"std\": \"3.89ms\", \"mean\": \"685.41us\", \"calls\": 164}, \"score.crop_cache.get.OCRScorerV0\": {\"std\": \"0.00us\", \"mean\": \"5.96us\", \"calls\": 1}, \"rewarder.sleep\": {\"std\": \"1.03ms\", \"mean\": \"14.74ms\", \"calls\": 164}} counters={\"agent_conn.reward\": {\"calls\": 1, \"mean\": 0.0, \"std\": 0}, \"control.env_id_change\": {\"calls\": 1, \"mean\": 1.0, \"std\": 0}, \"control.env_id_change.internet.SlitherIO-v0\": {\"calls\": 1, \"mean\": 1.0, \"std\": 0}, \"reward.vnc.updates.n\": {\"calls\": 164, \"mean\": 0.3719512195121951, \"std\": 4.763299737602485}, \"rewarder_protocol.messages\": {\"calls\": 11, \"mean\": 1.0, \"std\": 0.0}, \"rewarder_protocol.messages.v0.control.ping\": {\"calls\": 10, \"mean\": 1.0, \"std\": 0.0}, \"rewarder_protocol.messages.v0.env.reset\": {\"calls\": 1, \"mean\": 1.0, \"std\": 0}} gauges={} (export_time=102.76us)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:52,906] [INFO:root] [Rewarder] Rewarder fell behind by 1.6268916130065918s from target; losing 97 frames\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:52,917] [INFO:gym_controlplane.reward.reward] First score parsed: score=14. Our hardcoded initial_score=10\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:53,858] [INFO:universe.rewarder.remote] [Rewarder] Over past 1.00s, sent 3 reward messages to agent: reward=6.0 reward_min=0 reward_max=4 done=False info={'rewarder.vnc.updates.bytes': 1346, 'rewarder.vnc.updates.n': 1, 'rewarder.profile': '<2291 bytes>', 'rewarder.vnc.updates.pixels': 10000}\n"
]
}
],
"source": [
"env.action_space.sample()"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"[2017-05-04 01:03:56,268] Throttle fell behind by 33.82s; lost 2029.36 frames\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"[6.0] [False]\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:57,858] [INFO:universe.wrappers.logger] Stats for the past 5.00s: vnc_updates_ps=18.0 n=1 reaction_time=None observation_lag=None action_lag=None reward_ps=0.0 reward_total=0.0 vnc_bytes_ps[total]=275583.1 vnc_pixels_ps[total]=481065.2 reward_lag=None rewarder_message_lag=None fps=59.58\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:57,907] [INFO:universe.pyprofile] [pyprofile] period=5.00s timers={\"score.crop_cache.get.MatchImage\": {\"std\": \"14.46us\", \"mean\": \"31.89us\", \"calls\": 84}, \"reward.parsing.gameover\": {\"std\": \"81.40us\", \"mean\": \"197.57us\", \"calls\": 28}, \"vnc_env.VNCEnv.vnc_session.step\": {\"std\": \"52.84us\", \"mean\": \"108.84us\", \"calls\": 301}, \"reward.parsing.score\": {\"std\": \"2.80ms\", \"mean\": \"1.03ms\", \"calls\": 28}, \"rewarder.frame\": {\"std\": \"93.77ms\", \"mean\": \"22.55ms\", \"calls\": 301}, \"rewarder.sleep.missed\": {\"std\": \"0.00us\", \"mean\": \"1.63s\", \"calls\": 1}, \"score.crop_cache.readthrough.MatchImage\": {\"std\": \"24.33us\", \"mean\": \"38.70us\", \"calls\": 3}, \"score.crop_cache.readthrough.OCRScorerV0\": {\"std\": \"81.93us\", \"mean\": \"9.50ms\", \"calls\": 2}, \"rewarder.compute_reward\": {\"std\": \"940.87us\", \"mean\": \"364.28us\", \"calls\": 301}, \"score.crop_cache.get.OCRScorerV0\": {\"std\": \"33.67us\", \"mean\": \"88.71us\", \"calls\": 28}, \"rewarder.sleep\": {\"std\": \"993.50us\", \"mean\": \"15.75ms\", \"calls\": 300}} counters={\"agent_conn.reward\": {\"calls\": 3, \"mean\": 2.0, \"std\": 2.0}, \"reward.vnc.updates.n\": {\"calls\": 301, \"mean\": 0.09634551495016609, \"std\": 0.3066267006554015}, \"score.crop_cache.hit.MatchImage\": {\"calls\": 81, \"mean\": 1.0, \"std\": 0.0}, \"score.crop_cache.hit.OCRScorerV0\": {\"calls\": 26, \"mean\": 1.0, \"std\": 0.0}} gauges={\"reward_parser.score.last_score\": {\"calls\": 28, \"mean\": 15.357142857142854, \"value\": 16.0, \"std\": 1.3392724867071661}} (export_time=166.18us)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:03:57,908] [INFO:universe.rewarder.remote] [Rewarder] Over past 4.05s, sent 1 reward messages to agent: reward=0 reward_min=0 reward_max=0 done=False info={'rewarder.vnc.updates.bytes': 0, 'rewarder.vnc.updates.pixels': 0, 'rewarder.vnc.updates.n': 0, 'rewarder.profile': '<2137 bytes>'}\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:02,874] [INFO:universe.wrappers.logger] Stats for the past 5.02s: vnc_updates_ps=4.6 n=1 reaction_time=None observation_lag=None action_lag=None reward_ps=0.0 reward_total=0.0 vnc_bytes_ps[total]=6092.5 vnc_pixels_ps[total]=45078.5 reward_lag=None rewarder_message_lag=None fps=60.02\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:02,925] [INFO:universe.pyprofile] [pyprofile] period=5.02s timers={\"score.crop_cache.get.MatchImage\": {\"std\": \"13.05us\", \"mean\": \"29.72us\", \"calls\": 69}, \"reward.parsing.gameover\": {\"std\": \"31.76us\", \"mean\": \"172.76us\", \"calls\": 23}, \"reward.parsing.score\": {\"std\": \"49.61us\", \"mean\": \"268.42us\", \"calls\": 23}, \"rewarder.frame\": {\"std\": \"325.21us\", \"mean\": \"17.14ms\", \"calls\": 301}, \"vnc_env.VNCEnv.vnc_session.step\": {\"std\": \"70.75us\", \"mean\": \"111.72us\", \"calls\": 301}, \"rewarder.compute_reward\": {\"std\": \"160.77us\", \"mean\": \"288.34us\", \"calls\": 301}, \"score.crop_cache.get.OCRScorerV0\": {\"std\": \"19.83us\", \"mean\": \"83.46us\", \"calls\": 23}, \"rewarder.sleep\": {\"std\": \"378.90us\", \"mean\": \"15.83ms\", \"calls\": 301}} counters={\"agent_conn.reward\": {\"calls\": 1, \"mean\": 0.0, \"std\": 0}, \"reward.vnc.updates.n\": {\"calls\": 301, \"mean\": 0.0764119601328904, \"std\": 0.2660985088079445}, \"score.crop_cache.hit.MatchImage\": {\"calls\": 69, \"mean\": 1.0, \"std\": 0.0}, \"score.crop_cache.hit.OCRScorerV0\": {\"calls\": 23, \"mean\": 1.0, \"std\": 0.0}} gauges={\"reward_parser.score.last_score\": {\"calls\": 23, \"mean\": 16.0, \"value\": 16.0, \"std\": 0.0}} (export_time=150.92us)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:02,925] [INFO:universe.rewarder.remote] [Rewarder] Over past 5.02s, sent 1 reward messages to agent: reward=0 reward_min=0 reward_max=0 done=False info={'rewarder.vnc.updates.bytes': 0, 'rewarder.vnc.updates.pixels': 0, 'rewarder.vnc.updates.n': 0, 'rewarder.profile': '<1721 bytes>'}\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:04,991] [INFO:universe.utils] [gameover] Gameover screen detected: distance_n=[0.0, 0.0, 0.0] match_time=293us\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:04,991] [INFO:gym_controlplane.reward.reward] RESET CAUSE: gameover state reached\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:04,992] [INFO:universe.rewarder.remote] [Rewarder] Over past 2.07s, sent 1 reward messages to agent: reward=0.0 reward_min=0.0 reward_max=0.0 done=True info={'rewarder.vnc.updates.bytes': 4865, 'rewarder.vnc.updates.pixels': 1691, 'rewarder.vnc.updates.n': 1}\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:04,992] [INFO:root] [Rewarder] Resetting environment since done=True\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:04,992] [INFO:root] [Rewarder] Triggering a reset on EnvController\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:04,993] [INFO:root] [EnvStatus] Changing env_state: running (env_id=internet.SlitherIO-v0) -> resetting (env_id=internet.SlitherIO-v0) (episode_id: 2->3, fps=60)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:04,993] [INFO:root] [Rewarder] Blocking until env finishes resetting\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:04,993] [INFO:root] [EnvController] controlplane.py is resetting the environment\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:04,994] [INFO:root] [EnvController] Env state: env_id=internet.SlitherIO-v0 episode_id=3\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [Thu May 4 06:04:05 UTC 2017] [/usr/local/bin/sudoable-env-setup] Allowing outbound network traffic to non-private IPs for git-lfs. (Going to fetch files via git lfs.)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [Thu May 4 06:04:05 UTC 2017] [/usr/local/bin/sudoable-env-setup] Completion file /usr/local/openai/git-lfs/internet.SlitherIO-v0 exists; not git-lfs pulling\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [Thu May 4 06:04:05 UTC 2017] [/usr/local/bin/sudoable-env-setup] Allowing outbound network traffic to non-private IPs for internet.SlitherIO-v0\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:05,079] [INFO:gym_flashgames.launcher] [EnvController] Launching new Chrome process (attempt 0/10)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:05,079] [INFO:root] Replacing selenium_wrapper_server since we currently do it at every episode boundary\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:05,079] [ERROR:root] Closing server (via subprocess.close()) and all chromes (via pkill chromedriver || :; pkill chrome || :)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [init] [2017-05-04 06:04:05,094] init detected end of child process 467 with exit code 0, killed by SIGTERM: 15\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [init] [2017-05-04 06:04:05,104] init detected end of child process 688 with exit code 0, not killed by signal\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [init] [2017-05-04 06:04:05,105] init detected end of child process 482 with exit code 0, not killed by signal\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [init] [2017-05-04 06:04:05,110] init detected end of child process 757 with exit code 0, killed by SIGTERM: 15\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [init] [2017-05-04 06:04:05,150] init detected end of child process 703 with exit code 0, killed by SIGTERM: 15\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [nginx] 2017/05/04 06:04:05 [info] 65#65: *21 client 127.0.0.1 closed keepalive connection\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [nginx] 2017/05/04 06:04:05 [info] 65#65: *20 client 127.0.0.1 closed keepalive connection\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [init] [2017-05-04 06:04:05,238] init detected end of child process 470 with exit code 0, not killed by signal\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [init] [2017-05-04 06:04:05,239] init detected end of child process 478 with exit code 0, not killed by signal\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [init] [2017-05-04 06:04:05,239] init detected end of child process 479 with exit code 0, not killed by signal\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [init] [2017-05-04 06:04:05,239] init detected end of child process 481 with exit code 0, killed by SIGTERM: 15\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:05,246] [selenium_wrapper_server] Calling webdriver.Chrome()\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:06,565] [selenium_wrapper_server] Call to webdriver.Chrome() completed: 1.32s\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:06,566] [INFO:gym_flashgames.launcher] [EnvController] Navigating browser to url=http://localhost/internet.SlitherIO-v0\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:09,747] [INFO:root] [EnvController] Running command: /app/universe-envs/controlplane/bin/play_vexpect -e internet.SlitherIO-v0 -r vnc://127.0.0.1:5900 -d\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:10,337] [play_vexpect] No rewarder addresses were provided, so this env cannot connect to the remote's rewarder channel, and cannot send control messages (e.g. reset)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:10,337] [play_vexpect] Using the golang VNC implementation\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:10,337] [play_vexpect] Using VNCSession arguments: {'subsample_level': 2, 'fine_quality_level': 50, 'compress_level': 0, 'start_timeout': 7, 'encoding': 'zrle'}. (Customize by running \"env.configure(vnc_kwargs={...})\"\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:10,338] [play_vexpect] Printed stats will ignore clock skew. (This usually makes sense only when the environment and agent are on the same machine.)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:10,341] [play_vexpect] [0] Connecting to environment: vnc://127.0.0.1:5900 password=openai. If desired, you can manually connect a VNC viewer, such as TurboVNC. Most environments provide a convenient in-browser VNC client: http://None/viewer/?password=openai\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:10,341] [play_vexpect] [0] Connecting to environment details: vnc_address=127.0.0.1:5900 vnc_password=openai rewarder_address=None rewarder_password=openai\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m 2017/05/04 06:04:10 I0504 06:04:10.342057 1098 gymvnc.go:417] [0:127.0.0.1:5900] opening connection to VNC server\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] \n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Thu May 4 06:04:10 2017\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Connections: accepted: 127.0.0.1::35940\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] SConnection: Client needs protocol version 3.8\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] SConnection: Client requests security type VncAuth(2)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] VNCSConnST: Server default pixel format depth 24 (32bpp) little-endian rgb888\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] VNCSConnST: Client pixel format depth 24 (32bpp) little-endian bgr888\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m 2017/05/04 06:04:10 I0504 06:04:10.344597 1098 gymvnc.go:550] [0:127.0.0.1:5900] connection established\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:10,464] [play_vexpect] Waiting for any of [ready0, ready1, ready2] to activate\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:12,665] [play_vexpect] Applying transition: ClickTransition<ready1->[] x=0 y=0 buttonmask=0> for active state ready1. (Summary: plausible_states=[ready0, ready1, ready2] distance_m=[0.0049178139, 0.0, 0.0] match_time_m=['187us', '73us', '68us'])\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:12,666] [play_vexpect] Reaching start state: ready1\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:12,666] [play_vexpect] vexpect macro complete in 2.289935s\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] \n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Thu May 4 06:04:12 2017\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] Connections: closed: 127.0.0.1::35940 (Clean disconnection)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] EncodeManager: Framebuffer updates: 7\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] EncodeManager: ZRLE:\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] EncodeManager: Solid: 5 rects, 508 pixels\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] EncodeManager: 150 B (1:13.9467 ratio)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] EncodeManager: Indexed RLE: 7 rects, 68.018 kpixels\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] EncodeManager: 17.9219 KiB (1:14.8298 ratio)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] EncodeManager: Full Colour: 17 rects, 868.352 kpixels\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] EncodeManager: 2.48537 MiB (1:1.33288 ratio)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] EncodeManager: Total: 29 rects, 936.878 kpixels\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [tigervnc] EncodeManager: 2.50302 MiB (1:1.42797 ratio)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:13,001] [INFO:root] [EnvStatus] Changing env_state: resetting (env_id=internet.SlitherIO-v0) -> running (env_id=internet.SlitherIO-v0) (episode_id: 3->3, fps=60)\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:13,001] [INFO:root] [Rewarder] Unblocking since env reset finished\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:13,002] [INFO:root] [Rewarder] Clearing reward_parser state: env_id=internet.SlitherIO-v0 episode_id=2->3, env_state=running\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:13,003] [INFO:universe.rewarder.remote] [Rewarder] Over past 8.01s, sent 0 reward messages to agent: reward=0 reward_min=(empty) reward_max=(empty) done=False info={}\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:13,003] [INFO:universe.rewarder.remote] [Rewarder] Ending previous episode: episode_reward=6.0 episode_count=6 episode_duration=20.15\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:13,005] [INFO:universe.wrappers.logger] Stats for the past 10.13s: vnc_updates_ps=1.0 n=1 reaction_time=None observation_lag=None action_lag=None reward_ps=0.0 reward_total=0.0 vnc_bytes_ps[total]=1666.9 vnc_pixels_ps[total]=8964.3 reward_lag=None rewarder_message_lag=None fps=12.64\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:13,022] [INFO:gym_controlplane.reward.reward] First score parsed: score=10. Our hardcoded initial_score=10\n",
"\u001b[36muniverse-196mHq-0 |\u001b[0m [2017-05-04 06:04:13,022] [INFO:universe.pyprofile] [pyprofile] period=10.10s timers={\"score.crop_cache.get.MatchImage\": {\"std\": \"13.98us\", \"mean\": \"26.73us\", \"calls\": 33}, \"reward.parsing.gameover\": {\"std\": \"111.29us\", \"mean\": \"214.10us\", \"calls\": 11}, \"vnc_env.VNCEnv.vnc_session.step\": {\"std\": \"61.83us\", \"mean\": \"105.42us\", \"calls\": 125}, \"reward.parsing.score\": {\"std\": \"4.88ms\", \"mean\": \"1.73ms\", \"calls\": 11}, \"rewarder.frame\": {\"std\": \"1.27ms\", \"mean\": \"17.00ms\", \"calls\": 125}, \"score.crop_cache.readthrough.OCRScorerV0\": {\"std\": \"0.00us\", \"mean\": \"16.25ms\", \"calls\": 1}, \"score.crop_cache.readthrough.MatchImage\": {\"std\": \"21.96us\", \"mean\": \"44.98us\", \"calls\": 6}, \"rewarder.compute_reward\": {\"std\": \"1.62ms\", \"mean\": \"432.25us\", \"calls\": 125}, \"score.crop_cache.get.OCRScorerV0\": {\"std\": \"21.52us\", \"mean\": \"70.12us\", \"calls\": 11}, \"rewarder.sleep\": {\"std\": \"399.46us\", \"mean\": \"15.85ms\", \"calls\": 124}} counters={\"agent_conn.done\": {\"calls\": 1, \"mean\": 1.0, \"std\": 0}, \"agent_conn.reward\": {\"calls\": 2, \"mean\": 0.0, \"std\": 0.0}, \"reward.vnc.updates.n\": {\"calls\": 125, \"mean\": 0.296, \"std\": 2.4230878726760356}, \"score.crop_cache.hit.MatchImage\": {\"calls\": 27, \"mean\": 1.0, \"std\": 0.0}, \"score.crop_cache.hit.OCRScorerV0\": {\"calls\": 10, \"mean\": 1.0, \"std\": 0.0}} gauges={\"reward_parser.score.last_score\": {\"calls\": 11, \"mean\": 15.454545454545455, \"value\": 10, \"std\": 1.809068067466582}} (export_time=275.61us)\n"
]
}
],
"source": [
"a = vnc_a.vnc_event.PointerEvent(x=192, y=707, buttonmask=103)\n",
"s1,r,d,_ = env.step([[a]])\n",
"print(r, d)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
}
],
"metadata": {
"anaconda-cloud": {},
"kernelspec": {
"display_name": "Python [conda env:universe_p3]",
"language": "python",
"name": "conda-env-universe_p3-py"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.2"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment