Skip to content

Instantly share code, notes, and snippets.

@justheuristic

justheuristic/deephack-rl.ipynb

Created January 18, 2017 13:47
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save justheuristic/b02378cdc0e2bbf0467a91d5b1c11441 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save justheuristic/b02378cdc0e2bbf0467a91d5b1c11441 to your computer and use it in GitHub Desktop.
Download ZIP
Display the source blob
Display the rendered blob
Raw
deephack-rl.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"### This notebook plays atari skiing for the Deephack-RL contest.\n",
"\n",
"All the source code is [here](https://github.com/yandexdataschool/tinyverse/tree/deephack-rl)\n",
"\n",
"Unlike many other solutions, we only use one hack and i'm going to explain it shortly.\n",
"Otherwises it trains a simple feedforward A3C algorithm with tinyverse.\n",
"Below you will find a set of bash scripts that runs the algorithm, so it may be a good idea to first run in and than read the writeup.\n",
"\n",
"#### Reward trick\n",
"The original DQN wasn't very good at this game, and here's the reason: the reward function is very hard. \n",
"\n",
"OpenAI Gym has a very convenient interface for wrapping rewards, so that's what we did:\n",
"```\n",
"from gym.core import Wrapper\n",
"class RewardForPassingGates(Wrapper):\n",
" \"\"\"Skiing wrapper that rewards player +1 for passing gates only.\"\"\"\n",
" def _reset(self):\n",
" \"\"\"On game reset, remember the hash of initial score\"\"\"\n",
" s = self.env.reset()\n",
" self.prev_score_hash = hash(s[31:38,67:81].tobytes()) #hash of the image chunk with scoreboard\n",
" return s\n",
" def _step(self,action):\n",
" \"\"\"on each step, if score has changed, give +1 reward, else +0\"\"\"\n",
" s,_,done,info = self.env.step(action)\n",
" new_score_hash = hash(s[31:38,67:81].tobytes()) #hash of the same image chunk\n",
" \n",
" #reward = +1 if we have just crossed the gate, else 0\n",
" r = int(new_score_hash != self.prev_score_hash)\n",
" \n",
" #remember new score\n",
" self.prev_score_hash = new_score_hash\n",
" return s,r,done,info\n",
"```\n",
"\n",
"We can then use the environment thusly:\n",
"```\n",
"env = gym.make(\"Skiing-v0\")\n",
"env = RewardForPassingGates(env)\n",
"```\n",
"\n",
"Notably, so far as we know, the optimal policy under this reward is to get through all the gates as quickly as possible, which also gives the optimum to the original reward [we believe].\n",
"\n",
"\n",
"### Why we did that? What else could we do?\n",
"\n",
"The original reward function:\n",
"* -1~-10 negative reward on every tick, \n",
"* no immediate reward for passing gates\n",
"* some weird ~-10^4 reward at the very end, summarizing the whole performance.\n",
"\n",
"This is bad for two reasons: first, the rewards can be as large as -30k, which will blow up most gradient descent algorithms, should you use deep reinforcement learning.\n",
"\n",
"Second and main reason is that this reward function is as _sparse_ and as _delayed_ as it can be.\n",
"The problem is effectively almost from the Monte-Carlo domain (read [here])(http://people.inf.elte.hu/lorincz/Files/RL_2006/SuttonBook.pdf) because you can only evaluate the policy once you've played a full game.\n",
"\n",
"Temporal difference methods (Q-learning, SARSA, actor-critic) are extremely poorly suited for that kind of problem. The objective function for most of them tends to follow this pattern:\n",
"\n",
"$$ Q(s_t,a_t) = r_t + \\gamma \\cdot r_{t+1} + \\gamma^2 \\cdot r_{t+2} + ... $$\n",
"\n",
"Where $Q(s_t,a_t)$ is action value and $r_t$ is MDP reward at timestep t.\n",
"\n",
"\n",
"\n",
"For $ \\gamma = 0.99 $ , the value of _passing the first gate_ essentially becomes\n",
"\n",
"\n",
"$$ Q(s,a) = 0 + 0.99 \\cdot 0 + 0.99^2 \\cdot 0 + ... + 0.9 ^ 5000 \\cdot reward_at_the_end $$\n",
"\n",
"Assuming the progressed for 5000 ticks which is quite realistic. Note, however that __0.99^5k ~ 10^-22__.\n",
"\n",
"Conversely, _agent does not even assume that his actions influenced the final score_. No wonder DQN can't solve the game.\n",
"\n",
"As a teaser, we're working on a way that would allow agent to infer the action values in this kinds of environments, but it's far from ready yet and it may be [sadly] unscalable to practical environments.\n",
"\n",
"Now when you substitute the rewards with +1 for passing the gates only, the reward now behaves just like the reward for [breakout](https://gym.openai.com/envs/Breakout-v0). Agent receives +1 rewards every 20-50 frames. This kind of problem can be easily solved with any deep RL algorithm. _Our next challenge is to make it work as fast as we can_.\n",
"\n",
"### Algorithm\n",
"\n",
"Inspired by [this](http://rll.berkeley.edu/deeprlcourse/docs/nuts-and-bolts.pdf), we decided to start with something very simple for our problem.\n",
"\n",
"We decided to use [policy gradient methods](https://www.youtube.com/watch?v=KHZVXao4qXs) since they are known to have faster convergence properties. We started with Advantage Actor Critic (also explained in the video).\n",
"\n",
"The main idea of an algorithm is that you learn both policy $\\pi(a|s)$ (action probabilities) and state value function $V(s)$. In practice, these are the two outputs of the neural network:\n",
"<img src=\"https://s27.postimg.org/ge52gzx0j/a2c_deep.png\" width=800>\n",
"\n",
"The value function __V(s)__ is trained with a standard temporal difference loss (to minimize):\n",
"$$ (r(s,a) +\\gamma V(s') - V(s))^2 $$\n",
"\n",
"While the policy is optimized with an equivalent of policy gradient (to maximize):\n",
"$$ log \\pi(a|s) \\cdot(r(s,a) +\\gamma \\cdot V(s') - V(s)) $$\n",
"\n",
"While the last equation was derived through some pure math (logderivative trick followed by variance theorem), you can explain it with a very simple intuition:\n",
"* Critic __V(s)__ tries to estimate the state value, as in value iteration.\n",
"* Actor __pi(a|s)__ tries to maximize probability of actions that __exceed expectations__ of critic.\n",
"\n",
"$ (r(s,a) +\\gamma \\cdot V(s') - V(s)) $ can be viewed as (what_you_actually_got - what_you_should_have_got).\n",
"\n",
"Thus, if critic thought V(s) = 1, and actor picked an an action that yielded r = 2 and V(s') = 3, than actor exceeded expectations $r + \\gamma \\cdot V(s') - V(s) > 0$ and the probability of taking such action should increase.\n",
"\n",
"Instead of epsilon-greedy exploration, we regularized the actor with entropy, making it prefer to leave some randomness into it's actions, effectively exploring the environment. Idea mentioned [here](https://arxiv.org/abs/1602.01783).\n",
"\n",
"This algorithm would fully converge in 18~30 hours on other simple atari games like pong/breakout. To improve convergence, we used [eligibility traces](https://webdocs.cs.ualberta.ca/~sutton/book/ebook/node72.html) (also covered in the policy gradient video).\n",
"\n",
"The difference is essentially that if previously we used a one-step temporal difference\n",
"$$ (r(s,a) +\\gamma V(s') - V(s))^2 $$\n",
"\n",
"A 3-step eligibility trace would be:\n",
"$$ (r(s,a) +\\gamma r(s',a') + \\gamma^2 r(s'',a'') + \\gamma^3 V(s''') - V(s))^2 $$\n",
"\n",
"For stability reasons, we used a 25-step eligibility traces :) [More info](https://arxiv.org/abs/1506.02438) on them for actor-critic.\n",
"\n",
"Since the algorithm is on-policy, we only used a very small experience replay buffer. \n",
"The problem was that if replay buffer is large, than your agent will train on sessions that it recorded whence it was still stupid and an on-policy algorithm will learn that stupid policy. That's ... stupid.\n",
"\n",
"We only stored 5000 last sub-sessions(each is just 25 ticks) which is roughly equivalent to 20 games. Since we recorded sessions with 10 parallel [tinyverse](https://github.com/yandexdataschool/tinyverse) players, this buffer was completely renewed over 20-30 seconds depending on machine.\n",
"\n",
"\n",
"### Network architecture\n",
"\n",
"Like Pong or Breakout, Skiing with our new reward function can be viewed as a fully-observable MDP (roughly) as long as you know current __position__, __velocity and acceleration__ of all the objects. While we don't actually observe all the variables in the game state (~RAM), we know enough to build an optimal strategy, thus a full observability assumption is harmless.\n",
"\n",
"However, we can't see the object velocity from just one frame (or well, we could derive it from object shapes, but that's hard). We could also use recurrent neural networks as agent memory to let it remember all the past observations, but we decided to go wit ha simpler solution.\n",
"\n",
"Just like the [original DQN](https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf), we use a 4-frame buffer that stores last 4 images agent has seen. From these you can easily extract object motion and acceleration.\n",
"\n",
"We resize game images to (64,64), store tham in a frame buffer and feed the resulting (4,64,64) tensors to a simple neural network:\n",
"\n",
"```\n",
"conv0 = Conv2DLayer(<4-frame window>,32,5,stride=2,nonlinearity=elu)\n",
"conv1 = Conv2DLayer(conv0,32,5,stride=2,nonlinearity=elu)\n",
"conv2 = Conv2DLayer(conv1,64,5,stride=1,nonlinearity=elu)\n",
"dense = DenseLayer(dropout(conv2,0.1),512,nonlinearity=tanh)\n",
"\n",
"#actor head\n",
"policy_layer = DenseLayer(dense,n_actions,nonlinearity=T.nnet.softmax) \n",
"\n",
"#critic head\n",
"V_layer = DenseLayer(dense,1,nonlinearity=None)\n",
"\n",
"```\n",
"Some notes on the network architecture\n",
"* We use strides instead of max-pooling for computational simplicity, just like the original DQN\n",
"* Since we use smaller image size (than dqn), we can afford smaller convolutional layers.\n",
"* We used elu instead of relu because it has a betten mean and less of \"dying ReLU\" problem\n",
"* __We forgot to remove dropout from the policy__, probably at our peril. It could be better without it. On the other hand, dropout facilitates exploration... and adds noize to the policy radient :(\n",
"\n",
"We implemented the network and actor-critic with [Lasagne](https://github.com/Lasagne/Lasagne/) + [AgentNet](https://github.com/yandexdataschool/agentnet) and ran experiments with early version of [tinyverse](https://github.com/yandexdataschool/tinyverse) to speed up training.\n",
"\n",
"#### Improvement ideas\n",
"- dropout :DDD\n",
"- agent still sometimes misses the ultimate gate, giving him -30k reward. It may be a good idea to add more reward for the very last gate AND make sure agent can distinguish red gate from others\n",
"- try different framebuffer size or recurrent memory\n",
"- change image preprocessing so that it's clearer\n",
"- add intrinsic motivation-based exploration\n",
"- incorporate the original rewards (whatever they mean) with the new ones\n",
"- try [optimality tightening](https://arxiv.org/abs/1611.01606)\n",
"- train for 8 more hours >.<\n",
"\n",
"#### boasting\n",
"- Total coding time: 2h 11m (with tinyverse and lasagne/agentnet)\n",
"- Final model training time: ~8h (with GPU gf1080)\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Run stuff"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Starting job # 0 in a separate thread.\n"
]
}
],
"source": [
"%%bash --bg\n",
"#create several player processes. Each process plays games and saves results\n",
"#the loop below spawns 10 players. If you are doing this on a laptop, reducing to 2-4 is okay\n",
"for i in `seq 1 10`; \n",
"do\n",
" THEANO_FLAGS=device=cpu,floatX=float32 python tinyverse atari.py play -b 3 --port 6789 &\n",
"done "
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Starting job # 2 in a separate thread.\n"
]
}
],
"source": [
"%%bash --bg\n",
"#create learner process on GPU. batch size 10\n",
"THEANO_FLAGS=device=gpu4,floatX=float32 python tinyverse atari.py train -b 10 --port 6789 &"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### View stuff"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": false,
"scrolled": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Episode finished after 781 timesteps with reward=20\n",
"Episode finished after 731 timesteps with reward=20\n",
"Episode finished after 737 timesteps with reward=20\n",
"Episode finished after 758 timesteps with reward=20\n",
"Episode finished after 811 timesteps with reward=20\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[2017-01-18 08:11:03,138] Making new env: Skiing-v0\n",
"[2017-01-18 08:11:03,171] Clearing 12 monitor files from previous run (because force=True was provided)\n",
"[2017-01-18 08:11:03,188] Starting new video recorder writing to /home/hedgedir/tinyversers/skiverse/records/openaigym.video.0.20910.video000000.mp4\n",
"[2017-01-18 08:11:11,969] Starting new video recorder writing to /home/hedgedir/tinyversers/skiverse/records/openaigym.video.0.20910.video000001.mp4\n",
"[2017-01-18 08:11:41,854] Finished writing results. You can upload them to the scoreboard via gym.upload('/home/hedgedir/tinyversers/skiverse/records')\n"
]
}
],
"source": [
"%%bash\n",
"#play 5 games, show results (re-run whenever you want score)\n",
"#Tris particular score was recorded 7 hours through training\n",
"python tinyverse atari.py eval -n 5 --port 6789"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Upload stuff"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"[2017-01-18 08:23:38,440] Making new env: Skiing-v0\n",
"[2017-01-18 08:23:38,489] Clearing 6 monitor files from previous run (because force=True was provided)\n",
"[2017-01-18 08:23:38,516] Starting new video recorder writing to /home/hedgedir/tinyversers/skiverse/records/openaigym.video.0.2488.video000000.mp4\n",
"[2017-01-18 08:23:47,232] Starting new video recorder writing to /home/hedgedir/tinyversers/skiverse/records/openaigym.video.0.2488.video000001.mp4\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Episode finished after 735 timesteps with reward=-4168.0\n",
"Episode finished after 847 timesteps with reward=-4183.0\n",
"Episode finished after 729 timesteps with reward=-3646.0\n",
"Episode finished after 749 timesteps with reward=-4181.0\n",
"Episode finished after 5991 timesteps with reward=-30000.0\n",
"Episode finished after 730 timesteps with reward=-3629.0\n",
"Episode finished after 781 timesteps with reward=-3854.0\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[2017-01-18 08:25:25,809] Starting new video recorder writing to /home/hedgedir/tinyversers/skiverse/records/openaigym.video.0.2488.video000008.mp4\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Episode finished after 786 timesteps with reward=-3921.0\n",
"Episode finished after 749 timesteps with reward=-3716.0\n",
"Episode finished after 848 timesteps with reward=-4237.0\n",
"Episode finished after 781 timesteps with reward=-3906.0\n",
"Episode finished after 743 timesteps with reward=-3623.0\n",
"Episode finished after 771 timesteps with reward=-3834.0\n",
"Episode finished after 800 timesteps with reward=-4022.0\n",
"Episode finished after 751 timesteps with reward=-3746.0\n",
"Episode finished after 774 timesteps with reward=-3867.0\n",
"Episode finished after 790 timesteps with reward=-4441.0\n",
"Episode finished after 844 timesteps with reward=-4188.0\n",
"Episode finished after 794 timesteps with reward=-3979.0\n",
"Episode finished after 786 timesteps with reward=-3901.0\n",
"Episode finished after 731 timesteps with reward=-3644.0\n",
"Episode finished after 738 timesteps with reward=-3641.0\n",
"Episode finished after 743 timesteps with reward=-3659.0\n",
"Episode finished after 763 timesteps with reward=-3738.0\n",
"Episode finished after 737 timesteps with reward=-3691.0\n",
"Episode finished after 750 timesteps with reward=-3766.0\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[2017-01-18 08:27:47,617] Starting new video recorder writing to /home/hedgedir/tinyversers/skiverse/records/openaigym.video.0.2488.video000027.mp4\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Episode finished after 856 timesteps with reward=-4280.0\n",
"Episode finished after 864 timesteps with reward=-4252.0\n",
"Episode finished after 6015 timesteps with reward=-30000.0\n",
"Episode finished after 737 timesteps with reward=-3734.0\n",
"Episode finished after 783 timesteps with reward=-3902.0\n",
"Episode finished after 859 timesteps with reward=-4303.0\n",
"Episode finished after 739 timesteps with reward=-3674.0\n",
"Episode finished after 870 timesteps with reward=-4823.0\n",
"Episode finished after 714 timesteps with reward=-3693.0\n",
"Episode finished after 753 timesteps with reward=-3779.0\n",
"Episode finished after 774 timesteps with reward=-3931.0\n",
"Episode finished after 877 timesteps with reward=-4386.0\n",
"Episode finished after 797 timesteps with reward=-4022.0\n",
"Episode finished after 753 timesteps with reward=-3706.0\n",
"Episode finished after 741 timesteps with reward=-4183.0\n",
"Episode finished after 803 timesteps with reward=-4040.0\n",
"Episode finished after 752 timesteps with reward=-3718.0\n",
"Episode finished after 737 timesteps with reward=-3686.0\n",
"Episode finished after 765 timesteps with reward=-3876.0\n",
"Episode finished after 766 timesteps with reward=-3841.0\n",
"Episode finished after 762 timesteps with reward=-3846.0\n",
"Episode finished after 741 timesteps with reward=-3684.0\n",
"Episode finished after 787 timesteps with reward=-3911.0\n",
"Episode finished after 804 timesteps with reward=-4037.0\n",
"Episode finished after 795 timesteps with reward=-3927.0\n",
"Episode finished after 2897 timesteps with reward=-14964.0\n",
"Episode finished after 856 timesteps with reward=-4278.0\n",
"Episode finished after 739 timesteps with reward=-3716.0\n",
"Episode finished after 6015 timesteps with reward=-30000.0\n",
"Episode finished after 768 timesteps with reward=-3874.0\n",
"Episode finished after 752 timesteps with reward=-3741.0\n",
"Episode finished after 786 timesteps with reward=-3959.0\n",
"Episode finished after 780 timesteps with reward=-3889.0\n",
"Episode finished after 812 timesteps with reward=-3954.0\n",
"Episode finished after 746 timesteps with reward=-3723.0\n",
"Episode finished after 812 timesteps with reward=-3936.0\n",
"Episode finished after 740 timesteps with reward=-3674.0\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[2017-01-18 08:34:16,928] Starting new video recorder writing to /home/hedgedir/tinyversers/skiverse/records/openaigym.video.0.2488.video000064.mp4\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Episode finished after 786 timesteps with reward=-3869.0\n",
"Episode finished after 792 timesteps with reward=-4014.0\n",
"Episode finished after 790 timesteps with reward=-3904.0\n",
"Episode finished after 855 timesteps with reward=-4303.0\n",
"Episode finished after 726 timesteps with reward=-3618.0\n",
"Episode finished after 764 timesteps with reward=-3857.0\n",
"Episode finished after 861 timesteps with reward=-4280.0\n",
"Episode finished after 743 timesteps with reward=-3726.0\n",
"Episode finished after 854 timesteps with reward=-4280.0\n",
"Episode finished after 735 timesteps with reward=-3663.0\n",
"Episode finished after 745 timesteps with reward=-3684.0\n",
"Episode finished after 729 timesteps with reward=-3648.0\n",
"Episode finished after 788 timesteps with reward=-3864.0\n",
"Episode finished after 770 timesteps with reward=-3829.0\n",
"Episode finished after 738 timesteps with reward=-3706.0\n",
"Episode finished after 756 timesteps with reward=-3731.0\n",
"Episode finished after 803 timesteps with reward=-3927.0\n",
"Episode finished after 751 timesteps with reward=-3734.0\n",
"Episode finished after 5983 timesteps with reward=-30000.0\n",
"Episode finished after 754 timesteps with reward=-4297.0\n",
"Episode finished after 803 timesteps with reward=-3979.0\n",
"Episode finished after 742 timesteps with reward=-3723.0\n",
"Episode finished after 768 timesteps with reward=-3801.0\n",
"Episode finished after 774 timesteps with reward=-3871.0\n",
"Episode finished after 814 timesteps with reward=-3995.0\n",
"Episode finished after 786 timesteps with reward=-3914.0\n",
"Episode finished after 769 timesteps with reward=-3851.0\n",
"Episode finished after 743 timesteps with reward=-4186.0\n",
"Episode finished after 743 timesteps with reward=-3713.0\n",
"Episode finished after 786 timesteps with reward=-3901.0\n",
"Episode finished after 746 timesteps with reward=-3663.0\n",
"Episode finished after 861 timesteps with reward=-4215.0\n",
"Episode finished after 742 timesteps with reward=-3653.0\n",
"Episode finished after 768 timesteps with reward=-3774.0\n",
"Episode finished after 780 timesteps with reward=-3889.0\n",
"Episode finished after 851 timesteps with reward=-4275.0\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[2017-01-18 08:39:30,417] Finished writing results. You can upload them to the scoreboard via gym.upload('/home/hedgedir/tinyversers/skiverse/records')\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Episode finished after 745 timesteps with reward=-4149.0\n"
]
},
{
"data": {
"text/plain": [
"[-4168.0,\n",
" -4183.0,\n",
" -3646.0,\n",
" -4181.0,\n",
" -30000.0,\n",
" -3629.0,\n",
" -3854.0,\n",
" -3921.0,\n",
" -3716.0,\n",
" -4237.0,\n",
" -3906.0,\n",
" -3623.0,\n",
" -3834.0,\n",
" -4022.0,\n",
" -3746.0,\n",
" -3867.0,\n",
" -4441.0,\n",
" -4188.0,\n",
" -3979.0,\n",
" -3901.0,\n",
" -3644.0,\n",
" -3641.0,\n",
" -3659.0,\n",
" -3738.0,\n",
" -3691.0,\n",
" -3766.0,\n",
" -4280.0,\n",
" -4252.0,\n",
" -30000.0,\n",
" -3734.0,\n",
" -3902.0,\n",
" -4303.0,\n",
" -3674.0,\n",
" -4823.0,\n",
" -3693.0,\n",
" -3779.0,\n",
" -3931.0,\n",
" -4386.0,\n",
" -4022.0,\n",
" -3706.0,\n",
" -4183.0,\n",
" -4040.0,\n",
" -3718.0,\n",
" -3686.0,\n",
" -3876.0,\n",
" -3841.0,\n",
" -3846.0,\n",
" -3684.0,\n",
" -3911.0,\n",
" -4037.0,\n",
" -3927.0,\n",
" -14964.0,\n",
" -4278.0,\n",
" -3716.0,\n",
" -30000.0,\n",
" -3874.0,\n",
" -3741.0,\n",
" -3959.0,\n",
" -3889.0,\n",
" -3954.0,\n",
" -3723.0,\n",
" -3936.0,\n",
" -3674.0,\n",
" -3869.0,\n",
" -4014.0,\n",
" -3904.0,\n",
" -4303.0,\n",
" -3618.0,\n",
" -3857.0,\n",
" -4280.0,\n",
" -3726.0,\n",
" -4280.0,\n",
" -3663.0,\n",
" -3684.0,\n",
" -3648.0,\n",
" -3864.0,\n",
" -3829.0,\n",
" -3706.0,\n",
" -3731.0,\n",
" -3927.0,\n",
" -3734.0,\n",
" -30000.0,\n",
" -4297.0,\n",
" -3979.0,\n",
" -3723.0,\n",
" -3801.0,\n",
" -3871.0,\n",
" -3995.0,\n",
" -3914.0,\n",
" -3851.0,\n",
" -4186.0,\n",
" -3713.0,\n",
" -3901.0,\n",
" -3663.0,\n",
" -4215.0,\n",
" -3653.0,\n",
" -3774.0,\n",
" -3889.0,\n",
" -4275.0,\n",
" -4149.0]"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"#same as python tinyverse skiing_original_reward.py eval -n 100\n",
"\n",
"from skiing_original_reward import make_experiment\n",
"from tinyverse.database import Database\n",
"db = Database(port=6789)\n",
"exp = make_experiment(db)\n",
"\n",
"exp.evaluate(100)"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"[2017-01-18 08:41:40,022] [Skiing-v0] Uploading 100 episodes of training data\n",
"[2017-01-18 08:41:41,741] [Skiing-v0] Uploading videos of 5 training episodes (673553 bytes)\n",
"[2017-01-18 08:41:43,246] [Skiing-v0] Creating evaluation object from ./records/ with learning curve and training video\n"
]
},
{
"ename": "InvalidRequestError",
"evalue": "Request req_mQ9LAbuZQhKZAT8M0SSrFg: No such algorithm_id: a2c-baseline. (HINT: we recently changed algorithm_id from being a raw string to being an ID. Create a new algorithm at https://gym.openai.com/algorithms, or just omit the algorithm_id field going forward.)",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mInvalidRequestError\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m<ipython-input-13-48b48fdc0196>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mgym\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mgym\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mupload\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"./records/\"\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\"a2c-baseline\"\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mapi_key\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m\"sk_dRtyWNecT6mIQgExL7mZrw\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
"\u001b[0;32m/anaconda3/lib/python3.5/site-packages/gym/scoreboard/api.py\u001b[0m in \u001b[0;36mupload\u001b[0;34m(training_dir, algorithm_id, writeup, benchmark_id, api_key, ignore_open_monitors)\u001b[0m\n\u001b[1;32m 78\u001b[0m \u001b[0;31m# Single evalution upload\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 79\u001b[0m \u001b[0mbenchmark_run_id\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 80\u001b[0;31m \u001b[0mevaluation\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_upload\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtraining_dir\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0malgorithm_id\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mwriteup\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbenchmark_run_id\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mapi_key\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mignore_open_monitors\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 81\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 82\u001b[0m logger.info(\"\"\"\n",
"\u001b[0;32m/anaconda3/lib/python3.5/site-packages/gym/scoreboard/api.py\u001b[0m in \u001b[0;36m_upload\u001b[0;34m(training_dir, algorithm_id, writeup, benchmark_run_id, api_key, ignore_open_monitors)\u001b[0m\n\u001b[1;32m 127\u001b[0m \u001b[0mwriteup\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mwriteup\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 128\u001b[0m \u001b[0mgym_version\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0menv_info\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'gym_version'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 129\u001b[0;31m \u001b[0mapi_key\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mapi_key\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 130\u001b[0m )\n\u001b[1;32m 131\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/anaconda3/lib/python3.5/site-packages/gym/scoreboard/client/resource.py\u001b[0m in \u001b[0;36mcreate\u001b[0;34m(cls, api_key, idempotency_key, **params)\u001b[0m\n\u001b[1;32m 321\u001b[0m \u001b[0murl\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcls\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mclass_path\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 322\u001b[0m \u001b[0mheaders\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpopulate_headers\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0midempotency_key\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 323\u001b[0;31m \u001b[0mresponse\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mapi_key\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mrequestor\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrequest\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'post'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0murl\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mparams\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mheaders\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 324\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mconvert_to_gym_object\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mresponse\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mapi_key\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 325\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/anaconda3/lib/python3.5/site-packages/gym/scoreboard/client/api_requestor.py\u001b[0m in \u001b[0;36mrequest\u001b[0;34m(self, method, url, params, headers)\u001b[0m\n\u001b[1;32m 39\u001b[0m rbody, rcode, rheaders, my_api_key = self.request_raw(\n\u001b[1;32m 40\u001b[0m method.lower(), url, params, headers)\n\u001b[0;32m---> 41\u001b[0;31m \u001b[0mresp\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minterpret_response\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrbody\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrcode\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrheaders\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 42\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mresp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmy_api_key\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 43\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/anaconda3/lib/python3.5/site-packages/gym/scoreboard/client/api_requestor.py\u001b[0m in \u001b[0;36minterpret_response\u001b[0;34m(self, rbody, rcode, rheaders)\u001b[0m\n\u001b[1;32m 155\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 156\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0;36m200\u001b[0m \u001b[0;34m<=\u001b[0m \u001b[0mrcode\u001b[0m \u001b[0;34m<\u001b[0m \u001b[0;36m300\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 157\u001b[0;31m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mhandle_api_error\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrbody\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrcode\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mresp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrheaders\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 158\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 159\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mresp\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/anaconda3/lib/python3.5/site-packages/gym/scoreboard/client/api_requestor.py\u001b[0m in \u001b[0;36mhandle_api_error\u001b[0;34m(self, rbody, rcode, resp, rheaders)\u001b[0m\n\u001b[1;32m 53\u001b[0m raise error.InvalidRequestError(\n\u001b[1;32m 54\u001b[0m \u001b[0mresp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'detail'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtype\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 55\u001b[0;31m rbody, rcode, resp, rheaders)\n\u001b[0m\u001b[1;32m 56\u001b[0m \u001b[0;32melif\u001b[0m \u001b[0mrcode\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m401\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 57\u001b[0m raise error.AuthenticationError(\n",
"\u001b[0;31mInvalidRequestError\u001b[0m: Request req_mQ9LAbuZQhKZAT8M0SSrFg: No such algorithm_id: a2c-baseline. (HINT: we recently changed algorithm_id from being a raw string to being an ID. Create a new algorithm at https://gym.openai.com/algorithms, or just omit the algorithm_id field going forward.)"
]
}
],
"source": [
"import gym\n",
"gym.upload(\"./records/\",\"a2c-baseline\",api_key=\"sk_dRtyWNecT6mIQgExL7mZrw\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": false
},
"source": [
"# Some auxilary monitoring\n",
"\n",
"* The dashboard we used throughout training. A bit messy. Displays average reward per timestep * 5000."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"import numpy as np\n",
"from matplotlib import pyplot as plt\n",
"%matplotlib inline\n",
"import theano\n",
"import theano.tensor as T\n",
"import pickle\n",
"\n",
"import atari\n",
"from tinyverse.database import Database\n",
"\n",
"db = Database(port=6789)\n",
"experiment=atari.make_experiment(db)\n"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"n_sessions: 5253\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztvXd4HcW5+P+Z03TU5Sr3gjEYUwzGmA6il5AAqZB8gVRC\nbkhIu/cHISEkBEIu6TekkEBCGoQbwqUZG2MsqrExuDdsXOXe1HX6/P7Ycnb3FB3JknUsvZ/n0aM9\ns7O7s6Ojeect847SWiMIgiAMPHx93QBBEAShbxABIAiCMEARASAIgjBAEQEgCIIwQBEBIAiCMEAR\nASAIgjBAEQEgCIIwQBEBIAiCMEARASAIgjBACfR1A/IxdOhQPWHChG5f39bWRnl5ec81qJ8h/ZMf\n6Z/OkT7KT1/1zzvvvLNPaz2ss3pFLQAmTJjA4sWLu319fX09dXV1Pdegfob0T36kfzpH+ig/fdU/\nSqkthdQTE5AgCMIARQSAIAjCAEUEgCAIwgBFBIAgCMIARQSAIAjCAEUEgCAIwgBFBIAgCMIARQSA\nIAjCYWbRpgP8aNYamtrjfdqOol4IJgiC0N/Y3tjBx3+/AIBdzRF+ed0pfdYW0QAEQRAOI395c7N9\nPHvlrr5rCCIABEEQDiu/f3WjfRxNpPqwJSIABEE4QoglUuxpiRz252470E48mX+gjsSTpFLaVdYe\nS6C1zqhbWeK2vG/a13bojewmIgAEQTgimHrXbGbeO4+l2xoP2zPvfmYV5/73fH7y4rqcdQ62xZjy\n3dk89Fp6Zh+JJ5l61xz+viaWUX/80DIunDLc/nzBT+r55UvrAXhw/gaufvCNHnyD/IgAEASh22it\nSaYyZ7m9QcJ8zhsb9nXpuh8+t5oJtz/Pqh1NXX7mn017/e9f2ZizzpJtB426b2y2y/a2RAF4aWsi\no35LJEFlOMCTXzrLLvv5S+8RTSR5YM46lh1GAScCQBD6KUu3NbK7uXdNJv9cF2fSt2fxpzc2sWlf\nG6t2NDHh9ue5b9YaNu5tZcLtz7N484EefeYDc9Yxe+XOgupqrfnj65sAeHrpDgCiiSTz1+3JqLuv\nNcofX9tIa9QYtCPxpOv8hNuf54+vZQqC93a3AkZEz5TvvgDAf/1ruX2+I+a+jyUAThxd7So/9juz\n7eP2WKbg6A1EAAhCP+WaB9/g3B/P79VnzN5sxLF//9nVXPCTej7wq9cBeOjVjby4ejcAH/3dgi7f\nd3dzhMl3zmLhxv12WVU4bTu/5W/vZtjXI/Ek//WvZexxCL1tBzrsY2tQ/cOrG/nMn952CYEt+9t4\n6NWN/PD5NcxaYQiX7Y3pay1++PyajLL1pgAw2mD4ChY42r29sd0+1lrT3BGnKhwkFMg9/N751Mqc\n53oSEQCC0I+J5XBevvreXr791IpDurc1U85GOOij3TPzfX9va8Hmmzv+vYJ4UnP/7LUAxJMpmiPu\n5zkHd4C5q3fzxOIGfvTCWrvsP/7xTrpNAT8AP3nxPcBYjAWwcON+zn+gnofM6JyoOfPfYQqAK08c\nkbet2RzTZSE/U0ZUArB6Z4tdHomnSKQ0leEgABvvu5IPThuVcf1TS7azvzWa97k9gQgAQeiHxDoJ\nL7zxkUX8Y+FWmiO5V6KuaGjiHwu35jy/OU/0SiyR4lfz1tuffzVvPRf99BU+9ceFedtl8fJaY3a+\nZGsjkXiSNlPY3HXVVLvO1gPtrmsCPgUYg2dHLInWmpXbmwEYN7iM3S3uAXXdLmNg3uK5T3MkQSSe\ntGf2U0ZUuc5737uxPc70cTX2Z601kXgSS0H56mNL7HMtZn9XmtqMz6f4+cenZe2DDo8JqjfoVAAo\npcJKqUVKqWVKqVVKqe+b5ROVUguVUhuUUv9USoXM8hLz8wbz/ATHve4wy9cppS7rrZcShP6I1pqd\nTZlmiWys3dVcUD3LLp6ND/76db791Aqu+OVrWc9bwuNz50y0y75w7kS+84Hj8PqFfzb3Pft4/e4W\n8pHwaC0b9rTSYs7+K8IBzj56CAAH22MuO/1XH08PtA/MWWdrIN+69BgGl4d4dpn7Xa2Zu9Neb107\n5buzaTjYQXnIz6iaUtf5lR5nclNHnLGDy+x+6IgnSWm48Dgj0ue0CYPsupYWU+kwZwX8Puq/VcdJ\nY6p55taz+c4HjgPy/216ikI0gChwodZ6GnAycLlS6gzgx8DPtdZHAweBz5n1PwccNMt/btZDKTUV\nuA44Hrgc+I1Syt+TLyMI/Zkn393OmT96mSVbD3Za90O/zh9KOHawMai1Rjp3Nq7ZmV2YWNdee8po\nfv3JUwgHfXzz0mMZXB7Ke79Lfv4qP3txHRNuf54Jtz/PhT+pd9nzXzfNRF887ygAtuxvt81NlSUB\nvv+h4wH4ymNLuPGRRfZ18WT6Hhv2ttLYYQioYZUlJFKGUHFGLB1si7Nlf24tZtO+VqpKg3xw2kg+\nduoYuzzodw+bTR1xakqDjKgKA3CgzQj9HFpRwtE1Pjt6CdJCs6o06LrHhKHlPHPrOZw0poaZEwcD\nhvbT23QqALSB5eUImj8auBD4l1n+KHCNeXy1+Rnz/EVKKWWWP661jmqtNwEbgJk98haCMABYud2Y\neS7e3LkA6IyAz/jXb40Wlozsz29syphB27PykgBXnTSKtfdcQTjoZ1BZWgCYVpkMfvXyBvt44742\nO5IGYHlDE0rBx2aMBWDptoO2AKgIByh3LKSy7Phaa5SCT5jXnDCqioPmQFxdGuKj040B/EBbzG7T\n3pYoG/akn/vkl850tXH+ur2UBv2UBPw88LFpzP9WHeCO0OmIJWnqiDOkooRwyJjPWpFXNaVBBoUV\nzR3pPrb6zOnQ9nLSmBoqSgKMG1yWs05PUZAPQCnlV0otBfYAc4H3gUattdUTDcBo83g0sA3APN8E\nDHGWZ7lGEIROKAka/67rOjGheMkWp2+ZTiw7uBfv6tS7n13NVxy2bEg7gSs9g5mlXUDmTDcXljkm\nkUyxsynCkPIQo03Tyx9e22TbzstLAoysdptk2mMJ2mOGzf2oYeVUhQO8tXE/TebAO6gsyNDKEgAa\nDraT0jBxaDmxZIr9ppC4+LjhnDp+cEa7Jg4tt4+rzXdp7jDeuzkS569vbQYMH0PYjOrZ0WgKgLIg\npQFlD/rg9AHk75fSkJ+OeO+HghaUDVRrnQROVkrVAE8BU3qrQUqpm4GbAWpra6mvr+/2vVpbWw/p\n+v6O9E9+iq1/tm8zBqt/vdPAVcNyawEpT3jk3JfrCQfcU/HmdmOQWtewL+s7rt6f3QHprLv8faM9\n7y56k6Bjqt8WTz//U8f4eHCpcfz9s8I8tT7O0r2Z916weBnvr1bcvSBCeRAGh30sfNPwPRw7yMdb\n7xoRS++tWELzRh8fnBTk2feNwXTqXXO4dLwxlO3YspHmSIJ3tzbyxtvGg9evWkZLzGjT1/72FgC1\nwQibgP+ZY4RbnlbZnLUfgpGDdrllylm25j3qY5v5/fIIC3YY77L9/bX2e7/2jnHPTWtXopJx2iIJ\n+x6LtxltXrnkbXaEc8+/VTLGpm07qK/v2TUUXrqUDlpr3aiUmg+cCdQopQLmLH8MsN2sth0YCzQo\npQJANbDfUW7hvMb5jIeAhwBmzJih6+rquvRCTurr6zmU6/s70j/5Kbb+WZ5cDxsNZ2q+djV1xGHO\ni5SH/LTFksw88+wMu3xy3gtAih1tmse3VfLDa09gaEWJfb7k/f3w9lsZ93Y+d0HHGgIbNnLJhRe4\n6qRSGubNAuAjF53BLdeUUFESQCnFwZfeY+lL6/Ey5qhjzEifNbTF4dSJQ6irm8k5GxbSHktQO24U\nLF/NpXXnMLg8xPnna0qfXM4TixsAqG8wbPzTT5rKo6uNgT88fDzwHpecfxYH22P8+O3X2NJs1PvI\n2cfz1r+Ws63F+Hz6aady8tga7inZzHefXmW369hJE6irOwYwzEz+l15g1Jjx1NUdy+/eWwAYA/QF\nZ51mhI0uW8x+3yBgDx+86GzeeexVkiTtfnvnxXX417zPhy6pI+DPLQAGL32VykFl1NXNyFmnJygk\nCmiYOfNHKVUKXAKsAeYDHzWr3QQ8bR4/Y37GPP+yNjw8zwDXmVFCE4HJQNqDIwhCXqyZ/QXHDstb\nzwqZnFxrxKFHE+4Zdyql7QVLyZRm9qpd/P6V97PeozyUO06jNZIgm4XH59AGKkoCVIaDGG5AOHfy\nUPvczz8xjR9cbTh0myNx18Ko4abJxjCFpAyHbDjAoDLjgUopbr1gsl3fWu9QGQ4wdaQRtrlwk7EY\nqyzkZ/zgtCkHDJPNcSPT4Z1h07zmNZc526SUojTot8MzRzlMUSOqw5SafTXPDGGtrQoT9BkZPy0n\nd3NHnIqSQN7B32qzdwVxb1CID2AkMF8ptRx4G5irtX4O+P+AbyilNmDY+B826z8MDDHLvwHcDqC1\nXgU8AawGZgNfNk1LgiAUgDVoW79zYTkprVl/1FPfSkFsDbIAftMp3BFLkkxp2sx7PH3rOa5rnVkx\nWyIJSgPZvbwXH1cL4NIqAJed/dpTxnDDGeMJ+AxHqVNQDTPbNnf1btbsbOZgW5yaspAtSACGVGRG\nG1WUBPn9DacCaQdxOOinNOTnI9PTkTwBv88WJgCDTcf1J04bB8DJY2v4yoVH89mz0yGu1r0sARBL\npggHfSz57iUMLg+5fCFlpjAI+kDrdIRSLJnKuwI4fX3gsKSD6NQEpLVeDmRsWaO13kiWKB6tdQT4\nWI573Qvc2/VmCoJgDZDtjtj3u55eSSKl+dC0UVSXBjluZJUd/z7MHHxbPKGe1gB26vhBvGBuSGIt\nojrjR/NIJFN824xFrygJUBUO2PHrbdEENeZg2RrNLQB+/clTaOqIu7QBi6V3XWIvklJKUVUapDkS\npzSY1jacwgmMRV/lnjTK3s9GmZ+xg8sYN7iMrQfa8an0u1lOdICgXzHIYRazjktDfl7+5vnUVoWz\n3r805CNi9m9je5ypI6vsa50O4y+eN8l4jvnsaCJJKOAjGk9RUoAA+H9njO80BXVPICuBBeEIwZq5\nd5gzw1gixV8WbOEfC7dy3UNv2Qu22qLGADW5tgJw56KBtACoO3YYnzrdmPGOqDZi2Js64rSZoY0A\nVaUB14zVmf6hJRKnNMcUMhz0U2vGxXupKQu5Bt/q0iBNHQnX5ijDK41rf/oxY5XstoPtVJRkmqPO\nOModuWNpPdbgXRLw21pDyO8UAG4NwBnbf9SwiqyDP+AyAR1sj9nCENyRPdVmx1gyzXq3aIEawOUn\njMiaIqKnEQEgCEcIlinHmuHnWsRkmQ6sQd0SCBaWbTkc9POflx0LGKYd56ra3U0RKsMBykIB/vyZ\nmZw6fpDrWshvAuoKVeFAhgnICrm0VuE2thu2cy9///wZrvQQlsnJMsGEHbP+DA2gLP+CtWw4BUBj\ne5yasuzhnJZgsB5pC4B4ipJA8ax/FQEgCEcIlklgZ1OEWCLFM8syUwVc/otXafOYgLy2ZGugtxY5\ngTFAORcsbTvYYa9sPWF0NV++wDBptMWS/ODZ1cxeuYu1u1rIMinvMpYJyKkBjKwxnu2082eblft9\nikHlxiA8tCJkz+QtAeAcbL3Hx49yp2MuhHAw7ZxtbI/lFCLVpmCwTUAOv0EhGsDhonhaIghCXpzp\nDN7efID/caymtVi7q8WO4LEWP7V6NABbAIT8tj06Ek/aqRPAWCA2aViF/bksZAy+7bEEj7yxiVv+\nZmTZ7MQfXRBV4aChAcRTDKss4ZX/rLOf7Rxgq3MsKvOZJp59rendt7JqAI6Bd1hlCZefYGT5tFYP\nF0JpyE8kniSWSNEWS1LjaVNtldHnVnmmBpAsyAdwuOjSOgBBEPoOZ66bjXkycTa2GwP5kPIQPpWp\nAXQ4NACfTxHy+4gmUrbdHwz7/tDK9OBrDahN7e7UEdOHH7oKUFUaMH0ASUqDfsYPSTtTnSYWyy/g\n5QLH9orp9hpD2+b9af+Hc+ANm8b5zfd/oEttLQv52dGYpLHDEDY1nvUVFx9Xy98XbmW4qT15fQCx\nZCqrKauvKJ6WCIKQF2eWzGieVMF/e2sLYAyC5SHDvv7E4m184MSRlJcEXD4AMGzjkXjSFhwALdEE\nIX96cLcG1D2elMplwUP3AZQGA+xrjTJn1W7GDHKneXA6Z4d5IoMsqsJBnv7y2YysTguIsizrF5wh\npN3FCgO1zGVereQHV5/Ap04fb6exyDABJVKUlIsGIAhCF3FqAG+ZO05VhgMZYZ7bGzsI+X2EAj7K\nSvz8c/E2Igu20NQe5wvnHWVrALYACPiJJlKuVNNau52mZZ5EZxahXNneuoA1cHfEk65nesklAACm\nja1xfbZMRxc5tIO9LYe+wYrhA0jZZjVvZJLfp5g6Kr3ALMMElBAfgCAI3cAZF27Zu2d/7bysdcvM\ngak8FLAXjm07aJhDrGiiUoedPBJPcpcjBQK4TSblpgawu9k9iAZ7wAn86bMnOJ6Z+4b5BIAXy3lc\n4VicdbOZXvpQKAn4iCaSjpXS+efQXgGQTGk7E2sxUDwtEQQhL4mUtlMZ72+LMqo6nOGEtLAGJmfk\njJXmwOkDAGtQy/TmOgfj0pwaQHfexE3Q77PTN2RzkFrZRXO9azb2mwLSuanK4PIQT37pLGZ99dxu\nt9XSlpwb1OQj6Df+YFafJ1Ip/D2gNfUUIgAE4QghkUzZA3pTe5xwyJ/V1g3pBVHO8x2xJMsbGqk3\nN0O3BEA46CcaT1Ie8jPE4dR0DsahgI+gX2UIgGAPDWZWtE42AfD9Dx3PxKHlGTtz5eMTpxmRPbec\nP8lVfur4QS4TTVcpCfiIJVLp/Qk6ceiWm6ctn0EqRVEJAPEBCMIRQjypqSgxbP7NkQRjBpW5HJvn\nTh7Ka+uN3bQsc4lzgGqOJFw7hVmDbUnAR2s0QVssydjBZXaOfK89viTgZ2eTRwD00JomS9vIZgK6\ncEotF06p7dL9xg4u63KETyFYfhNrs5lcK4YtLCe5FWGVSKXw94AzuqcQDUAQipAn3t5mb9ZysC1G\nw8F24g4NANJmGYsHPjqNL5xrJC+z0h44TRTeDeCtPD3hoN92kDrt7N7BOOhXtEYTrhlsHp9tl7CE\nTT4ncDFgCU1LSHamAWT6AMiaH6mvEA1AEIqMSDzJfz25nHDQx9p7ruD8B+bTHEkwcWi5a4etsGew\nrK0qsWeoAdP27BQY3mghi5KAj/f3GusKmh11vNEqVkhmadBvm0B6zAQUSPsjihlLQM1asdP43El7\nlVIE/cp24CdTKTs5XTFQ3L0tCAOQfa3GbNyK3rEG5Xgy5crPX+qxvyil7AgTa5buMgF1ZN//N+y4\nzxRzDwHIHNwsAeAUPHm2C+gS1sCaZffKosLSirYeMCKqCllbEPIbfgMwHPHF5AMQASAIRUa2iByA\nRFK7ZvQlWQzw1szfSo9Q4dIA0gLgvmtPTN/HMdCf49iwxSsAQrbPwG8vgOqpCbslnF5bv7dnbthL\nOPtk5sTMPYSzEQr4bA0gpcUJLAiCSTyZYvKdL/CjD5/I9TON1MyxHAKgOeLOiGlpAP/4/Onp7JOm\nALCGGGcUUEs0QW1VCecfM4xPmmmgwa0BlDsWNnl9AJbpojTk5x9fOJ29LVFaNy/v0vvmwtJybrvo\nmB65X2/hFAAXZUlBkY2gQwOQMFBBEGys6JAfz15rl+XaCKQ9lnQN0JYp5qyjh9qhjR0x49rqLGmK\ntTYWcnkFjHNQKws5NYzcPoDxQ8qZMaGwGXAhWBrAlJGVndTsW5xaV7jAEKhQwGdvWVlsYaAiAASh\nD7Hy8jhXh+bSAMBwloYcA7EXK0mZlYvG2nnLWff/lrrTSLs0AKcA8PoAzM9t0Z7fqvD+j5zEWZOG\ncNyI7sfoHw6cfZKt/7MR8moAEgYqCAKkI3NCpunm2WU7MsI1nQT8Pntmnm0Asleomqai6eZGLs79\nar0bvbtSPjg0jMoStxZhtdG5sXtPcer4QfzjC2dkhLYWG66MogW21fIBaK3FByAIQpq9ZsRPZTjI\nyu1NfOWxJa7VuF6CfuXY4zZzALLSJVj70546fhAL7riQkoCf6ffMBeCPN53musZ5H6ePodyT6EyZ\nnoUhFYXn5OlvOLWlcIEecMsHYKXiEAEgCAKQ9gGEgz47csdaZJSNoN9nhx5ms0F/89JjOW3iYE4/\naohdNrLaMAcdU1vBe7tb7R20LJyzWpcw8OS5SZr2JO/6g4GEywTUJQ1A2/1XTAJg4P4lBaEIiJg+\nAKUUms6D4AN+ZUf4ZBuIS0N+Ljt+RNZrrzhhJAC1no1VXLNa1w5a7gHOmsEW6vzsjzgFZKE+gKBf\nFa0G0KkAUEqNVUrNV0qtVkqtUkrdZpbfrZTarpRaav5c6bjmDqXUBqXUOqXUZY7yy82yDUqp23vn\nlQThyMHKEulTRpy/F29EUNDnw/Ihhru4ufhtF01m2fcuZZDHxOTSAPLcU1saQBFtan64ybarWGeE\nAn5iSYcAKCIncCEmoATwTa31u0qpSuAdpdRc89zPtdY/cVZWSk0FrgOOB0YBLymlrODeB4FLgAbg\nbaXUM1rr1T3xIoJwJGIJAKVU1vDPiGfnr4Bf2QNPV2fiPp/Kuq+u9z73XH18xj7CkJ79Fnu+nt6k\nWwLAr1i6rZEP/+ZNoLg0gE4FgNZ6J7DTPG5RSq0BRue55Grgca11FNiklNoAzDTPbdBabwRQSj1u\n1hUBIAxYrDBQhXvHLwvvquCA38f4IWU0HOzoMVu8N9zzhjMnZK1nLSrLpyX0d7LtkdAZ1grq9Xta\ngeISAF36BimlJgCnAAvNoluVUsuVUo8opQaZZaOBbY7LGsyyXOWCMGCxZvixZIpEKlMDaPUkcAv5\nlW17TvZQ4pxCZ7KW6aKYBrDDjbXSGroWBeSkmPqv4CggpVQF8CTwNa11s1Lqt8A9gDZ//xT47KE2\nSCl1M3AzQG1tLfX19d2+V2tr6yFd39+R/snP4eifDZuNMNADjc28/e6yjPMvv/GWu/5762g+aAiN\nd5etJLR3bcY1XWXdgbS5J9/77tln7AWwetUKgnvWAAP7O7R44ZuU+PMP5q2trRzY695Gc8P696iP\nbOrNphVMQQJAKRXEGPz/rrX+N4DWerfj/B+A58yP24GxjsvHmGXkKbfRWj8EPAQwY8YMXVdXV0gT\ns1JfX8+hXN/fkf7Jz+Hon+f2LoOtDQRKShk+/ihYstJ1/qgpJ8KCt+3PJxw/lXEdcRY9vYpLzp7O\nqeMPPR3DoG2NsMjYKCbf+/5962JW7NvNGaeeYoeZDsjv0OznAbj0wrpOs4HW19czdswQ2L7VLpt6\n3BTqZozNc9Xho1MBoIw3fBhYo7X+maN8pOkfALgWsL65zwD/UEr9DMMJPBlYhGHmnKyUmogx8F8H\nfLKnXkQQjkSstAqReMrl8B1ZHWZnU4TGdveq4KDfx/87YzxnThrC0cN7Jm9OoU7de689geNGVnFa\nD+YAOpIpJBU0wG7PLmpHWhTQ2cANwAql1FKz7NvA9UqpkzFMQJuBLwJorVcppZ7AcO4mgC9rrZMA\nSqlbgTmAH3hEa72qB99FEI4onlrSwAsrdwEQTSTthGEAx4+qZmdTJGNRWMCnUEr12OAPhYd1Dq8M\n841LijtbZzGyaNMB1+cjygegtX6ddHZZJ7PyXHMvcG+W8ln5rhOEgcTX/5m2+UfiKdc6ACt3T1NH\npgbQ0wzksM7DQTjkp8WRQK+YBID85QWhF/j7wi1sM3eNKoRoIulaBxAO+lEKmto9GkAnTsfuUBY0\nhM1Rw8p7/N79kdMmDOKE0YVnLfWG2RaTAJBcQILQw2zd386dT61k6sgqZt12bkHXpHR6TQAYK4PD\nAb9rj15wp43uKarLgvz3R06ibsqwHr93f+R/bzmrS/VTnnDdYhIAogEIQg+zckcTkF7lWyitDjPB\nyOowAb9yCQVwx6H3JB8/bSzDPTmChJ7h2x84zvW5mJzAIgAEoYexInsmDCnLWcfKq+OkJZqgLOTn\nRx8+kZvPm0TApzKESDHNHoXCuOqkUdR/q87+7O8lId4dRAAIQg/T7sjw6eTzjy7m+88agW/OVbxW\nqoDWiCEArp85jlDAR8Dvy6IByL/skYhTcIsGIAj9GMuU452sv7RmN396YzPgzvsz1MzO2RZNuAb4\nbBpAbziBhd7H+3ctFkQACEIPYwmAfKl6nDH/V00bZV/nHOAD/iwCoBecwELv49QAfCIABKH/YiVw\na4+lnboJx4AfiSftz9+9aipnTTLSKrREEgR9zplipgmomGaPQuE4nffF5McRASAIPYylATgHb+eM\nf29L1DYBlQb9djbO1iwmoGz7AQhHHgHH37WY/DjF0xJB6Ce0RNL5fSyijuPWaMJe9BX0K3uhUKYJ\nyJdhAiqmwUMoHKfmVkx7KhdPSwShn2CFgToHb+fGLu0xpwDw2RpAMqVdM8WsGkARmQ+EwnEJgCLa\nUEcEgCD0MNGEMWi3x5IZZQCt0SQJ00Mc9PtcqQJCHiew15EsTuAjE6fdv5hyLxVPSwShn2DZ+yO5\nNIBogpj52bnHL7gH+GyzffEBHJk414SUlxRPBp7iaYkg9BOswb0jnkRrzYL39/PJPy60z7dGE/ZC\nsIBPuTQAlw8gy2xfBMCRT1U42NdNsBENQBB6GEsAJFOaWDLFd5927/IViSdJmakgfD63BuCKAsoy\n2AfFBCT0IKIBCEIP4zT3RGIp3t/b5jqfSOm0AFBuDSDo0gDSxx+fMYZ1u1qKahGR0DVuOX8So2qK\nK+GeCABB6GFiiRQ+ZaZ4zpIRNJHUtnPXrxQBv4+AT5HwRAG9s+WgffzNS4+ltqq4Bg+ha9x+xZS+\nbkIGok8KQg8TS6SoKTPy+2QTAPFUys4Rb03oLS0g6JjhO/cCkPBPoTcQASAIPUw0maK61HD0OdNB\nWCSTmqRpArKiQyw/gNMHcM81J9jHAVkAJvQC8q0ShB5Ea00skaLKFADehVwA8ZTG2g7Aig+3NADn\nQD/SYfLprY1ghIGNCABB6EGsHD81pgDoiKUy6iSSKYcT2CgbZKaEdg70zgVDsgBM6A3kWyUIPYi1\nCMwyAa0UAvpWAAAgAElEQVTd1ew6H/Qbzl5rHYBlAhpeWWKeT/9LloX8rusEoafpVAAopcYqpeYr\npVYrpVYppW4zywcrpeYqpdabvweZ5Uop9Sul1Aal1HKl1HTHvW4y669XSt3Ue68lCH2DtQbAEgAH\n22Ou8+GAn0Qy0wQ0zBQAztj/0TXpLSW9u4sJQk9QiAaQAL6ptZ4KnAF8WSk1FbgdmKe1ngzMMz8D\nXAFMNn9uBn4LhsAAvgecDswEvmcJDUHoL1gCoKbMEABNHXHX+YBfkUhlmoAsAeDcLrA0WDxJw4T+\nSacCQGu9U2v9rnncAqwBRgNXA4+a1R4FrjGPrwb+og3eAmqUUiOBy4C5WusDWuuDwFzg8h59G0Ho\nY6ykb2kNwC0A/D4f8WTaBOQzB/xBZtioM4FcMCCzfqF36ZIPQCk1ATgFWAjUaq13mqd2AbXm8Whg\nm+OyBrMsV7kg9Bu8JqBGjwko6FckUyl7IZglAErM2b5zFXFIQj+FXqbglcBKqQrgSeBrWutmp01S\na62VUnl2QC0cpdTNGKYjamtrqa+v7/a9WltbD+n6/o70T36s/nllW5wThvoZUtr5gLyl2ZjBb96w\nDoCGPcZq3uuODTGx2scfV0Rp2LGLFYm9ALzzzmL2vOdjc4OhKWxt2EF9/X7ACCm1KNa/k3yH8lPs\n/VOQAFBKBTEG/79rrf9tFu9WSo3UWu80TTx7zPLtwFjH5WPMsu1Anae83vssrfVDwEMAM2bM0HV1\ndd4qBVNfX8+hXN/fkf7JT319Paecfjafnv0ix9RW8OLXz+/0muUNjfDmG5xy0omUrlpCwhcCOvj0\nFaczZUQVj2+sZ8iwao6bWgtLl3D6zNM4praSpqXbYeVSBg8bTl3dKekbznkeoGj/TvIdyk+x908h\nUUAKeBhYo7X+mePUM4AVyXMT8LSj/EYzGugMoMk0Fc0BLlVKDTKdv5eaZYJQtFgrefe2RAuqb230\nEvArOuJJtjd2AOldoAK+7CYgy+HrnPULQm9TiAZwNnADsEIptdQs+zZwP/CEUupzwBbg4+a5WcCV\nwAagHfgMgNb6gFLqHuBts94PtNYHeuQtBKGXaDXz8RQahplIWnn+3XMra1FXwG84gbUnCuiCKcP5\n9FkT+I+6Sa7rRteUsqOpo9vtF4R8dCoAtNavA7m+/Rdlqa+BL+e41yPAI11poCD0JVFHbv9CSKTS\nO32dOLqaFdubAChxaACJZCojCijo93H3h47PuN8Tt5wpWoHQa0iYgSDkwTLpOPf0zVs/md7p64PT\nRtrl6Vw/ytwPwCj3d5Llc3RNKWMGleWtIwjdRQSAIOQhYaZ2cIZn5sPe6tHvY/q49DrHdLpnn7kf\ngJUKoidbKwhdQzaEEYQ8WMndCrXCxE2BEfApRlanZ+5Wlk+/z1wJ7DEBCUJfIBqAIOShUNu/t37A\nr+zVvU4CfkU8WbgJSBB6ExEAgpCHeKow00+6ftoHEApk/nsF/T6SKTEBCcWBCABByIPl1IV0mod8\nJK0oIEcY6LSxNfax36eIu/YDEAkg9B3iAxCEPCQdGkAkkcw6q3di+Qws087aey53DfLWfgCWD8Av\nAkDoQ0QACEIe4g4NIFWAP8DpA4D0Xr8WIb+PeDJF0rMSWBD6AjEBCUIeEg4NoBCHcCKZaQJyUhLw\nE42n7MVdSv4DhT5Evn6CkAenDyBZQCxowuEEzkZJ0Ec0kbR9AGICEvoSEQCCkIdEymkC6ry+1wTk\npSTgI5pIYSoKYgIS+hQRAIKQB8ukA4VpAPEcyeAsQgEfkXhSwkCFokAEgCDkoatOYCtnUK5ooaDf\nR0qnTUuyEEzoS0QACEIenI7fREECIEXQr3IO7EEzJUQsaQgKMQEJfYkIAEHIQ7yLUUCReNLe/CUb\n1j6/0bhxX1EAhL5EBIAg5MEZBZQqwAcQiafsDd6zYTmHo4kUShW+0Ywg9AYiAAQhD06zTyEaQCyR\nIpQjAgjSJqBoIinmH6HPEQEgCHlwRQEVshAslSKYJ11E0KEByBoAoa8RASAIeeiqBpBI6pyLwMDh\nBDZNQILQl4gAEIQ8dHUlcDyZsgf5bFgbw0TiYgIS+h4RAIKQB2cuoELWASRSOucqYMD2D0QTKVkD\nIPQ5IgAEIQ/OhWCFmIA60wDEBCQUE50KAKXUI0qpPUqplY6yu5VS25VSS82fKx3n7lBKbVBKrVNK\nXeYov9ws26CUur3nX0UQeh7nfgAFm4BypIGAtAkomkiJCUjocwrRAP4MXJ6l/Oda65PNn1kASqmp\nwHXA8eY1v1FK+ZVSfuBB4ApgKnC9WVcQihrXOoACksElkvlNQOkooKSYgIQ+p1MBoLV+FThQ4P2u\nBh7XWke11puADcBM82eD1nqj1joGPG7WFYSiJuFKBdG5BIintD3Lz0bQpQEcevsE4VA4FB/ArUqp\n5aaJaJBZNhrY5qjTYJblKheEPuGl1btZs7O503ouJ3AhJqBOFoJZIaLReEpWAQt9Tne3hPwtcA+g\nzd8/BT7bEw1SSt0M3AxQW1tLfX19t+/V2tp6SNf3dwZy/3x+dhsAf768PGed1tZWdu2O2J+XLV+B\nb9eavPdtammnNNWWs1+3NBtJ4Fo7IgR96ojv/4H8HSqEYu+fbgkArfVu61gp9QfgOfPjdmCso+oY\ns4w85d57PwQ8BDBjxgxdV1fXnSYCUF9fz6Fc398ZqP2jtYbZswDyvn99fT01g8sJ7NtLIqU5buoJ\n1J0wIu+9SxbXM7K2irq66VnPr9vVAm++ilZ+SsPBI77/B+p3qFCKvX+6ZQJSSo10fLwWsCKEngGu\nU0qVKKUmApOBRcDbwGSl1ESlVAjDUfxM95stCN3nYHu84LqJlLZz+xdkAkp1thDMMPvEkmICEvqe\nTjUApdRjQB0wVCnVAHwPqFNKnYxhAtoMfBFAa71KKfUEsBpIAF/WWifN+9wKzAH8wCNa61U9/jaC\nUAC7myOdVzJJJFOUBHy0x5IF7QfQaSoIM0Q0ntQSBST0OZ0KAK319VmKH85T/17g3izls4BZXWqd\nIPQC8WQB8ZwmiZQmHPQDcdeagFxEE6mcu4GBe69gGf+FvkZWAgsDjq4IgKQtANxrAnLRHktQXpJ7\nXuU0D8lCMKGvEQEgDDhiidwD+e7miHsbSNMEBJ2ngkilNJF4itI8G8IEnRqAqABCHyMCQBhw5FrQ\ntbs5wun3zeNnc9c56mp7hy+vD2DNzma27G+zP3fEjRDPslC+HcGcGkDX2y4IPYkIAGHAkcsEtKc5\nCsD8tXvtskRS59QArvjla5z/QL39uS2WAKAsjwnI6SAWE5DQ14gAEAYcuUxAGqPcOS4nUqm0D6AT\nE1BHzNQA8pqAxAcgFA8iAIQBRy5bfrYw/0TKqQGkNYeIae5x0h7r3ATk9ylbwORJGioIhwX5CgoD\njlw+AGuG79IAkukooPtmreWmRxaRSKZsc5ETSwCU5hEAkNYCRAMQ+hoRAMKAw6kBfO7Pb7Npn+HI\njSaMAVyRHpgTqXQUEMAr7+3l6Dtf4MH5GzLu2276APKFgQIETT+ACAChrxEBIAw4nLb8eWv3cM9z\nqwFjEZeXpMME5OSfi7dllNkaQB4fAKQjgSQKSOhrRAAIAw6vD8DepCVuCADnxDye1AT9vrzpHSw6\nCvABOJ8nqSCEvkYEgDAg2NHYYR97o3nS2zRaJqA0yZSRs6eQwbqtQBNQwPT+SjI4oa8RASD0exZt\nOsBZ97/M00uNDORJzzqAkGOXLsClAiRSKQJ+1SUNoDMnsJUPSBQAoa8RASD0O5Zua+SPr220P2/a\n1wrAa+v3AVk0AGuXriw+ACu7Zz4NIJXSPLNsBz98fg1+n8q7DgDSAkdMQEJfIwJAKDr+d/E23t16\nsNvXX/PgG/zw+fTOXWUhwyRjzdAzfACmkzdqxvYHfAqtNVprEilNwOfLu89vLJniq48tAWDMoNK8\ndcGpAYgAEPoWEQBC0fGf/1rOh3/zZo/dzxporQ1dvBqAFbVjaQDvbDnIxDtmsafdqNeZBhBzmJTi\nWbQIL+IDEIoFEQDCgCMzCsjjAzBZtd/UCPw+9rZkLvyy2Oc4N3VUVafPtzSOoJiAhD5GBIBwxHPC\n9+bw9X8uzSjX5ozfyvFjpXrwagBWcjgrCsgikkxrAGMHl+Z8/oU/fQWA6tIgv/5k9r2AnYQkDFQo\nEkQACEWFLmDfXS+t0QRPLdmeUW4N9N6NXJKpFD4Fd101FYCHX99EIpmy1wGkrzd+B/yKD00b1Wk7\nThlXY6eNyEdQnMBCkSACQCgqCtl310kqT31r4Lds9JYmkEgZi7s+e85Eu+7/vtOQYQKyTPsBn+o0\nth+gKhwsqM0iAIRiQQSAUFTECnCiOlmwcX/Oc1bSN8vEYykXySwbt7fHkhkmIEtx8Pt8lIc6FwCV\n4c7rQFoAFLK2QBB6ExEAQlHRmQbw9NLt/PKl9fbnWJ79fS1nrxWZY905Ya7udeJTmU7guHl9wK9c\nefxzUVmgBhAKWD4A+fcT+pbCpiyCcJhIdLJh+22PG87e2y6eDEDDgXYgvbjKSTyZPewzmdIZsfo+\npXL7AHyKQubqhWoAVhioaABCX9PpFEQp9YhSao9SaqWjbLBSaq5Sar35e5BZrpRSv1JKbVBKLVdK\nTXdcc5NZf71S6qbeeR3hSKezjdctrA1ZDrTFAaguC3KgLcaE25/PuFfMYwLKpgE8+W5Dhgko7QT2\nFaQBVHXRBOT3iwAQ+pZCdNA/A5d7ym4H5mmtJwPzzM8AVwCTzZ+bgd+CITCA7wGnAzOB71lCQxCc\nxAsUAFZcvjVoJ5IpGg62u+9lDvxxewtIbdf1zr6XNzSxuzniaYvxO+BT9urdfFSVdtEEJAvBhD6m\nUwGgtX4VOOApvhp41Dx+FLjGUf4XbfAWUKOUGglcBszVWh/QWh8E5pIpVASBZLIwAWAN/JbdPpZI\nZTX1QFoQWOeTDg3gmNoKu/6BtpjretsH4FNMHFqe0YbB5SHX5646gSUKSOhruusDqNVa7zSPdwG1\n5vFowLlTRoNZlqs8A6XUzRjaA7W1tdTX13ezidDa2npI1/d3irF/dram7fD52vbGW4toqPKzaYuh\nCUTiSd56+11XnTffWsjmCh/vbzYG9j379lNfX8+OXRHi0RT19fV8/UTNl3Yb9Rs9AqAjmgAUq1et\nJDQ8wP3nlnL7a+m00r5U3FV/+/pV1O9aQ2fs3mG0eeeOBurr93Zav5gpxu9QMVHs/XPITmCttVZK\ndX31Tu77PQQ8BDBjxgxdV1fX7XvV19dzKNf3d4qxf97b3QKvvwrAueednzlLnm3Y+E88eTrTxw3i\n+b3LYFsDSQ2TphwPb79jV50+YwZTRlTxSssq2LSZ8spq6urO4n+3v8veRLP97tPWvs6yhiYSnm/x\nyoPGs6efPI3zjhkGwL+3LWDRZkMhListZd2d53Hsd2YDcP2VdZ0mggN4q2MtbHmfCePHUVd3XNc6\nqMgoxu9QMVHs/dPdOLTdpmkH8/ces3w7MNZRb4xZlqtcEFzEnYnVHMde+/xfF2zh/hfWukI3mzrc\nM3JrIZh1n5gdFZRyOXW/dskxedvk9Bc8ccuZ3HHFFMDaLtLPC7edy7xvnl/Q4A+OVBDiAxD6mO4K\ngGcAK5LnJuBpR/mNZjTQGUCTaSqaA1yqlBpkOn8vNcsEwYUzCsiK3vnbW1s4/b55vLlhn33uqSXb\n+d0r77sidxrb3Sac9DoAMxrIFBZJTxTQuMFledvkHditdA9WdtHjRlYxaVhFxnWd3U/Gf6GvKSQM\n9DFgAXCsUqpBKfU54H7gEqXUeuBi8zPALGAjsAH4A/AfAFrrA8A9wNvmzw/MMkFwEXc4ga0FXN/5\nPyMCefXO5oz6Tg2gsd3QAL59pTFDt1cCe1YEGzn+06PvpGEVeZO9ec1Q4aDxb1NoyGqu+3Wy5EEQ\nep1OfQBa6+tznLooS10NfDnHfR4BHulS64QBh3NQjXsigrJF2TgXb1kmIGs2njYB5dcAAKaMqGLb\ngQ6y4Q0ZtTSA7goAa3+C7iS+E4SeRNaiC0WFcyXwk+82sLOpgykjKgEIBTK/rm4TkCEArMRtCU8q\nCFsDSGp7Na7FoLLcMfzeNQAlZjuS3RzALXmSEgEg9DGSCkIoKpyx/A/MWccr69Jhkt5UDWAkcbNo\n7IgRDqZX7drpoFPptQKQXQMoy5PszSssSg5RA7Ce3c3LBaHHEA1AKCqswdpiaUOjfRzPYjRv7ohT\nHjIG5KaOOGWhgG2ySabcC8Bitg8glTGrz5fHvyzkPhcOmE7gbo7gXtOWIPQVIgCEosK7eQuOj95s\nnQDNkYSdgqGxPU5p0G8P7nYyuAJ8AKUeAXDxccPtY6/vwXICd3XvAovTJhhZUC6aMryTmoLQu4gA\nEIoK76Ca0tp2mmYTAK3RhD1AN7XHCQd9tslmd3OEbQfaHesAUmitM6KAsuHc3KWixCsA3GGgXWXG\nhMGs+v5lnHX00G5dLwg9hfgAhKLCKwA06Xj5tmgi6zXWYN0STTAhVG5rAHc9vQpYxanjjRm31sb9\ns2oAIfdcyNIqRlSFM9YBWE7g7moAQEE7jAlCbyPfQqGo8O4HkNLadrbmFACOLJylQX/G7N5KHW0d\nGxqAe1C/8cwJxBIpFm46wGvr91FjRgWVBDOVZEsDkCAe4UhHTEBCUZGhAeh0WVssme0SVx7+cMif\nMbvf0ZiO74/EU1k1gHDQz60XTrY3lhldYywMGzMoc4FYIRu/C8KRgGgAQlGR4QQmHf3THsuuATi3\nYiwL+jM2b2mJpK8zNIDM/QAs7r32RCa+tpEPTx/D++vX8YUPnpJRJ5xFKxCEIxERAEJRkUxlOnot\nodAaza4BOKN0SrNoAE6tIppIkkxmagAWI6rDfOeqqQCcMTLAkIqSjDolAdEAhP6BTGWEoiJbjLyt\nAeTwATg1gHAWHwBApel0jcRTxFO6oB2+cmEJj0um1nZSUxCKG9EAhKLCu7pWqbQAyOkDKE1/jctC\n/qxpmSvCAVqiCcMElExlOIG7yjvfudgleAThSEQ0AKGoiOcxAVlRQJXhAGdNGmKfLw8F7Pw62aKA\nIB3LH4mnjFxAh7gh+5CKkqy5iQThSEK+wUJRkW1P4JjHCfzgJ6fzi+tOts+XBHwoc7FANh8AGBoA\nQEc8SdyzIYwgDFTkv0AoKuJZwkBtE5DpBA76fa7VuSVBn70qt3MNIGlmA5XdWARBBIBQVGSLArJk\nQoe5oCsUUK7snSUBv70oqzTkRymVoQVYAmDBxv3GQjDRAARBBIBQXGRbB+DFa74pcdjircyd3hm+\nlXrhHwu3GvcQDUAQRAAIxUUh+XUyBUA6Lt9apesVADWl7ogd0QAEQQSAUGQkkik6m5x7o2+cn620\nzl4TUHVpkPFDyjh5bA0AwUOMAhKE/oAIAKGoSKR0pyttQ57Z+4jqsH1smYB8HgEQ8PsYXVNKc8TY\nNlKcwIIgAkAoMhJJTWkovwDwxvBXl7pXAmcj6FdUlATsfYPFBCQIhygAlFKblVIrlFJLlVKLzbLB\nSqm5Sqn15u9BZrlSSv1KKbVBKbVcKTW9J15AKD5WNDTxid8vyJm8LR+xZCpjC0bAZRayNICXvnEe\nf/3cTFc9S3h45/cBnyUAYoCYgAQBekYDuEBrfbLWeob5+XZgntZ6MjDP/AxwBTDZ/LkZ+G0PPFso\nQu54ajkLNx1g9Y7mguov29bI3NW7ASNOP9ss3hn2GTRt/kcPr+TcycM89UwBoNwDfDDgoyIcsENK\nDzUVhCD0B3rjv+Bq4FHz+FHgGkf5X7TBW0CNUmpkLzz/kNFad3vD72Jh5fYmJtz+PA/O33DYn23Z\n8FuzJG872BZj8p2zeHPDPrvs6gff4At/WQwYAqA06Oe+a0/k6xcfY9dxagVeH4AT796+FkGfe/HY\noaaCEIT+wKEKAA28qJR6Ryl1s1lWq7XeaR7vAqyUiaOBbY5rG8yyouOWv73DUd+edUj30Frzu1fe\nZ39rtIda1TWu+p/XAXhgzjoAvvLYEu5+ZtVhebYVgdOeJXnb0m2NxJOa3726MePcwo37mb9uLy2R\nOJ88fRznH5ue3Vc4Uj7nS+Pg3b/XviagXNswSioIQTj0bKDnaK23K6WGA3OVUmudJ7XWWinVpam0\nKUhuBqitraW+vr7bjWttbe3y9Vpr5qxqB+j02pTW/HV1jPPHBJhQ7Z55bmlOcv+bEZ57ez3fmhHO\ncYfe56RhfubPn8+zy4x3qqvaa5/rTv8UQnOTsQPXkuUrKdu/znXu3d2GVtB08EDGsx94+m0ANu9v\np76+nm0t6VXBvnh6V6/XXn0l45k1JYrGqOb1114FIBGPuc6vX7uW5lj6q7h29aqMtnnprf7pT0gf\n5afY++eQBIDWerv5e49S6ilgJrBbKTVSa73TNPHsMatvB8Y6Lh9jlnnv+RDwEMCMGTN0XV1dt9tX\nX19PodcnU5qv/XMpH54+GjAGoo2B8VwytZaxg8uyXrO3Jcr8OS/x2vYkG+670nVu5fYmePN1Ir5S\n6urO7/Y75KMjZiQ2e/i1Tfxy3no23ndlOvxx9vMAVNcMYuqp02DOPABXf3Slf7rCHza8BQf2M27S\nMdSdMd517uCSBliyjBHDh1JXN8PV1mHDhsHuXXY739/bCm8Yg/3EUcPY0Lg74x0s5p4aZW9LlKmj\nqgAIvT4XYmkhMO2kE2iNJPjbmmUAnDLtJOqmDM/7Hr3VP/0J6aP8FHv/dFsPVkqVK6UqrWPgUmAl\n8Axwk1ntJuBp8/gZ4EYzGugMoMlhKuoz/rJgM2t3NbOrOcKzy3bwmT+9bZ/7wXOrueAn9fbn3c0R\nl2+gxYwpz7Z6NZowzB+5NjLvCa79zRucdPeL/HLeegD2t6UHPGtP23e2HOS39e/3Whuy4TMdsB1Z\nooCWbm0EsodhevvRmeJhUFn+3PvDKkvswR8yN2wP+pXLjCQ+AEE4NB9ALfC6UmoZsAh4Xms9G7gf\nuEQptR642PwMMAvYCGwA/gD8xyE8u0fY0djBXU+v4vJfvEYskZmEDIxBKZ5MsW5XC6ffN4+/L9pq\nn8vm5LSw7N/56hRCKqX561tbiMQz7elrd7W4PjdH4qze0czeliht5uDbHkvy5zc3H1Ibuks2H8Cj\nC7YAmWGaQIbj3bnCd1B5qEvP9orkoN9n7woGEgUkCHAIJiCt9UZgWpby/cBFWco18OXuPq832N6Y\ntitv2teas96mfW0s3LQfgBUNjYBh1nBuNu7FSl2cbRAslF+/vJ7/W7qDDXta+fXL61n47Yvz1o8l\nUlz5q9cAI1JmdE2p6x3BGGSdq2S11hkhk4eKlb65I8+7R+IpdjVF7AyfTu64YgrgzvEzsqprfhTt\nUQGCfp8rQkjWAQjCAF8JvMMxOH72z4td5+7+4FT7+MVVu7jraSOCxrlJuGUCykb9OsP14R2IusJP\nXnyPDXsMwbS7ufNoooMOE1AsmeL0owZn1Ikk0gPuwp0JJt4xiz0tkW63MRuRuCkAsgzu08xcPJF4\nkjN+NM9lYmuJJhhaUcIXz58EuE1AI02TVqF4rXJeE5BEAQnCABQAa3c1M/Wu2SzcuJ9dTemBz5sb\n5uyjhzLrq+cCxkBs8dv692lsj/Hw65vyagCPv21EvHZ328BsgsNZlu18Y4dbIB03oiojLNLSTACe\ned8QGM5+ANjTHKGpI7dw87KnJWLP+gGipjktm1ktagqFbMKh4UA7QxymHme8/8jqQ9cAZB2AILgZ\ncALg//1xIe2xJJ946C1+9EI6atXrgBxaUUJ1DsfjPc+t4Z7nVjN/3R677NX39vLqe+kQy+tnGgFP\n4aCfaCLJa+v3ZtwnH9k2QLdm1pDdtNTsGbTLSwKsuPtSfvqxaXz/Q8cDcKAtxiU/e4Vnl+2gw5Rf\ncU8O/pn3zePSn2eGWmZj6/52Zt47j4df32SXWYN8VgFglmXzaexoijCyJj3QO01Vo7uoAWTzAcg6\nAEFwM+D+C/a1xjqvhJFgrLo0uwCwBq9ZK3bZZTc+sogbH1lkf44ljCGoI5bknudWc8PDi1i7q7DU\nCJDdvNTsKDvQlvke3ll7eYmxO9ZHTh3DsErDdLViexPr97Ryx79XEDHbGHUMxtbM2Wlyem393gxf\nQiSeRGvNhr2GI/qpd7e7zgFEk5kCIJJHAwAI58gEOtjUDI53RPrkJYsJyKlRSDZQQRhgAqAze/yX\n6ibZxz6fojzk3l/2g9NGUVESoCqHYAD497sNNBxst8NAo4kUizYdAKC5I8G7Ww/mDA393tMrecNM\nkZDNvOSc4R9sL0AAOPLnWEnSdpmLtDriSSLmGLzKkbOnxdM2rTU3PLyIy3/xql227UA7U747m8cW\nbSNqaiVtjpDPvCYgSwPI4SAOB7N/JZVSvPj183js5jOynvfi/UsHfD6X41c0AEEYYALAmnV+85J0\njhmnaeEW0/l4huk8VUrZg/2X6iYxYUgZrdEEs1bkXr7wjSeWcc2Db7jMNdZsfXlDIx/+zZvcN2sN\nYAyu9zy3mqXbGkkkUzy6YAuf+uNCINOcA3Db40tte302DaDZozU4BVWZGQFjaUDJlKamxBgQY46Z\nelO7+x5WnzkF0pJtRiz/Pxdvswd0pw8g4jEBPbF4G0u2HnSda87hP8mVzhngmNpKqsL51wNYeIV9\nmakNWYgPQBAGiADY1xplwu3PM/WuOYCxaOj5r54DwK+uP8WuV10a5OVvns+Dn8zMVD2sosTOSNmZ\ng3Rfa8yVCtkSBlZ2zHe2HLTv8/Drm7jx4YUZA+Lelsyon9U7mznjR8aK3sb2zDY0dbjvUVWa1gAs\n+/c+R24iawyMOmbq3vsezPKcVrOtzR1xW9Nx+hEiDg2gPZbgv/61nGt/86brWbnWR+QTAF3BGwXk\nFRyyDkAQDj0X0BHBr192Z8QcUlHC8aOq2Xz/BwBYetcltr3/qGEVrrrWTHtweahLkTFOM4812G3e\n3ziyyDwAAA4MSURBVAYYOYTmrNrFqu1NAIQC/owZ/7+XZGTJAAyN5dllOzJs8pApmJyDnmUC2u/w\ngbTFM30AznskU9oVWpp+H6POvpaoy9zTFk3QEU+SNEffWDLFc8sNbSnoV8STKZIpjVKZK3UtSjwm\noN/fcGpWh3FnTBhazpqdzVw/cxyPLdrqCim12iMIA50BIQC85pJRNe6QwpqyzleZprRmxoRB9ueq\ncCCnGQOMQX9oRcjldN6y30jI5lOKL/71Hbt8aEXIZWKJJVIMNdcbbPrRlaza0Wxn99ze2MFXHlti\n3sc9080QAE4TkCUA2tIaQLv5SOcA2+bSXJK8t9u92th4N6N+Sms7rDSlNRf/7BV2OkJKY4kUyxsM\nc9GUEVW2sKgpDWbVLCDTCXzZ8SOy1uuMP336NDbsaeXso4dw7zUnZCx2kx3BBKEfm4CSKc3mJmNw\nWrerhaOGltvnxgzKntwtG7+87mTAcAAfP6ra9hnceOaEvNc1HOyg1rN6dX+W2TQY6ZNboukBcVeT\nkXNoeGUJSqmcZpFBHsHVkuEETl9nma/2Z4mCcvornMLglB/MtU1CzhmzZQLqiCdtTSeR1K7BHwwB\nYD2vsSNmaxr5BG5PmYBGVIc5Z/JQlFIZ+wODaACCAP1YA/ifl9fziwURjj1hP+t2t/Cflx3LA3PW\nMbyyJGd4ZzauPnk0V5+c3rbgkU+fxsJN+10RNtmIJlLUVoVdETYW3hBIpdxO1vMemA9gC61ckTHV\nZUGXUPFqAM5Zb1oDyCIAHKuDnQIgZuZAsrAcq5YJKKXhgBmNFMsS8hlLpmzH9MG2uO0bqMmT2C3X\nu/Y0nW08LwgDgX4rAH7xkpEh80t/N0wtU0dVsejOi3JuGFIox46o5NgRlcxfaywCy2fPHl5ZkrV8\njyetQ3s0mTXsc+M+w2eQa1bs3Rkrn48i6PeZdvjMxjoHfac2APDmRiMsNZ7UfOlv7/Lyuj1cfFw6\njXI2ZzUYjvZYIkWz6ZhujSZszaEmjwDuKQ1AEITO6bcmIAvLhHH8qCqGV4Zde8seCpaN/sTR1Tnr\nDHeYgEY5Uhl4NYCmjjitefIK5RoUh3kETLa01E5ybZfoHPS9bdt2IO1snr1qF7FEyiVocgmAEVVh\noomUKzR1p7kGwWu6cnK4NABBEPqxAKhyJP6aMqKS4ZU9uyvX8aOq+PrFx/DL607JWWdYRXqgmzis\nPGe91mgiqwbw6bMmAIYt/4YzxnPKOCOR2tjBpdz9wanc/5GT7LrWNoy5BnnAlQrBSSRupKr48G/e\n4PX1+7LWcbJuVzpzai4BUFsVJpZI0twRt/P7WGsYcqXYgNwrgXuKH334RH78kRN79RmCcKTQL01A\n8WSK5kiCS8cHuODU4/j4jLGdX9RFfD7FbRdPtj9XlgR4/qvnsrslwsd+twBw72M7cWg5b2zY77pH\ndWmQkoCPPS1R25ZuYYWogmHLv+eaE/ht/fss2dpIdWmQT5890VW/ptTwB1SEA3TEk8yckJkJ1AoF\nHVkddjlso4kUNzy8KKN+Lva1RpkwpIzN+9uzhqMCjKguIZZMEY9ppo2pZn9bzH5mfg2gdwXA9TPH\n9er9BeFIol9qAFaahBHlPq6fOc6eHfcWz33lHJ645UzGDSnjpDFpk5DTUTwuy7aS9d+qs1cfr9/d\n6tqwJBulpnnEeV/redbsvtJMAPfXz8/MuN66bmR1mN9+ajpHVfs4++ghGWklKsMBu11ghJt6mT4+\nHRKbzalbUxoiEjfi/icMMbQfywSUzwnvXQcgCELv0S//2waVhXjpG+cxY8ThUXBOGF3NcSONJGXO\n6JKUhnuuOYFPnzUhI/RRKSNO33JKr9nZbN8jF9YM3jmD/tvnT+eF2861005XhANUhoNZo1zKS4yy\nslCAK04cyV1nljK4vMTeF8HSGloiCW6/YortxJ4wNNN8NWlYhR1KOcyxR8J/1E3iT585zZUGe9wQ\nQ/gtbzAWvk3Mcj8LcQILwuGjXwqAoN/H0cMrqQr1baz3zImDueGM8dz9oeMzBraa0iB+n7Jn7vvb\nYkyurch2Gxtrh63BDt9CVTjIcSOr7IigfOGp1rkyx/qAkoDPjgyaNtbt0LaE08QhmQN2VWmQkdXG\nmginM/qcyUO54NjhLgEw3hQAa3e1UB7yc5TDH3LjmeNd1/e2D0AQhDT9UgD0Nd+9aiq/vO5kO4Ux\npBdPWc5paxbvzNczfkgZ188cx0M3nJr1vkcPrwRwhWFaDDGFwqg8efMtYeM02TijbiYOdQugMlNj\n8EYbWe9hLYpznrfMO84Q1aEVJbYwGeZZh/H9Dx3PgtsvtD+XhkQACMLhol86gfuaz50zMaPs8hNG\n8NiirVw3cyx3PrXSHoSdvoFxg8u4+bxJGddanH30EN6+8+KsA7Jlhhk9qBABkBZMzhn3UZ5IJStk\nNlv0kFN7cWZUte7t1ACqwsbeCq3RBNVlIddaDKWUKzOnhIEKwuFDBMBhYnB5iGe/cg5LzVTKlnbg\nHDzHZnEUO1FKZR38Ib0SN9d5SKc/cGsADgHgsc1bs/ia0iBfv/gYSoI+jh5Wwef/spjTJgxmzqrd\ngHu7xlrz+U4BUFMWtGf2NaXBvJvQiwlIEA4fIgAOMxWeWXjA72N0TSnbGzs4amh+H0A+RpiLzmrz\nCICUuWTZ6SB2mlyGVZZw3WljueqkUUA6x39NWZAbHLmPrBDV/7rsWA62xbjqpFF89+lV9vuA2xdR\nWxW2N7d3ppbIhjiBBeHwcdgFgFLqcuCXgB/4o9b6/sPdhr7EEgCDHLPwl791Plof2uD3rcuO5biR\nVZx3zLCcdS6ZOoK/vbWVsyYNscssTUQpQ8NwLi6ztIrqHHH7YweX8bfPnw4Ym+w4NRjn3r7O97LM\nY//35bNdi/XOOXoor2/Yl5G2WRCE3uOwCgCllB94ELgEaADeVko9o7VefTjb0ZdUlQYI+JQrU2hP\nJCYLB/185NQxeeucf8wwNtx7hSsVsuU7yJafp9LcT2BwAemyv3LRZNdny7RlmYKeufVs4knNqeb6\ngZPH1rjqP/Lp09jdHMmauVMQhN7hcGsAM4ENWuuNAEqpx4GrgQEjAMpCAZ780lkcPbz75p5DwZsH\n3/IZnDo+c+XwJ2aMpbkjzsnjajLOdUZtVZj//uhJnGIO9CeNyX+PUMDXqQ9EEISe5XALgNHANsfn\nBuD0w9yGPmfa2K4PqL3F8aOquOX8SXx4+uiMcx84aSQfOGlkt+/dGyk4BEHoOZR38+xefZhSHwUu\n11p/3vx8A3C61vpWR52bgZsBamtrT3388ce7/bzW1lYqKvpmpn0kIP2TH+mfzpE+yk9f9c8FF1zw\njtZ6Rmf1DrcGsB1wTgvHmGU2WuuHgIcAZsyYoevq6rr9sPr6eg7l+v6O9E9+pH86R/ooP8XeP4c7\n5OJtYLJSaqJSKgRcBzxzmNsgCIIgcJg1AK11Qil1KzAHIwz0Ea31qsPZBkEQBMHgsK8D0FrPAmYd\n7ucKgiAIbmTVjSAIwgBFBIAgCMIARQSAIAjCAEUEgCAIwgDlsC4E6ypKqb3AlkO4xVBgXw81pz8i\n/ZMf6Z/OkT7KT1/1z3itde7MkCZFLQAOFaXU4kJWww1UpH/yI/3TOdJH+Sn2/hETkCAIwgBFBIAg\nCMIApb8LgIf6ugFFjvRPfqR/Okf6KD9F3T/92gcgCIIg5Ka/awCCIAhCDvqlAFBKXa6UWqeU2qCU\nur2v29NXKKU2K6VWKKWWKqUWm2WDlVJzlVLrzd+DzHKllPqV2WfLlVLT+7b1vYNS6hGl1B6l1EpH\nWZf7RCl1k1l/vVLqpr54l94gR//crZTabn6PliqlrnScu8Psn3VKqcsc5f3yf1ApNVYpNV8ptVop\ntUopdZtZfmR+h7TW/eoHI8vo+8BRQAhYBkzt63b1UV9sBoZ6yv4buN08vh34sXl8JfACoIAzgIV9\n3f5e6pPzgOnAyu72CTAY2Gj+HmQeD+rrd+vF/rkb+FaWulPN/68SYKL5f+fvz/+DwEhgunlcCbxn\n9sMR+R3qjxqAve+w1joGWPsOCwZXA4+ax48C1zjK/6IN3gJqlFLd3w+ySNFavwoc8BR3tU8uA+Zq\nrQ9orQ8Cc4HLe7/1vU+O/snF1cDjWuuo1noTsAHj/6/f/g9qrXdqrd81j1uANRhb3R6R36H+KACy\n7TucueHtwEADLyql3jG32gSo1VrvNI93AbXm8UDut672yUDsq1tNE8YjlnmDAd4/SqkJwCnAQo7Q\n71B/FABCmnO01tOBK4AvK6XOc57Uhi4qYWAOpE+y8ltgEnAysBP4ad82p+9RSlUATwJf01o3O88d\nSd+h/igAOt13eKCgtd5u/t4DPIWhmu+2TDvm7z1m9YHcb13tkwHVV1rr3VrrpNY6BfwB43sEA7R/\nlFJBjMH/71rrf5vFR+R3qD8KANl3GFBKlSulKq1j4FJgJUZfWBEHNwFPm8fPADeaUQtnAE0Olba/\n09U+mQNcqpQaZJpDLjXL+iUeX9C1GN8jMPrnOqVUiVJqIjAZWEQ//h9USingYWCN1vpnjlNH5neo\nr73qvfGD4Xl/DyMS4c6+bk8f9cFRGNEXy4BVVj8AQ4B5wHrgJWCwWa6AB80+WwHM6Ot36KV+eQzD\njBHHsLt+rjt9AnwWw+m5AfhMX79XL/fPX833X44xoI101L/T7J91wBWO8n75Pwicg2HeWQ4sNX+u\nPFK/Q7ISWBAEYYDSH01AgiAIQgGIABAEQRigiAAQBEEYoIgAEARBGKCIABAEQRigiAAQBEEYoIgA\nEARBGKCIABAEQRig/P97eBy2AZni2QAAAABJRU5ErkJggg==\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x7f36b434b470>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"r\tV\t\tEnt\tLogits\n",
"+0.0 +2.636 (+0.336) +0.925 | 0.30(-0.01) 0.12(+0.00) 0.59(+0.00)\n",
"+0.0 +2.671 (+0.324) +0.831 | 0.32(-0.01) 0.06(+0.01) 0.62(+0.00)\n",
"+0.0 +2.694 (+0.335) +0.978 | 0.46(+0.11) 0.42(-0.09) 0.12(-0.02)\n",
"+0.0 +2.773 (+0.234) +0.566 | 0.07(+0.00) 0.09(+0.00) 0.83(-0.01)\n",
"+0.0 +2.810 (+0.218) +1.079 | 0.24(-0.06) 0.39(-0.10) 0.36(+0.16)\n",
"+0.0 +2.902 (+0.094) +1.051 | 0.31(+0.12) 0.47(-0.09) 0.22(-0.03)\n",
"+0.0 +2.976 (+0.005) +1.029 | 0.26(+0.08) 0.23(+0.07) 0.51(-0.15)\n",
"+0.0 +2.933 (+0.151) +1.046 | 0.34(-0.04) 0.46(+0.02) 0.20(+0.02)\n",
"+0.0 +2.947 (+0.183) +1.068 | 0.35(+0.13) 0.23(-0.04) 0.42(-0.09)\n",
"+0.0 +3.029 (+0.081) +0.662 | 0.26(+0.01) 0.72(-0.01) 0.02(+0.00)\n",
"+0.0 +3.058 (+0.085) +0.906 | 0.29(-0.00) 0.60(+0.00) 0.11(+0.00)\n",
"+1.0 +3.097 (+0.071) +1.065 | 0.31(-0.02) 0.45(+0.04) 0.24(-0.01)\n",
"+0.0 +2.136 (+0.035) +1.006 | 0.54(+0.08) 0.24(-0.12) 0.21(+0.04)\n",
"+0.0 +2.116 (+0.118) +1.090 | 0.31(-0.03) 0.39(+0.06) 0.30(-0.03)\n",
"+0.0 +2.163 (+0.068) +1.003 | 0.21(+0.01) 0.55(-0.00) 0.24(-0.00)\n",
"+0.0 +2.183 (+0.074) +1.075 | 0.27(+0.01) 0.29(-0.01) 0.44(-0.00)\n",
"+0.0 +2.202 (+0.080) +1.026 | 0.35(+0.02) 0.18(-0.05) 0.48(+0.03)\n",
"+0.0 +2.207 (+0.116) +0.820 | 0.33(-0.01) 0.05(+0.00) 0.62(+0.01)\n",
"+0.0 +2.238 (+0.099) +0.777 | 0.27(+0.03) 0.05(+0.01) 0.68(-0.04)\n",
"+0.0 +2.237 (+0.148) +0.693 | 0.20(-0.02) 0.05(-0.00) 0.75(+0.02)\n",
"+0.0 +2.292 (+0.083) +0.111 | 0.02(+0.00) 0.00(+0.00) 0.98(-0.00)\n",
"+0.0 +2.307 (+0.101) +0.127 | 0.02(+0.00) 0.01(+0.00) 0.98(-0.01)\n",
"+0.0 +2.308 (+0.147) +0.091 | 0.01(-0.00) 0.00(+0.00) 0.98(+0.00)\n",
"+0.0 +2.383 (+0.045) +0.903 | 0.27(+0.07) 0.12(-0.00) 0.61(-0.06)\n",
"+0.0 +2.430 (+0.000) +0.903 | 0.32(+0.00) 0.10(+0.01) 0.58(-0.01)\n"
]
},
{
"ename": "KeyboardInterrupt",
"evalue": "",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mKeyboardInterrupt\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m<ipython-input-12-a710b4fb1035>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m 33\u001b[0m \u001b[0mR\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 34\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mi\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m5000\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 35\u001b[0;31m \u001b[0ms\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mr\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0md\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mm\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdb\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mloads\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdb\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mredis\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlindex\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'sessions'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 36\u001b[0m \u001b[0mR\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0msum\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mr\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 37\u001b[0m \u001b[0mavg_reward_per_session\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mR\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/anaconda3/lib/python3.5/site-packages/redis/client.py\u001b[0m in \u001b[0;36mlindex\u001b[0;34m(self, name, index)\u001b[0m\n\u001b[1;32m 1203\u001b[0m \u001b[0mend\u001b[0m \u001b[0mof\u001b[0m \u001b[0mthe\u001b[0m \u001b[0mlist\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1204\u001b[0m \"\"\"\n\u001b[0;32m-> 1205\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexecute_command\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'LINDEX'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mindex\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1206\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1207\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mlinsert\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mwhere\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrefvalue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/anaconda3/lib/python3.5/site-packages/redis/client.py\u001b[0m in \u001b[0;36mexecute_command\u001b[0;34m(self, *args, **options)\u001b[0m\n\u001b[1;32m 570\u001b[0m \u001b[0mconnection\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpool\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_connection\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcommand_name\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0moptions\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 571\u001b[0m \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 572\u001b[0;31m \u001b[0mconnection\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msend_command\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 573\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mparse_response\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mconnection\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcommand_name\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0moptions\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 574\u001b[0m \u001b[0;32mexcept\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mConnectionError\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mTimeoutError\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/anaconda3/lib/python3.5/site-packages/redis/connection.py\u001b[0m in \u001b[0;36msend_command\u001b[0;34m(self, *args)\u001b[0m\n\u001b[1;32m 561\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0msend_command\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 562\u001b[0m \u001b[0;34m\"Pack and send a command to the Redis server\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 563\u001b[0;31m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msend_packed_command\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpack_command\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 564\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 565\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mcan_read\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtimeout\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/anaconda3/lib/python3.5/site-packages/redis/connection.py\u001b[0m in \u001b[0;36msend_packed_command\u001b[0;34m(self, command)\u001b[0m\n\u001b[1;32m 541\u001b[0m \u001b[0mcommand\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mcommand\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 542\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mitem\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcommand\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 543\u001b[0;31m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_sock\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msendall\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mitem\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 544\u001b[0m \u001b[0;32mexcept\u001b[0m \u001b[0msocket\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtimeout\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 545\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdisconnect\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;31mKeyboardInterrupt\u001b[0m: "
]
}
],
"source": [
"avg_reward_per_session = []\n",
"\n",
"from IPython.display import clear_output\n",
"while True:\n",
" if db.redis.exists(\"weights\"):\n",
" weights = db.loads(db.redis.get('weights'))\n",
" if any(map(np.any,map(np.isnan,weights))):\n",
" print(\"Weights broken!!!\")\n",
" \n",
" print(\"n_sessions:\",db.redis.llen(\"sessions\"))\n",
" plt.plot(avg_reward_per_session)\n",
" plt.xlabel(\"time (arbitrary ticks)\")\n",
" plt.ylabel(\"sum of rewards in 5k subsessions\")\n",
" plt.grid()\n",
" plt.show()\n",
" \n",
" \n",
" R=0\n",
" for i in range(5000)[::-1]:\n",
" s,a,r,d,m = db.loads(db.redis.lindex('sessions',i))\n",
" R += sum(r)\n",
" avg_reward_per_session.append(R)\n",
" clear_output(True)\n",
" "
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"root 6326 90.6 0.7 79039712 928644 pts/11 Rl+ 06:42 0:44 python tinyverse atari.py train -b 10\r\n",
"root 6475 90.9 0.2 71643972 347380 pts/8 Rl 06:42 0:28 python tinyverse atari.py play -b 1\r\n",
"root 6476 87.8 0.2 71643852 348072 pts/8 Rl 06:42 0:27 python tinyverse atari.py play -b 1\r\n",
"root 6477 89.0 0.2 71643968 349932 pts/8 Rl 06:42 0:27 python tinyverse atari.py play -b 1\r\n",
"root 6478 87.4 0.2 71643552 347084 pts/8 Rl 06:42 0:27 python tinyverse atari.py play -b 1\r\n",
"root 6479 87.4 0.2 71643540 354384 pts/8 Rl 06:42 0:27 python tinyverse atari.py play -b 1\r\n",
"root 6480 84.1 0.2 71643848 347368 pts/8 Rl 06:42 0:26 python tinyverse atari.py play -b 1\r\n",
"root 6481 91.0 0.2 71643512 349676 pts/8 Rl 06:42 0:28 python tinyverse atari.py play -b 1\r\n",
"root 6482 87.4 0.2 71643512 347188 pts/8 Sl 06:42 0:27 python tinyverse atari.py play -b 1\r\n",
"root 6483 86.0 0.2 71643848 348648 pts/8 Rl 06:42 0:26 python tinyverse atari.py play -b 1\r\n",
"root 6484 86.0 0.2 71643844 351244 pts/8 Rl 06:42 0:26 python tinyverse atari.py play -b 1\r\n",
"root 6485 90.8 0.2 71643840 347592 pts/8 Rl 06:42 0:28 python tinyverse atari.py play -b 1\r\n",
"root 6486 97.8 0.2 71645432 348740 pts/8 Rl 06:42 0:30 python tinyverse atari.py play -b 1\r\n",
"root 6487 89.5 0.2 71643848 349260 pts/8 Rl 06:42 0:27 python tinyverse atari.py play -b 1\r\n",
"root 6488 87.6 0.2 71643540 346468 pts/8 Rl 06:42 0:27 python tinyverse atari.py play -b 1\r\n",
"root 6489 85.3 0.2 71643588 349764 pts/8 Rl 06:42 0:26 python tinyverse atari.py play -b 1\r\n",
"root 6490 87.1 0.2 71643844 347828 pts/8 Rl 06:42 0:27 python tinyverse atari.py play -b 1\r\n",
"root 6491 87.3 0.2 71643388 346572 pts/8 Rl 06:42 0:27 python tinyverse atari.py play -b 1\r\n",
"root 6492 86.1 0.2 71643840 347660 pts/8 Rl 06:42 0:26 python tinyverse atari.py play -b 1\r\n",
"root 6493 87.7 0.2 71643856 352008 pts/8 Rl 06:42 0:27 python tinyverse atari.py play -b 1\r\n",
"root 6494 86.0 0.2 71643520 349620 pts/8 Rl 06:42 0:26 python tinyverse atari.py play -b 1\r\n",
"root 6495 91.6 0.2 71643856 347504 pts/8 Rl 06:42 0:28 python tinyverse atari.py play -b 1\r\n",
"root 6496 88.1 0.2 71643576 347772 pts/8 Rl 06:42 0:27 python tinyverse atari.py play -b 1\r\n",
"root 6497 85.1 0.2 71643844 348192 pts/8 Rl 06:42 0:26 python tinyverse atari.py play -b 1\r\n",
"root 6498 89.2 0.2 71643500 346368 pts/8 Rl 06:42 0:27 python tinyverse atari.py play -b 1\r\n",
"root 6499 91.4 0.2 71643508 347040 pts/8 Rl 06:42 0:28 python tinyverse atari.py play -b 1\r\n",
"root 11217 0.0 0.0 4448 796 pts/16 Ss+ 06:43 0:00 /bin/sh -c ps aux | grep tinyverse\r\n",
"root 11219 0.0 0.0 10472 2136 pts/16 R+ 06:43 0:00 grep tinyverse\r\n"
]
}
],
"source": [
"!ps aux | grep tinyverse"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# massacre processes\n",
"\n",
"* Will soon make a cleaner version through ```python tinyverse clear```"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"!kill -9 $(ps aux | grep tinyverse | awk '{print $2}') #uncomment to hard kill all\n",
"from tinyverse.database import Database\n",
"db = Database()\n",
"for key in db.redis.keys():\n",
" db.redis.delete(key)"
]
}
],
"metadata": {
"anaconda-cloud": {},
"kernelspec": {
"display_name": "Python [conda root]",
"language": "python",
"name": "conda-root-py"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.2"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment