Skip to content

Instantly share code, notes, and snippets.

@mserranom

mserranom/nyse.ipynb

Last active May 2, 2018 16:51
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save mserranom/4b7c5cff941f3f6831daf4141ab397a7 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save mserranom/4b7c5cff941f3f6831daf4141ab397a7 to your computer and use it in GitHub Desktop.
Download ZIP
nyse.ipynb
Display the source blob
Display the rendered blob
Raw
nyse.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# NYSE & Blurr\n",
"\n",
"In this guide we will train a machine learning model that predicts closing price of a stock based on historical data. We will transform time-series stock data into features to train this model. \n",
"\n",
"## Prerequisites\n",
"\n",
"It's recommended to have a basic understanding of how Blurr works. Following [tutorials 1](http://productml-blurr.readthedocs.io/en/latest/Streaming%20DTC%20Tutorial/) and [2](http://productml-blurr.readthedocs.io/en/latest/Window%20DTC%20Tutorial/) should provide enough background context.\n",
"\n",
"\n",
"## Preparation\n",
"\n",
"Let's start by installing `Blurr` and other required dependencies:"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"installing blurr, keras and tensorflow...\n",
"done.\n"
]
}
],
"source": [
"import sys\n",
"\n",
"print(\"installing blurr, keras and tensorflow...\")\n",
"!{sys.executable} -m pip install blurr-dev --quiet\n",
"!{sys.executable} -m pip install keras --quiet\n",
"!{sys.executable} -m pip install tensorflow --quiet\n",
"print(\"done.\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## The Dataset\n",
"\n",
"This walkthrough is based on [New York Stock Exchange Data](https://www.kaggle.com/dgawlik/nyse/data) made available for [Kaggle challenges](https://www.kaggle.com/dgawlik/nyse).\n",
"\n",
"Let's start by downloading and having a peek at the available data: "
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"--2018-05-02 16:26:07-- http://demo.productml.com/data/prices-split-adjusted.json.zip\n",
"Resolving demo.productml.com (demo.productml.com)... 52.218.160.7\n",
"Connecting to demo.productml.com (demo.productml.com)|52.218.160.7|:80... connected.\n",
"HTTP request sent, awaiting response... 200 OK\n",
"Length: 22162139 (21M) [application/zip]\n",
"Saving to: ‘prices-split-adjusted.json.zip’\n",
"\n",
"prices-split-adjust 100%[===================>] 21.13M 314KB/s in 65s \n",
"\n",
"2018-05-02 16:27:13 (334 KB/s) - ‘prices-split-adjusted.json.zip’ saved [22162139/22162139]\n",
"\n",
"Archive: prices-split-adjusted.json.zip\n",
" inflating: ./prices-split-adjusted.json \n"
]
}
],
"source": [
"!wget http://demo.productml.com/data/prices-split-adjusted.json.zip\n",
"!unzip -o prices-split-adjusted.json.zip -d ."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>close</th>\n",
" <th>date</th>\n",
" <th>high</th>\n",
" <th>low</th>\n",
" <th>open</th>\n",
" <th>symbol</th>\n",
" <th>volume</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>125.839996</td>\n",
" <td>2016-01-05</td>\n",
" <td>126.250000</td>\n",
" <td>122.309998</td>\n",
" <td>123.430000</td>\n",
" <td>WLTW</td>\n",
" <td>2163600</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>119.980003</td>\n",
" <td>2016-01-06</td>\n",
" <td>125.540001</td>\n",
" <td>119.940002</td>\n",
" <td>125.239998</td>\n",
" <td>WLTW</td>\n",
" <td>2386400</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>114.949997</td>\n",
" <td>2016-01-07</td>\n",
" <td>119.739998</td>\n",
" <td>114.930000</td>\n",
" <td>116.379997</td>\n",
" <td>WLTW</td>\n",
" <td>2489500</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>116.620003</td>\n",
" <td>2016-01-08</td>\n",
" <td>117.440002</td>\n",
" <td>113.500000</td>\n",
" <td>115.480003</td>\n",
" <td>WLTW</td>\n",
" <td>2006300</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>114.970001</td>\n",
" <td>2016-01-11</td>\n",
" <td>117.330002</td>\n",
" <td>114.089996</td>\n",
" <td>117.010002</td>\n",
" <td>WLTW</td>\n",
" <td>1408600</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" close date high low open symbol volume\n",
"0 125.839996 2016-01-05 126.250000 122.309998 123.430000 WLTW 2163600\n",
"1 119.980003 2016-01-06 125.540001 119.940002 125.239998 WLTW 2386400\n",
"2 114.949997 2016-01-07 119.739998 114.930000 116.379997 WLTW 2489500\n",
"3 116.620003 2016-01-08 117.440002 113.500000 115.480003 WLTW 2006300\n",
"4 114.970001 2016-01-11 117.330002 114.089996 117.010002 WLTW 1408600"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import pandas as pd\n",
"\n",
"stocks = pd.read_json(\"./prices-split-adjusted.json\", lines=True)\n",
"stocks.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This dataset contains data for each market day.\n",
"\n",
"Our **goal is to predict closing price** of a stock for any given day based on historical data. In order to do that, we need to transform our original data source into **features** that can be used for training.\n",
"\n",
"We'll calculate **moving averages** and other aggregate data for different **time windows**: one, three and seven days.\n",
"\n",
"## Blurr Templates\n",
"\n",
"We perform initial transformation of our data with [nyse-streaming-dtc.yml](./nyse-streaming-dtc.yml). Data is then aggregated by time using [nyse-window-dtc.yml](./nyse-window-dtc.yml)."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Type: 'Blurr:Transform:Streaming'\r\n",
"Version: '2018-03-01'\r\n",
"Description: New York Store Exchange Transformations\r\n",
"Name: nyse\r\n",
"\r\n",
"Import:\r\n",
" - { Module: datetime, Identifiers: [ datetime ] }\r\n",
"\r\n",
"Identity: source.symbol\r\n",
"\r\n",
"Time: datetime.strptime(source.date, '%Y-%m-%d')\r\n",
"\r\n",
"Stores:\r\n",
" - Type: 'Blurr:Store:Memory'\r\n",
" Name: memory\r\n",
"\r\n",
"Aggregates:\r\n",
" - Type: 'Blurr:Aggregate:Block'\r\n",
" Name: stats\r\n",
" Store: memory\r\n",
" Split: True\r\n",
" When: source.symbol in ['AAPL', 'MSFT', 'GOOG', 'FB']\r\n",
" Fields:\r\n",
" - Name: close\r\n",
" Type: float\r\n",
" Value: source.close\r\n",
"\r\n",
" - Name: volatility\r\n",
" Type: float\r\n",
" Value: (float(source.high) / float(source.low)) - 1\r\n",
"\r\n",
" - Name: volume\r\n",
" Type: float\r\n",
" Value: source.volume"
]
}
],
"source": [
"!cat nyse-streaming-dtc.yml"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Streaming DTC**\n",
"\n",
"We're predicting values for tech companies only (Apple, Facebook, Microsoft, Google):\n",
"\n",
"```yaml\n",
"When: source.symbol in ['AAPL', 'MSFT', 'GOOG', 'FB']\n",
"```\n",
"\n",
"Each record in the original dataset represents a single day, which is the same we need to feed our window transformation. By setting `Split: True` we'll create a new aggregate for each single dataset record.\n"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Type: Blurr:Transform:Window\r\n",
"Version: '2018-03-01'\r\n",
"Name: moving_averages\r\n",
"\r\n",
"SourceDTC: nyse\r\n",
"\r\n",
"Anchor:\r\n",
" Condition: nyse.stats.volatility < 0.04\r\n",
"\r\n",
"Aggregates:\r\n",
"\r\n",
"\r\n",
" - Type: Blurr:Aggregate:Window\r\n",
" Name: close\r\n",
" WindowType: count\r\n",
" WindowValue: 1\r\n",
" Source: nyse.stats\r\n",
" Fields:\r\n",
" - Name: value\r\n",
" Type: float\r\n",
" Value: anchor.close\r\n",
"\r\n",
" - Type: Blurr:Aggregate:Window\r\n",
" Name: last\r\n",
" WindowType: count\r\n",
" WindowValue: -1\r\n",
" Source: nyse.stats\r\n",
" Fields:\r\n",
" - Name: close\r\n",
" Type: float\r\n",
" Value: source.close[0]\r\n",
" - Name: volume\r\n",
" Type: float\r\n",
" Value: source.volume[0]\r\n",
" - Name: volatility\r\n",
" Type: float\r\n",
" Value: source.volatility[0]\r\n",
"\r\n",
" - Type: Blurr:Aggregate:Window\r\n",
" Name: last_3\r\n",
" WindowType: count\r\n",
" WindowValue: -3\r\n",
" Source: nyse.stats\r\n",
" Fields:\r\n",
" - Name: close_avg\r\n",
" Type: float\r\n",
" Value: sum(source.close) / len(source.close)\r\n",
" - Name: volume_avg\r\n",
" Type: float\r\n",
" Value: sum(source.volume) / len(source.volume)\r\n",
" - Name: volatility_avg\r\n",
" Type: float\r\n",
" Value: sum(source.volatility) / len(source.volatility)\r\n",
" - Name: max_volatility\r\n",
" Type: float\r\n",
" Value: max(source.volatility)\r\n",
" - Name: min_volatility\r\n",
" Type: float\r\n",
" Value: min(source.volatility)\r\n",
"\r\n",
" - Type: Blurr:Aggregate:Window\r\n",
" Name: last_7\r\n",
" WindowType: count\r\n",
" WindowValue: -7\r\n",
" Source: nyse.stats\r\n",
" Fields:\r\n",
" - Name: close_avg\r\n",
" Type: float\r\n",
" Value: sum(source.close) / len(source.close)\r\n",
" - Name: volume_avg\r\n",
" Type: float\r\n",
" Value: sum(source.volume) / len(source.volume)\r\n",
" - Name: volatility_avg\r\n",
" Type: float\r\n",
" Value: sum(source.volatility) / len(source.volatility)\r\n",
" - Name: max_volatility\r\n",
" Type: float\r\n",
" Value: max(source.volatility)\r\n",
" - Name: min_volatility\r\n",
" Type: float\r\n",
" Value: min(source.volatility)"
]
}
],
"source": [
"!cat 'nyse-window-dtc.yml'"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Window DTC**\n",
"\n",
"We'll use a very rough criteria to remove outliers: our model will only work when closing price changes less than a 4%:\n",
"\n",
"```yaml\n",
"Anchor:\n",
" Condition: nyse.stats.volatility < 0.04\n",
"```\n",
"\n",
"We're using [moving averages](https://www.investopedia.com/terms/m/movingaverage.asp) to generate features based on historical data about a stock:\n",
"\n",
"```yaml\n",
"- Type: Blurr:Aggregate:Window\n",
" Name: last_7\n",
" WindowType: count\n",
" WindowValue: -7\n",
" Source: nyse.stats\n",
" Fields:\n",
" - Name: close_avg\n",
" Type: float\n",
" Value: sum(source.close) / len(source.close)\n",
"```\n",
"\n",
"\n",
"## Transforming Data\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Running syntax validation on nyse-streaming-dtc.yml\n",
"Document is valid\n",
"Running syntax validation on nyse-window-dtc.yml\n",
"Document is valid\n"
]
}
],
"source": [
"from blurr_util import print_head, validate, transform\n",
"\n",
"validate('nyse-streaming-dtc.yml')\n",
"validate('nyse-window-dtc.yml')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's run our Streaming DTC for informational purposes only, so we can preview the result of the transformation:"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"transform(log_files=[\"./prices-split-adjusted.json\"],\n",
" stream_dtc='./nyse-streaming-dtc.yml',\n",
" output_file=\"./nyse-streaming-dtc-out.log\")"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<table><tr><th>_identity<th><th>_start_time<th><th>_end_time<th><th>close<th><th>volatility<th><th>volume<th></tr><tr><td>AAPL<td><td>2010-01-04T00:00:00<td><td>2010-01-04T00:00:00<td><td>30.5728568571<td><td>0.009982083954336085<td><td>123432400.0<td></tr><tr><td>AAPL<td><td>2010-01-05T00:00:00<td><td>2010-01-05T00:00:00<td><td>30.6257132857<td><td>0.010973036651542811<td><td>150476200.0<td></tr><tr><td>AAPL<td><td>2010-01-06T00:00:00<td><td>2010-01-06T00:00:00<td><td>30.1385707143<td><td>0.02125739461533116<td><td>138040000.0<td></tr><tr><td>AAPL<td><td>2010-01-07T00:00:00<td><td>2010-01-07T00:00:00<td><td>30.0828571428<td><td>0.014111461035875461<td><td>119282800.0<td></tr></table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"print_head(\"./nyse-streaming-dtc-out.log\")"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [],
"source": [
"transform(log_files=[\"./prices-split-adjusted.json\"],\n",
" stream_dtc='./nyse-streaming-dtc.yml',\n",
" window_dtc='./nyse-window-dtc.yml',\n",
" output_file=\"./nyse-processed-data.csv\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's now preview the data that will be used to **train our model**"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>close._identity</th>\n",
" <th>close.value</th>\n",
" <th>last._identity</th>\n",
" <th>last.close</th>\n",
" <th>last.volatility</th>\n",
" <th>last.volume</th>\n",
" <th>last_3._identity</th>\n",
" <th>last_3.close_avg</th>\n",
" <th>last_3.max_volatility</th>\n",
" <th>last_3.min_volatility</th>\n",
" <th>last_3.volatility_avg</th>\n",
" <th>last_3.volume_avg</th>\n",
" <th>last_7._identity</th>\n",
" <th>last_7.close_avg</th>\n",
" <th>last_7.max_volatility</th>\n",
" <th>last_7.min_volatility</th>\n",
" <th>last_7.volatility_avg</th>\n",
" <th>last_7.volume_avg</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>AAPL</td>\n",
" <td>30.092857</td>\n",
" <td>AAPL</td>\n",
" <td>29.674286</td>\n",
" <td>0.016229</td>\n",
" <td>148614900.0</td>\n",
" <td>AAPL</td>\n",
" <td>29.990953</td>\n",
" <td>0.021828</td>\n",
" <td>0.014063</td>\n",
" <td>0.017373</td>\n",
" <td>1.253583e+08</td>\n",
" <td>AAPL</td>\n",
" <td>30.198979</td>\n",
" <td>0.021828</td>\n",
" <td>0.009982</td>\n",
" <td>0.015492</td>\n",
" <td>129615200.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>AAPL</td>\n",
" <td>29.918571</td>\n",
" <td>AAPL</td>\n",
" <td>30.092857</td>\n",
" <td>0.033464</td>\n",
" <td>151473000.0</td>\n",
" <td>AAPL</td>\n",
" <td>29.927619</td>\n",
" <td>0.033464</td>\n",
" <td>0.016229</td>\n",
" <td>0.023840</td>\n",
" <td>1.385484e+08</td>\n",
" <td>AAPL</td>\n",
" <td>30.130408</td>\n",
" <td>0.033464</td>\n",
" <td>0.010973</td>\n",
" <td>0.018847</td>\n",
" <td>133621000.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>AAPL</td>\n",
" <td>29.418571</td>\n",
" <td>AAPL</td>\n",
" <td>29.918571</td>\n",
" <td>0.006889</td>\n",
" <td>108223500.0</td>\n",
" <td>AAPL</td>\n",
" <td>29.895238</td>\n",
" <td>0.033464</td>\n",
" <td>0.006889</td>\n",
" <td>0.018861</td>\n",
" <td>1.361038e+08</td>\n",
" <td>AAPL</td>\n",
" <td>30.029388</td>\n",
" <td>0.033464</td>\n",
" <td>0.006889</td>\n",
" <td>0.018263</td>\n",
" <td>127584900.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>AAPL</td>\n",
" <td>30.719999</td>\n",
" <td>AAPL</td>\n",
" <td>29.418571</td>\n",
" <td>0.027833</td>\n",
" <td>148516900.0</td>\n",
" <td>AAPL</td>\n",
" <td>29.810000</td>\n",
" <td>0.033464</td>\n",
" <td>0.006889</td>\n",
" <td>0.022729</td>\n",
" <td>1.360711e+08</td>\n",
" <td>AAPL</td>\n",
" <td>29.926531</td>\n",
" <td>0.033464</td>\n",
" <td>0.006889</td>\n",
" <td>0.019203</td>\n",
" <td>129081600.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>AAPL</td>\n",
" <td>30.247143</td>\n",
" <td>AAPL</td>\n",
" <td>30.719999</td>\n",
" <td>0.038361</td>\n",
" <td>182501900.0</td>\n",
" <td>AAPL</td>\n",
" <td>30.019047</td>\n",
" <td>0.038361</td>\n",
" <td>0.006889</td>\n",
" <td>0.024361</td>\n",
" <td>1.464141e+08</td>\n",
" <td>AAPL</td>\n",
" <td>30.017551</td>\n",
" <td>0.038361</td>\n",
" <td>0.006889</td>\n",
" <td>0.022667</td>\n",
" <td>138112900.0</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" close._identity close.value last._identity last.close last.volatility \\\n",
"0 AAPL 30.092857 AAPL 29.674286 0.016229 \n",
"1 AAPL 29.918571 AAPL 30.092857 0.033464 \n",
"2 AAPL 29.418571 AAPL 29.918571 0.006889 \n",
"3 AAPL 30.719999 AAPL 29.418571 0.027833 \n",
"4 AAPL 30.247143 AAPL 30.719999 0.038361 \n",
"\n",
" last.volume last_3._identity last_3.close_avg last_3.max_volatility \\\n",
"0 148614900.0 AAPL 29.990953 0.021828 \n",
"1 151473000.0 AAPL 29.927619 0.033464 \n",
"2 108223500.0 AAPL 29.895238 0.033464 \n",
"3 148516900.0 AAPL 29.810000 0.033464 \n",
"4 182501900.0 AAPL 30.019047 0.038361 \n",
"\n",
" last_3.min_volatility last_3.volatility_avg last_3.volume_avg \\\n",
"0 0.014063 0.017373 1.253583e+08 \n",
"1 0.016229 0.023840 1.385484e+08 \n",
"2 0.006889 0.018861 1.361038e+08 \n",
"3 0.006889 0.022729 1.360711e+08 \n",
"4 0.006889 0.024361 1.464141e+08 \n",
"\n",
" last_7._identity last_7.close_avg last_7.max_volatility \\\n",
"0 AAPL 30.198979 0.021828 \n",
"1 AAPL 30.130408 0.033464 \n",
"2 AAPL 30.029388 0.033464 \n",
"3 AAPL 29.926531 0.033464 \n",
"4 AAPL 30.017551 0.038361 \n",
"\n",
" last_7.min_volatility last_7.volatility_avg last_7.volume_avg \n",
"0 0.009982 0.015492 129615200.0 \n",
"1 0.010973 0.018847 133621000.0 \n",
"2 0.006889 0.018263 127584900.0 \n",
"3 0.006889 0.019203 129081600.0 \n",
"4 0.006889 0.022667 138112900.0 "
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"window_out = pd.read_csv(\"./nyse-processed-data.csv\")\n",
"window_out.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Modelling\n",
"\n",
"**Blurr** is about Data Preparation and Feature Engineering. Modeling is included here for illustration purpose, and the reader can use any modeling library or tool for such purpose.\n",
"\n",
"Let's start by importing the output of our Window DTC as the source dataset. We're dropping unnecessary `_identity` columns:"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>last.close</th>\n",
" <th>last.volatility</th>\n",
" <th>last.volume</th>\n",
" <th>last_3.close_avg</th>\n",
" <th>last_3.max_volatility</th>\n",
" <th>last_3.min_volatility</th>\n",
" <th>last_3.volatility_avg</th>\n",
" <th>last_3.volume_avg</th>\n",
" <th>last_7.close_avg</th>\n",
" <th>last_7.max_volatility</th>\n",
" <th>last_7.min_volatility</th>\n",
" <th>last_7.volatility_avg</th>\n",
" <th>last_7.volume_avg</th>\n",
" <th>close</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>29.674286</td>\n",
" <td>0.016229</td>\n",
" <td>148614900.0</td>\n",
" <td>29.990953</td>\n",
" <td>0.021828</td>\n",
" <td>0.014063</td>\n",
" <td>0.017373</td>\n",
" <td>1.253583e+08</td>\n",
" <td>30.198979</td>\n",
" <td>0.021828</td>\n",
" <td>0.009982</td>\n",
" <td>0.015492</td>\n",
" <td>129615200.0</td>\n",
" <td>30.092857</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>30.092857</td>\n",
" <td>0.033464</td>\n",
" <td>151473000.0</td>\n",
" <td>29.927619</td>\n",
" <td>0.033464</td>\n",
" <td>0.016229</td>\n",
" <td>0.023840</td>\n",
" <td>1.385484e+08</td>\n",
" <td>30.130408</td>\n",
" <td>0.033464</td>\n",
" <td>0.010973</td>\n",
" <td>0.018847</td>\n",
" <td>133621000.0</td>\n",
" <td>29.918571</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>29.918571</td>\n",
" <td>0.006889</td>\n",
" <td>108223500.0</td>\n",
" <td>29.895238</td>\n",
" <td>0.033464</td>\n",
" <td>0.006889</td>\n",
" <td>0.018861</td>\n",
" <td>1.361038e+08</td>\n",
" <td>30.029388</td>\n",
" <td>0.033464</td>\n",
" <td>0.006889</td>\n",
" <td>0.018263</td>\n",
" <td>127584900.0</td>\n",
" <td>29.418571</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>29.418571</td>\n",
" <td>0.027833</td>\n",
" <td>148516900.0</td>\n",
" <td>29.810000</td>\n",
" <td>0.033464</td>\n",
" <td>0.006889</td>\n",
" <td>0.022729</td>\n",
" <td>1.360711e+08</td>\n",
" <td>29.926531</td>\n",
" <td>0.033464</td>\n",
" <td>0.006889</td>\n",
" <td>0.019203</td>\n",
" <td>129081600.0</td>\n",
" <td>30.719999</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>30.719999</td>\n",
" <td>0.038361</td>\n",
" <td>182501900.0</td>\n",
" <td>30.019047</td>\n",
" <td>0.038361</td>\n",
" <td>0.006889</td>\n",
" <td>0.024361</td>\n",
" <td>1.464141e+08</td>\n",
" <td>30.017551</td>\n",
" <td>0.038361</td>\n",
" <td>0.006889</td>\n",
" <td>0.022667</td>\n",
" <td>138112900.0</td>\n",
" <td>30.247143</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" last.close last.volatility last.volume last_3.close_avg \\\n",
"0 29.674286 0.016229 148614900.0 29.990953 \n",
"1 30.092857 0.033464 151473000.0 29.927619 \n",
"2 29.918571 0.006889 108223500.0 29.895238 \n",
"3 29.418571 0.027833 148516900.0 29.810000 \n",
"4 30.719999 0.038361 182501900.0 30.019047 \n",
"\n",
" last_3.max_volatility last_3.min_volatility last_3.volatility_avg \\\n",
"0 0.021828 0.014063 0.017373 \n",
"1 0.033464 0.016229 0.023840 \n",
"2 0.033464 0.006889 0.018861 \n",
"3 0.033464 0.006889 0.022729 \n",
"4 0.038361 0.006889 0.024361 \n",
"\n",
" last_3.volume_avg last_7.close_avg last_7.max_volatility \\\n",
"0 1.253583e+08 30.198979 0.021828 \n",
"1 1.385484e+08 30.130408 0.033464 \n",
"2 1.361038e+08 30.029388 0.033464 \n",
"3 1.360711e+08 29.926531 0.033464 \n",
"4 1.464141e+08 30.017551 0.038361 \n",
"\n",
" last_7.min_volatility last_7.volatility_avg last_7.volume_avg close \n",
"0 0.009982 0.015492 129615200.0 30.092857 \n",
"1 0.010973 0.018847 133621000.0 29.918571 \n",
"2 0.006889 0.018263 127584900.0 29.418571 \n",
"3 0.006889 0.019203 129081600.0 30.719999 \n",
"4 0.006889 0.022667 138112900.0 30.247143 "
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"import pandas as pd\n",
"\n",
"def import_dataset():\n",
" data = pd.read_csv(\"./nyse-processed-data.csv\")\n",
" data[\"close\"] = data[\"close.value\"] # Moving close to the last column\n",
" data.drop(['close.value'], 1, inplace=True) \n",
" data.drop(['close._identity'], 1, inplace=True) \n",
" data.drop(['last._identity'], 1, inplace=True) \n",
" data.drop(['last_3._identity'], 1, inplace=True) \n",
" data.drop(['last_7._identity'], 1, inplace=True) \n",
" return data\n",
"\n",
"dataset = import_dataset()\n",
"dataset.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Each column represents a Feature, except the rightmost column which represents the Output we're trying to predic"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"#features=13\n"
]
}
],
"source": [
"feature_count = len(dataset.columns) - 1\n",
"print(\"#features=\" + str(feature_count))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We're splitting our dataset into Input Variables (`X`) and the Output Variable (`Y`) using pandas' [`iloc` function](http://pandas.pydata.org/pandas-docs/version/0.17.0/generated/pandas.DataFrame.iloc.html):"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(5978, 13)\n"
]
}
],
"source": [
"X = dataset.iloc[:, 0:feature_count].values\n",
"print(X.shape)"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(5978,)\n"
]
}
],
"source": [
"Y = dataset.iloc[:, feature_count].values\n",
"print(Y.shape)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We need to split between train and test datasets for training and evaluation of the model:"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [],
"source": [
"from sklearn.model_selection import train_test_split\n",
"\n",
"X_train_raw, X_test_raw, Y_train_raw, Y_test_raw = train_test_split(X, Y, test_size = 0.2)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Finally, we need to scale our data before training:"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/opt/conda/lib/python3.6/site-packages/sklearn/preprocessing/data.py:321: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.\n",
" warnings.warn(DEPRECATION_MSG_1D, DeprecationWarning)\n",
"/opt/conda/lib/python3.6/site-packages/sklearn/preprocessing/data.py:356: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.\n",
" warnings.warn(DEPRECATION_MSG_1D, DeprecationWarning)\n",
"/opt/conda/lib/python3.6/site-packages/sklearn/preprocessing/data.py:356: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.\n",
" warnings.warn(DEPRECATION_MSG_1D, DeprecationWarning)\n"
]
}
],
"source": [
"from sklearn.preprocessing import MinMaxScaler\n",
"\n",
"scaler = MinMaxScaler()\n",
"X_train = scaler.fit_transform(X_train_raw)\n",
"X_test = scaler.transform(X_test_raw)\n",
"Y_train = scaler.fit_transform(Y_train_raw)\n",
"Y_test = scaler.transform(Y_test_raw)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"It's now time to build and train our model:"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Using TensorFlow backend.\n"
]
}
],
"source": [
"# Importing the Keras libraries and packages\n",
"\n",
"import keras\n",
"from keras.models import Sequential\n",
"from keras.layers import Dense"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Train on 4303 samples, validate on 479 samples\n",
"Epoch 1/70\n",
"4303/4303 [==============================] - 0s 88us/step - loss: 0.1013 - acc: 2.3240e-04 - val_loss: 0.0889 - val_acc: 0.0000e+00\n",
"Epoch 2/70\n",
"4303/4303 [==============================] - 0s 7us/step - loss: 0.0939 - acc: 2.3240e-04 - val_loss: 0.0803 - val_acc: 0.0000e+00\n",
"Epoch 3/70\n",
"4303/4303 [==============================] - 0s 9us/step - loss: 0.0825 - acc: 2.3240e-04 - val_loss: 0.0678 - val_acc: 0.0000e+00\n",
"Epoch 4/70\n",
"4303/4303 [==============================] - 0s 8us/step - loss: 0.0669 - acc: 2.3240e-04 - val_loss: 0.0535 - val_acc: 0.0000e+00\n",
"Epoch 5/70\n",
"4303/4303 [==============================] - 0s 8us/step - loss: 0.0518 - acc: 2.3240e-04 - val_loss: 0.0443 - val_acc: 0.0000e+00\n",
"Epoch 6/70\n",
"4303/4303 [==============================] - 0s 8us/step - loss: 0.0414 - acc: 2.3240e-04 - val_loss: 0.0338 - val_acc: 0.0000e+00\n",
"Epoch 7/70\n",
"4303/4303 [==============================] - 0s 17us/step - loss: 0.0287 - acc: 2.3240e-04 - val_loss: 0.0204 - val_acc: 0.0000e+00\n",
"Epoch 8/70\n",
"4303/4303 [==============================] - 0s 8us/step - loss: 0.0154 - acc: 4.6479e-04 - val_loss: 0.0083 - val_acc: 0.0000e+00\n",
"Epoch 9/70\n",
"4303/4303 [==============================] - 0s 9us/step - loss: 0.0049 - acc: 4.6479e-04 - val_loss: 0.0017 - val_acc: 0.0000e+00\n",
"Epoch 10/70\n",
"4303/4303 [==============================] - 0s 10us/step - loss: 0.0010 - acc: 4.6479e-04 - val_loss: 7.4279e-04 - val_acc: 0.0000e+00\n",
"Epoch 11/70\n",
"4303/4303 [==============================] - 0s 10us/step - loss: 7.9136e-04 - acc: 4.6479e-04 - val_loss: 8.5862e-04 - val_acc: 0.0000e+00\n",
"Epoch 12/70\n",
"4303/4303 [==============================] - 0s 9us/step - loss: 8.6798e-04 - acc: 4.6479e-04 - val_loss: 7.8711e-04 - val_acc: 0.0000e+00\n",
"Epoch 13/70\n",
"4303/4303 [==============================] - 0s 10us/step - loss: 7.2163e-04 - acc: 4.6479e-04 - val_loss: 6.3454e-04 - val_acc: 0.0000e+00\n",
"Epoch 14/70\n",
"4303/4303 [==============================] - 0s 9us/step - loss: 5.7548e-04 - acc: 4.6479e-04 - val_loss: 5.5086e-04 - val_acc: 0.0000e+00\n",
"Epoch 15/70\n",
"4303/4303 [==============================] - 0s 7us/step - loss: 5.2260e-04 - acc: 4.6479e-04 - val_loss: 5.2969e-04 - val_acc: 0.0000e+00\n",
"Epoch 16/70\n",
"4303/4303 [==============================] - 0s 9us/step - loss: 5.0104e-04 - acc: 4.6479e-04 - val_loss: 4.9973e-04 - val_acc: 0.0000e+00\n",
"Epoch 17/70\n",
"4303/4303 [==============================] - 0s 7us/step - loss: 4.7037e-04 - acc: 4.6479e-04 - val_loss: 4.6564e-04 - val_acc: 0.0000e+00\n",
"Epoch 18/70\n",
"4303/4303 [==============================] - 0s 8us/step - loss: 4.4491e-04 - acc: 4.6479e-04 - val_loss: 4.4358e-04 - val_acc: 0.0000e+00\n",
"Epoch 19/70\n",
"4303/4303 [==============================] - 0s 9us/step - loss: 4.2526e-04 - acc: 4.6479e-04 - val_loss: 4.2346e-04 - val_acc: 0.0000e+00\n",
"Epoch 20/70\n",
"4303/4303 [==============================] - 0s 9us/step - loss: 4.0683e-04 - acc: 4.6479e-04 - val_loss: 4.0460e-04 - val_acc: 0.0000e+00\n",
"Epoch 21/70\n",
"4303/4303 [==============================] - 0s 7us/step - loss: 3.8903e-04 - acc: 4.6479e-04 - val_loss: 3.8761e-04 - val_acc: 0.0000e+00\n",
"Epoch 22/70\n",
"4303/4303 [==============================] - 0s 8us/step - loss: 3.7150e-04 - acc: 4.6479e-04 - val_loss: 3.6949e-04 - val_acc: 0.0000e+00\n",
"Epoch 23/70\n",
"4303/4303 [==============================] - 0s 10us/step - loss: 3.5385e-04 - acc: 4.6479e-04 - val_loss: 3.5096e-04 - val_acc: 0.0000e+00\n",
"Epoch 24/70\n",
"4303/4303 [==============================] - 0s 7us/step - loss: 3.3511e-04 - acc: 4.6479e-04 - val_loss: 3.3146e-04 - val_acc: 0.0000e+00\n",
"Epoch 25/70\n",
"4303/4303 [==============================] - 0s 8us/step - loss: 3.1418e-04 - acc: 4.6479e-04 - val_loss: 3.0959e-04 - val_acc: 0.0000e+00\n",
"Epoch 26/70\n",
"4303/4303 [==============================] - 0s 8us/step - loss: 2.9245e-04 - acc: 4.6479e-04 - val_loss: 2.8844e-04 - val_acc: 0.0000e+00\n",
"Epoch 27/70\n",
"4303/4303 [==============================] - 0s 8us/step - loss: 2.6991e-04 - acc: 4.6479e-04 - val_loss: 2.6561e-04 - val_acc: 0.0000e+00\n",
"Epoch 28/70\n",
"4303/4303 [==============================] - 0s 8us/step - loss: 2.4877e-04 - acc: 4.6479e-04 - val_loss: 2.4738e-04 - val_acc: 0.0000e+00\n",
"Epoch 29/70\n",
"4303/4303 [==============================] - 0s 8us/step - loss: 2.2982e-04 - acc: 4.6479e-04 - val_loss: 2.2889e-04 - val_acc: 0.0000e+00\n",
"Epoch 30/70\n",
"4303/4303 [==============================] - 0s 8us/step - loss: 2.1222e-04 - acc: 4.6479e-04 - val_loss: 2.1304e-04 - val_acc: 0.0000e+00\n",
"Epoch 31/70\n",
"4303/4303 [==============================] - 0s 7us/step - loss: 1.9655e-04 - acc: 4.6479e-04 - val_loss: 1.9797e-04 - val_acc: 0.0000e+00\n",
"Epoch 32/70\n",
"4303/4303 [==============================] - 0s 8us/step - loss: 1.8246e-04 - acc: 4.6479e-04 - val_loss: 1.8552e-04 - val_acc: 0.0000e+00\n",
"Epoch 33/70\n",
"4303/4303 [==============================] - 0s 9us/step - loss: 1.6935e-04 - acc: 4.6479e-04 - val_loss: 1.7353e-04 - val_acc: 0.0000e+00\n",
"Epoch 34/70\n",
"4303/4303 [==============================] - 0s 7us/step - loss: 1.5819e-04 - acc: 4.6479e-04 - val_loss: 1.6356e-04 - val_acc: 0.0000e+00\n",
"Epoch 35/70\n",
"4303/4303 [==============================] - 0s 7us/step - loss: 1.4714e-04 - acc: 4.6479e-04 - val_loss: 1.5206e-04 - val_acc: 0.0000e+00\n",
"Epoch 36/70\n",
"4303/4303 [==============================] - 0s 8us/step - loss: 1.3734e-04 - acc: 4.6479e-04 - val_loss: 1.4514e-04 - val_acc: 0.0000e+00\n",
"Epoch 37/70\n",
"4303/4303 [==============================] - 0s 8us/step - loss: 1.2823e-04 - acc: 4.6479e-04 - val_loss: 1.3507e-04 - val_acc: 0.0000e+00\n",
"Epoch 38/70\n",
"4303/4303 [==============================] - 0s 7us/step - loss: 1.1964e-04 - acc: 4.6479e-04 - val_loss: 1.2785e-04 - val_acc: 0.0000e+00\n",
"Epoch 39/70\n",
"4303/4303 [==============================] - 0s 6us/step - loss: 1.1129e-04 - acc: 4.6479e-04 - val_loss: 1.1993e-04 - val_acc: 0.0000e+00\n",
"Epoch 40/70\n",
"4303/4303 [==============================] - 0s 7us/step - loss: 1.0401e-04 - acc: 4.6479e-04 - val_loss: 1.1278e-04 - val_acc: 0.0000e+00\n",
"Epoch 41/70\n",
"4303/4303 [==============================] - 0s 9us/step - loss: 9.7596e-05 - acc: 4.6479e-04 - val_loss: 1.0628e-04 - val_acc: 0.0000e+00\n",
"Epoch 42/70\n",
"4303/4303 [==============================] - 0s 7us/step - loss: 9.1281e-05 - acc: 4.6479e-04 - val_loss: 1.0266e-04 - val_acc: 0.0000e+00\n",
"Epoch 43/70\n",
"4303/4303 [==============================] - 0s 7us/step - loss: 8.5867e-05 - acc: 4.6479e-04 - val_loss: 9.4637e-05 - val_acc: 0.0000e+00\n",
"Epoch 44/70\n",
"4303/4303 [==============================] - 0s 9us/step - loss: 8.0901e-05 - acc: 4.6479e-04 - val_loss: 9.0048e-05 - val_acc: 0.0000e+00\n",
"Epoch 45/70\n",
"4303/4303 [==============================] - 0s 8us/step - loss: 7.6543e-05 - acc: 4.6479e-04 - val_loss: 8.8481e-05 - val_acc: 0.0000e+00\n",
"Epoch 46/70\n",
"4303/4303 [==============================] - 0s 6us/step - loss: 7.2598e-05 - acc: 4.6479e-04 - val_loss: 8.1160e-05 - val_acc: 0.0000e+00\n",
"Epoch 47/70\n",
"4303/4303 [==============================] - 0s 9us/step - loss: 6.8590e-05 - acc: 4.6479e-04 - val_loss: 7.7573e-05 - val_acc: 0.0000e+00\n",
"Epoch 48/70\n",
"4303/4303 [==============================] - 0s 10us/step - loss: 6.5812e-05 - acc: 4.6479e-04 - val_loss: 7.6484e-05 - val_acc: 0.0000e+00\n",
"Epoch 49/70\n",
"4303/4303 [==============================] - 0s 8us/step - loss: 6.2704e-05 - acc: 4.6479e-04 - val_loss: 7.0473e-05 - val_acc: 0.0000e+00\n",
"Epoch 50/70\n",
"4303/4303 [==============================] - 0s 10us/step - loss: 5.9942e-05 - acc: 4.6479e-04 - val_loss: 6.9676e-05 - val_acc: 0.0000e+00\n",
"Epoch 51/70\n",
"4303/4303 [==============================] - 0s 9us/step - loss: 5.8576e-05 - acc: 4.6479e-04 - val_loss: 6.5183e-05 - val_acc: 0.0000e+00\n",
"Epoch 52/70\n",
"4303/4303 [==============================] - 0s 10us/step - loss: 5.6660e-05 - acc: 4.6479e-04 - val_loss: 6.3196e-05 - val_acc: 0.0000e+00\n",
"Epoch 53/70\n",
"4303/4303 [==============================] - 0s 7us/step - loss: 5.5070e-05 - acc: 4.6479e-04 - val_loss: 6.6809e-05 - val_acc: 0.0000e+00\n",
"Epoch 54/70\n",
"4303/4303 [==============================] - 0s 8us/step - loss: 5.3106e-05 - acc: 4.6479e-04 - val_loss: 5.9363e-05 - val_acc: 0.0000e+00\n",
"Epoch 55/70\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"4303/4303 [==============================] - 0s 7us/step - loss: 5.1329e-05 - acc: 4.6479e-04 - val_loss: 5.8611e-05 - val_acc: 0.0000e+00\n",
"Epoch 56/70\n",
"4303/4303 [==============================] - 0s 8us/step - loss: 4.9965e-05 - acc: 4.6479e-04 - val_loss: 5.7405e-05 - val_acc: 0.0000e+00\n",
"Epoch 57/70\n",
"4303/4303 [==============================] - 0s 7us/step - loss: 4.8528e-05 - acc: 4.6479e-04 - val_loss: 5.6071e-05 - val_acc: 0.0000e+00\n",
"Epoch 58/70\n",
"4303/4303 [==============================] - 0s 9us/step - loss: 4.7505e-05 - acc: 4.6479e-04 - val_loss: 5.3840e-05 - val_acc: 0.0000e+00\n",
"Epoch 59/70\n",
"4303/4303 [==============================] - 0s 8us/step - loss: 4.6505e-05 - acc: 4.6479e-04 - val_loss: 5.3323e-05 - val_acc: 0.0000e+00\n",
"Epoch 60/70\n",
"4303/4303 [==============================] - 0s 8us/step - loss: 4.5437e-05 - acc: 4.6479e-04 - val_loss: 5.1991e-05 - val_acc: 0.0000e+00\n",
"Epoch 61/70\n",
"4303/4303 [==============================] - 0s 9us/step - loss: 4.4684e-05 - acc: 4.6479e-04 - val_loss: 5.0997e-05 - val_acc: 0.0000e+00\n",
"Epoch 62/70\n",
"4303/4303 [==============================] - 0s 10us/step - loss: 4.3866e-05 - acc: 4.6479e-04 - val_loss: 4.9680e-05 - val_acc: 0.0000e+00\n",
"Epoch 63/70\n",
"4303/4303 [==============================] - 0s 9us/step - loss: 4.2916e-05 - acc: 4.6479e-04 - val_loss: 4.8643e-05 - val_acc: 0.0000e+00\n",
"Epoch 64/70\n",
"4303/4303 [==============================] - 0s 8us/step - loss: 4.2157e-05 - acc: 4.6479e-04 - val_loss: 4.8342e-05 - val_acc: 0.0000e+00\n",
"Epoch 65/70\n",
"4303/4303 [==============================] - 0s 8us/step - loss: 4.1420e-05 - acc: 4.6479e-04 - val_loss: 4.6409e-05 - val_acc: 0.0000e+00\n",
"Epoch 66/70\n",
"4303/4303 [==============================] - 0s 7us/step - loss: 4.1111e-05 - acc: 4.6479e-04 - val_loss: 4.5707e-05 - val_acc: 0.0000e+00\n",
"Epoch 67/70\n",
"4303/4303 [==============================] - 0s 5us/step - loss: 4.0508e-05 - acc: 4.6479e-04 - val_loss: 4.4937e-05 - val_acc: 0.0000e+00\n",
"Epoch 68/70\n",
"4303/4303 [==============================] - 0s 6us/step - loss: 3.9723e-05 - acc: 4.6479e-04 - val_loss: 4.5177e-05 - val_acc: 0.0000e+00\n",
"Epoch 69/70\n",
"4303/4303 [==============================] - 0s 10us/step - loss: 3.9179e-05 - acc: 4.6479e-04 - val_loss: 4.4589e-05 - val_acc: 0.0000e+00\n",
"Epoch 70/70\n",
"4303/4303 [==============================] - 0s 8us/step - loss: 3.8605e-05 - acc: 4.6479e-04 - val_loss: 4.3353e-05 - val_acc: 0.0000e+00\n"
]
},
{
"data": {
"text/plain": [
"<keras.callbacks.History at 0x7f9686209470>"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"#Initializing Neural Network\n",
"\n",
"model = Sequential()\n",
"model.add(Dense(units = 36, kernel_initializer = 'uniform', activation = 'relu', input_dim = feature_count))\n",
"model.add(Dense(units = 36, kernel_initializer = 'uniform', activation = 'relu'))\n",
"model.add(Dense(units = 1, kernel_initializer = 'uniform', activation = 'linear'))\n",
"\n",
"# Compiling Neural Network\n",
"model.compile(loss='mse',optimizer='adam', metrics=['accuracy'])\n",
"\n",
"# Fitting our model \n",
"model.fit(X_train, Y_train, batch_size = 512, epochs = 70, validation_split=0.1, verbose=1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can measure the quality of our model using [MSE](https://en.wikipedia.org/wiki/Mean_squared_error) and [RMSE](https://en.wikipedia.org/wiki/Root-mean-square_deviation):"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Model Score: 0.00004 MSE (0.01 RMSE)\n"
]
}
],
"source": [
"import math\n",
"\n",
"score = model.evaluate(X_test, Y_test, verbose=0)\n",
"print('Model Score: %.5f MSE (%.2f RMSE)' % (score[0], math.sqrt(score[0])))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Finally, let's plot prediction vs actual data.\n",
"\n",
"Prior to normalisation, we undo scaling and perform a sort for graph quality:"
]
},
{
"cell_type": "code",
"execution_count": 47,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEKCAYAAAAIO8L1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8VOW9x/HPLzsJkJAQEBI2BVkKEjAqAi4VV2rBjSveVpHS0k2t9VbF2tbb3i5qrVZrtUVR6K3FWhSlvWqrorWiYgNSQdawhyWEQAIh++S5f5wTGiDAAJmcGfJ9v17zmnOe88zM78lJ5pfnPOc8x5xziIiIHCwu6ABERCQ6KUGIiEizlCBERKRZShAiItIsJQgREWmWEoSIiDRLCUJERJqlBCEiIs1SghARkWYlBB3AiejcubPr3bt30GGIiMSURYsW7XTOZR+tXkwniN69e1NQUBB0GCIiMcXMNoZTT4eYRESkWUoQIiLSLCUIERFpVkyPQTSnrq6OoqIiqqurgw7lpJGSkkJubi6JiYlBhyIireikSxBFRUV06NCB3r17Y2ZBhxPznHOUlpZSVFREnz59gg5HRFpRRA8xmdm3zexTM1tmZrPNLMXM+pjZQjNbY2Z/NLMkv26yv17ob+99PJ9ZXV1NVlaWkkMLMTOysrLUIxNpgyKWIMwsB7gNyHfODQbigYnAA8Ajzrl+wG5giv+SKcBu51xf4BG/3vF+9omELgfRz1OkbYr0IHUC0M7MEoBUYBtwETDH3z4LuMpfHu+v428fY/pmEhE5UEMD9XfcBevWRfyjIpYgnHNbgIeATXiJoRxYBJQ55+r9akVAjr+cA2z2X1vv18+KVHyRFB8fT15eHoMHD2bChAlUVlYe93u98847XHnllQDMmzeP+++//7B1y8rKeOKJJ/avb926leuuu+64P1tEos/u515l8CNfYs6DMZwgzKwTXq+gD9AdSAOuaKaqa3zJEbY1fd+pZlZgZgUlJSUtFW6LateuHUuWLGHZsmUkJSXxm9/85oDtzjkaGhqO+X3HjRvHtGnTDrv94ATRvXt35syZc9j6IhJ7ptyXSyF96TP5woh/ViQPMV0MrHfOlTjn6oCXgJFAhn/ICSAX2OovFwE9APzt6cCug9/UOTfdOZfvnMvPzj7qVCKBO++88ygsLGTDhg0MHDiQb3zjGwwfPpzNmzfzt7/9jXPPPZfhw4czYcIEKioqAHj99dcZMGAAo0eP5qWXXtr/XjNnzuSWW24BoLi4mKuvvpqhQ4cydOhQ3n//faZNm8batWvJy8vjzjvvZMOGDQwePBjwBu8nT57MkCFDGDZsGG+//fb+97zmmmu4/PLL6devH3fddVcr/4REJFxFq/Yxd30e/3X6nznznMifhBrJT9gEjDCzVKAKGAMUAG8D1wHPA5OAV/z68/z1D/zt851zh/Qgjsntt8OSJSf0FofIy4Nf/jKsqvX19bz22mtcfvnlAKxatYpnn32WJ554gp07d/LjH/+YN998k7S0NB544AEefvhh7rrrLr7yla8wf/58+vbty/XXX9/se992221ccMEFzJ07l1AoREVFBffffz/Lli1jid/mDRs27K//61//GoClS5eycuVKLr30UlavXg3AkiVL+Pjjj0lOTqZ///7ceuut9OjR43h/QiISIc/duQQYxZTvdm2Vz4vkGMRCvMHmxcBS/7OmA3cDd5hZId4Ywwz/JTOALL/8DuDwx1KiXFVVFXl5eeTn59OzZ0+mTPFO1OrVqxcjRowA4MMPP2T58uWMGjWKvLw8Zs2axcaNG1m5ciV9+vShX79+mBlf/OIXm/2M+fPn8/Wvfx3wxjzS09OPGNN7773HjTfeCMCAAQPo1avX/gQxZswY0tPTSUlJYdCgQWzcGNY8XiLSSkpK4HOfg2l/HsU5SYs5fdLIVvnciPZRnHP3AfcdVLwOOLuZutXAhBYNIMz/9Fta4xjEwdLS0vYvO+e45JJLmD179gF1lixZEpHTSo/UGUtOTt6/HB8fT319/WHrikjre/T+Kl5/LZkf832+Om4X8GSrfK7mYgrIiBEjWLBgAYWFhQBUVlayevVqBgwYwPr161m7di3AIQmk0ZgxY3jySe+XJBQKsWfPHjp06MDevXubrX/++efz3HPPAbB69Wo2bdpE//79W7pZItLCKstqeeKxei5w73Dvl7bT+fH/brXPVoIISHZ2NjNnzuSGG27gjDPOYMSIEaxcuZKUlBSmT5/O5z73OUaPHk2vXr2aff2jjz7K22+/zZAhQzjzzDP59NNPycrKYtSoUQwePJg777zzgPrf+MY3CIVCDBkyhOuvv56ZM2ce0HMQkej00BVvsbu+Az/4ThXMmAFdW2f8AcBOdBw4SPn5+e7gGwatWLGCgQMHBhTRyUs/V5HWt+/p2bT/yg0MS1/Lot2n0VJHn81skXMu/2j11IMQEYlG69fz8FdWAPCFycktlhyOhRKEiEgU+ujRD/gBPwLgmz/LDSQGJQgRkSjjdpdx/a9GAfDWW5CSEkwcShAiIlFm1tQFbGjoxV1f2MJFFwUXhxKEiEgUcZVV/M+8MxiYVMhPZ+Uc/QURpAQhIhJFCu6dy7raHtx5YzHx8cHGogQRIXPnzsXMWLly5RHrzZw5k61btx6xzpE0nQ5cRGLcnj088psUEq2Oqx8aFXQ0ShCRMnv2bEaPHs3zzz9/xHonmiBE5OQQCsGtw95jdvU1XHNBKRkZQUekBBERFRUVLFiwgBkzZhyQIB588EGGDBnC0KFDmTZtGnPmzKGgoIAvfOEL5OXlUVVVRe/evdm5cycABQUFXHjhhQB89NFHjBw5kmHDhjFy5EhWrVoVRNNEJEJ+9qXVPL5uLGNyV/HkS6cEHQ4Q4cn6ghbUbN8vv/wyl19+OaeffjqZmZksXryY4uJiXn75ZRYuXEhqaiq7du0iMzOTxx9/nIceeoj8/CNf1DhgwADeffddEhISePPNN/nud7/Liy++2IItE5Gg1L78Kj/73QWMT3uTuR8NxjoFHZHnpE4QQZk9eza33347ABMnTmT27Nk0NDQwefJkUlNTAcjMzDym9ywvL2fSpEmsWbMGM6Ourq7F4xaRANTWcveXS6kkjS8/nod16xx0RPud1AkiiNm+S0tLmT9/PsuWLcPMCIVCmBnXXnttWNN4JyQk7L8daXV19f7y73//+3z2s59l7ty5bNiwYf+hJxGJbcvvfY5nSq+hf/c9jL0pepIDaAyixc2ZM4ebbrqJjRs3smHDBjZv3kyfPn3IzMzkmWeeobKyEoBdu7y7qR48RXfv3r1ZtGgRwAGHkMrLy8nJ8c6JnjlzZiu1RkQiqbISxv9qDInxDfz5nY7ERdk3csTCMbP+ZrakyWOPmd1uZplm9oaZrfGfO/n1zcweM7NCM/vEzIZHKrZImj17NldfffUBZddeey1bt25l3Lhx5Ofnk5eXx0MPPQTAzTffzNe+9rX9g9T33Xcf3/rWtzjvvPOIb3IS9F133cU999zDqFGjCIVCrdomEYmM2d9ZRGFNT+4f9gL9+gUdzaFaZbpvM4sHtgDnAN8Edjnn7jezaUAn59zdZjYWuBUY69d71Dl3zpHeV9N9tx79XEVa1tYNtfQ+1TjbCpi/YzBJWR1a7bOjbbrvMcBa59xGYDwwyy+fBVzlL48Hfuc8HwIZZtatleITEWlVr17+GHUukd/csqxVk8OxaK0EMRFovHdmV+fcNgD/uYtfngNsbvKaIr9MROTksmABr63qQ25qKZ955MtBR3NYEU8QZpYEjAP+dLSqzZQdcvzLzKaaWYGZFZSUlDT7RrF8l7xopJ+nSMuq/d3zvMnFXDGhAxYXwJ2AwtQaPYgrgMXOuWJ/vbjx0JH/vMMvLwJ6NHldLnDIHBTOuenOuXznXH52dvYhH5aSkkJpaam+1FqIc47S0lJSgpqQXuQk9MriHuwhnauvTwo6lCNqjesgbuDfh5cA5gGTgPv951ealN9iZs/jDVKXNx6KOha5ubkUFRVxuN6FHLuUlBRyc4O5o5XIyaiw1Jto6YILAg7kKCKaIMwsFbgE+GqT4vuBF8xsCrAJmOCXv4p3BlMhUAlMPp7PTExMpE+fPscds4hIpH2yuwcZiRWkprYPOpQjimiCcM5VAlkHlZXindV0cF2HdwqsiMjJq6GBt/ecyZDsLUD/oKM5oii7bk9E5ORWv2Y9Oxo689n8iqBDOSolCBGRVlSyuRpHHKfkBHy7uDAoQYiItKLtRfUAnNI1+s+0VIIQEWlFxdu82Zq7nhK91z80UoIQEWlFH3zi3RNGh5hEROQAT73ek3N5nz79ov92PEoQIiKtxDnYuTeJC/g7ltou6HCOSglCRKSVVFVBXSiejLR6yIn+uUiVIEREWkl5ufec3j0NEhODDSYMShAiIq3ktZeqAOgWI3e6UYIQEWkFDZXVPPWdlXRlO1fe/ZmgwwmLEoSISCuY8cX5fFg9jK9evon4sZcFHU5YlCBERCLtV7/itbnVdE7YzX3zzgw6mrApQYiIRFJBAR/e9hxzuYYbb04gLjH6L5BrpAQhIhJJb77JRJ4nPt4x7Scdgo7mmChBiIhEUGXJPjbSm699zejSJehojo0ShIhIBG0v9ibly88POJDjENEEYWYZZjbHzFaa2QozO9fMMs3sDTNb4z938uuamT1mZoVm9omZDY9kbCIirWFdcRoQExdOHyLSPYhHgdedcwOAocAKYBrwlnOuH/CWvw5wBdDPf0wFnoxwbCIiEffq+oHEU89ZZwUdybGLWIIws47A+cAMAOdcrXOuDBgPzPKrzQKu8pfHA79zng+BDDOLkesNRUQOVb1yA0+svZSru35ARkbQ0Ry7SPYgTgVKgGfN7GMze9rM0oCuzrltAP5z47BNDrC5yeuL/LIDmNlUMysws4KSkpIIhi8icgJqa3njnO9RQwrX3xpjo9O+SCaIBGA48KRzbhiwj38fTmpOc7dXOuSefM656c65fOdcfnZ2dstEKiLS0lau5Cd7bqFflzLGfrt/0NEcl0gmiCKgyDm30F+fg5cwihsPHfnPO5rU79Hk9bnA1gjGJyISMcV/X8lihnPtuHpSU4OO5vhELEE457YDm82sMXWOAZYD84BJftkk4BV/eR5wk3820wigvPFQlIhILGlogFt+dToAN90Wg4MPvkjf8+5W4DkzSwLWAZPxktILZjYF2ARM8Ou+CowFCoFKv66ISMx59PslzFmTx7RTZjJwyM1Bh3PczLlDDvPHjPz8fFdQUBB0GCIi+239ZCe5QzPpaZtZ9/ci4s4bFXRIhzCzRc65o166pyupRURa0As/WoEjjlfuXxmVyeFYKEGIiLSQ5Qt2c9eL53BOyhKG3nlp0OGcMCUIEZEW8oOvFlNHEnfemwzW3Jn7sUUJQkSkBdRX1fHm8m5M6f4a135vYNDhtAglCBGRE+EczJnDM71+SLlL57KbugYdUYuJ9GmuIiInr7o6mDSJwtkf8cu4/yM5McTl9wwLOqoWox6EiMjx+OADGi65jMdmd2Zw/AqKUk9n9h/j6dAx9sceGqkHISJyrL73PWp/8iAT4l5iHlcy8hx4/nno0ePoL40l6kGIiByL8nI++cmfOSNtHfMaruSBB+C9906+5ADqQYiIhKesjNAjj/HUo5XcyXukJScz/RH48pdPijNam6UEISJyNJ98wqIL/4uv7f4pBZzF0Jyd/HZOe84ZEXRgkaUEISJyOBUVVEy9g5mzk7iHl9hn7Xnufx03/Gfnk7bX0JQShIhIc5Ys4cX/fJFbV/w32+jO2UOqmPWCMWBA0IG1HiUIEZGDffQRMy94lsnVT9IxuZoX/wBXX92uTfQamlKCEBE5SN0dd/PD2pkM6lvD+wUppKcHHVEwlCBERJoqLOTpBQPYQC/+71HabHKAMK6DMLNUM/u+mT3lr/czsyvDeXMz22BmS81siZkV+GWZZvaGma3xnzv55WZmj5lZoZl9YmbDT6RhIiLHw/3yUX7CvYwYXsMVVwQdTbDCuVDuWaAGONdfLwJ+fAyf8VnnXF6TuxdNA95yzvUD3vLXAa4A+vmPqcCTx/AZIiIn7h//4JVfb2YLuUz+anKbG3M4WDgJ4jTn3INAHYBzrgo4kR/beGCWvzwLuKpJ+e+c50Mgw8y6ncDniIgcm3ff5ZfcTq/ceiZPDjqY4IWTIGrNrB3gAMzsNLweRTgc8DczW2RmU/2yrs65bQD+cxe/PAfY3OS1RX6ZiEireGtBMn/nQr72zQQSE4OOJnjhDFLfB7wO9DCz54BRwM1hvv8o59xWM+sCvGFmK49Qt7leiTukkpdopgL07NkzzDBERI7u75tPA+Cb3ww4kChx1B6Ec+4N4Bq8pDAbyHfOvRPOmzvntvrPO4C5wNlAceOhI/95h1+9CGg63VUusLWZ95zunMt3zuVnZ2eHE4aISFg2V2SQm7CNDh2CjiQ6hHMW0/nAZ4C9wB5gkF92tNelmVmHxmXgUmAZMA+Y5FebBLziL88DbvLPZhoBlDceihIRaQ01dXGkxNUFHUbUCOcQ051NllPwegGLgIuO8rquwFzzTgNIAP7gnHvdzP4JvGBmU4BNwAS//qvAWKAQqAQ0RCQiraqmPp7keCWIRkdNEM65zzddN7MewINhvG4dMLSZ8lJgTDPlDtCRPxEJTE19PMlx9UGHETWO54ZBRcDglg5ERCRoNfUJJMcrQTQ6ag/CzH7Fv88migPygH9FMigRkSDUhOJJTlKCaBTOGERBk+V6YLZzbkGE4hERCUxNKJFOCRqDaBTOGMSso9URETkZ1NTHkZwQCjqMqHHYBGFmS2nmQjW8C9qcc+6MiEUlItLatm6lpi6O5MSGoCOJGkfqQYQ1Y6uIyElh4ULKGEH703WVXKPDJgjn3MbWDEREJDA7dvDK115jO1eTf0V10NFEjXCupB5hZv80swozqzWzkJntaY3gREQirr6ebfmfZ+KOR+mUvI9JU1OCjihqhHMW0+PAROBPQD5wE9A3kkGJiLSaxx/n6c2XUk07Zj7bQFpa0AFFj7BuOeqcKzSzeOdcCHjWzN6PcFwiIpG1cyf1V0/g8feG8hN7gFEjHdffcDzXDp+8wkkQlWaWBCwxsweBbYByrIjEtNqfPsSt701kOl/lsosb+P0f2vjt45oRTrq80a93C7APb0ruayMZlIhIRO3axZW/upTpfJUJE+C1v8bRuXPQQUWfcHoQw4FXnXN7gB9GOB4RkYh777Y/8kb917l5/C5++/vMNn/v6cMJpwcxDlhtZv9rZp8zs7DGLUREotUvXh1Ix/gKHpieSVJS0NFEr3DuKDcZ76ylPwH/Caw1s6cjHZiISKRsqUhn5Cnr6dIl6EiiW1hD9s65OuA14Hm8mwWNj2RQIiIR88EHlNZ1JLOrDoYcTTgXyl1uZjPx7vR2HfA00C3CcYmIRMaf/kQpWWSdrcu5jiacHsTNwMvA6c65Sc65V51zYU+YbmbxZvaxmf3FX+9jZgvNbI2Z/dE/hRYzS/bXC/3tvY+9OSIiR7b0/b2Uk0H/wYlBhxL1whmDmOice9k5V3Ocn/EtYEWT9QeAR5xz/YDdwBS/fAqw2znXF3jEryci0nLq65lZMJjEuHquvz7oYKJfRC8bNLNc4HN4h6UwMwMuAub4VWYBV/nL4/11/O1j/PoiIi0itH4Tvw9N5PN5m3XdQxgifV35L4G7gMYJ1rOAsiaHqIqAHH85B9gM4G8v9+sfwMymmlmBmRWUlJREMnYROcn8a9LD7KAr14zTbUXDEVaCMLN2Ztb/WN7YzK4EdjjnFjUtbqaqC2Pbvwucm+6cy3fO5WdnZx9LSCLSlpWX86OFlwHw2a/0CziY2BDOWUyfB5YAr/vreWY2L4z3HgWMM7MNeKfHXoTXo8hocrFdLrDVXy7Cm8YDf3s6sCvsloiIHEHxSwv4c8NYxo0qpXv3oKOJDeH0IP4bOBsoA3DOLQF6H+1Fzrl7nHO5zrneeNOFz3fOfQF4G+90WYBJwCv+8jx/HX/7fOdcc7c8FRE5ZrPnJNBAPD97RPd7CFc4CaLeOVfegp95N3CHmRXijTHM8MtnAFl++R3AtBb8TBFpw/bucfzg9ZEMTV7JoLM0GXW4wrmUcJmZ/ScQb2b9gNuAY7ofhHPuHeAdf3kdXo/k4DrVwIRjeV8RkXC8OWMjext6c9+N64ABQYcTM8LpQdwKfAaoAWYDe4DbIxmUiEhL+mB+FUnUMPYbvYMOJaYctQfhnKsE7gXuNbN4IM3/b19EJCZ8uLwDw1lMcm/1Ho5FOGcx/cHMOppZGvApsMrM7ox8aCIiJ66uDgo2ZjMiYRFkZAQdTkwJ5xDTIP9mQVcBrwI98e4yJyIS9f717GKqQsmMGN8V3Rno2ISTIBLNLBEvQbziT/2t009FJPo5x//8LIFU9nHhzy4LOpqYE06C+C2wAUgD3jWzXngD1SIiUW3r7+czb8MZ3JXzB7r26xh0ODEnnEHqx4DHmhRtNLPPRi4kEZGWsWrxPgBG339lwJHEpnAGqdPN7OHGCfLM7Bd4vQkRkai2ep33P3C/EYfM+ylhCOcQ0zPAXuA//Mce4NlIBiUicqLcnr384//KSaGK3FOTgg4nJoVzJfVpzrlrm6z/0MyWRCogEZGWMO8HBTwXuoHrzlhFXNwxTUYtvnB6EFVmNrpxxcxGAVWRC0lE5MS99553suVT83Xv6eMVTg/i68AsM0vHu2fDLrz7VIuIRK1XlvdjaOpqMrJODzqUmBXOWUxLgKFm1tFf1ymuIhLVQnsrWVPVg+/m/xVQgjheh00QZnbHYcoBcM49HKGYREROyKO3rQWGcOr5PYIOJaYdqQfRodWiEBFpAc7BF66rYfZLQzgrfhFf/OEZQYcU0w6bIJxzP2zNQERETkTlnnrGn7+bN/+VzVW8zMwXOpDcPjHosGJaOBfKzTKzjCbrnczsmTBel2JmH5nZv8zsUzP7oV/ex8wWmtkaM/ujmSX55cn+eqG/vffxN0tE2opQCB68s4ScrCre+lcWP89+kBdfSyX9mjFBhxbzwjnN9QznXFnjinNuNzAsjNfVABc554YCecDlZjYCeAB4xDnXD9gNTPHrTwF2O+f6Ao/49UREmrVhdS2z7lzGuV3XcvdD2ZxZv5C/XvsU39lwC3GXXxp0eCeFcBJEnJl1alwxs0zCO/vJOecq/NVE/+GAi4A5fvksvFliAcb76/jbx5hpbl4R+beyDWXcP/Zdzu60mj79k7j5ocGUlsKDZ/yeN9b04ZI5X4XU1KDDPGmEcx3EL4D3zWwO3hf8fwA/CefN/TvQLQL6Ar8G1gJlzrl6v0oRkOMv5wCbAZxz9WZWDmQBO8NrioicrPbtrOKpry/mu3OGUcX55Ccv5RdnzeaC8RkM+9Jw4rp9MegQT0rh9AR+Z2YFeP/5G3CNc255OG/unAsBef4YxlxgYHPV/OfmeguH3HfCzKYCUwF69uwZThgiEsPm/KGWm240qhpGMaL9Mp54KpFhE4cAQ4IO7aQXTg8CPyGElRQO8/oyM3sHGAFkmFmC34vIBbb61YqAHkCRmSUA6XhXbR/8XtOB6QD5+fm6cZHISco5ePVV+NLNDfRvWMFPb1rFxU9dT2KSjjy3lnDGII6LmWU3nv1kZu2Ai4EVwNvAdX61ScAr/vI8fx1/+3znnBKASBu0eTMMGhDiyiuhU10xz5z+AFc8+x9KDq0srB7EceqGN4dTPF4iesE59xczWw48b2Y/Bj4GZvj1ZwD/a2aFeD2HiRGMTUSi1OaV+7h0xB7WlWfyBLcwZVKIpIceh7iI/T8rhxGxBOGc+4RmTod1zq0Dzm6mvBqYEKl4RCT6Ff99JedclMHehva8ed6POO8XX4Kzzgo6rDYrkj0IEZFj8v2p2ylu6Mfff/5PRn8nrJMlJYLUZxORqDD//6p4evX5TOhTwOjvjAg6HEEJQkQC9sG8Ei7ouY4xV7bjVNbx28frj/4iaRVKECLS6pyDv/1vMbec8S6XjG/Hys1p/KD7Uyz81T9JHzsq6PDEpzEIEWlVZWUw9fwV/GnpQJJJ53M9l/LY7C7kjPxK0KHJQZQgRKRV3X/DEv60NI9vn/oyP5rZi/bn6SylaKUEISKt5u3/eY8HXh/NuR2W8fA/z4fMzKBDkiNQghCRVvGPP5fx+R/k0T1uO3/b2B866WY+0U6D1CISUa4+xBPX/53PXxVHMjUs+MNG2is5xAT1IEQkYhq2bGPEwHL+ufcCRndYwszvraX39dcGHZaESQlCRCLm0Svf4J97b2Ly6DU8/c5Q4uLzgg5JjoEShIhEzLPLz+bcrFXMeLc/uj9k7NEYhIhExMaVVSytHcC1Z25UcohRShAiEhFrfr8QgLPObxdwJHK8lCBEJCK2FFYBkDN2aMCRyPFSghCRiNi1LxmArC7xAUcixyuStxztYWZvm9kKM/vUzL7ll2ea2RtmtsZ/7uSXm5k9ZmaFZvaJmQ2PVGwiEnn1Ie85IVkJIlZFsgdRD/yXc24gMAL4ppkNAqYBbznn+gFv+esAVwD9/MdU4MkIxiYiERbyZ+2OT9SBilgVsT3nnNvmnFvsL+8FVgA5wHhgll9tFnCVvzwe+J3zfAhkmFm3SMUnIpEV8nsQ8UnqQcSqVkntZtYb7/7UC4Guzrlt4CURoItfLQfY3ORlRX6ZiMSg/QlCPYiYFfE9Z2btgReB251ze45UtZky18z7TTWzAjMrKCkpaakwRaSFhRq8P+m4eF0EEasimiDMLBEvOTznnHvJLy5uPHTkP+/wy4uAHk1engtsPfg9nXPTnXP5zrn87OzsyAUvIickFII4QrpILoZF8iwmA2YAK5xzDzfZNA+Y5C9PAl5pUn6TfzbTCKC88VCUiMSeUAjiCQUdhpyASM7FNAq4EVhqZkv8su8C9wMvmNkUYBMwwd/2KjAWKAQqgckRjE1EIizUYEoQMS5iCcI59x7NjysAjGmmvgO+Gal4RKR1qQcR+3R6gYhERKhBCSLWKUGISESEQka8NQQdhpwAJQgRiQiNQcQ+JQgRiYhQCPUgYpwShIhEhHoQsU8JQkQiItSgMYhYpwSob2VxAAARcElEQVQhIhHh9SCUIGKZEoSIRESoQWMQsU4JQkQiItQQR7xpDCKWKUGISERoDCL2KUGISEQoQcQ+JQgRaXErVsCHu/opQcS4SM7mKiJtQHExrFkDJSWwZAnMmF7Plu0JxJHOd3JnAQOCDlGOkxKEiByz8nL4+X/v49V5dXy8LuOAbZfxFrfxFtec8gF9H/h6QBFKS1CCEJGw7N0LK19azt9nrufBBaMoqcvgPN7lp7zG8OTldDklju79O9D1rJ7w5W9A7weDDllOkBKEiBygrg4++giWL/cfC3axYgVsrsgEBgGD6J+0jheumsWF3x4GZ9wN6eno3qInHyUIkTbMOdi4EdauqufT93bz3rsh3l3SkeI9qQCkWhUD3HouiFvFwNPrGDQ6k4Ffv5DT8k4lIeFbAUcvkRaxBGFmzwBXAjucc4P9skzgj0BvYAPwH8653f79qx/Fu+VoJXCzc25xpGITaavWL97NX/9cw8KP4li3vJp/FWVSXt8e76sgm55sZAxzGZv8FqNyNtKzbxJx542CW26BjIyjvb2cZCLZg5gJPA78rknZNOAt59z9ZjbNX78buALo5z/OAZ70n0XkONXUwItPFvOPF0soWJ7KhrIMdjZkAnAK2+hDERMzFzLs1HJO72/0HdaBHvldoedI6HUDxOks+LYukvekftfMeh9UPB640F+eBbyDlyDGA7/z70v9oZllmFk359y2SMUncrIqK4Mf/Vc5L/yhji3VXelICmcmLeW6nqsZeEYil51XyekD47G8oZAzMuhwJYq19hhE18YvfefcNjPr4pfnAJub1Cvyyw5JEGY2FZgK0LNnz8hGKxKFnIMtW2DzZli7FtaurmfHmj2UbKxkx/YQn2zJYndtOiPjPmT6hL9w2U/OJ77vKA0iyzGLlkHq5n5zXXMVnXPTgekA+fn5zdYROdksXAg//G4161bVsb00kfLqlCZbE8jE0YW9dGEHl7T7F186u4BLn5uE9R0RWMwS+1o7QRQ3Hjoys27ADr+8COjRpF4usLWVYxOJGs7Bp5/Cay9XM2dGOR9t6Eo7GhjLX7mUrfTLKOHUXg306RnitAGJJA8dAAMHQr/h0KEDMC7oJshJoLUTxDxgEnC///xKk/JbzOx5vMHpco0/SFtSWwvr1oRY8V4pb71Ww98+6MCaHRlACmeyjJ+mTmfKpUV0+dYNcOZlfhIQiaxInuY6G29AurOZFQH34SWGF8xsCrAJmOBXfxXvFNdCvNNcJ0cqLpEg1dTAunXw/vuw4tMGCv+5i+WfOjaUZVDnEoEupLKPUSzg2wl/4fJrUulz+3g49/tBhy5tUCTPYrrhMJvGNFPXAd+MVCwikVZd7Z09tHOn99hT7ti4qpptayrYvqmG9eugcHsaWyrScf4kysnU0pftDGYN12aXMmgg9BuZzbAL00keNBByLtapphKoaBmkFol6NTWwaBG89qrj44U1FK5xlJUbZRUJ1NQf/KdkQDsSSKArxfRkE2MSN9Cn8y5O7VLBmaeV0b8/xI84Cz77WcjMDKJJIkekBCFyJM6xuaCYV367nYeez2HjvmziCTGI1QxhFVmUkk45GfEVZGRAZpbRpauR1j2dXv1TyB6QRVzPXOjVC3J1zYHEFiUIkSZ27oSP3ynn4zdLWbywlsUrU1lT3RM4hf5xq3m63zNcPWYPmZ/p5n3pd8uH007zpqHQdQZyklGCkDalZk8N6z6poPDTGkq31rBnawU7N1dRtAUWburG8j09gHQgnV5sYHjqCr5y8UrG3ZxJ/+uGQPLdQTdBpNUoQUjscQ5Xuovy1cUULd/DhvWOkhJH1d4Q+8rr2LIjiYqyeir3OapqjKqaeHbXprGlvivbOIUGsg54O6OBbmzjM6nruXHg25w1Opm8MVlkjewPuZeqZyBtlhKEBM8579hOURFs2ULthq18ujKewq2prNueSmFxB3bsSaGsKoldNWmU1qdTSha1B33RN2rPXjrG76NdQh3tEutplxIio1MtgzptpWf2Ok7vWU3f3vVk5ySRfmoW6QO7k9CtO1gOMLp12y4SxZQgpGXU1kJlJezbBxUVsGsXlJUR2lVOWXENu0sb2L0bysocu3c2sLuknh07oHhvKjsq0ygOdWYHXShmJLs58Iye7PhScpJLyWhXQ9/OVZyTXkFW56106Z5A7qlJ9OiTQPdeiaR2TqXdKem079wBM11IJnKilCDEU1/v3XV+40bYtAl27PC+7CsrqS2vory0nr1lIfaUNVBSlkhReQe2VKSzo7oDu2rS2NPQngraU0kq+0hjD93ZzWfYS8cjfmynpAq6dNxH18x6BndxjMmpo0uvPfQfmsKAIYn0OdXo2DELDtNbEJHIUYJoC7Zvh5UrYfFiWL0atm71ruryH9W7q1hV0Z1tdGMtp1FIX7aQwxZy2ERPtpCz/+Kug6UnVZLVsYqOKXW0bxcio10D3VMdHTPiyMjcR6cuNXTKTiSjSyKduiTRKTuBjE5Gp06QnQ1JSe2B9q378xCRsChBxLp9+7xZ3VauhE2bqNu0jZ1ba6korWFvaS27S+opLYujlCxKyaIkKY/StM+zk2yKGzpTXNuJHdUdCRG//y1TUxrIzWkgJ8e4qFccfU41srKgY0fvkZUFubnQvTukpKQCqcG1X0QiRgkiWtTVecfwG4/jNy43ru/bBzt2UFVUyq7N+yjdUk3ptlp2rKtgfUNPljKEpYxjJQOoI+mwH9MxxZGV4X3h554CZ3aFbt1gyBDvS79nT8jJicNMUzyItHVKEIdTXw9VVd4kOzU13iBsTc2Bj9par07TL/ODvtjdvkqq99axuzyOyooGqiod1ZX+czVUVRvV1bCnIY3ddKKMDPaRRiWplJPu/+ffY38PoOow/63nnlLH0GFxjD0jnl69vMk+O3Twrt/KyvIemZmQnKxTNkUkPG0zQRQV4ea/Tem8BVSsL6Fyb4h9lUZllbGvKo7K2gQqQ0n7v6gbn6toRy1J1JC8/3knnangFGpIpoZkqkk54LmG5MMev29OfFwDacn1pKWE6JgWIisjRM9OjmFZkNUdMrs3kNU5bv+XfufO0Ls3dOiQGLmfl4i0SW0yQTz9rU/43kuXUMyNYb8mKSFEu6QQyYkNJCU6kpMcSYmQ1SlEdgcjuV0cKalxJKfFk5yaQEo7IzkZkpMhNRU6dYK0NEhJgXbt/v3cuNyhQ2OdOMwOf4hIRKS1tMkE0X382VxSE8+ZFzWQ3imO1FTvy/twz+3aQUJCPDQZyBUROdm1yQQx9qbOjL0p6ChERKJbVJ2qYmaXm9kqMys0s2lBxyMi0pZFTYIws3jg18AVwCDgBjMbFGxUIiJtV9QkCOBsoNA5t845Vws8D4wPOCYRkTYrmhJEDrC5yXqRXyYiIgGIpgTR3BVc7pBKZlPNrMDMCkpKSlohLBGRtimaEkQR0KPJei6w9eBKzrnpzrl851x+dnZ2qwUnItLWRFOC+CfQz8z6mHel2ERgXsAxiYi0WVFzHYRzrt7MbgH+indF2jPOuU8DDktEpM0y5w45zB8zzKwE2HicL+8M7GzBcIKktkQntSU6qS3Qyzl31GP0MZ0gToSZFTjn8oOOoyWoLdFJbYlOakv4omkMQkREoogShIiINKstJ4jpQQfQgtSW6KS2RCe1JUxtdgxCRESOrC33IERE5AjaZIKItWnFzayHmb1tZivM7FMz+5Zfnmlmb5jZGv+5k19uZvaY375PzGx4sC04kJnFm9nHZvYXf72PmS302/FH/0JJzCzZXy/0t/cOMu6DmVmGmc0xs5X+vjk3hvfJt/3frWVmNtvMUmJpv5jZM2a2w8yWNSk75n1hZpP8+mvMbFKUtOPn/u/YJ2Y218wymmy7x2/HKjO7rEl5y3zHOefa1APvIry1wKlAEvAvYFDQcR0l5m7AcH+5A7Aab0r0B4Fpfvk04AF/eSzwGt78ViOAhUG34aD23AH8AfiLv/4CMNFf/g3wdX/5G8Bv/OWJwB+Djv2gdswCvuwvJwEZsbhP8CbFXA+0a7I/bo6l/QKcDwwHljUpO6Z9AWQC6/znTv5ypyhox6VAgr/8QJN2DPK/v5KBPv73WuOtL1vkOy7wX84AfpHOBf7aZP0e4J6g4zrGNrwCXAKsArr5Zd2AVf7yb4EbmtTfXy/oB94cW28BFwF/8f9Idzb5A9i/f/Cuqj/XX07w61nQbfDj6eh/qdpB5bG4TxpnUs70f85/AS6Ltf0C9D7oi/WY9gVwA/DbJuUH1AuqHQdtuxp4zl8+4Lurcb+05HdcWzzEFNPTivvd+WHAQqCrc24bgP/cxa8WzW38JXAX0OCvZwFlzrl6f71prPvb4W8v9+tHg1OBEuBZ/3DZ02aWRgzuE+fcFuAhYBOwDe/nvIjY3C9NHeu+iNp91MSX8Ho/0ArtaIsJIqxpxaORmbUHXgRud87tOVLVZsoCb6OZXQnscM4talrcTFUXxragJeAdCnjSOTcM2Id3GONworYt/rH58XiHKboDaXh3djxYLOyXcBwu/qhul5ndC9QDzzUWNVOtRdvRFhNEWNOKRxszS8RLDs85517yi4vNrJu/vRuwwy+P1jaOAsaZ2Qa8OwZehNejyDCzxokjm8a6vx3+9nRgV2sGfARFQJFzbqG/PgcvYcTaPgG4GFjvnCtxztUBLwEjic390tSx7ouo3Uf+gPmVwBecf9yIVmhHW0wQMTetuJkZMANY4Zx7uMmmeUDjmRaT8MYmGstv8s/WGAGUN3a1g+Scu8c5l+uc6433c5/vnPsC8DZwnV/t4HY0tu86v35U/EfnnNsObDaz/n7RGGA5MbZPfJuAEWaW6v+uNbYl5vbLQY51X/wVuNTMOvm9qkv9skCZ2eXA3cA451xlk03zgIn+WWV9gH7AR7Tkd1zQA0tBPPDOYliNN9J/b9DxhBHvaLwu4ifAEv8xFu+471vAGv85069vwK/99i0F8oNuQzNtupB/n8V0qv+LXQj8CUj2y1P89UJ/+6lBx31QG/KAAn+/vIx35ktM7hPgh8BKYBnwv3hnxsTMfgFm442f1OH9Bz3lePYF3jH+Qv8xOUraUYg3ptD4t/+bJvXv9duxCriiSXmLfMfpSmoREWlWWzzEJCIiYVCCEBGRZilBiIhIs5QgRESkWUoQIiLSLCUIkSbM7GdmdqGZXXVCs2Ae/+ffbGaPt/bnijRHCULkQOfgzXN1AfCPgGMRCVTC0auInPzM7Od4M5j2AT4ATgPGmNkcoAz4Gt48OMudcxPN7Gy8aULaAVV4F1WtMrObgavwplweDPwCb8rlG4EaYKxzbpeZvYN30dPZeDPDfsk599FBMWXjTbPd0y+63Tm3IDI/AZFDKUGIAM65O83sT3hf5HcA7zjnRgGY2Vagj3OupsnNWlYC5zvn6s3sYuCnwLX+tsF4M+6m4F0Fe7dzbpiZPQLchJdYANKccyPN7HzgGf91TT0KPOKce8/MeuJN+zCw5Vsv0jwlCJF/G4b3X/0AvLmIGn0CPGdmL+NNqQHeBHWzzKwf3jQoiU3qv+2c2wvsNbNy4M9++VLgjCb1ZgM45941s45N7xTmuxgY5E2PBEBHM+vgv7dIxClBSJtnZnnATLxZL3cCqV6xLcG7+crn8O70NQ74vpl9BvgfvERwtX+PjneavGVNk+WGJusNHPg3d/A8Nwevx+HdmKfqeNolcqI0SC1tnnNuiXMuj3/fynU+cJlfVgP0cM69jXejowygPV4PYov/Fjcf50dfD2Bmo/FmFC0/aPvfgFsaV/xEJtJqlCBE2D8gvNs51wAMcM41HmKKB35vZkuBj/HGBMrw7nf8MzNb4Nc5HrvN7H28gegpzWy/Dcj3b1a/HG+gXKTVaDZXkQD4ZzF9xzlXEHQsIoejHoSIiDRLPQgREWmWehAiItIsJQgREWmWEoSIiDRLCUJERJqlBCEiIs1SghARkWb9P6ZenI8bBthHAAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"import matplotlib.pyplot as plt2\n",
"\n",
"prediction_sorted = scaler.inverse_transform(model.predict(X_test))\n",
"prediction_sorted.sort(axis=0)\n",
"\n",
"Y_test_sorted = scaler.inverse_transform(Y_test.copy().reshape(-1, 1))\n",
"Y_test_sorted.sort(axis=0)\n",
"\n",
"plt2.plot(prediction_sorted, color='red', label='Prediction')\n",
"plt2.plot(Y_test_sorted, color='blue', label='Actual')\n",
"plt2.xlabel('#sample')\n",
"plt2.ylabel('close value')\n",
"plt2.legend(loc='best')\n",
"plt2.show()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.5"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment