Skip to content

Instantly share code, notes, and snippets.

@tuxedocat

tuxedocat/theano_lr.ipynb

Created June 1, 2015 03:14
Show Gist options
  • Save tuxedocat/187320ab535632df2fb7 to your computer and use it in GitHub Desktop.
Save tuxedocat/187320ab535632df2fb7 to your computer and use it in GitHub Desktop.
Download ZIP
Display the source blob
Display the rendered blob
Raw
theano_lr.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Theano Tutorial: Logistic Regression"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Using gpu device 0: Quadro K5000\n"
]
}
],
"source": [
"from theano import *\n",
"import theano.tensor as T\n",
"import numpy as np"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 変数宣言と関数オブジェクト"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"ロジスティックシグモイド関数 [0,1]\n",
"$$s(x) = \\frac{1}{1 + e^{-x}}$$"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# Theanoライブラリの特徴はC++コードの自動生成、テンソルデータ型とそれに対する記号処理ができる点\n",
"# ※解析的な微分ができる（自動微分）⇒ 多層ニューラルネットの活性化関数を容易に記述できる\n",
"\n",
"# Theanoで記号処理をするには、まず変数を宣言する\n",
"# dmatrixは倍精度浮動小数点数を要素とする行列で、xはそのインスタンス\n",
"x = T.dmatrix('x')\n",
"s = 1 / (1 + T.exp(-x))\n",
"\n",
"# 記号処理を用いた関数オブジェクトを生成する\n",
"logistic_sigmoid = function([x], s)"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"array([[ 0.73105858, 0.5 , 0.5 , 0.5 ],\n",
" [ 0.5 , 0.73105858, 0.5 , 0.5 ],\n",
" [ 0.5 , 0.5 , 0.73105858, 0.5 ],\n",
" [ 0.5 , 0.5 , 0.5 , 0.73105858]])"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"logistic_sigmoid(np.eye(4))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"ロジスティックシグモイド関数は以下のように双曲線関数でも表すことができる\n",
"$$s2(x) = \\frac{1 + \\tanh(\\frac{x}{2})}{2}$$"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"s2 = ((1 + T.tanh(x/2) )/ 2)\n",
"hyperbolic_tangent_01 = function([x], s2)"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"array([[ 0.73105858, 0.5 , 0.5 , 0.5 ],\n",
" [ 0.5 , 0.73105858, 0.5 , 0.5 ],\n",
" [ 0.5 , 0.5 , 0.73105858, 0.5 ],\n",
" [ 0.5 , 0.5 , 0.5 , 0.73105858]])"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# 同じになる\n",
"hyperbolic_tangent_01(np.eye(4))"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"関数は複数の出力を同時に返すことができる。 \n",
"例えば、要素ごとの差分、差分の絶対値、差分の自乗の3つを返す関数を作ってみる。"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"a, b = T.dmatrices('a', 'b') # 浮動小数点数値型行列を複数宣言できる\n",
"diff = a - b # 要素ごとの差\n",
"abs_diff = abs(diff)\n",
"diff_squared = diff**2\n",
"triple_f = function([a, b], [diff, abs_diff, diff_squared]) # function([変数],[関数値])"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"[array([[ 1., 0.],\n",
" [-1., -2.]]), array([[ 1., 0.],\n",
" [ 1., 2.]]), array([[ 1., 0.],\n",
" [ 1., 4.]])]"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"triple_f([[1, 1], [1, 1]], [[0, 1], [2, 3]])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 関数のデフォルト値\n",
"Paramクラスを使う。"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"34.0\n",
"38.0\n"
]
}
],
"source": [
"x, y = T.dscalars('x', 'y')\n",
"z = x + y\n",
"function_with_defaults = function([x, Param(y, default=1)], z)\n",
"print(function_with_defaults(33))\n",
"print(function_with_defaults(33,5))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Shared Variables\n",
"Accumulator (加算器)のように、初期値があり、それが関数呼び出しごとに操作される場合、Shared Variable (共有変数)が利用できる。\n",
"\n",
"以下の例では、`state`変数は共有変数であり、複数のfunctionオブジェクト間で共有される。\n",
"`updates`パラメータとして、`(共有変数, 関数適用後の式)`のリストを与える必要がある。\n",
"\n",
"※DNNでは、これを用いて行列演算時に破壊的操作を行うことで、中間オブジェクトの生成を無くして効率化できる。\n",
"GPUは共有メモリ型の並列計算に特化しているので、この特性が活きてくる。"
]
},
{
"cell_type": "code",
"execution_count": 67,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"state = shared(0)\n",
"increment = T.iscalar('increment')\n",
"accumulator = function([increment], state, updates=[(state, state+increment)])"
]
},
{
"cell_type": "code",
"execution_count": 68,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"initial state: 0\n",
"+1: 1\n",
"+300: 301\n"
]
}
],
"source": [
"print(\"initial state: \", state.get_value())\n",
"accumulator(1)\n",
"print(\"+1: \", state.get_value())\n",
"accumulator(300)\n",
"print(\"+300: \", state.get_value())"
]
},
{
"cell_type": "code",
"execution_count": 69,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"before decrement: 301\n",
"-2: 299\n"
]
}
],
"source": [
"# 上記の加算器の状態を、減算器で利用してみる\n",
"decrementor = function([increment], state, updates=[(state, state-increment)])\n",
"print(\"before decrement: \", state.get_value())\n",
"decrementor(2)\n",
"print(\"-2: \", state.get_value())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### givenの項は一旦省略"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Random Variables\n",
"機械学習では確率変数が頻繁に登場する。\n",
"実装時にはある分布に従う疑似乱数を用いる。\n",
"Theanoでは記号処理ベースであるため、よくある疑似乱数を利用したコードとは異なるやり方になる。"
]
},
{
"cell_type": "code",
"execution_count": 70,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"from theano.tensor.shared_randomstreams import RandomStreams\n",
"srng = RandomStreams(seed=234)\n",
"rv_uniform = srng.uniform((2,2))\n",
"rv_normal = srng.uniform((2,2))\n",
"f = function([], rv_uniform)\n",
"g = function([], rv_normal, no_default_updates=True)"
]
},
{
"cell_type": "code",
"execution_count": 71,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[[ 0.12672381 0.97091597]\n",
" [ 0.13989098 0.88754827]]\n",
"[[ 0.31971416 0.47584376]\n",
" [ 0.24129163 0.42046082]]\n"
]
}
],
"source": [
"# no_default_updates = Falseのとき、関数呼び出しのたびに変化する\n",
"f_val0 = f()\n",
"f_val1 = f()\n",
"print(f_val0)\n",
"print(f_val1)\n",
"\n",
"# nearly_zeros中で3回呼び出されるが、それぞれ値は異なる\n",
"nearly_zeros = function([], rv_uniform + rv_uniform - 2*rv_uniform)"
]
},
{
"cell_type": "code",
"execution_count": 72,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[[ 0.12309219 0.71399385]\n",
" [ 0.1424956 0.36686867]]\n",
"[[ 0.12309219 0.71399385]\n",
" [ 0.1424956 0.36686867]]\n"
]
}
],
"source": [
"# no_default_updates = Trueのとき、同じ値が返る\n",
"g_val0 = g()\n",
"g_val1 = g()\n",
"print(g_val0)\n",
"print(g_val1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"それぞれの疑似乱数の変数に、異なるシードを与えることができる。"
]
},
{
"cell_type": "code",
"execution_count": 73,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"rng1 = rv_uniform.rng.get_value(borrow=True) # rngオブジェクトを取得\n",
"rng1.seed(89234) # シードを変更\n",
"rv_uniform.rng.set_value(rng1, borrow=True) # rngオブジェクトを変数に与えてシードを変更\n",
"\n",
"# 同じRandomStreamで生成された変数のシードを変更する場合\n",
"srng.seed(902340)"
]
},
{
"cell_type": "code",
"execution_count": 75,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[[ 0. 0.]\n",
" [ 0. 0.]]\n",
"[[ 0.6272043 0.90458977]\n",
" [ 0.14363919 0.89279932]]\n",
"[[ 0.23219827 0.25305995]\n",
" [ 0.02116774 0.65845078]]\n"
]
}
],
"source": [
"state_after_v0 = rv_uniform.rng.get_value().get_state()\n",
"print(nearly_zeros())\n",
"v1 = f()\n",
"rng = rv_uniform.rng.get_value(borrow=True)\n",
"rng.set_state(state_after_v0)\n",
"rv_uniform.rng.set_value(rng, borrow=True)\n",
"v2 = f(); print(v2)\n",
"v3 = f(); print(v3)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Copying Random State Between Theano Graphsはいったん省略"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## ロジスティック回帰"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"簡単な例として、2クラスロジスティック回帰をTheanoを利用して実装する。\n",
"\n",
"各データの次元数を$n$とすると、縦ベクトル $x \\in \\mathbb{ R^n }$と表せる。 \n",
"入力データはこれを並べた行列で、$$X = \\begin{bmatrix} {x_1}^T \\\\ {x_2}^T \\\\ \\vdots \\\\ {x_N}^T \\end{bmatrix}$$とする。\n",
"また、ラベルは$y \\in \\{-1, 1 \\}$で $$y = \\begin{bmatrix} y_1 \\\\ y_2 \\\\ \\vdots \\\\ y_N \\end{bmatrix} $$のように二値の縦ベクトルとする。"
]
},
{
"cell_type": "code",
"execution_count": 81,
"metadata": {
"collapsed": false,
"scrolled": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Initial model:\n",
"weight vector: [ 1.43402735e+00 -6.83763407e-01 -6.91706179e-01 -9.88391923e-01\n",
" 1.61743603e+00 1.05490538e-01 -1.52768086e-01 -3.33501711e-01\n",
" -2.57143192e-04 -1.35070663e+00 -1.11617336e+00 -3.46099598e-01\n",
" -1.23153270e+00 -8.32843425e-01 -2.66742074e+00 -6.29815393e-01\n",
" -1.75071877e+00 1.69944262e-01 -7.63241706e-01 4.34970606e-01\n",
" 3.77181530e-01 1.86429507e+00 -4.39847987e-01 -1.36277336e+00\n",
" -1.41005261e-01 -1.11291440e-01 1.79312582e-01 -1.21739457e-01\n",
" 8.98378009e-01 1.09015795e-01 1.41068037e+00 2.68356120e-01\n",
" 3.98595146e-01 -6.63000414e-01 -7.75145262e-01 1.04212506e+00\n",
" -4.08172012e-02 -6.09885249e-02 6.62543734e-01 -4.65899573e-01\n",
" -1.10613682e-01 2.12845129e-01 1.99291730e+00 8.44344742e-01\n",
" 1.24562809e+00 -1.04319599e+00 6.42654466e-01 3.16487566e-01\n",
" 1.66839032e+00 7.46084820e-01 -1.81275554e-01 1.34757010e+00\n",
" 2.54506448e-02 -1.33260464e+00 -4.25462196e-01 -5.50170240e-02\n",
" 1.79757651e+00 2.44164960e-01 -3.77325260e-01 -3.88829150e-02\n",
" -8.33293325e-01 5.38216263e-01 9.85236926e-01 1.90044268e-01\n",
" 1.11452151e+00 1.11903336e-01 -1.17989550e+00 -5.46541397e-01\n",
" -3.64901399e-01 7.40013568e-02 -1.56217794e-02 1.16670740e+00\n",
" -5.58589327e-01 -2.00941175e-01 1.70628794e+00 -8.38591454e-01\n",
" -2.69937021e-01 -2.46489794e-01 2.81071940e-01 -5.88888396e-01\n",
" 9.82979914e-01 -2.36867944e-01 -1.78489189e-01 -1.67382164e+00\n",
" -5.85082825e-01 -6.37670529e-02 -3.38238095e-01 -1.12425305e-01\n",
" -2.57794826e-01 -6.10803004e-01 -1.98241007e-01 -5.38541670e-01\n",
" -1.06270669e+00 -4.91472973e-01 -1.90057412e+00 -6.77921416e-01\n",
" -1.81898816e+00 1.29351072e+00 1.13576182e+00 1.87356482e-01\n",
" -3.44233399e-01 1.15154730e+00 -3.77298797e-01 -5.77787700e-01\n",
" -5.18470244e-01 -9.95743893e-01 -9.68267374e-01 -8.44817549e-01\n",
" -6.03667730e-01 -1.04453633e+00 -7.37623954e-01 -5.42801087e-01\n",
" 1.82234685e-01 -8.25152214e-01 1.60486478e+00 8.46026651e-01\n",
" -2.44052666e-01 -1.17395635e+00 1.89839386e+00 -1.64975724e+00\n",
" 4.32729740e-01 -1.26994337e-01 6.35878186e-01 -2.91479773e-01\n",
" 9.38854579e-01 1.56977604e+00 -1.58898922e+00 -1.89225464e+00\n",
" 3.21740433e-01 1.10067018e+00 -3.22765274e+00 5.84397861e-01\n",
" -4.36560383e-01 -1.59968333e-01 1.13659381e+00 1.29211995e+00\n",
" 8.69643848e-01 5.29781152e-01 -4.17704263e-01 1.57690483e+00\n",
" -4.12269234e-01 3.19971272e-01 1.94898181e-02 -1.17298495e-04\n",
" 7.80499854e-01 -4.83594741e-01 1.65365604e-01 8.55992861e-01\n",
" 2.18265358e-01 7.47933420e-01 -1.63487170e+00 8.52934840e-01\n",
" 1.14791312e+00 7.81615871e-01 7.24566720e-01 -1.37764290e+00\n",
" 1.63298689e+00 1.64234925e-02 -2.91239085e-01 -5.04673730e-01\n",
" -7.30085746e-01 -8.12310298e-01 6.35759590e-01 -1.46535789e+00\n",
" -5.89073415e-01 7.60230797e-02 -9.50910252e-01 -1.69714704e+00\n",
" -2.76775887e-01 -1.78789317e-01 -3.03144510e-02 -2.70731626e-01\n",
" 8.35531532e-03 -1.34698671e-01 -9.67081893e-01 -2.22128554e-01\n",
" -1.32587434e-01 5.99300073e-01 1.50931890e-01 -1.73019971e-01\n",
" -1.26736156e+00 -1.07101646e+00 1.27694078e+00 4.50522851e-01\n",
" -8.24241093e-01 -1.76354362e+00 -1.27053412e+00 -4.24919336e-02\n",
" -2.17140419e+00 -3.91360054e-01 3.12649684e-01 -1.87347040e+00\n",
" 8.14220021e-01 -8.90793084e-01 1.30982412e+00 -1.25864076e+00\n",
" 1.54519529e-01 1.66581814e+00 4.04040518e-01 1.55170698e-01\n",
" 4.24357389e-01 1.28057200e-01 -1.36378241e-01 -1.84691112e+00\n",
" 1.15045977e+00 2.44802090e-01 2.03131247e-01 3.53505121e-01\n",
" -1.06909153e+00 3.60346517e-01 3.00480049e-01 5.58756486e-01\n",
" 2.53765534e-02 -9.44433836e-01 1.08655699e+00 -2.92898204e-02\n",
" -1.46073874e-01 3.63613221e-01 2.27478982e+00 -1.13804160e+00\n",
" -3.57949764e-01 -4.00459068e-01 -1.76548147e+00 1.54152138e+00\n",
" 5.25175235e-01 1.69688572e+00 1.07646846e-01 4.68010045e-01\n",
" 9.01149841e-01 -2.09659573e+00 1.06888582e-01 1.07730868e+00\n",
" 6.21767625e-01 -3.23610621e-01 -3.68142885e-02 -1.23580357e+00\n",
" 8.15231479e-01 -1.81120006e+00 4.02581734e-01 -1.58116570e+00\n",
" -2.02016962e-01 -4.91553930e-01 1.10872535e+00 -1.28777217e+00\n",
" 1.58009392e+00 -1.98453681e+00 1.69194842e+00 -2.10208011e-01\n",
" 2.70896949e-01 -2.23007844e+00 -9.82696848e-01 3.94314863e-01\n",
" -1.78728560e-01 -8.23413903e-02 1.04410012e+00 1.67544424e-01\n",
" 5.14809062e-01 1.83662157e-01 -5.59921468e-01 -7.34448689e-01\n",
" 1.38598102e+00 2.28772299e-01 -6.43325036e-01 8.33640130e-01\n",
" 2.42034024e-02 -8.60626581e-01 -1.77965348e+00 -1.68725726e-02\n",
" 2.04659744e-01 -1.95329826e+00 7.48889738e-01 -1.36494006e-01\n",
" 2.03747226e-01 1.21400805e+00 -1.07165766e-01 1.27566844e+00\n",
" -4.08906770e-01 -1.26469441e-01 1.26236718e+00 1.94319200e+00\n",
" -1.12657559e+00 -8.78329106e-01 -8.14544620e-01 -2.34658296e+00\n",
" -3.81350745e-01 8.06803151e-01 -1.14845512e+00 -5.22983494e-01\n",
" 4.12756558e-01 5.34792763e-01 6.17150763e-01 -9.27758173e-01\n",
" -3.21171071e-01 -2.90508324e-01 -7.64511876e-01 -3.80814224e-01\n",
" 2.25337970e-01 5.08674603e-01 -6.39184657e-01 8.38110922e-01\n",
" 5.65464067e-02 8.25476941e-01 7.46263539e-01 9.92747525e-01\n",
" -8.35560533e-01 9.15915192e-01 -1.75112811e+00 1.29077709e+00\n",
" -4.97087509e-01 2.60354523e-01 -2.55815998e+00 -2.27502770e+00\n",
" -1.01748682e+00 -3.65618416e-01 -5.26020380e-01 -2.43708952e-01\n",
" 5.71322679e-01 1.21902181e+00 7.55128789e-01 -7.91395285e-01\n",
" -1.31103762e-01 1.55548996e+00 1.47274889e-01 5.33357380e-01\n",
" 1.02425516e+00 -1.07267698e+00 -1.70030302e+00 6.04323416e-01\n",
" 1.79396344e+00 -4.02745622e-01 1.62265029e+00 1.34535366e-02\n",
" 4.17600215e-01 -3.02237850e-01 -7.78435734e-01 1.03307829e+00\n",
" -1.52700583e+00 1.26803154e+00 -3.29882487e-02 2.16787221e+00\n",
" 2.72493731e-03 6.41514245e-01 -1.11303483e-01 -8.97300103e-01\n",
" 4.23756275e-03 -3.55009661e-01 -2.06092357e-01 -1.16746858e-01\n",
" -1.81537860e+00 -9.56359052e-01 5.03279464e-01 -5.28806141e-01\n",
" 1.76825566e+00 1.61344333e+00 -1.90089026e+00 -1.13537146e+00\n",
" 3.46605334e-01 -1.09330446e+00 4.70696260e-02 -1.31370411e-01\n",
" 2.20454815e-02 2.99130624e-02 9.16510197e-01 1.28471638e+00\n",
" -1.87968333e+00 2.18799526e+00 7.29187473e-01 -5.66745789e-01\n",
" 4.75191775e-01 -2.87714538e-01 9.36601221e-01 -8.59914675e-01\n",
" -3.50813644e-01 -1.77770535e+00 3.05799278e-01 6.00395405e-02\n",
" -1.05279738e-01 -6.33478985e-01 -4.17143817e-01 8.65005065e-01\n",
" -3.71711454e-02 1.78586803e-01 -1.40825071e+00 1.29690693e+00\n",
" 3.74239774e-01 -7.58568114e-01 -6.42218597e-01 -1.01711737e+00\n",
" 4.32817713e-01 -7.25988270e-01 -5.76227858e-01 6.22368546e-01\n",
" 1.81348536e-01 2.52158436e-01 -1.55231073e+00 -5.60366439e-01\n",
" 1.16543088e+00 -9.01393300e-01 -7.88955718e-01 -1.11451596e+00\n",
" 4.36752323e-01 -2.46323275e-01 -3.10100913e-03 -1.02098861e+00\n",
" -1.48300257e-01 -3.57560022e-01 2.30498414e-01 2.79514024e+00\n",
" 1.13220571e+00 -8.25087800e-01 7.42493254e-02 3.77065632e-01\n",
" -3.89765890e-01 1.27053929e+00 -8.07248303e-01 2.08405512e-01\n",
" 1.54610541e+00 7.38781223e-01 2.45794676e-01 2.25851633e+00\n",
" -6.19701450e-01 -6.93061826e-01 -2.18093606e-01 1.56053380e+00\n",
" 3.03503196e-01 1.06405471e+00 -8.41562082e-01 1.63668596e-01\n",
" -2.63830339e+00 2.56211922e-01 -2.33799209e+00 1.65266340e-01\n",
" -8.80872890e-01 1.84516022e+00 3.92449348e-01 -2.68050340e-01\n",
" 2.06614669e-01 1.80807293e+00 -2.04137922e+00 -3.26849025e-01\n",
" 1.84764275e+00 1.45258451e+00 -1.19571193e+00 9.24424982e-02\n",
" -2.00650254e-02 -1.31858468e+00 -5.72515210e-02 2.08309425e+00\n",
" -1.33788272e-01 -4.10933741e-01 -6.24917248e-01 4.66854469e-01\n",
" -1.01928798e+00 -2.25104688e-01 9.08524631e-01 -1.01978896e-01\n",
" -9.01940814e-01 3.44311158e-01 5.18470252e-01 -3.24163419e-01\n",
" 4.73736512e-01 8.67053676e-01 1.79648701e-01 -1.09028327e+00\n",
" -1.18957071e+00 1.66266018e+00 -4.75468295e-01 3.43482986e-01\n",
" -9.18391112e-01 -1.79356195e-01 -2.77743994e-01 -3.99288035e-01\n",
" 1.21195142e+00 -5.46886766e-01 2.97309309e-01 1.17861127e+00\n",
" 1.22824769e+00 3.39867226e-02 9.94929572e-01 -5.85024964e-01\n",
" 2.62946560e-01 6.38188697e-01 8.81200845e-01 -1.87980193e-01\n",
" -3.81540350e-01 2.48413971e+00 3.47241458e-02 2.08711813e-01\n",
" 6.65291320e-01 1.65080295e+00 -2.69574057e+00 1.53570499e+00\n",
" 1.36689734e-01 7.82756388e-01 -9.68504308e-01 -1.02890663e-01\n",
" 4.55193940e-01 8.64400246e-01 -3.58902858e-01 -1.01211179e-01\n",
" -1.36050884e+00 1.81328909e-01 7.88763425e-01 -1.51427392e+00\n",
" 2.14858390e-01 1.64972712e-01 1.69542864e+00 9.28101727e-01\n",
" -1.01364000e-01 -2.12110659e+00 -6.11777772e-01 -3.05644873e-01\n",
" -7.92440419e-01 1.19810228e-01 3.00345828e-01 1.22956417e+00\n",
" -2.18633990e+00 9.29144677e-01 -1.61009844e+00 -1.09463657e-01\n",
" -1.05004255e+00 5.87577591e-01 -1.25990708e-01 1.74409546e+00\n",
" 3.63606353e-01 2.17248098e-01 -1.20678342e-01 3.54775428e-01\n",
" 1.12820796e-01 -4.45576735e-01 -7.92960174e-02 2.71588859e-01\n",
" 1.94868477e+00 2.92553730e-01 1.29050951e+00 1.58941311e+00\n",
" 1.29144959e+00 -8.48535645e-01 4.33050345e-01 -2.05941580e-01\n",
" 1.05985638e+00 4.06712682e-01 1.12728658e+00 -1.21031526e-01\n",
" 5.09396838e-01 1.30212677e+00 2.28140887e-01 5.47424469e-02\n",
" 2.66588379e-01 -4.03474764e-01 -1.37547559e+00 2.03621814e-01\n",
" 8.62739577e-01 9.41234574e-01 1.02469078e+00 -1.55606124e+00\n",
" -1.04309601e+00 -2.05086228e+00 -7.98049523e-02 -6.68232639e-03\n",
" 3.39291077e-01 -1.11561655e+00 1.17315155e+00 3.57678865e-01\n",
" 6.06921509e-01 1.74871739e+00 5.59354513e-01 -1.92683640e+00\n",
" 1.22142919e+00 -6.48741171e-01 -5.23869594e-01 -5.00660812e-01\n",
" 1.41561701e+00 -1.22846925e+00 7.46822512e-01 -9.76046798e-01\n",
" -1.54295150e+00 -6.38822966e-01 1.46958167e-01 -8.22068129e-01\n",
" 7.27877343e-01 1.56182416e+00 -3.09438969e-01 1.50977667e+00\n",
" 2.07013661e+00 -4.54247460e-01 2.43673443e-01 7.95719208e-01\n",
" -8.02707551e-01 2.57435026e+00 -1.53976525e+00 -1.35096567e+00\n",
" -9.55517186e-01 -3.40166192e+00 5.51981397e-01 7.41826225e-01\n",
" -2.41374390e+00 1.21521532e+00 6.09481611e-01 7.68289503e-01\n",
" -2.78705100e-01 -1.48395092e+00 1.38560656e+00 -5.95066463e-01\n",
" 1.52671646e+00 1.53417098e+00 9.16142120e-01 -7.49500084e-01\n",
" 1.39522800e+00 3.55380666e-01 -3.31170937e-01 -9.89039440e-01\n",
" 3.86274851e-01 1.27044899e-01 9.56885005e-01 1.17077841e+00\n",
" -2.62329087e+00 1.49377927e+00 -2.38363692e-01 6.24451391e-01\n",
" -1.43382031e+00 4.39720588e-01 6.31241703e-01 1.32911845e-01\n",
" 2.27929618e-01 8.48483135e-01 -1.59598375e-01 2.03044030e-01\n",
" -1.76675438e-02 1.97559505e-01 1.53469183e+00 6.57383077e-01\n",
" -3.99662188e-01 1.04928303e+00 2.99845603e-01 -1.48423517e-01\n",
" 8.85929808e-01 -1.42027690e+00 -7.13927575e-01 2.46777012e-04\n",
" 1.06886361e+00 5.71880677e-02 1.81791807e+00 -9.14467506e-01\n",
" 9.67195189e-01 3.94493220e-01 -1.63013621e+00 -1.25859598e+00\n",
" -1.12997429e+00 4.44636228e-02 3.29495938e-01 -1.07726493e+00\n",
" -1.87112777e-01 -2.85130527e-01 8.98701185e-01 -7.20509684e-01\n",
" -1.08738986e+00 1.46266935e+00 -7.54956397e-01 -9.44545594e-01\n",
" 4.26881609e-01 -3.84792241e-01 -1.18682821e+00 -1.36091508e-01\n",
" 4.81619038e-01 -1.13249975e+00 3.75912511e-01 -5.15973103e-01\n",
" -1.11943975e-01 -8.96281775e-01 2.16656997e+00 -4.57959370e-01\n",
" -1.20716629e-01 -1.21421312e+00 2.54293481e-01 -3.54072141e-01\n",
" -4.90544312e-01 -1.28232589e-01 4.32012453e-02 -4.73238298e-01\n",
" 7.03031724e-01 8.08436765e-01 -3.31213413e-01 5.54655133e-01\n",
" -6.82095908e-01 -7.58729285e-01 -6.65635175e-01 1.00143433e+00\n",
" 1.49278314e+00 3.47028496e-01 -1.79395770e-01 -2.70052032e-02\n",
" -9.16235885e-01 -8.74599667e-01 -8.70242234e-01 -1.90304801e+00\n",
" 1.09040764e+00 1.95984479e+00 9.42653606e-01 3.55382856e-01\n",
" 2.61587434e-01 2.46682405e-01 1.25049573e+00 -9.03597986e-01\n",
" 9.96713474e-01 2.15174179e-01 -1.00800123e+00 -6.36728839e-01\n",
" 3.10336538e-01 8.64550545e-01 -1.12597412e+00 -1.63986845e+00\n",
" 1.41761910e+00 2.31741922e+00 -1.45594835e+00 -1.24973193e+00\n",
" -5.50074332e-01 -3.08841845e-01 -5.06496448e-01 -4.24909399e-01\n",
" 6.81696929e-01 -4.86921670e-01 -8.70820209e-01 -1.22200371e+00\n",
" 1.37952217e-01 1.34998225e+00 -1.97858874e+00 -3.79918487e-01\n",
" 5.02370978e-01 1.15984016e+00 -1.05939077e+00 -2.72836244e-01\n",
" -1.23605763e+00 -1.26504599e+00 -9.62755764e-02 -4.16387834e-01\n",
" -5.52784401e-02 -5.50033071e-01 -3.26660980e-01 -2.34271260e-01\n",
" 6.95347528e-01 1.03480920e+00 -1.02397570e+00 8.92656424e-01\n",
" 1.79372700e-01 -1.25989691e+00 1.15126055e+00 -5.10208206e-01\n",
" -8.77879584e-01 7.38112661e-01 2.93756672e-01 8.72568717e-01\n",
" 9.53461843e-01 -4.95909654e-01 -1.67512500e+00 -1.20236851e-01\n",
" 8.39349058e-01 5.47703371e-01 2.00470329e-01 -1.59893086e+00\n",
" 7.99797635e-01 2.61475146e-01 6.79985059e-01 -2.84122665e-01\n",
" -9.66464480e-01 2.94170057e-01 7.65700345e-03 1.56761209e+00\n",
" 1.82155175e+00 -2.42791490e-01 -9.86115743e-01 -1.30432267e+00\n",
" 5.23281125e-01 1.79138911e+00 6.28858214e-01 -1.01785881e+00\n",
" 4.52121985e-01 -1.65795470e+00 2.43593276e-01 -9.72010265e-01\n",
" -4.96427938e-02 1.25077461e+00 -7.39725422e-01 -2.81993921e-01]\n",
"Reg: 0.0\n",
"Final model:\n",
"weight vector: [ 5.08938442e-02 -2.37791189e-02 3.15372117e-02 -2.92743356e-02\n",
" -4.19801790e-02 -3.55935894e-02 -2.04589224e-01 4.63749704e-02\n",
" -1.50362288e-01 -1.08259517e-03 -4.75568261e-02 -3.24263121e-02\n",
" -1.03219771e-01 3.93757426e-02 5.66442044e-02 -1.92488097e-01\n",
" 1.44582619e-01 -1.38102229e-01 -8.09215078e-02 6.00852802e-02\n",
" -1.94679574e-01 1.54510670e-01 -6.22948636e-02 -6.97076644e-02\n",
" -2.20580708e-01 9.31168602e-02 -2.70744917e-02 -7.55246096e-02\n",
" -1.68532701e-02 1.11694376e-01 -1.39929017e-01 -5.80997149e-02\n",
" -3.16196161e-02 -3.58730449e-02 2.86265753e-02 -6.59281974e-02\n",
" -3.37857890e-02 -1.01259516e-02 2.58764992e-01 8.76245160e-02\n",
" 4.22719315e-02 3.66152819e-02 8.83006685e-03 1.21125381e-01\n",
" -9.64545323e-02 -1.47227293e-02 3.46139972e-02 -6.94525247e-02\n",
" -1.48743071e-02 1.64269843e-01 -9.92278550e-02 5.74864517e-02\n",
" 8.62735290e-02 -4.41572896e-02 2.31291552e-02 -1.56756632e-01\n",
" -1.42889758e-01 1.07673779e-01 1.18933549e-01 1.27092639e-02\n",
" 5.91964501e-02 1.45134844e-02 -3.93186397e-03 1.52083804e-01\n",
" 1.16525105e-01 2.08715968e-02 9.72458459e-02 8.56972125e-02\n",
" 1.04996306e-01 -1.14644062e-01 -6.27195385e-02 1.29658587e-02\n",
" 2.37175912e-02 -7.68230379e-02 -1.13572746e-01 4.39149514e-02\n",
" 1.45761781e-02 2.18794115e-02 -1.66427062e-01 -2.80067994e-02\n",
" 8.60964509e-02 -1.91393487e-02 3.54417932e-02 -1.54845862e-01\n",
" -1.35915240e-02 -1.04344426e-01 -1.54442179e-01 7.54935671e-02\n",
" 1.52745157e-02 -3.56648719e-05 6.88578070e-02 -5.87026844e-02\n",
" -7.88380512e-02 -1.06454259e-01 2.32037678e-03 -9.63398717e-02\n",
" 7.76402526e-02 -2.40387880e-02 6.79181091e-02 -1.02767935e-01\n",
" 1.65209422e-02 -1.00685536e-01 -3.00025683e-02 -2.71777228e-02\n",
" -7.48033442e-03 1.08319342e-01 -1.04242142e-01 5.74646910e-02\n",
" -2.07688085e-02 7.74472613e-02 -1.38731548e-01 -5.17851374e-02\n",
" -1.96808053e-02 -4.11228312e-02 1.37265970e-01 -1.96300482e-01\n",
" 1.01163523e-01 2.31479961e-01 1.33852016e-01 4.86004903e-03\n",
" 6.42834651e-02 1.65767678e-01 1.44580981e-01 -2.06016222e-03\n",
" -1.43940315e-01 2.90133085e-02 -1.79135703e-01 -6.98958195e-02\n",
" -4.24244662e-03 1.07941060e-01 -1.46814575e-01 -1.61840477e-02\n",
" 4.70487079e-02 3.87246757e-02 -5.21262653e-02 7.37359583e-02\n",
" 7.38174169e-02 -1.55580581e-01 -6.05672765e-02 -8.25982510e-02\n",
" -3.59270569e-02 -1.46747006e-02 1.65769443e-03 1.16191284e-01\n",
" -3.58828836e-02 -6.17010179e-02 -6.79453518e-02 -1.85475034e-02\n",
" -5.83853705e-02 -5.34920375e-02 5.19693333e-03 2.59914830e-02\n",
" -1.78796344e-01 -1.23758101e-01 -6.41686371e-02 -3.23842371e-03\n",
" 9.27884691e-02 -2.06636456e-01 2.61697475e-02 2.30452679e-01\n",
" -1.90780544e-01 1.44198877e-02 -3.09410446e-02 -1.27527553e-01\n",
" -1.34995597e-01 4.91673100e-03 1.34740997e-01 -1.96618789e-02\n",
" 3.29984627e-02 3.93476596e-02 -1.68899691e-02 -7.08626378e-03\n",
" -9.56163592e-02 1.29585307e-01 -1.28025073e-01 1.59790691e-02\n",
" -1.05990522e-01 3.64211671e-02 -4.58332274e-02 -5.39691641e-02\n",
" -2.48497531e-03 -2.64985876e-02 -1.57086638e-01 1.05228712e-01\n",
" 1.49147501e-02 1.58620866e-01 -3.30868850e-02 -4.45355623e-02\n",
" 7.72188149e-02 -1.35510344e-01 -1.67011201e-02 -7.35883544e-02\n",
" 7.43431461e-04 -4.66223828e-02 -5.69759695e-02 7.04833790e-02\n",
" -1.51902488e-01 -9.75537887e-03 6.30991746e-02 7.99776092e-03\n",
" -1.38566247e-01 -7.22913784e-02 9.40037300e-02 -1.83501141e-01\n",
" 9.90245691e-02 1.03680736e-01 8.18577477e-02 3.90874348e-02\n",
" -2.95718319e-02 -6.14346517e-02 -1.48169815e-01 2.31329414e-02\n",
" -7.58272828e-02 -9.84066460e-02 2.13119132e-02 -5.67215240e-02\n",
" -2.37277309e-02 -2.66853994e-01 5.11752308e-02 -5.26857448e-02\n",
" 9.88108446e-03 5.91620708e-02 2.29472702e-01 5.83109065e-02\n",
" 7.21636840e-02 -5.15995459e-02 1.50616528e-01 6.36811539e-02\n",
" -1.42737499e-01 9.87128037e-02 1.18039618e-01 -1.68817414e-01\n",
" 8.46541761e-02 9.21870477e-02 -1.00928832e-01 5.04742474e-03\n",
" 1.32439179e-01 -1.11797187e-01 -2.32502376e-03 -3.71735747e-02\n",
" -1.28143969e-01 1.74850332e-01 -4.55523530e-02 -1.03221047e-01\n",
" -6.87316465e-02 -1.44058702e-01 -5.79601985e-02 1.46510110e-01\n",
" 3.15381145e-02 3.67293978e-02 8.28114963e-02 1.40355232e-01\n",
" -6.21338183e-02 -4.23154097e-02 -1.77387517e-02 1.76992203e-01\n",
" -1.11741891e-01 -1.12874878e-01 2.26000130e-02 -4.32748750e-02\n",
" 1.36230570e-01 2.16218624e-02 -1.15681275e-01 5.30612646e-02\n",
" -1.37064719e-01 -1.25600663e-02 -5.32708453e-02 -1.56316668e-01\n",
" -2.44411522e-01 6.43964153e-02 5.59987987e-02 3.47502974e-02\n",
" 6.20721761e-02 2.88452244e-03 -5.06243364e-02 -5.91216334e-02\n",
" 1.19795959e-01 7.78388832e-03 1.17284064e-01 8.54991604e-02\n",
" -1.68810347e-01 2.46380572e-02 1.86412756e-02 -4.56742281e-02\n",
" 6.55661106e-02 5.42128892e-02 -1.01536630e-01 4.04070855e-02\n",
" -1.15448955e-01 1.06792777e-01 -8.94558423e-02 -4.87769020e-02\n",
" 1.06177571e-01 4.01304177e-02 -8.92056961e-02 1.08404447e-02\n",
" 5.68525757e-02 -4.35921535e-02 3.16158695e-01 -5.12267799e-02\n",
" 1.46750614e-01 -6.44860166e-02 3.65669273e-02 9.42611199e-02\n",
" -3.01542239e-02 2.00815962e-02 4.65826088e-02 -9.49604108e-02\n",
" -9.52952288e-02 4.45354167e-02 -1.17618397e-02 -2.17581634e-01\n",
" 9.34376678e-02 -1.91742593e-01 2.42375759e-02 -2.32977155e-02\n",
" -1.94380979e-02 -4.05988239e-02 -5.70355724e-03 -1.47067905e-01\n",
" -3.71706200e-02 5.01482665e-02 -6.12369372e-02 -6.05502307e-02\n",
" -1.04856268e-01 5.07202963e-02 -8.07256922e-03 1.61524184e-01\n",
" -7.62400216e-02 4.32162899e-02 1.53487606e-01 1.16303543e-01\n",
" -9.74065605e-02 -7.49241748e-02 2.64838543e-02 -9.25019355e-02\n",
" 4.02053731e-03 3.67619137e-02 -2.09978867e-02 1.20947573e-01\n",
" -5.04959863e-03 -2.57215359e-02 1.87869045e-01 -4.54034098e-02\n",
" 1.05505109e-01 1.15917889e-01 1.01019537e-03 8.10575796e-02\n",
" 6.15996287e-02 -6.86831713e-02 1.15471822e-01 -1.58806165e-01\n",
" 8.59257187e-02 -6.80194560e-02 5.25809557e-02 1.29241793e-01\n",
" 2.96442257e-01 5.86340471e-02 -1.28423183e-01 -6.01744040e-02\n",
" 1.14950987e-01 -1.04963361e-01 -1.25614692e-02 -7.76409872e-02\n",
" -1.91352495e-02 -6.84886136e-02 -1.01946908e-01 7.26190380e-02\n",
" 7.37416629e-03 -4.53055978e-02 -2.66747019e-02 5.12159581e-02\n",
" -1.05212593e-01 6.66124574e-02 1.31252812e-01 3.63420298e-02\n",
" -7.14641192e-02 9.70643636e-03 7.18976380e-02 -1.23604353e-02\n",
" -1.24176108e-02 1.04690136e-01 5.39828740e-03 9.76624749e-02\n",
" 6.50910260e-02 7.12631412e-02 5.39964790e-02 -9.50968340e-02\n",
" -1.36926965e-01 -1.67338572e-01 1.71024317e-01 -4.31746660e-02\n",
" -3.39770440e-01 -2.12476333e-02 1.39387962e-02 -1.20733909e-01\n",
" -7.10107818e-02 -1.40207470e-01 -1.53325651e-01 -1.44034680e-01\n",
" -1.42187172e-02 4.82190337e-03 -9.58835595e-02 -1.25239836e-01\n",
" -3.11037821e-02 2.41683622e-02 -9.50108114e-03 -3.75821400e-02\n",
" 2.65741259e-02 -3.57189265e-02 -4.50321567e-02 7.31199908e-02\n",
" 7.68755832e-02 2.22411881e-02 -3.51710996e-02 -8.43992963e-02\n",
" -1.09629981e-01 8.29998118e-02 -1.07159941e-01 8.63919904e-02\n",
" -8.05284620e-02 1.39233426e-01 8.50788929e-02 -5.69528307e-02\n",
" -1.74747289e-03 6.02786603e-02 -1.29662675e-01 1.51356597e-01\n",
" 1.14494638e-02 -4.95706687e-02 9.41938160e-02 -8.14363465e-02\n",
" -8.82947986e-02 -7.91736730e-02 -9.91572754e-02 9.43910732e-03\n",
" 3.94720959e-02 5.21605495e-02 -6.60198127e-02 5.05199717e-03\n",
" 1.72427126e-01 1.99759250e-02 -4.67214130e-02 -1.36060459e-01\n",
" 2.12096243e-02 6.88640712e-02 -1.34119417e-01 2.64990332e-02\n",
" -1.65715417e-02 -5.53480523e-02 -2.56264650e-01 9.34297309e-02\n",
" 1.17750733e-01 6.82551963e-02 1.42595746e-01 2.17898743e-03\n",
" -1.22132586e-01 3.41369622e-02 9.97796267e-03 1.82852063e-02\n",
" -1.23661233e-01 -3.14363828e-02 1.45924115e-01 -2.74168188e-02\n",
" -1.70743963e-01 -4.24041414e-02 1.50026647e-01 1.13942805e-01\n",
" 9.83950225e-02 -2.28521568e-02 -8.74596363e-02 1.44233965e-02\n",
" 4.17401583e-02 2.00039699e-01 2.06897253e-02 5.31776953e-02\n",
" -5.48230966e-02 -1.48835571e-03 1.84402687e-01 -3.82521930e-02\n",
" -3.82894806e-02 -1.65995319e-01 3.23331283e-02 -1.12758032e-01\n",
" -7.04617555e-02 -1.43410773e-01 -7.58885157e-02 8.32975736e-02\n",
" -3.66208141e-02 -1.15579199e-01 -1.95759310e-02 6.23899274e-02\n",
" 1.10706862e-02 -1.97582027e-01 -3.18123691e-02 -1.24772821e-01\n",
" -8.19364596e-02 3.85059456e-02 2.52371285e-03 1.02048618e-01\n",
" -2.11977080e-01 -1.00900694e-01 1.92402556e-02 -3.68415330e-02\n",
" -1.07850992e-01 6.38785209e-02 -7.08707948e-02 1.21634759e-01\n",
" 1.45721454e-01 -9.42186109e-02 -1.55991677e-01 1.80152591e-01\n",
" 1.57254945e-01 -3.65897804e-02 -2.99067393e-02 -7.98993747e-02\n",
" -5.96615634e-02 1.38419220e-01 -2.18027629e-02 4.22037533e-02\n",
" 2.54258235e-01 1.65882478e-01 1.13829852e-01 1.22233607e-01\n",
" 6.70457515e-02 -3.56047258e-02 7.51963834e-02 -3.86212081e-02\n",
" 1.21353012e-02 9.45267045e-02 -1.42723216e-01 4.02014121e-02\n",
" 1.10302614e-01 -3.70178700e-02 -1.12214083e-01 -7.95965522e-02\n",
" 1.17638701e-02 -5.71211597e-02 8.05794183e-03 1.28764174e-01\n",
" 9.32423931e-02 1.26698056e-01 -2.54690099e-02 3.36396802e-02\n",
" -6.65502944e-02 -7.26224694e-02 -1.17771867e-01 -9.83117710e-02\n",
" -1.01189894e-01 1.25703006e-01 -1.87956188e-02 3.42859688e-02\n",
" -4.58310287e-02 -2.91817450e-02 -6.22237882e-02 9.88892296e-02\n",
" 1.67045851e-01 9.66810634e-02 -1.29928330e-01 8.08320503e-02\n",
" 7.16967510e-02 -2.07392222e-02 4.79989001e-02 9.64173565e-02\n",
" 3.15275901e-03 -3.11517624e-02 1.41608703e-01 -1.32065953e-01\n",
" 7.95776676e-02 1.01443107e-01 -3.97219607e-02 1.35702927e-01\n",
" 8.62457079e-03 4.71091523e-02 3.80673443e-03 -1.04136022e-01\n",
" -2.28789813e-01 -1.07083453e-01 -1.30958785e-02 6.37186715e-02\n",
" -1.64349934e-01 2.59403277e-02 -1.27339395e-01 -6.93078629e-02\n",
" -1.75796585e-01 -7.07985538e-02 1.09507694e-03 2.62531939e-02\n",
" -2.14016719e-03 -4.60804416e-02 1.38238801e-01 -4.77353705e-02\n",
" 4.17105210e-02 5.18725428e-02 -1.05532433e-01 3.25341374e-02\n",
" 1.23307384e-01 -1.58771892e-01 5.48099568e-02 -1.07696953e-01\n",
" -1.73781393e-02 -7.11075289e-02 -1.59958570e-01 -5.96227908e-02\n",
" -7.25574801e-02 -1.33758699e-01 8.28637436e-02 1.22675785e-03\n",
" 4.00161862e-02 -5.21819751e-02 -3.64242706e-02 7.55397997e-02\n",
" 8.23295048e-03 4.40921881e-02 5.57156803e-02 5.84725693e-02\n",
" 3.53546171e-02 -7.51859268e-03 3.79308301e-02 1.89543076e-02\n",
" 7.10311752e-02 -1.02858478e-01 1.40085299e-01 1.08367833e-03\n",
" 1.54873327e-01 9.56025311e-03 -1.65840164e-01 -1.63639525e-01\n",
" 1.39766315e-01 -9.12268448e-02 4.92465642e-03 5.62187928e-02\n",
" -1.95392868e-02 -1.22974898e-01 -1.01128124e-02 2.41926164e-02\n",
" 1.50253120e-01 -1.35249154e-02 -2.07806839e-01 2.50919996e-02\n",
" -5.83207062e-02 1.43024910e-01 4.31592059e-02 -9.82545628e-02\n",
" -1.13476631e-01 4.07274255e-02 -4.07917733e-02 1.88026086e-02\n",
" -1.77803485e-02 -7.30364824e-02 8.35099271e-02 -1.01066884e-01\n",
" 5.69377272e-02 8.64202530e-03 -8.27043957e-02 2.49491926e-02\n",
" 9.80547837e-02 8.74192948e-02 -1.55865276e-01 1.56006969e-01\n",
" -9.45179284e-02 1.87778876e-03 7.11275425e-02 -1.50052068e-01\n",
" 2.60634964e-02 -1.21370185e-01 -5.61218773e-02 4.76169083e-02\n",
" -1.06103357e-01 -2.24678701e-02 9.64560447e-02 -1.44790677e-01\n",
" 1.44041573e-01 2.59831347e-02 -1.48692886e-01 3.72903347e-02\n",
" -6.69701088e-02 -1.28525324e-02 -6.16591619e-02 -8.79610722e-03\n",
" 3.54905586e-02 2.55419573e-02 2.89435038e-02 5.43433365e-02\n",
" -1.00521002e-02 -2.13155995e-02 -9.77105738e-02 1.39976732e-01\n",
" -9.79831918e-02 1.63311248e-01 1.66223474e-01 -9.66849925e-02\n",
" 2.07154661e-01 1.85989485e-02 -7.17770321e-02 1.85046175e-02\n",
" -9.49511470e-03 -6.56468459e-02 -1.63668363e-01 5.46138983e-02\n",
" 1.92151676e-02 -5.51495319e-03 1.90691541e-02 -2.82951975e-01\n",
" 1.47836394e-01 2.12018747e-02 4.19543213e-02 6.73306764e-02\n",
" -2.07177648e-01 -5.57087178e-02 1.12369418e-01 5.22906940e-02\n",
" 4.09439991e-02 -3.82106037e-03 6.62684610e-02 7.75153683e-02\n",
" -1.82720335e-01 -3.46726867e-02 4.21394155e-02 -8.04155683e-02\n",
" -4.72100955e-03 1.34064117e-01 8.30033406e-02 1.62328213e-01\n",
" 4.20271828e-02 2.11395400e-01 4.02218106e-02 3.17014654e-02\n",
" 1.19848496e-01 9.19098433e-02 4.46309330e-02 1.41435345e-02\n",
" 9.41579559e-02 -1.59552092e-01 8.30743482e-04 6.69200847e-03\n",
" 9.17753249e-02 2.06699110e-03 -9.96837063e-02 -6.65841926e-02\n",
" 3.63179370e-02 7.66102584e-02 7.21451114e-02 5.88863707e-03\n",
" 2.37700611e-01 5.65460338e-02 2.73612392e-02 8.10616946e-02\n",
" 6.08917820e-02 -7.73588309e-02 -1.60888764e-01 1.28279857e-01\n",
" 1.05560326e-01 5.25876364e-02 -1.09809369e-01 3.24588206e-02\n",
" -1.04049065e-01 5.48738197e-02 -6.59409666e-02 2.55417377e-01\n",
" -3.69764015e-02 -3.47014603e-02 -3.21324952e-02 -3.00577655e-02\n",
" 1.09005328e-01 4.87301212e-02 -1.19949481e-02 -1.73671001e-02\n",
" -1.43853354e-01 -3.65072555e-02 -4.50227619e-02 5.23457696e-02\n",
" -1.46412423e-02 -1.71380226e-01 1.43650814e-01 -8.89799415e-02\n",
" -6.28181604e-02 -9.95530697e-02 -9.49923094e-02 3.62329642e-02\n",
" 8.24353924e-02 2.23326761e-02 1.07033108e-01 -4.17383760e-02]\n",
"Reg: 0.01085252892999154\n",
"Target values for D: [0 0 1 0 1 0 0 0 1 1 0 1 1 1 0 1 1 0 1 0 0 1 1 0 1 0 1 1 1 0 1 1 1 0 0 1 1\n",
" 1 1 1 1 0 0 1 1 1 1 0 0 0 1 0 1 1 1 0 0 1 1 1 1 0 1 1 1 0 1 1 1 0 1 0 1 0\n",
" 1 0 0 0 1 0 0 1 1 0 1 1 0 1 1 0 1 1 0 1 1 1 0 1 1 1 0 0 0 1 0 1 1 1 1 0 0\n",
" 0 0 1 0 0 0 0 0 0 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 1 1 1 1 1 1 1 1 0 1 0 0 0\n",
" 1 0 1 1 1 0 0 0 0 1 0 1 0 1 0 0 1 0 1 1 0 1 1 1 1 1 0 0 1 0 0 1 1 0 0 1 1\n",
" 0 1 1 0 0 1 1 0 0 1 1 1 1 0 1 1 0 0 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 1 1 1 0\n",
" 1 1 0 0 1 0 1 1 1 0 1 1 1 0 0 0 1 0 1 0 0 1 1 1 1 0 1 0 1 1 1 1 0 0 1 0 0\n",
" 1 0 1 1 1 1 1 1 1 0 1 1 0 1 1 1 0 0 1 0 0 0 1 0 0 1 1 1 0 1 1 0 0 1 0 1 0\n",
" 1 1 1 0 1 0 0 0 0 0 0 0 1 0 0 0 1 0 0 1 1 1 1 1 1 1 0 1 1 1 0 1 1 1 0 0 0\n",
" 0 1 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 1 0 1 0 0 1 1 1 1 0 1 0 1 1 0 0 0 1\n",
" 0 0 1 0 0 0 0 1 0 1 0 1 1 1 1 0 0 1 0 1 1 0 0 1 1 0 0 1 1 1]\n",
"Prediction on D: [0 0 1 0 1 0 0 0 1 1 0 1 1 1 0 1 1 0 1 0 0 1 1 0 1 0 1 1 1 0 1 1 1 0 0 1 1\n",
" 1 1 1 1 0 0 1 1 1 1 0 0 0 1 0 1 1 1 0 0 1 1 1 1 0 1 1 1 0 1 1 1 0 1 0 1 0\n",
" 1 0 0 0 1 0 0 1 1 0 1 1 0 1 1 0 1 1 0 1 1 1 0 1 1 1 0 0 0 1 0 1 1 1 1 0 0\n",
" 0 0 1 0 0 0 0 0 0 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 1 1 1 1 1 1 1 1 0 1 0 0 0\n",
" 1 0 1 1 1 0 0 0 0 1 0 1 0 1 0 0 1 0 1 1 0 1 1 1 1 1 0 0 1 0 0 1 1 0 0 1 1\n",
" 0 1 1 0 0 1 1 0 0 1 1 1 1 0 1 1 0 0 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 1 1 1 0\n",
" 1 1 0 0 1 0 1 1 1 0 1 1 1 0 0 0 1 0 1 0 0 1 1 1 1 0 1 0 1 1 1 1 0 0 1 0 0\n",
" 1 0 1 1 1 1 1 1 1 0 1 1 0 1 1 1 0 0 1 0 0 0 1 0 0 1 1 1 0 1 1 0 0 1 0 1 0\n",
" 1 1 1 0 1 0 0 0 0 0 0 0 1 0 0 0 1 0 0 1 1 1 1 1 1 1 0 1 1 1 0 1 1 1 0 0 0\n",
" 0 1 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 1 0 1 0 0 1 1 1 1 0 1 0 1 1 0 0 0 1\n",
" 0 0 1 0 0 0 0 1 0 1 0 1 1 1 1 0 0 1 0 1 1 0 0 1 1 0 0 1 1 1]\n"
]
}
],
"source": [
"import numpy\n",
"import theano\n",
"import theano.tensor as T\n",
"rng = numpy.random\n",
"\n",
"N = 400 # 事例数\n",
"feats = 784 # 特徴次元数\n",
"lmd = 0.01 # 正則化パラメータ\n",
"alpha = 0.1 # 学習率\n",
"D = (rng.randn(N, feats), rng.randint(size=N, low=0, high=2))\n",
"training_steps = 10000 # 学習時のIteration数\n",
"\n",
"x = T.matrix('x') # データ X=[xN^T]\n",
"y = T.vector('y') # ラベル y=[0 1 0 0....]^T\n",
"w = theano.shared(rng.randn(feats), name='w') # 重みベクトル\n",
"b = theano.shared(0., name='b') # 正則化項\n",
"print(\"Initial model:\")\n",
"print(\"weight vector: {w}\\nReg: {b}\".format(w=w.get_value(), b=b.get_value()))\n",
"\n",
"# 数式の実装\n",
"p_1 = 1 / (1 + T.exp(-T.dot(x, w) - b)) # 出力ラベルが1である確率\n",
"prediction = p_1 > 0.5 # True or False\n",
"loss = -y * T.log(p_1) - (1 - y) * T.log(1 - p_1) # クロスエントロピー損失関数\n",
"cost = loss.mean() + lmd * (w ** 2).sum() # 目的関数\n",
"gw, gb = T.grad(cost, [w, b]) # 勾配ベクトル\n",
"\n",
"# 数式のコンパイル\n",
"train = theano.function(\n",
" inputs=[x,y],\n",
" outputs=[prediction, loss],\n",
" updates=((w, w - alpha * gw), (b, b - alpha * gb)),\n",
" allow_input_downcast=True)\n",
"predict = theano.function(inputs=[x], outputs=prediction, allow_input_downcast=True)\n",
"\n",
"# 学習\n",
"for i in range(training_steps):\n",
" pred, err = train(D[0], D[1])\n",
"\n",
"print(\"Final model:\")\n",
"print(\"weight vector: {w}\\nReg: {b}\".format(w=w.get_value(), b=b.get_value()))\n",
"print(\"Target values for D:\", D[1])\n",
"print(\"Prediction on D:\", predict(D[0]))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.4.3"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment