Skip to content

Instantly share code, notes, and snippets.

@Orbifold

Orbifold/MathematicaLSTM.nb

Last active August 16, 2019 14:29
Show Gist options
  • Save Orbifold/7cb5128c06e014f9854c7048b9d2bf5c to your computer and use it in GitHub Desktop.
Save Orbifold/7cb5128c06e014f9854c7048b9d2bf5c to your computer and use it in GitHub Desktop.
Download ZIP
Learning to add with LSTM
Raw
MathematicaLSTM.nb
(* Content-type: application/vnd.wolfram.mathematica *)
(*** Wolfram Notebook File ***)
(* http://www.wolfram.com/nb *)
(* CreatedBy='Mathematica 11.3' *)
(*CacheID: 234*)
(* Internal cache information:
NotebookFileLineBreakTest
NotebookFileLineBreakTest
NotebookDataPosition[ 158, 7]
NotebookDataLength[ 74633, 1521]
NotebookOptionsPosition[ 72178, 1475]
NotebookOutlinePosition[ 72518, 1490]
CellTagsIndexPosition[ 72475, 1487]
WindowFrame->Normal*)
(* Beginning of Notebook Content *)
Notebook[{
Cell[CellGroupData[{
Cell["Learning to add with LSTM", "Title",
CellChangeTimes->{{3.744369183855311*^9,
3.744369217364102*^9}},ExpressionUUID->"32c93783-d88e-47ee-b059-\
234c2b3e852c"],
Cell["\<\
I know, Wolfram\[CloseCurlyQuote]s stuff ain\[CloseCurlyQuote]t the most \
popular when it comes to machine learning and AI. What pleases in their \
sandbox is the ease with which one can experiment with ideas without having \
to go very deep into various frameworks.
Their neural network API is based on MXNet and is straightforward to use. \
Below, for example, is a handful of lines of code through which addition is \
learned. Can\[CloseCurlyQuote]t beat the fun here.\
\>", "Text",
CellFrame->{{0, 0}, {2, 0}},
CellChangeTimes->{{3.744369226702145*^9, 3.744369267904488*^9}, {
3.774949544639586*^9,
3.774949722126099*^9}},ExpressionUUID->"fdafed7b-2746-4880-a7ac-\
555a1c373c29"],
Cell["\<\
First you need a bit of data of course. Just a matter of assembling a \
dictionary with simple additions and their result.
This rule takes two integers and returns a rule corresponding to the addition:\
\>", "Text",
CellChangeTimes->{{3.7749497426878633`*^9,
3.774949846658722*^9}},ExpressionUUID->"7f486880-4547-412d-a459-\
9dd2dd515d7f"],
Cell[BoxData[
RowBox[{
RowBox[{
RowBox[{"makeRule", "[",
RowBox[{"a_", ",", "b_"}], "]"}], ":=",
RowBox[{
RowBox[{
RowBox[{"IntegerString", "[", "a", "]"}], "<>", "\"\<+\>\"", "<>",
RowBox[{"IntegerString", "[", "b", "]"}]}], "\[Rule]",
RowBox[{"a", "+", "b"}]}]}], ";"}]], "Input",
CellChangeTimes->{{3.774949861777101*^9,
3.774949862797042*^9}},ExpressionUUID->"810ceca5-690e-4a65-9ebf-\
009bc803d42c"],
Cell["\<\
With this, one can compile a dataset consisting of all combinations from zero \
to hundred:\
\>", "Text",
CellChangeTimes->{{3.774949865828495*^9,
3.774949921082429*^9}},ExpressionUUID->"87620808-0bd8-41cc-adbf-\
5fbaafa21e1c"],
Cell[CellGroupData[{
Cell[BoxData[{
RowBox[{
RowBox[{"trainingData", "=",
RowBox[{"Flatten", "@",
RowBox[{"Table", "[",
RowBox[{
RowBox[{"makeRule", "[",
RowBox[{"i", ",", "j"}], "]"}], ",",
RowBox[{"{",
RowBox[{"i", ",", "0", ",", "99"}], "}"}], ",",
RowBox[{"{",
RowBox[{"j", ",", "0", ",", "99"}], "}"}]}], "]"}]}]}],
";"}], "\[IndentingNewLine]",
RowBox[{
RowBox[{"RandomSample", "[",
RowBox[{"trainingData", ",", "4"}], "]"}], "//", "InputForm"}]}], "Input",
CellLabel->"In[4]:=",ExpressionUUID->"164ded93-e4a5-44b0-9f61-7217f358b550"],
Cell["\<\
{\"76+87\" -> 163, \"87+14\" -> 101, \"91+22\" -> 113, \"78+36\" -> 114}\
\>", "Output",
CellChangeTimes->{3.744368401286501*^9, 3.774949850045703*^9,
3.774949885033712*^9},
CellLabel->
"Out[5]//InputForm=",ExpressionUUID->"26549ffc-05b3-47e0-bebc-8f5232a439de"]
}, Open ]],
Cell["\<\
A neural network does not understand strings so one has to convert things to \
a tensor. In this case a vector corresponding to the characters in the input \
string:\
\>", "Text",
CellChangeTimes->{{3.7749499422600727`*^9,
3.774949996666527*^9}},ExpressionUUID->"09f1e4f1-10af-464a-8aea-\
407542e4cc14"],
Cell[BoxData[
RowBox[{
RowBox[{"enc", "=",
RowBox[{"NetEncoder", "[",
RowBox[{"{",
RowBox[{"\"\<Characters\>\"", ",",
RowBox[{"{",
RowBox[{"DigitCharacter", ",", "\"\<+\>\""}], "}"}]}], "}"}], "]"}]}],
";"}]], "Input",
CellLabel->"In[6]:=",ExpressionUUID->"0f002a64-20e9-4089-9e5b-a92d8698de6a"],
Cell["For instance the following addition is converted to a vector", "Text",
CellChangeTimes->{{3.77495000996383*^9,
3.774950036102777*^9}},ExpressionUUID->"b3df4772-496a-4542-b1f4-\
f757a6b3354f"],
Cell[CellGroupData[{
Cell[BoxData[
RowBox[{"enc", "@", "\"\<97+60\>\""}]], "Input",
CellChangeTimes->{3.7749500224147987`*^9},
CellLabel->"In[7]:=",ExpressionUUID->"b2bebf00-70c9-4655-8456-b0e77432e0dd"],
Cell[BoxData[
RowBox[{"{",
RowBox[{"10", ",", "8", ",", "11", ",", "7", ",", "1"}], "}"}]], "Output",
CellChangeTimes->{3.744368438905314*^9, 3.774950022996832*^9},
CellLabel->"Out[7]=",ExpressionUUID->"aa992927-f8fc-48a3-a496-b5f9be46b744"]
}, Open ]],
Cell["\<\
Finally, the actual neural network is assembled using a couple of LSTM \
layers. You are free to take anything you like, composing neural networks are \
a bit of an art rather than a science. A LSTM layer has an internal memory \
across the input axis, which makes sense since the output of an addition is \
directly related to previous input (unlike, say, poems or music):\
\>", "Text",
CellChangeTimes->{{3.7749500512162647`*^9,
3.77495020221589*^9}},ExpressionUUID->"eb26b3a5-fcf9-4d50-b88f-\
8eed51775e1a"],
Cell[CellGroupData[{
Cell[BoxData[
RowBox[{"net", "=",
RowBox[{"NetChain", "[",
RowBox[{
RowBox[{"{",
RowBox[{
RowBox[{"UnitVectorLayer", "[", "]"}], ",",
RowBox[{"LongShortTermMemoryLayer", "[", "40", "]"}], ",",
RowBox[{"LongShortTermMemoryLayer", "[", "20", "]"}], ",",
RowBox[{"SequenceLastLayer", "[", "]"}], ",",
RowBox[{"LinearLayer", "[", "]"}]}], "}"}], ",",
RowBox[{"\"\<Input\>\"", "\[Rule]", "enc"}], ",",
RowBox[{"\"\<Output\>\"", "\[Rule]", "\"\<Real\>\""}]}], "]"}]}]], "Input",\
CellLabel->"In[8]:=",ExpressionUUID->"4b7f764e-3a64-4d6c-8aea-62f73c9dfa5c"],
Cell[BoxData[
TagBox[
TemplateBox[{RowBox[{
StyleBox[
TagBox["NetChain", "SummaryHead"], "NonInterpretableSummary"],
StyleBox["[", "NonInterpretableSummary"],
DynamicModuleBox[{Typeset`open = True},
PanelBox[
PaneSelectorBox[{False -> GridBox[{{
PaneBox[
ButtonBox[
DynamicBox[
FEPrivate`FrontEndResource[
"FEBitmaps", "SquarePlusIconMedium"],
ImageSizeCache -> {12., {0., 12.}}], Appearance -> None,
ButtonFunction :> (Typeset`open = True), Evaluator ->
Automatic, Method -> "Preemptive"],
Alignment -> {Center, Center}, ImageSize -> {Automatic, 24}],
OverlayBox[{
GraphicsBox[{
Thickness[0.014925373134328358`], {
GrayLevel[0.6],
Thickness[0.018686567164179105`],
Opacity[1.],
JoinForm[{"Miter", 10.}],
JoinedCurveBox[{{{0, 2, 0}}}, {{{23., 91.5}, {30., 91.5}}},
CurveClosed -> {0}]}, {
GrayLevel[0.6],
Thickness[0.018686567164179105`],
Opacity[1.],
JoinForm[{"Miter", 10.}],
JoinedCurveBox[{{{0, 2, 0}}}, {{{37., 91.5}, {44., 91.5}}},
CurveClosed -> {0}]}, {
FaceForm[{
GrayLevel[0.9],
Opacity[1.]}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}, {0, 1, 0}}}, {{{
15.5, 103.5}, {23.5, 103.5}, {23.5, 79.5}, {15.5,
79.5}}}]}, {
FaceForm[{
GrayLevel[0.6],
Opacity[1.]}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}, {0, 1, 0}}, {{0, 2,
0}, {0, 1, 0}, {0, 1, 0}}}, {{{24., 104.}, {15., 104.}, {
15., 79.}, {24., 79.}}, {{23., 80.}, {16., 80.}, {16.,
103.}, {23., 103.}}}]}, {
FaceForm[{
GrayLevel[0.9],
Opacity[1.]}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}, {0, 1, 0}}}, {{{
29.5, 103.5}, {37.5, 103.5}, {37.5, 79.5}, {29.5,
79.5}}}]}, {
FaceForm[{
GrayLevel[0.6],
Opacity[1.]}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}, {0, 1, 0}}, {{0, 2,
0}, {0, 1, 0}, {0, 1, 0}}}, {{{38., 104.}, {29., 104.}, {
29., 79.}, {38., 79.}}, {{37., 80.}, {30., 80.}, {30.,
103.}, {37., 103.}}}]}, {
FaceForm[{
GrayLevel[0.9],
Opacity[1.]}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}, {0, 1, 0}}}, {{{
43.5, 103.5}, {51.5, 103.5}, {51.5, 79.5}, {43.5,
79.5}}}]}, {
FaceForm[{
GrayLevel[0.6],
Opacity[1.]}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}, {0, 1, 0}}, {{0, 2,
0}, {0, 1, 0}, {0, 1, 0}}}, {{{52., 104.}, {43., 104.}, {
43., 79.}, {52., 79.}}, {{51., 80.}, {44., 80.}, {44.,
103.}, {51., 103.}}}]}, {
FaceForm[{
GrayLevel[0.6],
Opacity[1.]}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}}}, {{{28., 91.5}, {
25., 95.}, {25., 88.}}}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}}}, {{{42., 91.5}, {
39., 95.}, {39., 88.}}}]}}, BaseStyle -> GrayLevel[0.6],
ImageMargins -> 0, ImageSize -> {36, Automatic},
PlotRangePadding -> 0],
StyleBox[
"uninitialized", LineColor -> RGBColor[0.66, 0, 0],
FrontFaceColor -> RGBColor[0.66, 0, 0], BackFaceColor ->
RGBColor[0.66, 0, 0], GraphicsColor -> RGBColor[0.66, 0, 0],
FontFamily -> "Roboto", FontSize -> 8, FontColor ->
RGBColor[0.66, 0, 0], Background -> GrayLevel[1, 0.65]]},
Alignment -> {Center,
Scaled[0.0001]}],
GridBox[{{
StyleBox[
TemplateBox[{
TemplateBox[{
StyleBox["\"Input\"",
GrayLevel[0], StripOnInput -> False], "\" \"",
"\"port\""}, "RowDefault"], "\":\""}, "RowDefault"],
"SummaryItemAnnotation"],
StyleBox["\"string\"", "SummaryItem"]}, {
StyleBox[
TemplateBox[{
TemplateBox[{
StyleBox["\"Output\"",
GrayLevel[0], StripOnInput -> False], "\" \"",
"\"port\""}, "RowDefault"], "\":\""}, "RowDefault"],
"SummaryItemAnnotation"],
StyleBox["\"real\"", "SummaryItem"]}, {
StyleBox[
TemplateBox[{"\"Number of layers\"", "\":\""},
"RowDefault"], "SummaryItemAnnotation"],
StyleBox["5", "SummaryItem"]}}, BaselinePosition -> {2, 1},
GridBoxAlignment -> {
"Columns" -> {{Left}}, "Rows" -> {{Automatic}}},
GridBoxItemSize -> {
"Columns" -> {{Automatic}}, "Rows" -> {{Automatic}}},
GridBoxSpacings -> {
"Columns" -> {{2}}, "Rows" -> {{Automatic}}}]}},
BaselinePosition -> {1, 3},
GridBoxAlignment -> {"Rows" -> {{Top}}},
GridBoxItemSize -> {
"Columns" -> {{Automatic}}, "Rows" -> {{Automatic}}},
GridBoxSpacings -> {"Columns" -> {
Offset[0.], {
Offset[0.5599999999999999]},
Offset[0.]}, "Rows" -> {
Offset[0.], {
Offset[0.4]},
Offset[0.]}}], True -> GridBox[{{
PaneBox[
ButtonBox[
DynamicBox[
FEPrivate`FrontEndResource[
"FEBitmaps", "SquareMinusIconMedium"],
ImageSizeCache -> {12., {0., 12.}}], Appearance -> None,
ButtonFunction :> (Typeset`open = False), Evaluator ->
Automatic, Method -> "Preemptive"],
Alignment -> {Center, Center}, ImageSize -> {Automatic, 24}],
OverlayBox[{
GraphicsBox[{
Thickness[0.014925373134328358`], {
GrayLevel[0.6],
Thickness[0.018686567164179105`],
Opacity[1.],
JoinForm[{"Miter", 10.}],
JoinedCurveBox[{{{0, 2, 0}}}, {{{23., 91.5}, {30., 91.5}}},
CurveClosed -> {0}]}, {
GrayLevel[0.6],
Thickness[0.018686567164179105`],
Opacity[1.],
JoinForm[{"Miter", 10.}],
JoinedCurveBox[{{{0, 2, 0}}}, {{{37., 91.5}, {44., 91.5}}},
CurveClosed -> {0}]}, {
FaceForm[{
GrayLevel[0.9],
Opacity[1.]}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}, {0, 1, 0}}}, {{{
15.5, 103.5}, {23.5, 103.5}, {23.5, 79.5}, {15.5,
79.5}}}]}, {
FaceForm[{
GrayLevel[0.6],
Opacity[1.]}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}, {0, 1, 0}}, {{0, 2,
0}, {0, 1, 0}, {0, 1, 0}}}, {{{24., 104.}, {15., 104.}, {
15., 79.}, {24., 79.}}, {{23., 80.}, {16., 80.}, {16.,
103.}, {23., 103.}}}]}, {
FaceForm[{
GrayLevel[0.9],
Opacity[1.]}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}, {0, 1, 0}}}, {{{
29.5, 103.5}, {37.5, 103.5}, {37.5, 79.5}, {29.5,
79.5}}}]}, {
FaceForm[{
GrayLevel[0.6],
Opacity[1.]}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}, {0, 1, 0}}, {{0, 2,
0}, {0, 1, 0}, {0, 1, 0}}}, {{{38., 104.}, {29., 104.}, {
29., 79.}, {38., 79.}}, {{37., 80.}, {30., 80.}, {30.,
103.}, {37., 103.}}}]}, {
FaceForm[{
GrayLevel[0.9],
Opacity[1.]}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}, {0, 1, 0}}}, {{{
43.5, 103.5}, {51.5, 103.5}, {51.5, 79.5}, {43.5,
79.5}}}]}, {
FaceForm[{
GrayLevel[0.6],
Opacity[1.]}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}, {0, 1, 0}}, {{0, 2,
0}, {0, 1, 0}, {0, 1, 0}}}, {{{52., 104.}, {43., 104.}, {
43., 79.}, {52., 79.}}, {{51., 80.}, {44., 80.}, {44.,
103.}, {51., 103.}}}]}, {
FaceForm[{
GrayLevel[0.6],
Opacity[1.]}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}}}, {{{28., 91.5}, {
25., 95.}, {25., 88.}}}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}}}, {{{42., 91.5}, {
39., 95.}, {39., 88.}}}]}}, BaseStyle -> GrayLevel[0.6],
ImageMargins -> 0, ImageSize -> {36, Automatic},
PlotRangePadding -> 0],
StyleBox[
"uninitialized", LineColor -> RGBColor[0.66, 0, 0],
FrontFaceColor -> RGBColor[0.66, 0, 0], BackFaceColor ->
RGBColor[0.66, 0, 0], GraphicsColor -> RGBColor[0.66, 0, 0],
FontFamily -> "Roboto", FontSize -> 8, FontColor ->
RGBColor[0.66, 0, 0], Background -> GrayLevel[1, 0.65]]},
Alignment -> {Center,
Scaled[0.0001]}],
GridBox[{{
DynamicModuleBox[{
NeuralNetworks`Private`NetChain`assoc3 =
Association[
"Type" -> "Chain", "Nodes" ->
Association[
"1" -> Association[
"Type" -> "UnitVector", "Arrays" -> Association[],
"Parameters" ->
Association["ClassCount" -> 11, "$Dimensions" -> {
NeuralNetworks`LengthVar[1973845059]}], "Inputs" ->
Association["Input" -> NeuralNetworks`TensorT[{
NeuralNetworks`LengthVar[1973845059]},
NeuralNetworks`IndexIntegerT[11]]], "Outputs" ->
Association["Output" -> NeuralNetworks`TensorT[{
NeuralNetworks`LengthVar[1973845059], 11},
NeuralNetworks`RealT]]], "2" ->
Association[
"Type" -> "LongShortTermMemory", "Arrays" ->
Association[
"InputGateInputWeights" ->
NeuralNetworks`TensorT[{40, 11}, NeuralNetworks`RealT],
"InputGateStateWeights" ->
NeuralNetworks`TensorT[{40, 40}, NeuralNetworks`RealT],
"InputGateBiases" ->
NeuralNetworks`TensorT[{40}, NeuralNetworks`RealT],
"OutputGateInputWeights" ->
NeuralNetworks`TensorT[{40, 11}, NeuralNetworks`RealT],
"OutputGateStateWeights" ->
NeuralNetworks`TensorT[{40, 40}, NeuralNetworks`RealT],
"OutputGateBiases" ->
NeuralNetworks`TensorT[{40}, NeuralNetworks`RealT],
"ForgetGateInputWeights" ->
NeuralNetworks`TensorT[{40, 11}, NeuralNetworks`RealT],
"ForgetGateStateWeights" ->
NeuralNetworks`TensorT[{40, 40}, NeuralNetworks`RealT],
"ForgetGateBiases" ->
NeuralNetworks`TensorT[{40}, NeuralNetworks`RealT],
"MemoryGateInputWeights" ->
NeuralNetworks`TensorT[{40, 11}, NeuralNetworks`RealT],
"MemoryGateStateWeights" ->
NeuralNetworks`TensorT[{40, 40}, NeuralNetworks`RealT],
"MemoryGateBiases" ->
NeuralNetworks`TensorT[{40}, NeuralNetworks`RealT]],
"Parameters" ->
Association[
"OutputSize" -> 40, "Dropout" ->
NeuralNetworks`ValidatedParameter[None], "$InputSize" ->
11, "$SequenceLength" ->
NeuralNetworks`LengthVar[1973845059],
"$CellStateConnectedQ" -> False], "Inputs" ->
Association["Input" -> NeuralNetworks`TensorT[{
NeuralNetworks`LengthVar[1973845059], 11},
NeuralNetworks`RealT]], "Outputs" ->
Association["Output" -> NeuralNetworks`TensorT[{
NeuralNetworks`LengthVar[1973845059], 40},
NeuralNetworks`RealT]], "States" ->
Association[
"State" ->
NeuralNetworks`TensorT[{40}, NeuralNetworks`RealT],
"CellState" ->
NeuralNetworks`TensorT[{40}, NeuralNetworks`RealT]]], "3" ->
Association[
"Type" -> "LongShortTermMemory", "Arrays" ->
Association[
"InputGateInputWeights" ->
NeuralNetworks`TensorT[{20, 40}, NeuralNetworks`RealT],
"InputGateStateWeights" ->
NeuralNetworks`TensorT[{20, 20}, NeuralNetworks`RealT],
"InputGateBiases" ->
NeuralNetworks`TensorT[{20}, NeuralNetworks`RealT],
"OutputGateInputWeights" ->
NeuralNetworks`TensorT[{20, 40}, NeuralNetworks`RealT],
"OutputGateStateWeights" ->
NeuralNetworks`TensorT[{20, 20}, NeuralNetworks`RealT],
"OutputGateBiases" ->
NeuralNetworks`TensorT[{20}, NeuralNetworks`RealT],
"ForgetGateInputWeights" ->
NeuralNetworks`TensorT[{20, 40}, NeuralNetworks`RealT],
"ForgetGateStateWeights" ->
NeuralNetworks`TensorT[{20, 20}, NeuralNetworks`RealT],
"ForgetGateBiases" ->
NeuralNetworks`TensorT[{20}, NeuralNetworks`RealT],
"MemoryGateInputWeights" ->
NeuralNetworks`TensorT[{20, 40}, NeuralNetworks`RealT],
"MemoryGateStateWeights" ->
NeuralNetworks`TensorT[{20, 20}, NeuralNetworks`RealT],
"MemoryGateBiases" ->
NeuralNetworks`TensorT[{20}, NeuralNetworks`RealT]],
"Parameters" ->
Association[
"OutputSize" -> 20, "Dropout" ->
NeuralNetworks`ValidatedParameter[None], "$InputSize" ->
40, "$SequenceLength" ->
NeuralNetworks`LengthVar[1973845059],
"$CellStateConnectedQ" -> False], "Inputs" ->
Association["Input" -> NeuralNetworks`TensorT[{
NeuralNetworks`LengthVar[1973845059], 40},
NeuralNetworks`RealT]], "Outputs" ->
Association["Output" -> NeuralNetworks`TensorT[{
NeuralNetworks`LengthVar[1973845059], 20},
NeuralNetworks`RealT]], "States" ->
Association[
"State" ->
NeuralNetworks`TensorT[{20}, NeuralNetworks`RealT],
"CellState" ->
NeuralNetworks`TensorT[{20}, NeuralNetworks`RealT]]], "4" ->
Association[
"Type" -> "SequenceLast", "Arrays" -> Association[],
"Parameters" -> Association[], "Inputs" ->
Association["Input" -> NeuralNetworks`TensorT[{
NeuralNetworks`LengthVar[1973845059], 20},
NeuralNetworks`RealT]], "Outputs" ->
Association[
"Output" ->
NeuralNetworks`TensorT[{20}, NeuralNetworks`RealT]]], "5" ->
Association[
"Type" -> "Linear", "Arrays" ->
Association[
"Weights" ->
NeuralNetworks`TensorT[{1, 20}, NeuralNetworks`RealT],
"Biases" -> NeuralNetworks`Nullable[
NeuralNetworks`TensorT[{1}, NeuralNetworks`RealT]]],
"Parameters" ->
Association[
"OutputDimensions" -> {}, "$OutputSize" -> 1,
"$InputSize" -> 20, "$InputDimensions" -> {20}], "Inputs" ->
Association[
"Input" ->
NeuralNetworks`TensorT[{20}, NeuralNetworks`RealT]],
"Outputs" ->
Association[
"Output" ->
NeuralNetworks`TensorT[{}, NeuralNetworks`RealT]]]],
"Edges" -> {
NeuralNetworks`NetPath["Nodes", "1", "Inputs", "Input"] ->
NeuralNetworks`NetPath["Inputs", "Input"],
NeuralNetworks`NetPath["Nodes", "2", "Inputs", "Input"] ->
NeuralNetworks`NetPath["Nodes", "1", "Outputs", "Output"],
NeuralNetworks`NetPath["Nodes", "3", "Inputs", "Input"] ->
NeuralNetworks`NetPath["Nodes", "2", "Outputs", "Output"],
NeuralNetworks`NetPath["Nodes", "4", "Inputs", "Input"] ->
NeuralNetworks`NetPath["Nodes", "3", "Outputs", "Output"],
NeuralNetworks`NetPath["Nodes", "5", "Inputs", "Input"] ->
NeuralNetworks`NetPath["Nodes", "4", "Outputs", "Output"],
NeuralNetworks`NetPath["Outputs", "Output"] ->
NeuralNetworks`NetPath[
"Nodes", "5", "Outputs", "Output"]}, "Inputs" ->
Association[
"Input" -> Association["Form" -> NeuralNetworks`TensorT[{
NeuralNetworks`LengthVar[1973845059]},
NeuralNetworks`IndexIntegerT[11]], "Type" -> "Characters",
"Encoding" -> NeuralNetworks`ValidatedParameter[
MXNetLink`CharacterEncodingData[
1, RawArray["UnsignedInteger8",{128, 0, 0, 128, 128, 128,
128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128,
128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128,
128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128,
128, 128, 128, 128, 128, 128, 11, 128, 128, 128, 128, 1,
2, 3, 4, 5, 6, 7, 8, 9, 10, 128, 128, 128, 128, 128, 128,
128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128,
128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128,
128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128,
128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128,
128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128,
128, 128, 128, 128, 128, 128, 128, 128, 128}],
RawArray["UnsignedInteger8",{48, 49, 50, 51, 52, 53, 54,
55, 56, 57, 43}], 11, {DigitCharacter, "+"}]],
"OutputForm" -> "Index", "IgnoreCase" -> False, "Count" ->
11, "TargetLength" -> All, "$TargetLength" ->
NeuralNetworks`LengthVar[1601547397], "$Padding" ->
Automatic, "$Version" -> "12.0.10"]], "Outputs" ->
Association[
"Output" ->
NeuralNetworks`TensorT[{}, NeuralNetworks`RealT]],
"InteriorStates" ->
Association[{2, "State"} ->
NeuralNetworks`NetPath["Nodes", "2", "States", "State"], {
2, "CellState"} ->
NeuralNetworks`NetPath[
"Nodes", "2", "States", "CellState"], {3, "State"} ->
NeuralNetworks`NetPath["Nodes", "3", "States", "State"], {
3, "CellState"} ->
NeuralNetworks`NetPath[
"Nodes", "3", "States", "CellState"]]],
NeuralNetworks`Private`NetChain`opart,
NeuralNetworks`Private`NetChain`part,
NeuralNetworks`Private`NetChain`selected = Null},
DynamicBox[
GridBox[{{
NeuralNetworks`Private`NetChain`MouseClickBoxes[
TagBox[
GridBox[{{
TagBox[
TagBox[
"\"\"", Annotation[#, {"Inputs", "Input"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]],
TagBox[
TagBox[
"\"Input\"",
Annotation[#, {"Inputs", "Input"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]],
TagBox[
TagBox[
TagBox[
GridBox[{{"\"string\""}, {
TemplateBox[{"\"vector of \"",
StyleBox["\"n\"", Italic, StripOnInput -> False], "\" \"",
TemplateBox[{"\"indices\"", "\" \"",
StyleBox[
TemplateBox[{"\"(\"", "\"\[VeryThinSpace]\"", "\"range\"",
"\":\"", "\" \"",
TemplateBox[{"1", "\"..\"", "11"}, "RowDefault"],
"\"\[VeryThinSpace]\"", "\")\""}, "RowDefault"],
GrayLevel[0.5], StripOnInput -> False]}, "RowDefault"]},
"RowDefault"]}},
GridBoxAlignment -> {"Columns" -> {{Left}}},
BaselinePosition -> 2, DefaultBaseStyle -> "Column",
GridBoxItemSize -> {
"Columns" -> {{Automatic}}, "Rows" -> {{Automatic}}}],
"Column"], Annotation[#, {"Inputs", "Input"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]]}, {
TagBox[
TagBox[
StyleBox["\"1\"",
GrayLevel[0.5], StripOnInput -> False],
Annotation[#, {"Nodes", "1"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]],
TagBox[
TagBox[
StyleBox["UnitVectorLayer",
GrayLevel[0], StripOnInput -> False],
Annotation[#, {"Nodes", "1"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]],
TagBox[
TagBox[
TemplateBox[{"\"matrix\"", "\" \"",
StyleBox[
TemplateBox[{"\"(\"", "\"\[VeryThinSpace]\"", "\"size\"",
"\":\"", "\" \"",
TemplateBox[{"\[Times]", "\"\[Times]\"",
StyleBox["\"n\"", Italic, StripOnInput -> False],
"\"11\""}, "RowWithSeparators"], "\"\[VeryThinSpace]\"",
"\")\""}, "RowDefault"],
GrayLevel[0.5], StripOnInput -> False]}, "RowDefault"],
Annotation[#, {"Nodes", "1"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]]}, {
TagBox[
TagBox[
StyleBox["\"2\"",
GrayLevel[0.5], StripOnInput -> False],
Annotation[#, {"Nodes", "2"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]],
TagBox[
TagBox[
StyleBox["LongShortTermMemoryLayer",
RGBColor[0.66, 0, 0], StripOnInput -> False],
Annotation[#, {"Nodes", "2"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]],
TagBox[
TagBox[
TemplateBox[{"\"matrix\"", "\" \"",
StyleBox[
TemplateBox[{"\"(\"", "\"\[VeryThinSpace]\"", "\"size\"",
"\":\"", "\" \"",
TemplateBox[{"\[Times]", "\"\[Times]\"",
StyleBox["\"n\"", Italic, StripOnInput -> False],
"\"40\""}, "RowWithSeparators"], "\"\[VeryThinSpace]\"",
"\")\""}, "RowDefault"],
GrayLevel[0.5], StripOnInput -> False]}, "RowDefault"],
Annotation[#, {"Nodes", "2"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]]}, {
TagBox[
TagBox[
StyleBox["\"3\"",
GrayLevel[0.5], StripOnInput -> False],
Annotation[#, {"Nodes", "3"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]],
TagBox[
TagBox[
StyleBox["LongShortTermMemoryLayer",
RGBColor[0.66, 0, 0], StripOnInput -> False],
Annotation[#, {"Nodes", "3"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]],
TagBox[
TagBox[
TemplateBox[{"\"matrix\"", "\" \"",
StyleBox[
TemplateBox[{"\"(\"", "\"\[VeryThinSpace]\"", "\"size\"",
"\":\"", "\" \"",
TemplateBox[{"\[Times]", "\"\[Times]\"",
StyleBox["\"n\"", Italic, StripOnInput -> False],
"\"20\""}, "RowWithSeparators"], "\"\[VeryThinSpace]\"",
"\")\""}, "RowDefault"],
GrayLevel[0.5], StripOnInput -> False]}, "RowDefault"],
Annotation[#, {"Nodes", "3"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]]}, {
TagBox[
TagBox[
StyleBox["\"4\"",
GrayLevel[0.5], StripOnInput -> False],
Annotation[#, {"Nodes", "4"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]],
TagBox[
TagBox[
StyleBox["SequenceLastLayer",
GrayLevel[0], StripOnInput -> False],
Annotation[#, {"Nodes", "4"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]],
TagBox[
TagBox[
TemplateBox[{"\"vector\"", "\" \"",
StyleBox[
TemplateBox[{"\"(\"", "\"\[VeryThinSpace]\"", "\"size\"",
"\":\"", "\" \"",
TemplateBox[{"\[Times]", "\"\[Times]\"", "\"20\""},
"RowWithSeparators"], "\"\[VeryThinSpace]\"", "\")\""},
"RowDefault"],
GrayLevel[0.5], StripOnInput -> False]}, "RowDefault"],
Annotation[#, {"Nodes", "4"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]]}, {
TagBox[
TagBox[
StyleBox["\"5\"",
GrayLevel[0.5], StripOnInput -> False],
Annotation[#, {"Nodes", "5"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]],
TagBox[
TagBox[
StyleBox["LinearLayer",
RGBColor[0.66, 0, 0], StripOnInput -> False],
Annotation[#, {"Nodes", "5"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]],
TagBox[
TagBox[
"\"real\"", Annotation[#, {"Nodes", "5"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]]}, {
TagBox[
TagBox[
"\"\"", Annotation[#, {"Outputs", "Output"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]],
TagBox[
TagBox[
"\"Output\"",
Annotation[#, {"Outputs", "Output"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]],
TagBox[
TagBox[
"\"real\"",
Annotation[#, {"Outputs", "Output"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]]}},
GridBoxAlignment -> {"Columns" -> {{Left}}}, AutoDelete ->
False, GridBoxItemSize -> {
"Columns" -> {{Automatic}}, "Rows" -> {{Automatic}}},
GridBoxSpacings -> {"Columns" -> {{1.1}}}], "Grid"],
If[
ListQ[
NeuralNetworks`Private`NetChain`part = MouseAnnotation[]],
If[NeuralNetworks`Private`NetChain`opart ===
NeuralNetworks`Private`NetChain`part,
NeuralNetworks`Private`NetChain`selected = Null;
NeuralNetworks`Private`NetChain`opart = Null,
NeuralNetworks`Private`NetChain`selected =
Part[NeuralNetworks`Private`NetChain`assoc3,
Apply[Sequence, NeuralNetworks`Private`NetChain`part]];
NeuralNetworks`Private`NetChain`opart =
NeuralNetworks`Private`NetChain`part; Null]; Null]]},
NeuralNetworks`Private`NetChain`fmtSelected[
NeuralNetworks`Private`NetChain`selected,
NeuralNetworks`Private`NetChain`part,
NeuralNetworks`Private`GetSharedArrays[
NeuralNetworks`Private`NetChain`assoc3]]},
GridBoxSpacings -> {"Columns" -> {{1}}},
GridBoxAlignment -> {"Columns" -> {{Left}}},
GridFrameMargins -> {{0, 0}, {0, 0}}],
ImageSizeCache -> {316., {54.4169921875, 59.5830078125}},
TrackedSymbols :> {
NeuralNetworks`Private`NetChain`selected}],
Initialization :> {NetChain}]}}, BaselinePosition ->
Automatic,
GridBoxAlignment -> {
"Columns" -> {{Left}}, "Rows" -> {{Automatic}}},
GridBoxItemSize -> {
"Columns" -> {{Automatic}}, "Rows" -> {{Automatic}}},
GridBoxSpacings -> {
"Columns" -> {{2}}, "Rows" -> {{Automatic}}}]}},
BaselinePosition -> {1, 1},
GridBoxAlignment -> {"Rows" -> {{Top}}},
GridBoxItemSize -> {
"Columns" -> {{Automatic}}, "Rows" -> {{Automatic}}},
GridBoxSpacings -> {"Columns" -> {
Offset[0.], {
Offset[0.5599999999999999]},
Offset[0.]}, "Rows" -> {
Offset[0.], {
Offset[0.4]},
Offset[0.]}}]},
Dynamic[Typeset`open], ImageSize -> Automatic],
BaseStyle -> {
ShowStringCharacters -> False, NumberMarks -> False, PrintPrecision ->
3, ShowSyntaxStyles -> False}, BaselinePosition -> Baseline]],
StyleBox["]", "NonInterpretableSummary"]}]},
"CopyTag",
DisplayFunction->(#& ),
InterpretationFunction->("NetChain[<>]"& )],
False,
Editable->False,
SelectWithContents->True,
Selectable->False]], "Output",
CellChangeTimes->{3.744368483088208*^9, 3.774950207458829*^9},
CellLabel->"Out[8]=",ExpressionUUID->"d2637d9d-9020-48c7-8e3d-106da36cbcb5"]
}, Open ]],
Cell["\<\
The training is also straightforward and we use 10% of the data for \
validation:\
\>", "Text",
CellChangeTimes->{{3.774950226939666*^9,
3.774950249331353*^9}},ExpressionUUID->"18f7bcbd-cfed-4c36-b314-\
a2a81b8d6f53"],
Cell[CellGroupData[{
Cell[BoxData[
RowBox[{"trained", "=",
RowBox[{"NetTrain", "[",
RowBox[{"net", ",", "trainingData", ",",
RowBox[{"BatchSize", "\[Rule]", "64"}], ",",
RowBox[{"MaxTrainingRounds", "\[Rule]", "200"}], ",",
RowBox[{"ValidationSet", "\[Rule]",
RowBox[{"Scaled", "[", "0.1", "]"}]}]}], "]"}]}]], "Input",
CellLabel->"In[9]:=",ExpressionUUID->"d79543c6-4a5d-4906-b458-c250d99304ae"],
Cell[BoxData[
TagBox[
TemplateBox[{RowBox[{
StyleBox[
TagBox["NetChain", "SummaryHead"], "NonInterpretableSummary"],
StyleBox["[", "NonInterpretableSummary"],
DynamicModuleBox[{Typeset`open = False},
PanelBox[
PaneSelectorBox[{False -> GridBox[{{
PaneBox[
ButtonBox[
DynamicBox[
FEPrivate`FrontEndResource[
"FEBitmaps", "SquarePlusIconMedium"]],
ButtonFunction :> (Typeset`open = True), Appearance -> None,
Evaluator -> Automatic, Method -> "Preemptive"],
Alignment -> {Center, Center}, ImageSize -> {Automatic, 24}],
GraphicsBox[{
Thickness[0.014925373134328358`], {
RGBColor[0.729, 0.729, 0.729],
Thickness[0.018686567164179105`],
Opacity[1.],
JoinForm[{"Miter", 10.}],
JoinedCurveBox[{{{0, 2, 0}}}, {{{23., 91.5}, {30., 91.5}}},
CurveClosed -> {0}]}, {
RGBColor[0.729, 0.729, 0.729],
Thickness[0.018686567164179105`],
Opacity[1.],
JoinForm[{"Miter", 10.}],
JoinedCurveBox[{{{0, 2, 0}}}, {{{37., 91.5}, {44., 91.5}}},
CurveClosed -> {0}]}, {
FaceForm[{
RGBColor[0.631, 0.8160000000000001, 0.498],
Opacity[1.]}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}, {0, 1, 0}}}, {{{15.5,
103.5}, {23.5, 103.5}, {23.5, 79.5}, {15.5, 79.5}}}]}, {
FaceForm[{
RGBColor[0.40800000000000003`, 0.627, 0.196],
Opacity[1.]}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}, {0, 1, 0}}, {{0, 2,
0}, {0, 1, 0}, {0, 1, 0}}}, {{{24., 104.}, {15., 104.}, {
15., 79.}, {24., 79.}}, {{23., 80.}, {16., 80.}, {16.,
103.}, {23., 103.}}}]}, {
FaceForm[{
RGBColor[0.38, 0.753, 0.89],
Opacity[1.]}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}, {0, 1, 0}}}, {{{29.5,
103.5}, {37.5, 103.5}, {37.5, 79.5}, {29.5, 79.5}}}]}, {
FaceForm[{
RGBColor[0.24300000000000002`, 0.608, 0.71],
Opacity[1.]}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}, {0, 1, 0}}, {{0, 2,
0}, {0, 1, 0}, {0, 1, 0}}}, {{{38., 104.}, {29., 104.}, {
29., 79.}, {38., 79.}}, {{37., 80.}, {30., 80.}, {30.,
103.}, {37., 103.}}}]}, {
FaceForm[{
RGBColor[0.584, 0.51, 0.8390000000000001],
Opacity[1.]}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}, {0, 1, 0}}}, {{{43.5,
103.5}, {51.5, 103.5}, {51.5, 79.5}, {43.5, 79.5}}}]}, {
FaceForm[{
RGBColor[0.4, 0.396, 0.659],
Opacity[1.]}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}, {0, 1, 0}}, {{0, 2,
0}, {0, 1, 0}, {0, 1, 0}}}, {{{52., 104.}, {43., 104.}, {
43., 79.}, {52., 79.}}, {{51., 80.}, {44., 80.}, {44.,
103.}, {51., 103.}}}]}, {
FaceForm[{
RGBColor[0.549, 0.549, 0.549],
Opacity[1.]}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}}}, {{{28., 91.5}, {25.,
95.}, {25., 88.}}}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}}}, {{{42., 91.5}, {39.,
95.}, {39., 88.}}}]}}, ImageMargins -> 0,
ImageSize -> {36, Automatic}, PlotRangePadding -> 0],
GridBox[{{
StyleBox[
TemplateBox[{
TemplateBox[{
StyleBox["\"Input\"",
GrayLevel[0], StripOnInput -> False], "\" \"",
"\"port\""}, "RowDefault"], "\":\""}, "RowDefault"],
"SummaryItemAnnotation"],
StyleBox["\"string\"", "SummaryItem"]}, {
StyleBox[
TemplateBox[{
TemplateBox[{
StyleBox["\"Output\"",
GrayLevel[0], StripOnInput -> False], "\" \"",
"\"port\""}, "RowDefault"], "\":\""}, "RowDefault"],
"SummaryItemAnnotation"],
StyleBox["\"real\"", "SummaryItem"]}, {
StyleBox[
TemplateBox[{"\"Number of layers\"", "\":\""},
"RowDefault"], "SummaryItemAnnotation"],
StyleBox["5", "SummaryItem"]}}, BaselinePosition -> {2, 1},
GridBoxAlignment -> {
"Columns" -> {{Left}}, "Rows" -> {{Automatic}}},
GridBoxItemSize -> {
"Columns" -> {{Automatic}}, "Rows" -> {{Automatic}}},
GridBoxSpacings -> {
"Columns" -> {{2}}, "Rows" -> {{Automatic}}}]}},
GridBoxAlignment -> {"Rows" -> {{Top}}},
GridBoxItemSize -> {
"Columns" -> {{Automatic}}, "Rows" -> {{Automatic}}},
GridFrameMargins -> {{0, 0}, {0, 0}}, BaselinePosition -> {1, 3}],
True -> GridBox[{{
PaneBox[
ButtonBox[
DynamicBox[
FEPrivate`FrontEndResource[
"FEBitmaps", "SquareMinusIconMedium"]],
ButtonFunction :> (Typeset`open = False), Appearance -> None,
Evaluator -> Automatic, Method -> "Preemptive"],
Alignment -> {Center, Center}, ImageSize -> {Automatic, 24}],
GraphicsBox[{
Thickness[0.014925373134328358`], {
RGBColor[0.729, 0.729, 0.729],
Thickness[0.018686567164179105`],
Opacity[1.],
JoinForm[{"Miter", 10.}],
JoinedCurveBox[{{{0, 2, 0}}}, {{{23., 91.5}, {30., 91.5}}},
CurveClosed -> {0}]}, {
RGBColor[0.729, 0.729, 0.729],
Thickness[0.018686567164179105`],
Opacity[1.],
JoinForm[{"Miter", 10.}],
JoinedCurveBox[{{{0, 2, 0}}}, {{{37., 91.5}, {44., 91.5}}},
CurveClosed -> {0}]}, {
FaceForm[{
RGBColor[0.631, 0.8160000000000001, 0.498],
Opacity[1.]}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}, {0, 1, 0}}}, {{{15.5,
103.5}, {23.5, 103.5}, {23.5, 79.5}, {15.5, 79.5}}}]}, {
FaceForm[{
RGBColor[0.40800000000000003`, 0.627, 0.196],
Opacity[1.]}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}, {0, 1, 0}}, {{0, 2,
0}, {0, 1, 0}, {0, 1, 0}}}, {{{24., 104.}, {15., 104.}, {
15., 79.}, {24., 79.}}, {{23., 80.}, {16., 80.}, {16.,
103.}, {23., 103.}}}]}, {
FaceForm[{
RGBColor[0.38, 0.753, 0.89],
Opacity[1.]}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}, {0, 1, 0}}}, {{{29.5,
103.5}, {37.5, 103.5}, {37.5, 79.5}, {29.5, 79.5}}}]}, {
FaceForm[{
RGBColor[0.24300000000000002`, 0.608, 0.71],
Opacity[1.]}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}, {0, 1, 0}}, {{0, 2,
0}, {0, 1, 0}, {0, 1, 0}}}, {{{38., 104.}, {29., 104.}, {
29., 79.}, {38., 79.}}, {{37., 80.}, {30., 80.}, {30.,
103.}, {37., 103.}}}]}, {
FaceForm[{
RGBColor[0.584, 0.51, 0.8390000000000001],
Opacity[1.]}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}, {0, 1, 0}}}, {{{43.5,
103.5}, {51.5, 103.5}, {51.5, 79.5}, {43.5, 79.5}}}]}, {
FaceForm[{
RGBColor[0.4, 0.396, 0.659],
Opacity[1.]}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}, {0, 1, 0}}, {{0, 2,
0}, {0, 1, 0}, {0, 1, 0}}}, {{{52., 104.}, {43., 104.}, {
43., 79.}, {52., 79.}}, {{51., 80.}, {44., 80.}, {44.,
103.}, {51., 103.}}}]}, {
FaceForm[{
RGBColor[0.549, 0.549, 0.549],
Opacity[1.]}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}}}, {{{28., 91.5}, {25.,
95.}, {25., 88.}}}],
FilledCurveBox[{{{0, 2, 0}, {0, 1, 0}}}, {{{42., 91.5}, {39.,
95.}, {39., 88.}}}]}}, ImageMargins -> 0,
ImageSize -> {36, Automatic}, PlotRangePadding -> 0],
GridBox[{{
DynamicModuleBox[{
NeuralNetworks`Private`NetChain`assoc3 =
Association[
"Type" -> "Chain", "Nodes" ->
Association[
"1" -> Association[
"Type" -> "UnitVector", "Arrays" -> Association[],
"Parameters" ->
Association["ClassCount" -> 11, "$Dimensions" -> {
NeuralNetworks`LengthVar[1973845059]}], "Inputs" ->
Association["Input" -> NeuralNetworks`TensorT[{
NeuralNetworks`LengthVar[1973845059]},
NeuralNetworks`IndexIntegerT[11]]], "Outputs" ->
Association["Output" -> NeuralNetworks`TensorT[{
NeuralNetworks`LengthVar[1973845059], 11},
NeuralNetworks`RealT]]], "2" ->
Association[
"Type" -> "LongShortTermMemory", "Arrays" ->
Association[
"InputGateInputWeights" ->
NeuralNetworks`Private`DummyArray[{40, 11}],
"InputGateStateWeights" ->
NeuralNetworks`Private`DummyArray[{40, 40}],
"InputGateBiases" ->
NeuralNetworks`Private`DummyArray[{40}],
"OutputGateInputWeights" ->
NeuralNetworks`Private`DummyArray[{40, 11}],
"OutputGateStateWeights" ->
NeuralNetworks`Private`DummyArray[{40, 40}],
"OutputGateBiases" ->
NeuralNetworks`Private`DummyArray[{40}],
"ForgetGateInputWeights" ->
NeuralNetworks`Private`DummyArray[{40, 11}],
"ForgetGateStateWeights" ->
NeuralNetworks`Private`DummyArray[{40, 40}],
"ForgetGateBiases" ->
NeuralNetworks`Private`DummyArray[{40}],
"MemoryGateInputWeights" ->
NeuralNetworks`Private`DummyArray[{40, 11}],
"MemoryGateStateWeights" ->
NeuralNetworks`Private`DummyArray[{40, 40}],
"MemoryGateBiases" ->
NeuralNetworks`Private`DummyArray[{40}]], "Parameters" ->
Association[
"OutputSize" -> 40, "Dropout" ->
NeuralNetworks`ValidatedParameter[None], "$InputSize" ->
11, "$SequenceLength" ->
NeuralNetworks`LengthVar[1973845059],
"$CellStateConnectedQ" -> False], "Inputs" ->
Association["Input" -> NeuralNetworks`TensorT[{
NeuralNetworks`LengthVar[1973845059], 11},
NeuralNetworks`RealT]], "Outputs" ->
Association["Output" -> NeuralNetworks`TensorT[{
NeuralNetworks`LengthVar[1973845059], 40},
NeuralNetworks`RealT]], "States" ->
Association[
"State" ->
NeuralNetworks`TensorT[{40}, NeuralNetworks`RealT],
"CellState" ->
NeuralNetworks`TensorT[{40}, NeuralNetworks`RealT]]], "3" ->
Association[
"Type" -> "LongShortTermMemory", "Arrays" ->
Association[
"InputGateInputWeights" ->
NeuralNetworks`Private`DummyArray[{20, 40}],
"InputGateStateWeights" ->
NeuralNetworks`Private`DummyArray[{20, 20}],
"InputGateBiases" ->
NeuralNetworks`Private`DummyArray[{20}],
"OutputGateInputWeights" ->
NeuralNetworks`Private`DummyArray[{20, 40}],
"OutputGateStateWeights" ->
NeuralNetworks`Private`DummyArray[{20, 20}],
"OutputGateBiases" ->
NeuralNetworks`Private`DummyArray[{20}],
"ForgetGateInputWeights" ->
NeuralNetworks`Private`DummyArray[{20, 40}],
"ForgetGateStateWeights" ->
NeuralNetworks`Private`DummyArray[{20, 20}],
"ForgetGateBiases" ->
NeuralNetworks`Private`DummyArray[{20}],
"MemoryGateInputWeights" ->
NeuralNetworks`Private`DummyArray[{20, 40}],
"MemoryGateStateWeights" ->
NeuralNetworks`Private`DummyArray[{20, 20}],
"MemoryGateBiases" ->
NeuralNetworks`Private`DummyArray[{20}]], "Parameters" ->
Association[
"OutputSize" -> 20, "Dropout" ->
NeuralNetworks`ValidatedParameter[None], "$InputSize" ->
40, "$SequenceLength" ->
NeuralNetworks`LengthVar[1973845059],
"$CellStateConnectedQ" -> False], "Inputs" ->
Association["Input" -> NeuralNetworks`TensorT[{
NeuralNetworks`LengthVar[1973845059], 40},
NeuralNetworks`RealT]], "Outputs" ->
Association["Output" -> NeuralNetworks`TensorT[{
NeuralNetworks`LengthVar[1973845059], 20},
NeuralNetworks`RealT]], "States" ->
Association[
"State" ->
NeuralNetworks`TensorT[{20}, NeuralNetworks`RealT],
"CellState" ->
NeuralNetworks`TensorT[{20}, NeuralNetworks`RealT]]], "4" ->
Association[
"Type" -> "SequenceLast", "Arrays" -> Association[],
"Parameters" -> Association[], "Inputs" ->
Association["Input" -> NeuralNetworks`TensorT[{
NeuralNetworks`LengthVar[1973845059], 20},
NeuralNetworks`RealT]], "Outputs" ->
Association[
"Output" ->
NeuralNetworks`TensorT[{20}, NeuralNetworks`RealT]]], "5" ->
Association[
"Type" -> "Linear", "Arrays" ->
Association[
"Weights" -> NeuralNetworks`Private`DummyArray[{1, 20}],
"Biases" -> NeuralNetworks`Private`DummyArray[{1}]],
"Parameters" ->
Association[
"OutputDimensions" -> {}, "$OutputSize" -> 1,
"$InputSize" -> 20, "$InputDimensions" -> {20}], "Inputs" ->
Association[
"Input" ->
NeuralNetworks`TensorT[{20}, NeuralNetworks`RealT]],
"Outputs" ->
Association[
"Output" ->
NeuralNetworks`TensorT[{}, NeuralNetworks`RealT]]]],
"Edges" -> {
NeuralNetworks`NetPath["Nodes", "1", "Inputs", "Input"] ->
NeuralNetworks`NetPath["Inputs", "Input"],
NeuralNetworks`NetPath["Nodes", "2", "Inputs", "Input"] ->
NeuralNetworks`NetPath["Nodes", "1", "Outputs", "Output"],
NeuralNetworks`NetPath["Nodes", "3", "Inputs", "Input"] ->
NeuralNetworks`NetPath["Nodes", "2", "Outputs", "Output"],
NeuralNetworks`NetPath["Nodes", "4", "Inputs", "Input"] ->
NeuralNetworks`NetPath["Nodes", "3", "Outputs", "Output"],
NeuralNetworks`NetPath["Nodes", "5", "Inputs", "Input"] ->
NeuralNetworks`NetPath["Nodes", "4", "Outputs", "Output"],
NeuralNetworks`NetPath["Outputs", "Output"] ->
NeuralNetworks`NetPath[
"Nodes", "5", "Outputs", "Output"]}, "Inputs" ->
Association[
"Input" -> Association["Form" -> NeuralNetworks`TensorT[{
NeuralNetworks`LengthVar[1973845059]},
NeuralNetworks`IndexIntegerT[11]], "Type" -> "Characters",
"Encoding" -> NeuralNetworks`ValidatedParameter[
MXNetLink`CharacterEncodingData[
1, RawArray["UnsignedInteger8",{128, 0, 0, 128, 128, 128,
128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128,
128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128,
128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128,
128, 128, 128, 128, 128, 128, 11, 128, 128, 128, 128, 1,
2, 3, 4, 5, 6, 7, 8, 9, 10, 128, 128, 128, 128, 128, 128,
128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128,
128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128,
128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128,
128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128,
128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128,
128, 128, 128, 128, 128, 128, 128, 128, 128}],
RawArray["UnsignedInteger8",{48, 49, 50, 51, 52, 53, 54,
55, 56, 57, 43}], 11, {DigitCharacter, "+"}]],
"OutputForm" -> "Index", "IgnoreCase" -> False, "Count" ->
11, "TargetLength" -> All, "$TargetLength" ->
NeuralNetworks`LengthVar[1601547397], "$Padding" ->
Automatic, "$Version" -> "12.0.10"]], "Outputs" ->
Association[
"Output" ->
NeuralNetworks`TensorT[{}, NeuralNetworks`RealT]],
"InteriorStates" ->
Association[{2, "State"} ->
NeuralNetworks`NetPath["Nodes", "2", "States", "State"], {
2, "CellState"} ->
NeuralNetworks`NetPath[
"Nodes", "2", "States", "CellState"], {3, "State"} ->
NeuralNetworks`NetPath["Nodes", "3", "States", "State"], {
3, "CellState"} ->
NeuralNetworks`NetPath[
"Nodes", "3", "States", "CellState"]]],
NeuralNetworks`Private`NetChain`opart,
NeuralNetworks`Private`NetChain`part,
NeuralNetworks`Private`NetChain`selected = Null},
DynamicBox[
GridBox[{{
NeuralNetworks`Private`NetChain`MouseClickBoxes[
TagBox[
GridBox[{{
TagBox[
TagBox[
"\"\"", Annotation[#, {"Inputs", "Input"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]],
TagBox[
TagBox[
"\"Input\"",
Annotation[#, {"Inputs", "Input"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]],
TagBox[
TagBox[
TagBox[
GridBox[{{"\"string\""}, {
TemplateBox[{"\"vector of \"",
StyleBox["\"n\"", Italic, StripOnInput -> False], "\" \"",
TemplateBox[{"\"indices\"", "\" \"",
StyleBox[
TemplateBox[{"\"(\"", "\"\[VeryThinSpace]\"", "\"range\"",
"\":\"", "\" \"",
TemplateBox[{"1", "\"..\"", "11"}, "RowDefault"],
"\"\[VeryThinSpace]\"", "\")\""}, "RowDefault"],
GrayLevel[0.5], StripOnInput -> False]}, "RowDefault"]},
"RowDefault"]}},
GridBoxAlignment -> {"Columns" -> {{Left}}},
BaselinePosition -> 2, DefaultBaseStyle -> "Column",
GridBoxItemSize -> {
"Columns" -> {{Automatic}}, "Rows" -> {{Automatic}}}],
"Column"], Annotation[#, {"Inputs", "Input"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]]}, {
TagBox[
TagBox[
StyleBox["\"1\"",
GrayLevel[0.5], StripOnInput -> False],
Annotation[#, {"Nodes", "1"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]],
TagBox[
TagBox[
StyleBox["UnitVectorLayer",
GrayLevel[0], StripOnInput -> False],
Annotation[#, {"Nodes", "1"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]],
TagBox[
TagBox[
TemplateBox[{"\"matrix\"", "\" \"",
StyleBox[
TemplateBox[{"\"(\"", "\"\[VeryThinSpace]\"", "\"size\"",
"\":\"", "\" \"",
TemplateBox[{"\[Times]", "\"\[Times]\"",
StyleBox["\"n\"", Italic, StripOnInput -> False],
"\"11\""}, "RowWithSeparators"], "\"\[VeryThinSpace]\"",
"\")\""}, "RowDefault"],
GrayLevel[0.5], StripOnInput -> False]}, "RowDefault"],
Annotation[#, {"Nodes", "1"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]]}, {
TagBox[
TagBox[
StyleBox["\"2\"",
GrayLevel[0.5], StripOnInput -> False],
Annotation[#, {"Nodes", "2"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]],
TagBox[
TagBox[
StyleBox["LongShortTermMemoryLayer",
GrayLevel[0], StripOnInput -> False],
Annotation[#, {"Nodes", "2"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]],
TagBox[
TagBox[
TemplateBox[{"\"matrix\"", "\" \"",
StyleBox[
TemplateBox[{"\"(\"", "\"\[VeryThinSpace]\"", "\"size\"",
"\":\"", "\" \"",
TemplateBox[{"\[Times]", "\"\[Times]\"",
StyleBox["\"n\"", Italic, StripOnInput -> False],
"\"40\""}, "RowWithSeparators"], "\"\[VeryThinSpace]\"",
"\")\""}, "RowDefault"],
GrayLevel[0.5], StripOnInput -> False]}, "RowDefault"],
Annotation[#, {"Nodes", "2"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]]}, {
TagBox[
TagBox[
StyleBox["\"3\"",
GrayLevel[0.5], StripOnInput -> False],
Annotation[#, {"Nodes", "3"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]],
TagBox[
TagBox[
StyleBox["LongShortTermMemoryLayer",
GrayLevel[0], StripOnInput -> False],
Annotation[#, {"Nodes", "3"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]],
TagBox[
TagBox[
TemplateBox[{"\"matrix\"", "\" \"",
StyleBox[
TemplateBox[{"\"(\"", "\"\[VeryThinSpace]\"", "\"size\"",
"\":\"", "\" \"",
TemplateBox[{"\[Times]", "\"\[Times]\"",
StyleBox["\"n\"", Italic, StripOnInput -> False],
"\"20\""}, "RowWithSeparators"], "\"\[VeryThinSpace]\"",
"\")\""}, "RowDefault"],
GrayLevel[0.5], StripOnInput -> False]}, "RowDefault"],
Annotation[#, {"Nodes", "3"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]]}, {
TagBox[
TagBox[
StyleBox["\"4\"",
GrayLevel[0.5], StripOnInput -> False],
Annotation[#, {"Nodes", "4"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]],
TagBox[
TagBox[
StyleBox["SequenceLastLayer",
GrayLevel[0], StripOnInput -> False],
Annotation[#, {"Nodes", "4"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]],
TagBox[
TagBox[
TemplateBox[{"\"vector\"", "\" \"",
StyleBox[
TemplateBox[{"\"(\"", "\"\[VeryThinSpace]\"", "\"size\"",
"\":\"", "\" \"",
TemplateBox[{"\[Times]", "\"\[Times]\"", "\"20\""},
"RowWithSeparators"], "\"\[VeryThinSpace]\"", "\")\""},
"RowDefault"],
GrayLevel[0.5], StripOnInput -> False]}, "RowDefault"],
Annotation[#, {"Nodes", "4"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]]}, {
TagBox[
TagBox[
StyleBox["\"5\"",
GrayLevel[0.5], StripOnInput -> False],
Annotation[#, {"Nodes", "5"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]],
TagBox[
TagBox[
StyleBox["LinearLayer",
GrayLevel[0], StripOnInput -> False],
Annotation[#, {"Nodes", "5"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]],
TagBox[
TagBox[
"\"real\"", Annotation[#, {"Nodes", "5"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]]}, {
TagBox[
TagBox[
"\"\"", Annotation[#, {"Outputs", "Output"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]],
TagBox[
TagBox[
"\"Output\"",
Annotation[#, {"Outputs", "Output"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]],
TagBox[
TagBox[
"\"real\"",
Annotation[#, {"Outputs", "Output"}, "Mouse"]& ],
MouseAppearanceTag["LinkHand"]]}},
GridBoxAlignment -> {"Columns" -> {{Left}}}, AutoDelete ->
False, GridBoxItemSize -> {
"Columns" -> {{Automatic}}, "Rows" -> {{Automatic}}},
GridBoxSpacings -> {"Columns" -> {{1.1}}}], "Grid"],
If[
ListQ[
NeuralNetworks`Private`NetChain`part = MouseAnnotation[]],
If[NeuralNetworks`Private`NetChain`opart ===
NeuralNetworks`Private`NetChain`part,
NeuralNetworks`Private`NetChain`selected = Null;
NeuralNetworks`Private`NetChain`opart = Null,
NeuralNetworks`Private`NetChain`selected =
Part[NeuralNetworks`Private`NetChain`assoc3,
Apply[Sequence, NeuralNetworks`Private`NetChain`part]];
NeuralNetworks`Private`NetChain`opart =
NeuralNetworks`Private`NetChain`part; Null]; Null]]},
NeuralNetworks`Private`NetChain`fmtSelected[
NeuralNetworks`Private`NetChain`selected,
NeuralNetworks`Private`NetChain`part,
NeuralNetworks`Private`GetSharedArrays[
NeuralNetworks`Private`NetChain`assoc3]]},
GridBoxSpacings -> {"Columns" -> {{1}}},
GridBoxAlignment -> {"Columns" -> {{Left}}},
GridFrameMargins -> {{0, 0}, {0, 0}}],
TrackedSymbols :> {
NeuralNetworks`Private`NetChain`selected}],
Initialization :> {NetChain}]}}, BaselinePosition ->
Automatic,
GridBoxAlignment -> {
"Columns" -> {{Left}}, "Rows" -> {{Automatic}}},
GridBoxItemSize -> {
"Columns" -> {{Automatic}}, "Rows" -> {{Automatic}}},
GridBoxSpacings -> {
"Columns" -> {{2}}, "Rows" -> {{Automatic}}}]}},
GridBoxAlignment -> {"Rows" -> {{Top}}},
GridBoxItemSize -> {
"Columns" -> {{Automatic}}, "Rows" -> {{Automatic}}},
GridFrameMargins -> {{0, 0}, {0, 0}},
BaselinePosition -> {1, 1}]},
Dynamic[Typeset`open], ImageSize -> Automatic], BaselinePosition ->
Baseline,
BaseStyle -> {
ShowStringCharacters -> False, NumberMarks -> False, PrintPrecision ->
3, ShowSyntaxStyles -> False}]],
StyleBox["]", "NonInterpretableSummary"]}]},
"CopyTag",
DisplayFunction->(#& ),
InterpretationFunction->("NetChain[<>]"& )],
False,
Editable->False,
SelectWithContents->True,
Selectable->False]], "Output",
CellChangeTimes->{3.744368600926662*^9, 3.774950398693987*^9},
CellLabel->"Out[9]=",ExpressionUUID->"f760596d-e1c6-453d-b9e5-2ac891663384"]
}, Open ]],
Cell["Now you can give any addition with numbers below hundred:", "Text",
CellChangeTimes->{{3.7749502665012503`*^9,
3.7749502872232933`*^9}},ExpressionUUID->"06a7399b-c9c8-41b2-ab0c-\
ee931fe28486"],
Cell[CellGroupData[{
Cell[BoxData[
RowBox[{"trained", "[",
RowBox[{"{",
RowBox[{
"\"\<5+2\>\"", ",", "\"\<10+25\>\"", ",", "\"\<9+11\>\"", ",",
"\"\<44+44\>\""}], "}"}], "]"}]], "Input",
CellLabel->"In[10]:=",ExpressionUUID->"15bd3ce5-fd47-4b57-bfe5-d24b50b5ae3d"],
Cell[BoxData[
RowBox[{"{",
RowBox[{
"6.387209892272949`", ",", "34.81212615966797`", ",", "20.33081817626953`",
",", "87.83650970458984`"}], "}"}]], "Output",
CellChangeTimes->{3.744368605721415*^9, 3.7749537167808332`*^9},
CellLabel->"Out[10]=",ExpressionUUID->"c3d0195d-aa3c-493b-8f95-6fb094e84a9d"]
}, Open ]],
Cell["Simple, powerful and enticing at the same time.", "Text",
CellChangeTimes->{{3.774953728294293*^9,
3.774953744423419*^9}},ExpressionUUID->"b147231a-8dc4-4aca-adb8-\
b0db0015e514"]
}, Open ]]
},
WindowSize->{962, 1070},
WindowMargins->{{501, Automatic}, {Automatic, 161}},
FrontEndVersion->"12.0 for Mac OS X x86 (64-bit) (April 8, 2019)",
StyleDefinitions->"Default.nb"
]
(* End of Notebook Content *)
(* Internal cache information *)
(*CellTagsOutline
CellTagsIndex->{}
*)
(*CellTagsIndex
CellTagsIndex->{}
*)
(*NotebookFileOutline
Notebook[{
Cell[CellGroupData[{
Cell[580, 22, 168, 3, 98, "Title",ExpressionUUID->"32c93783-d88e-47ee-b059-234c2b3e852c"],
Cell[751, 27, 703, 14, 137, "Text",ExpressionUUID->"fdafed7b-2746-4880-a7ac-555a1c373c29"],
Cell[1457, 43, 354, 7, 58, "Text",ExpressionUUID->"7f486880-4547-412d-a459-9dd2dd515d7f"],
Cell[1814, 52, 443, 12, 30, "Input",ExpressionUUID->"810ceca5-690e-4a65-9ebf-009bc803d42c"],
Cell[2260, 66, 241, 6, 35, "Text",ExpressionUUID->"87620808-0bd8-41cc-adbf-5fbaafa21e1c"],
Cell[CellGroupData[{
Cell[2526, 76, 593, 16, 52, "Input",ExpressionUUID->"164ded93-e4a5-44b0-9f61-7217f358b550"],
Cell[3122, 94, 279, 6, 45, "Output",ExpressionUUID->"26549ffc-05b3-47e0-bebc-8f5232a439de"]
}, Open ]],
Cell[3416, 103, 317, 7, 58, "Text",ExpressionUUID->"09f1e4f1-10af-464a-8aea-407542e4cc14"],
Cell[3736, 112, 332, 9, 30, "Input",ExpressionUUID->"0f002a64-20e9-4089-9e5b-a92d8698de6a"],
Cell[4071, 123, 201, 3, 35, "Text",ExpressionUUID->"b3df4772-496a-4542-b1f4-f757a6b3354f"],
Cell[CellGroupData[{
Cell[4297, 130, 185, 3, 30, "Input",ExpressionUUID->"b2bebf00-70c9-4655-8456-b0e77432e0dd"],
Cell[4485, 135, 247, 4, 34, "Output",ExpressionUUID->"aa992927-f8fc-48a3-a496-b5f9be46b744"]
}, Open ]],
Cell[4747, 142, 524, 9, 81, "Text",ExpressionUUID->"eb26b3a5-fcf9-4d50-b88f-8eed51775e1a"],
Cell[CellGroupData[{
Cell[5296, 155, 614, 14, 52, "Input",ExpressionUUID->"4b7f764e-3a64-4d6c-8aea-62f73c9dfa5c"],
Cell[5913, 171, 33322, 646, 153, "Output",ExpressionUUID->"d2637d9d-9020-48c7-8e3d-106da36cbcb5"]
}, Open ]],
Cell[39250, 820, 231, 6, 35, "Text",ExpressionUUID->"18f7bcbd-cfed-4c36-b314-a2a81b8d6f53"],
Cell[CellGroupData[{
Cell[39506, 830, 405, 8, 30, "Input",ExpressionUUID->"d79543c6-4a5d-4906-b458-c250d99304ae"],
Cell[39914, 840, 31225, 602, 75, "Output",ExpressionUUID->"f760596d-e1c6-453d-b9e5-2ac891663384"]
}, Open ]],
Cell[71154, 1445, 203, 3, 35, "Text",ExpressionUUID->"06a7399b-c9c8-41b2-ab0c-ee931fe28486"],
Cell[CellGroupData[{
Cell[71382, 1452, 260, 6, 30, "Input",ExpressionUUID->"15bd3ce5-fd47-4b57-bfe5-d24b50b5ae3d"],
Cell[71645, 1460, 313, 6, 34, "Output",ExpressionUUID->"c3d0195d-aa3c-493b-8f95-6fb094e84a9d"]
}, Open ]],
Cell[71973, 1469, 189, 3, 35, "Text",ExpressionUUID->"b147231a-8dc4-4aca-adb8-b0db0015e514"]
}, Open ]]
}
]
*)
@Orbifold
Copy link
Author

Surprisingly easy to learn simple additions. This is based on a few LSTM layers.
The tricky part is to understand the syntax and meaning of command. Like hot-encoding is UnitVectorLayer. Wolfram is so opinionated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment