Skip to content

Instantly share code, notes, and snippets.

@luisquintanilla

luisquintanilla/Multiclass-PFI-AutoML.ipynb

Created September 21, 2022 03:56
Show Gist options
  • Save luisquintanilla/8dce371fb171830f2db264e8903496fa to your computer and use it in GitHub Desktop.
Save luisquintanilla/8dce371fb171830f2db264e8903496fa to your computer and use it in GitHub Desktop.
Download ZIP
Multiclass AutoML PFI
Display the source blob
Display the rendered blob
Raw
Multiclass-PFI-AutoML.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Install NuGet packages"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [
{
"data": {
"text/html": [
"<div><div></div><div></div><div><strong>Installed Packages</strong><ul><li><span>Microsoft.Data.Analysis, 0.20.0-preview.22356.1</span></li><li><span>Microsoft.ML.AutoML, 0.20.0-preview.22356.1</span></li><li><span>Plotly.NET.CSharp, 0.0.1</span></li><li><span>Plotly.NET.Interactive, 3.0.2</span></li></ul></div></div>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/markdown": [
"Loading extensions from `Microsoft.ML.AutoML.Interactive.dll`"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/markdown": [
"Loading extensions from `Microsoft.Data.Analysis.Interactive.dll`"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/markdown": [
"Loading extensions from `Plotly.NET.Interactive.dll`"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"#r \"nuget: Plotly.NET.Interactive, 3.0.2\"\n",
"#r \"nuget: Plotly.NET.CSharp, 0.0.1\"\n",
"#r \"nuget:Microsoft.ML.AutoML, 0.20.0-preview.22356.1\"\n",
"#r \"nuget: Microsoft.Data.Analysis, 0.20.0-preview.22356.1\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Add using statements"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [],
"source": [
"using System;\n",
"using System.IO;\n",
"using System.Collections.Generic;\n",
"using System.Linq;\n",
"using Microsoft.ML;\n",
"using Microsoft.ML.Data;\n",
"using Microsoft.ML.AutoML;\n",
"using Microsoft.Data.Analysis;"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Define data schema"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [],
"source": [
"public class Data\n",
"{\n",
" public float Feature1 {get;set;}\n",
" public float Feature2 {get;set;}\n",
" public float Label {get;set;}\n",
"}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Define method to generate random data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [],
"source": [
"public IEnumerable<Data> GenerateData(int nExamples = 10000,double bias = 0, double weight1 = 1, double weight2 = 2, int seed = 1)\n",
"{\n",
" var rng = new Random(seed);\n",
" var max = bias + 4.5 * weight1 + 4.5 * weight2 + 0.5;\n",
" for (int i = 0; i < nExamples; i++)\n",
" {\n",
" var data = new Data\n",
" {\n",
" Feature1 = (float)(rng.Next(10) * (rng.NextDouble() - 0.5)),\n",
" Feature2 = (float)(rng.Next(10) * (rng.NextDouble() - 0.5)),\n",
" };\n",
"\n",
" // Create a noisy label.\n",
" var value = (float)\n",
" (bias + weight1 * data.Feature1 + weight2 * data.Feature2 +\n",
" rng.NextDouble() - 0.5);\n",
"\n",
" if (value < max / 3)\n",
" data.Label = 0;\n",
" else if (value < 2 * max / 3)\n",
" data.Label = 1;\n",
" else\n",
" data.Label = 2;\n",
" yield return data;\n",
" }\n",
"}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Initialize MLContext"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [],
"source": [
"var mlContext = new MLContext(seed:1);"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Generate data samples"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [],
"source": [
"var data = GenerateData();"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [
{
"data": {
"text/html": [
"<table><thead><tr><th><i>index</i></th><th>Feature1</th><th>Feature2</th><th>Label</th></tr></thead><tbody><tr><td>0</td><td><div class=\"dni-plaintext\">-0.77851206</div></td><td><div class=\"dni-plaintext\">1.0864165</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>1</td><td><div class=\"dni-plaintext\">-0.58366495</div></td><td><div class=\"dni-plaintext\">-3.5886018</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>2</td><td><div class=\"dni-plaintext\">-0</div></td><td><div class=\"dni-plaintext\">1.4693015</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>3</td><td><div class=\"dni-plaintext\">-1.3036246</div></td><td><div class=\"dni-plaintext\">1.2255092</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>4</td><td><div class=\"dni-plaintext\">-3.6595037</div></td><td><div class=\"dni-plaintext\">-0.118028924</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>5</td><td><div class=\"dni-plaintext\">0.29378363</div></td><td><div class=\"dni-plaintext\">0.9690853</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>6</td><td><div class=\"dni-plaintext\">1.08367</div></td><td><div class=\"dni-plaintext\">-2.9183135</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>7</td><td><div class=\"dni-plaintext\">2.8699489</div></td><td><div class=\"dni-plaintext\">0</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>8</td><td><div class=\"dni-plaintext\">0.9875927</div></td><td><div class=\"dni-plaintext\">-0.6803071</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>9</td><td><div class=\"dni-plaintext\">1.06429</div></td><td><div class=\"dni-plaintext\">-1.021925</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>10</td><td><div class=\"dni-plaintext\">-1.6528636</div></td><td><div class=\"dni-plaintext\">0.57508487</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>11</td><td><div class=\"dni-plaintext\">0.4498368</div></td><td><div class=\"dni-plaintext\">0</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>12</td><td><div class=\"dni-plaintext\">-2.7188668</div></td><td><div class=\"dni-plaintext\">-0.2852031</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>13</td><td><div class=\"dni-plaintext\">0.14619994</div></td><td><div class=\"dni-plaintext\">1.4936732</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>14</td><td><div class=\"dni-plaintext\">-1.9584187</div></td><td><div class=\"dni-plaintext\">-0.32100672</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>15</td><td><div class=\"dni-plaintext\">-0.8794435</div></td><td><div class=\"dni-plaintext\">3.13346</div></td><td><div class=\"dni-plaintext\">1</div></td></tr><tr><td>16</td><td><div class=\"dni-plaintext\">-0.71636426</div></td><td><div class=\"dni-plaintext\">-0</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>17</td><td><div class=\"dni-plaintext\">-0.3895614</div></td><td><div class=\"dni-plaintext\">0</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>18</td><td><div class=\"dni-plaintext\">2.189632</div></td><td><div class=\"dni-plaintext\">-0</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>19</td><td><div class=\"dni-plaintext\">0.745925</div></td><td><div class=\"dni-plaintext\">-0.798151</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td colspan=\"4\"><i>... (more)</i></td></tr></tbody></table>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"data"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Load data samples into IDataView"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [],
"source": [
"var idv = mlContext.Data.LoadFromEnumerable(data);"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Prepare data for training"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [],
"source": [
"var featureColumns = new string[] { nameof(Data.Feature1), nameof(Data.Feature2) };"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [],
"source": [
"var dataPrepPipeline = \n",
" mlContext.Transforms.Concatenate(\"Features\", featureColumns)\n",
" .Append(mlContext.Transforms.Conversion.MapValueToKey(\"Label\"))\n",
" .Append(mlContext.Transforms.NormalizeMinMax(\"Features\"));"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Apply data transformations"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [],
"source": [
"var dataPrepTransformer = dataPrepPipeline.Fit(idv);"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [],
"source": [
"var transformedData = dataPrepTransformer.Transform(idv);"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Only display active columns"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [
{
"data": {
"text/html": [
"<table><thead><tr><th><i>index</i></th><th>Name</th><th>Index</th><th>IsHidden</th><th>Type</th><th>Annotations</th></tr></thead><tbody><tr><td>0</td><td>Feature1</td><td><div class=\"dni-plaintext\">0</div></td><td><div class=\"dni-plaintext\">False</div></td><td><table><thead><tr><th>RawType</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\">System.Single</div></td></tr></tbody></table></td><td><table><thead><tr><th>Schema</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\">[ ]</div></td></tr></tbody></table></td></tr><tr><td>1</td><td>Feature2</td><td><div class=\"dni-plaintext\">1</div></td><td><div class=\"dni-plaintext\">False</div></td><td><table><thead><tr><th>RawType</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\">System.Single</div></td></tr></tbody></table></td><td><table><thead><tr><th>Schema</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\">[ ]</div></td></tr></tbody></table></td></tr><tr><td>2</td><td>Label</td><td><div class=\"dni-plaintext\">3</div></td><td><div class=\"dni-plaintext\">False</div></td><td><table><thead><tr><th>Count</th><th>RawType</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\">3</div></td><td><div class=\"dni-plaintext\">System.UInt32</div></td></tr></tbody></table></td><td><table><thead><tr><th>Schema</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\">[ { KeyValues: Vector&lt;Single, 3&gt;: Name: KeyValues, Index: 0, IsHidden: False, Type: { Vector&lt;Single, 3&gt;: Dimensions: [ 3 ], IsKnownSize: True, ItemType: { Single: RawType: System.Single }, Size: 3, RawType: Microsoft.ML.Data.VBuffer&lt;System.Single&gt; }, Annotations: { : Schema: [ ] } } ]</div></td></tr></tbody></table></td></tr><tr><td>3</td><td>Features</td><td><div class=\"dni-plaintext\">5</div></td><td><div class=\"dni-plaintext\">False</div></td><td><table><thead><tr><th>Dimensions</th><th>IsKnownSize</th><th>ItemType</th><th>Size</th><th>RawType</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\">[ 2 ]</div></td><td><div class=\"dni-plaintext\">True</div></td><td><div class=\"dni-plaintext\">{ Single: RawType: System.Single }</div></td><td><div class=\"dni-plaintext\">2</div></td><td><div class=\"dni-plaintext\">Microsoft.ML.Data.VBuffer&lt;System.Single&gt;</div></td></tr></tbody></table></td><td><table><thead><tr><th>Schema</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\">[ { IsNormalized: Boolean: Name: IsNormalized, Index: 0, IsHidden: False, Type: { Boolean: RawType: System.Boolean }, Annotations: { : Schema: [ ] } }, { SlotNames: Vector&lt;String, 2&gt;: Name: SlotNames, Index: 1, IsHidden: False, Type: { Vector&lt;String, 2&gt;: Dimensions: [ 2 ], IsKnownSize: True, ItemType: { String: RawType: System.ReadOnlyMemory&lt;System.Char&gt; }, Size: 2, RawType: Microsoft.ML.Data.VBuffer&lt;System.ReadOnlyMemory&lt;System.Char&gt;&gt; }, Annotations: { : Schema: [ ] } } ]</div></td></tr></tbody></table></td></tr></tbody></table>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"transformedData.Schema.Where(x => x.IsHidden == false)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Define AutoML Pipeline"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [],
"source": [
"var autoMLPipeline = \n",
" new MultiModelPipeline()\n",
" .Append(mlContext.Auto().MultiClassification());\n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Define AutoML Experiment"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [],
"source": [
"var experiment = \n",
" mlContext.Auto().CreateExperiment()\n",
" .SetPipeline(autoMLPipeline)\n",
" .SetEvaluateMetric(MulticlassClassificationMetric.MicroAccuracy,labelColumn:\"Label\")\n",
" .SetTrainingTimeInSeconds(60)\n",
" .SetDataset(transformedData,fold:5);"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Initialize monitor (only works in notebooks)\n",
"\n",
"To log outputs in a console app, use the following code:\n",
"\n",
"```csharp\n",
"mlContext.Log += (object? sender, LoggingEventArgs e) =>\n",
"{\n",
" if (e.Source.Contains(\"AutoMLExperiment\")) Console.WriteLine(e.RawMessage);\n",
"};\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [],
"source": [
"var monitor = new NotebookMonitor();"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [],
"source": [
"experiment.SetMonitor(monitor);"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Run experiment"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [
{
"data": {
"text/html": [
"<div><h3>Best Trial</h3><p>Id: 18</p><p>Trainer: FastForestOva</p><p>Parameters: {\r\n",
" &quot;0&quot;: {\r\n",
" &quot;NumberOfTrees&quot;: 5,\r\n",
" &quot;NumberOfLeaves&quot;: 7,\r\n",
" &quot;FeatureFraction&quot;: 1,\r\n",
" &quot;LabelColumnName&quot;: &quot;Label&quot;,\r\n",
" &quot;FeatureColumnName&quot;: &quot;Features&quot;\r\n",
" }\r\n",
"}</p><h3>Active Trial</h3><p>Id: 20</p><p>Trainer: LbfgsMaximumEntropyMulti</p><p>Parameters: {\r\n",
" &quot;0&quot;: {\r\n",
" &quot;NumberOfLeaves&quot;: 6,\r\n",
" &quot;MinimumExampleCountPerLeaf&quot;: 24,\r\n",
" &quot;NumberOfTrees&quot;: 4,\r\n",
" &quot;MaximumBinCountPerFeature&quot;: 465,\r\n",
" &quot;FeatureFraction&quot;: 0.99999999,\r\n",
" &quot;LearningRate&quot;: 0.005305832399633224,\r\n",
" &quot;LabelColumnName&quot;: &quot;Label&quot;,\r\n",
" &quot;FeatureColumnName&quot;: &quot;Features&quot;\r\n",
" }\r\n",
"}</p></div><div><h3>Plot Metrics over Trials</h3></div>\n",
"<div>\n",
" <div id=\"036273aa-00d3-4c0a-addd-b1a4e2c0b836\"><!-- Plotly chart will be drawn inside this DIV --></div>\r\n",
"<script type=\"text/javascript\">\r\n",
"\r\n",
" var renderPlotly_036273aa00d34c0aadddb1a4e2c0b836 = function() {\r\n",
" var fsharpPlotlyRequire = requirejs.config({context:'fsharp-plotly',paths:{plotly:'https://cdn.plot.ly/plotly-2.6.3.min'}}) || require;\r\n",
" fsharpPlotlyRequire(['plotly'], function(Plotly) {\r\n",
"\r\n",
" var data = [{\"type\":\"scatter\",\"name\":\"Plot Metrics over Trials.\",\"mode\":\"markers\",\"x\":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19],\"y\":[0.9105367793240556,0.9105367793240556,0.9105367793240556,0.9835984095427436,0.9840954274353877,0.9587475149105368,0.9845924453280318,0.9105367793240556,0.9696819085487077,0.9105367793240556,0.9786282306163022,0.9840954274353877,0.9592445328031809,0.9826043737574552,0.9105367793240556,0.9771371769383698,0.98558648111332,0.9105367793240556,0.9860834990059643,0.9711729622266402],\"marker\":{},\"line\":{},\"showlegend\":false}];\r\n",
" var layout = {\"width\":600,\"height\":600,\"template\":{\"layout\":{\"title\":{\"x\":0.05},\"font\":{\"color\":\"rgba(42, 63, 95, 1.0)\"},\"paper_bgcolor\":\"rgba(255, 255, 255, 1.0)\",\"plot_bgcolor\":\"rgba(229, 236, 246, 1.0)\",\"autotypenumbers\":\"strict\",\"colorscale\":{\"diverging\":[[0.0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1.0,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}},\"geo\":{\"showland\":true,\"landcolor\":\"rgba(229, 236, 246, 1.0)\",\"showlakes\":true,\"lakecolor\":\"rgba(255, 255, 255, 1.0)\",\"subunitcolor\":\"rgba(255, 255, 255, 1.0)\",\"bgcolor\":\"rgba(255, 255, 255, 1.0)\"},\"mapbox\":{\"style\":\"light\"},\"polar\":{\"bgcolor\":\"rgba(229, 236, 246, 1.0)\",\"radialaxis\":{\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"ticks\":\"\"},\"angularaxis\":{\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"ticks\":\"\",\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"gridwidth\":2.0,\"zerolinecolor\":\"rgba(255, 255, 255, 1.0)\",\"backgroundcolor\":\"rgba(229, 236, 246, 1.0)\",\"showbackground\":true},\"yaxis\":{\"ticks\":\"\",\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"gridwidth\":2.0,\"zerolinecolor\":\"rgba(255, 255, 255, 1.0)\",\"backgroundcolor\":\"rgba(229, 236, 246, 1.0)\",\"showbackground\":true},\"zaxis\":{\"ticks\":\"\",\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"gridwidth\":2.0,\"zerolinecolor\":\"rgba(255, 255, 255, 1.0)\",\"backgroundcolor\":\"rgba(229, 236, 246, 1.0)\",\"showbackground\":true}},\"ternary\":{\"aaxis\":{\"ticks\":\"\",\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\"},\"baxis\":{\"ticks\":\"\",\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\"},\"caxis\":{\"ticks\":\"\",\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\"},\"bgcolor\":\"rgba(229, 236, 246, 1.0)\"},\"xaxis\":{\"title\":{\"standoff\":15},\"ticks\":\"\",\"automargin\":true,\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"zerolinecolor\":\"rgba(255, 255, 255, 1.0)\",\"zerolinewidth\":2.0},\"yaxis\":{\"title\":{\"standoff\":15},\"ticks\":\"\",\"automargin\":true,\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"zerolinecolor\":\"rgba(255, 255, 255, 1.0)\",\"zerolinewidth\":2.0},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"shapedefaults\":{\"line\":{\"color\":\"rgba(42, 63, 95, 1.0)\"}},\"colorway\":[\"rgba(99, 110, 250, 1.0)\",\"rgba(239, 85, 59, 1.0)\",\"rgba(0, 204, 150, 1.0)\",\"rgba(171, 99, 250, 1.0)\",\"rgba(255, 161, 90, 1.0)\",\"rgba(25, 211, 243, 1.0)\",\"rgba(255, 102, 146, 1.0)\",\"rgba(182, 232, 128, 1.0)\",\"rgba(255, 151, 255, 1.0)\",\"rgba(254, 203, 82, 1.0)\"]},\"data\":{\"bar\":[{\"marker\":{\"line\":{\"color\":\"rgba(229, 236, 246, 1.0)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"error_x\":{\"color\":\"rgba(42, 63, 95, 1.0)\"},\"error_y\":{\"color\":\"rgba(42, 63, 95, 1.0)\"}}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"rgba(229, 236, 246, 1.0)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}}}],\"carpet\":[{\"aaxis\":{\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"endlinecolor\":\"rgba(42, 63, 95, 1.0)\",\"minorgridcolor\":\"rgba(255, 255, 255, 1.0)\",\"startlinecolor\":\"rgba(42, 63, 95, 1.0)\"},\"baxis\":{\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"endlinecolor\":\"rgba(42, 63, 95, 1.0)\",\"minorgridcolor\":\"rgba(255, 255, 255, 1.0)\",\"startlinecolor\":\"rgba(42, 63, 95, 1.0)\"}}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}}}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"pie\":[{\"automargin\":true}],\"scatter\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}},\"line\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scattergl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"rgba(235, 240, 248, 1.0)\"},\"line\":{\"color\":\"rgba(255, 255, 255, 1.0)\"}},\"header\":{\"fill\":{\"color\":\"rgba(200, 212, 227, 1.0)\"},\"line\":{\"color\":\"rgba(255, 255, 255, 1.0)\"}}}]}},\"xaxis\":{\"title\":{\"text\":\"Trial\"},\"showgrid\":false},\"yaxis\":{\"title\":{\"text\":\"Metric\"},\"showgrid\":false}};\r\n",
" var config = {\"responsive\":true};\r\n",
" Plotly.newPlot('036273aa-00d3-4c0a-addd-b1a4e2c0b836', data, layout, config);\r\n",
"});\r\n",
" };\r\n",
" if ((typeof(requirejs) !== typeof(Function)) || (typeof(requirejs.config) !== typeof(Function))) {\r\n",
" var script = document.createElement(\"script\");\r\n",
" script.setAttribute(\"src\", \"https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.6/require.min.js\");\r\n",
" script.onload = function(){\r\n",
" renderPlotly_036273aa00d34c0aadddb1a4e2c0b836();\r\n",
" };\r\n",
" document.getElementsByTagName(\"head\")[0].appendChild(script);\r\n",
" }\r\n",
" else {\r\n",
" renderPlotly_036273aa00d34c0aadddb1a4e2c0b836();\r\n",
" }\r\n",
"</script>\r\n",
"\n",
" \n",
"</div> \n",
"<div><h3>All Trials Table</h3></div><table id=\"table_637993145954321918\"><thead><tr><th><i>index</i></th><th>Trial</th><th>Metric</th><th>Trainer</th><th>Parameters</th></tr></thead><tbody><tr><td><i><div class=\"dni-plaintext\">0</div></i></td><td><div class=\"dni-plaintext\">0</div></td><td><div class=\"dni-plaintext\">0.91053677</div></td><td>SdcaMaximumEntropyMulti</td><td>{&quot;0&quot;:{&quot;L1Regularization&quot;:1,&quot;L2Regularization&quot;:0.1,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">1</div></i></td><td><div class=\"dni-plaintext\">1</div></td><td><div class=\"dni-plaintext\">0.91053677</div></td><td>SdcaMaximumEntropyMulti</td><td>{&quot;0&quot;:{&quot;L1Regularization&quot;:0.2834022,&quot;L2Regularization&quot;:0.056204632,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">2</div></i></td><td><div class=\"dni-plaintext\">2</div></td><td><div class=\"dni-plaintext\">0.91053677</div></td><td>SdcaLogisticRegressionOva</td><td>{&quot;0&quot;:{&quot;L1Regularization&quot;:1,&quot;L2Regularization&quot;:0.1,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">3</div></i></td><td><div class=\"dni-plaintext\">3</div></td><td><div class=\"dni-plaintext\">0.9835984</div></td><td>LbfgsLogisticRegressionOva</td><td>{&quot;0&quot;:{&quot;L1Regularization&quot;:1,&quot;L2Regularization&quot;:1,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">4</div></i></td><td><div class=\"dni-plaintext\">4</div></td><td><div class=\"dni-plaintext\">0.98409545</div></td><td>FastForestOva</td><td>{&quot;0&quot;:{&quot;NumberOfTrees&quot;:4,&quot;NumberOfLeaves&quot;:4,&quot;FeatureFraction&quot;:1,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">5</div></i></td><td><div class=\"dni-plaintext\">5</div></td><td><div class=\"dni-plaintext\">0.9587475</div></td><td>FastTreeOva</td><td>{&quot;0&quot;:{&quot;NumberOfLeaves&quot;:4,&quot;MinimumExampleCountPerLeaf&quot;:20,&quot;NumberOfTrees&quot;:4,&quot;MaximumBinCountPerFeature&quot;:255,&quot;FeatureFraction&quot;:1,&quot;LearningRate&quot;:0.09999999999999998,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">6</div></i></td><td><div class=\"dni-plaintext\">6</div></td><td><div class=\"dni-plaintext\">0.98459244</div></td><td>LbfgsMaximumEntropyMulti</td><td>{&quot;0&quot;:{&quot;L1Regularization&quot;:1,&quot;L2Regularization&quot;:1,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">7</div></i></td><td><div class=\"dni-plaintext\">7</div></td><td><div class=\"dni-plaintext\">0.91053677</div></td><td>SdcaMaximumEntropyMulti</td><td>{&quot;0&quot;:{&quot;L1Regularization&quot;:3.528554,&quot;L2Regularization&quot;:0.17792128,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">8</div></i></td><td><div class=\"dni-plaintext\">8</div></td><td><div class=\"dni-plaintext\">0.9696819</div></td><td>LightGbmMulti</td><td>{&quot;0&quot;:{&quot;NumberOfLeaves&quot;:4,&quot;MinimumExampleCountPerLeaf&quot;:20,&quot;LearningRate&quot;:1,&quot;NumberOfTrees&quot;:4,&quot;SubsampleFraction&quot;:1,&quot;MaximumBinCountPerFeature&quot;:255,&quot;FeatureFraction&quot;:1,&quot;L1Regularization&quot;:2E-10,&quot;L2Regularization&quot;:1,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">9</div></i></td><td><div class=\"dni-plaintext\">9</div></td><td><div class=\"dni-plaintext\">0.91053677</div></td><td>SdcaLogisticRegressionOva</td><td>{&quot;0&quot;:{&quot;L1Regularization&quot;:1.2639047,&quot;L2Regularization&quot;:0.03125,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">10</div></i></td><td><div class=\"dni-plaintext\">10</div></td><td><div class=\"dni-plaintext\">0.9786282</div></td><td>LbfgsLogisticRegressionOva</td><td>{&quot;0&quot;:{&quot;L1Regularization&quot;:1.6473554,&quot;L2Regularization&quot;:3.644816,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">11</div></i></td><td><div class=\"dni-plaintext\">11</div></td><td><div class=\"dni-plaintext\">0.98409545</div></td><td>FastForestOva</td><td>{&quot;0&quot;:{&quot;NumberOfTrees&quot;:4,&quot;NumberOfLeaves&quot;:4,&quot;FeatureFraction&quot;:0.96048987,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">12</div></i></td><td><div class=\"dni-plaintext\">12</div></td><td><div class=\"dni-plaintext\">0.95924455</div></td><td>FastTreeOva</td><td>{&quot;0&quot;:{&quot;NumberOfLeaves&quot;:4,&quot;MinimumExampleCountPerLeaf&quot;:20,&quot;NumberOfTrees&quot;:4,&quot;MaximumBinCountPerFeature&quot;:354,&quot;FeatureFraction&quot;:0.99359328389078,&quot;LearningRate&quot;:0.04157699059538498,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">13</div></i></td><td><div class=\"dni-plaintext\">13</div></td><td><div class=\"dni-plaintext\">0.9826044</div></td><td>LbfgsMaximumEntropyMulti</td><td>{&quot;0&quot;:{&quot;L1Regularization&quot;:1.3323684,&quot;L2Regularization&quot;:3.8816857,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">14</div></i></td><td><div class=\"dni-plaintext\">14</div></td><td><div class=\"dni-plaintext\">0.91053677</div></td><td>SdcaMaximumEntropyMulti</td><td>{&quot;0&quot;:{&quot;L1Regularization&quot;:1.6197896,&quot;L2Regularization&quot;:0.045458466,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">15</div></i></td><td><div class=\"dni-plaintext\">15</div></td><td><div class=\"dni-plaintext\">0.97713715</div></td><td>LightGbmMulti</td><td>{&quot;0&quot;:{&quot;NumberOfLeaves&quot;:4,&quot;MinimumExampleCountPerLeaf&quot;:20,&quot;LearningRate&quot;:0.9999997766729865,&quot;NumberOfTrees&quot;:4,&quot;SubsampleFraction&quot;:0.9490243618001709,&quot;MaximumBinCountPerFeature&quot;:282,&quot;FeatureFraction&quot;:0.99999999,&quot;L1Regularization&quot;:2.0746147925314597E-10,&quot;L2Regularization&quot;:0.9999997766729865,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">16</div></i></td><td><div class=\"dni-plaintext\">16</div></td><td><div class=\"dni-plaintext\">0.98558646</div></td><td>LbfgsLogisticRegressionOva</td><td>{&quot;0&quot;:{&quot;L1Regularization&quot;:0.60703355,&quot;L2Regularization&quot;:0.27436227,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">17</div></i></td><td><div class=\"dni-plaintext\">17</div></td><td><div class=\"dni-plaintext\">0.91053677</div></td><td>SdcaLogisticRegressionOva</td><td>{&quot;0&quot;:{&quot;L1Regularization&quot;:0.79119885,&quot;L2Regularization&quot;:0.39210808,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">18</div></i></td><td><div class=\"dni-plaintext\">18</div></td><td><div class=\"dni-plaintext\">0.9860835</div></td><td>FastForestOva</td><td>{&quot;0&quot;:{&quot;NumberOfTrees&quot;:5,&quot;NumberOfLeaves&quot;:7,&quot;FeatureFraction&quot;:1,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">19</div></i></td><td><div class=\"dni-plaintext\">19</div></td><td><div class=\"dni-plaintext\">0.971173</div></td><td>FastTreeOva</td><td>{&quot;0&quot;:{&quot;NumberOfLeaves&quot;:6,&quot;MinimumExampleCountPerLeaf&quot;:24,&quot;NumberOfTrees&quot;:4,&quot;MaximumBinCountPerFeature&quot;:465,&quot;FeatureFraction&quot;:0.99999999,&quot;LearningRate&quot;:0.005305832399633224,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr></tbody></table>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"monitor.SetUpdate(monitor.Display());\n",
"var expResult = await experiment.RunAsync();"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Get best model"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [],
"source": [
"var bestModel = expResult.Model; "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Calculate PFI"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [],
"source": [
"var pfi = \n",
" mlContext.MulticlassClassification.PermutationFeatureImportance(bestModel,transformedData,permutationCount:3);"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Display most important features\n",
"\n",
"Using mean micro-accuracy as the metric since that's what AutoML used as the metric to optimize during training."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [
{
"data": {
"text/html": [
"<table><thead><tr><th><i>index</i></th><th>Item1</th><th>Item2</th></tr></thead><tbody><tr><td>0</td><td>Feature1</td><td><div class=\"dni-plaintext\">-0.05460000000000006</div></td></tr><tr><td>1</td><td>Feature2</td><td><div class=\"dni-plaintext\">-0.13686666666666672</div></td></tr></tbody></table>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"pfi.Select(x => Tuple.Create(x.Key,x.Value.MicroAccuracy.Mean))\n",
" .OrderByDescending(x => x.Item2)\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": ".NET (C#)",
"language": "C#",
"name": ".net-csharp"
},
"language_info": {
"name": "C#"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
@luisquintanilla
Copy link
Author

luisquintanilla commented Sep 23, 2022 •
edited
Loading

T

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment