Skip to content

Instantly share code, notes, and snippets.

@luisquintanilla

luisquintanilla/Multiclass-PFI-AutoML-Featurizer.ipynb

Created September 21, 2022 04:18
Show Gist options
  • Save luisquintanilla/16ef56e2563bd2ffef66a6e05cb6e2a8 to your computer and use it in GitHub Desktop.
Save luisquintanilla/16ef56e2563bd2ffef66a6e05cb6e2a8 to your computer and use it in GitHub Desktop.
Download ZIP
AutoML Multiclass Classification Experiment using AutoML Featurizer
Display the source blob
Display the rendered blob
Raw
Multiclass-PFI-AutoML-Featurizer.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Install NuGet packages"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [
{
"data": {
"text/html": [
"<div><div></div><div></div><div><strong>Installed Packages</strong><ul><li><span>Microsoft.Data.Analysis, 0.20.0-preview.22356.1</span></li><li><span>Microsoft.ML.AutoML, 0.20.0-preview.22356.1</span></li><li><span>Plotly.NET.CSharp, 0.0.1</span></li><li><span>Plotly.NET.Interactive, 3.0.2</span></li></ul></div></div>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"#r \"nuget: Plotly.NET.Interactive, 3.0.2\"\n",
"#r \"nuget: Plotly.NET.CSharp, 0.0.1\"\n",
"#r \"nuget:Microsoft.ML.AutoML, 0.20.0-preview.22356.1\"\n",
"#r \"nuget: Microsoft.Data.Analysis, 0.20.0-preview.22356.1\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Add using statements"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [],
"source": [
"using System;\n",
"using System.IO;\n",
"using System.Collections.Generic;\n",
"using System.Linq;\n",
"using Microsoft.ML;\n",
"using Microsoft.ML.Data;\n",
"using Microsoft.ML.AutoML;\n",
"using Microsoft.Data.Analysis;"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Define data schema"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [],
"source": [
"public class Data\n",
"{\n",
" public float Feature1 {get;set;}\n",
" public float Feature2 {get;set;}\n",
" public float Label {get;set;}\n",
"}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Define method to generate random data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [],
"source": [
"public IEnumerable<Data> GenerateData(int nExamples = 10000,double bias = 0, double weight1 = 1, double weight2 = 2, int seed = 1)\n",
"{\n",
" var rng = new Random(seed);\n",
" var max = bias + 4.5 * weight1 + 4.5 * weight2 + 0.5;\n",
" for (int i = 0; i < nExamples; i++)\n",
" {\n",
" var data = new Data\n",
" {\n",
" Feature1 = (float)(rng.Next(10) * (rng.NextDouble() - 0.5)),\n",
" Feature2 = (float)(rng.Next(10) * (rng.NextDouble() - 0.5)),\n",
" };\n",
"\n",
" // Create a noisy label.\n",
" var value = (float)\n",
" (bias + weight1 * data.Feature1 + weight2 * data.Feature2 +\n",
" rng.NextDouble() - 0.5);\n",
"\n",
" if (value < max / 3)\n",
" data.Label = 0;\n",
" else if (value < 2 * max / 3)\n",
" data.Label = 1;\n",
" else\n",
" data.Label = 2;\n",
" yield return data;\n",
" }\n",
"}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Initialize MLContext"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [],
"source": [
"var mlContext = new MLContext(seed:1);"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Generate data samples"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [],
"source": [
"var data = GenerateData();"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [
{
"data": {
"text/html": [
"<table><thead><tr><th><i>index</i></th><th>Feature1</th><th>Feature2</th><th>Label</th></tr></thead><tbody><tr><td>0</td><td><div class=\"dni-plaintext\">-0.77851206</div></td><td><div class=\"dni-plaintext\">1.0864165</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>1</td><td><div class=\"dni-plaintext\">-0.58366495</div></td><td><div class=\"dni-plaintext\">-3.5886018</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>2</td><td><div class=\"dni-plaintext\">-0</div></td><td><div class=\"dni-plaintext\">1.4693015</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>3</td><td><div class=\"dni-plaintext\">-1.3036246</div></td><td><div class=\"dni-plaintext\">1.2255092</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>4</td><td><div class=\"dni-plaintext\">-3.6595037</div></td><td><div class=\"dni-plaintext\">-0.118028924</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>5</td><td><div class=\"dni-plaintext\">0.29378363</div></td><td><div class=\"dni-plaintext\">0.9690853</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>6</td><td><div class=\"dni-plaintext\">1.08367</div></td><td><div class=\"dni-plaintext\">-2.9183135</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>7</td><td><div class=\"dni-plaintext\">2.8699489</div></td><td><div class=\"dni-plaintext\">0</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>8</td><td><div class=\"dni-plaintext\">0.9875927</div></td><td><div class=\"dni-plaintext\">-0.6803071</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>9</td><td><div class=\"dni-plaintext\">1.06429</div></td><td><div class=\"dni-plaintext\">-1.021925</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>10</td><td><div class=\"dni-plaintext\">-1.6528636</div></td><td><div class=\"dni-plaintext\">0.57508487</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>11</td><td><div class=\"dni-plaintext\">0.4498368</div></td><td><div class=\"dni-plaintext\">0</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>12</td><td><div class=\"dni-plaintext\">-2.7188668</div></td><td><div class=\"dni-plaintext\">-0.2852031</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>13</td><td><div class=\"dni-plaintext\">0.14619994</div></td><td><div class=\"dni-plaintext\">1.4936732</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>14</td><td><div class=\"dni-plaintext\">-1.9584187</div></td><td><div class=\"dni-plaintext\">-0.32100672</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>15</td><td><div class=\"dni-plaintext\">-0.8794435</div></td><td><div class=\"dni-plaintext\">3.13346</div></td><td><div class=\"dni-plaintext\">1</div></td></tr><tr><td>16</td><td><div class=\"dni-plaintext\">-0.71636426</div></td><td><div class=\"dni-plaintext\">-0</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>17</td><td><div class=\"dni-plaintext\">-0.3895614</div></td><td><div class=\"dni-plaintext\">0</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>18</td><td><div class=\"dni-plaintext\">2.189632</div></td><td><div class=\"dni-plaintext\">-0</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td>19</td><td><div class=\"dni-plaintext\">0.745925</div></td><td><div class=\"dni-plaintext\">-0.798151</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td colspan=\"4\"><i>... (more)</i></td></tr></tbody></table>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"data"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Load data samples into IDataView"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [],
"source": [
"var idv = mlContext.Data.LoadFromEnumerable(data);"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Define AutoML Pipeline"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [],
"source": [
"var numericCols = new string[] { nameof(Data.Feature1), nameof(Data.Feature2) };"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [],
"source": [
"var autoMLPipeline = \n",
" mlContext.Auto().Featurizer(idv,numericColumns:numericCols)\n",
" .Append(mlContext.Transforms.Conversion.MapValueToKey(\"Label\"))\n",
" .Append(mlContext.Auto().MultiClassification());\n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Define AutoML Experiment"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [],
"source": [
"var experiment = \n",
" mlContext.Auto().CreateExperiment()\n",
" .SetPipeline(autoMLPipeline)\n",
" .SetEvaluateMetric(MulticlassClassificationMetric.MicroAccuracy,labelColumn:\"Label\")\n",
" .SetTrainingTimeInSeconds(60)\n",
" .SetDataset(idv,fold:5);"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Initialize monitor (only works in notebooks)\n",
"\n",
"To log outputs in a console app, use the following code:\n",
"\n",
"```csharp\n",
"mlContext.Log += (object? sender, LoggingEventArgs e) =>\n",
"{\n",
" if (e.Source.Contains(\"AutoMLExperiment\")) Console.WriteLine(e.RawMessage);\n",
"};\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [],
"source": [
"var monitor = new NotebookMonitor();"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [],
"source": [
"experiment.SetMonitor(monitor);"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Run experiment"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [
{
"data": {
"text/html": [
"<div><h3>Best Trial</h3><p>Id: 6</p><p>Trainer: ReplaceMissingValues=&gt;Concatenate=&gt;LbfgsMaximumEntropyMulti</p><p>Parameters: {\r\n",
" &quot;0&quot;: {\r\n",
" &quot;OutputColumnNames&quot;: [\r\n",
" &quot;Feature1&quot;,\r\n",
" &quot;Feature2&quot;\r\n",
" ],\r\n",
" &quot;InputColumnNames&quot;: [\r\n",
" &quot;Feature1&quot;,\r\n",
" &quot;Feature2&quot;\r\n",
" ]\r\n",
" },\r\n",
" &quot;1&quot;: {\r\n",
" &quot;InputColumnNames&quot;: [\r\n",
" &quot;Feature1&quot;,\r\n",
" &quot;Feature2&quot;\r\n",
" ],\r\n",
" &quot;OutputColumnName&quot;: &quot;Features&quot;\r\n",
" },\r\n",
" &quot;2&quot;: {},\r\n",
" &quot;3&quot;: {\r\n",
" &quot;L1Regularization&quot;: 1,\r\n",
" &quot;L2Regularization&quot;: 1,\r\n",
" &quot;LabelColumnName&quot;: &quot;Label&quot;,\r\n",
" &quot;FeatureColumnName&quot;: &quot;Features&quot;\r\n",
" }\r\n",
"}</p><h3>Active Trial</h3><p>Id: 19</p><p>Trainer: ReplaceMissingValues=&gt;Concatenate=&gt;FastForestOva</p><p>Parameters: {\r\n",
" &quot;0&quot;: {\r\n",
" &quot;OutputColumnNames&quot;: [\r\n",
" &quot;Feature1&quot;,\r\n",
" &quot;Feature2&quot;\r\n",
" ],\r\n",
" &quot;InputColumnNames&quot;: [\r\n",
" &quot;Feature1&quot;,\r\n",
" &quot;Feature2&quot;\r\n",
" ]\r\n",
" },\r\n",
" &quot;1&quot;: {\r\n",
" &quot;InputColumnNames&quot;: [\r\n",
" &quot;Feature1&quot;,\r\n",
" &quot;Feature2&quot;\r\n",
" ],\r\n",
" &quot;OutputColumnName&quot;: &quot;Features&quot;\r\n",
" },\r\n",
" &quot;2&quot;: {},\r\n",
" &quot;3&quot;: {\r\n",
" &quot;NumberOfTrees&quot;: 209,\r\n",
" &quot;NumberOfLeaves&quot;: 4,\r\n",
" &quot;FeatureFraction&quot;: 0.9150315,\r\n",
" &quot;LabelColumnName&quot;: &quot;Label&quot;,\r\n",
" &quot;FeatureColumnName&quot;: &quot;Features&quot;\r\n",
" }\r\n",
"}</p></div><div><h3>Plot Metrics over Trials</h3></div>\n",
"<div>\n",
" <div id=\"1576b49d-dac2-47b4-98e9-5bd5e8a037fc\"><!-- Plotly chart will be drawn inside this DIV --></div>\r\n",
"<script type=\"text/javascript\">\r\n",
"\r\n",
" var renderPlotly_1576b49ddac247b498e95bd5e8a037fc = function() {\r\n",
" var fsharpPlotlyRequire = requirejs.config({context:'fsharp-plotly',paths:{plotly:'https://cdn.plot.ly/plotly-2.6.3.min'}}) || require;\r\n",
" fsharpPlotlyRequire(['plotly'], function(Plotly) {\r\n",
"\r\n",
" var data = [{\"type\":\"scatter\",\"name\":\"Plot Metrics over Trials.\",\"mode\":\"markers\",\"x\":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18],\"y\":[0.9363817097415507,0.9567594433399602,0.9204771371769384,0.98558648111332,0.9840954274353877,0.9587475149105368,0.989065606361829,0.9766401590457257,0.9696819085487077,0.9657057654075547,0.9860834990059643,0.9850894632206759,0.9821073558648111,0.98558648111332,0.9507952286282306,0.9110337972166997,0.9105367793240556,0.981610337972167,0.989065606361829],\"marker\":{},\"line\":{},\"showlegend\":false}];\r\n",
" var layout = {\"width\":600,\"height\":600,\"template\":{\"layout\":{\"title\":{\"x\":0.05},\"font\":{\"color\":\"rgba(42, 63, 95, 1.0)\"},\"paper_bgcolor\":\"rgba(255, 255, 255, 1.0)\",\"plot_bgcolor\":\"rgba(229, 236, 246, 1.0)\",\"autotypenumbers\":\"strict\",\"colorscale\":{\"diverging\":[[0.0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1.0,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}},\"geo\":{\"showland\":true,\"landcolor\":\"rgba(229, 236, 246, 1.0)\",\"showlakes\":true,\"lakecolor\":\"rgba(255, 255, 255, 1.0)\",\"subunitcolor\":\"rgba(255, 255, 255, 1.0)\",\"bgcolor\":\"rgba(255, 255, 255, 1.0)\"},\"mapbox\":{\"style\":\"light\"},\"polar\":{\"bgcolor\":\"rgba(229, 236, 246, 1.0)\",\"radialaxis\":{\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"ticks\":\"\"},\"angularaxis\":{\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"ticks\":\"\",\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"gridwidth\":2.0,\"zerolinecolor\":\"rgba(255, 255, 255, 1.0)\",\"backgroundcolor\":\"rgba(229, 236, 246, 1.0)\",\"showbackground\":true},\"yaxis\":{\"ticks\":\"\",\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"gridwidth\":2.0,\"zerolinecolor\":\"rgba(255, 255, 255, 1.0)\",\"backgroundcolor\":\"rgba(229, 236, 246, 1.0)\",\"showbackground\":true},\"zaxis\":{\"ticks\":\"\",\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"gridwidth\":2.0,\"zerolinecolor\":\"rgba(255, 255, 255, 1.0)\",\"backgroundcolor\":\"rgba(229, 236, 246, 1.0)\",\"showbackground\":true}},\"ternary\":{\"aaxis\":{\"ticks\":\"\",\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\"},\"baxis\":{\"ticks\":\"\",\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\"},\"caxis\":{\"ticks\":\"\",\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\"},\"bgcolor\":\"rgba(229, 236, 246, 1.0)\"},\"xaxis\":{\"title\":{\"standoff\":15},\"ticks\":\"\",\"automargin\":true,\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"zerolinecolor\":\"rgba(255, 255, 255, 1.0)\",\"zerolinewidth\":2.0},\"yaxis\":{\"title\":{\"standoff\":15},\"ticks\":\"\",\"automargin\":true,\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"zerolinecolor\":\"rgba(255, 255, 255, 1.0)\",\"zerolinewidth\":2.0},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"shapedefaults\":{\"line\":{\"color\":\"rgba(42, 63, 95, 1.0)\"}},\"colorway\":[\"rgba(99, 110, 250, 1.0)\",\"rgba(239, 85, 59, 1.0)\",\"rgba(0, 204, 150, 1.0)\",\"rgba(171, 99, 250, 1.0)\",\"rgba(255, 161, 90, 1.0)\",\"rgba(25, 211, 243, 1.0)\",\"rgba(255, 102, 146, 1.0)\",\"rgba(182, 232, 128, 1.0)\",\"rgba(255, 151, 255, 1.0)\",\"rgba(254, 203, 82, 1.0)\"]},\"data\":{\"bar\":[{\"marker\":{\"line\":{\"color\":\"rgba(229, 236, 246, 1.0)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"error_x\":{\"color\":\"rgba(42, 63, 95, 1.0)\"},\"error_y\":{\"color\":\"rgba(42, 63, 95, 1.0)\"}}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"rgba(229, 236, 246, 1.0)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}}}],\"carpet\":[{\"aaxis\":{\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"endlinecolor\":\"rgba(42, 63, 95, 1.0)\",\"minorgridcolor\":\"rgba(255, 255, 255, 1.0)\",\"startlinecolor\":\"rgba(42, 63, 95, 1.0)\"},\"baxis\":{\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"endlinecolor\":\"rgba(42, 63, 95, 1.0)\",\"minorgridcolor\":\"rgba(255, 255, 255, 1.0)\",\"startlinecolor\":\"rgba(42, 63, 95, 1.0)\"}}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}}}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"pie\":[{\"automargin\":true}],\"scatter\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}},\"line\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scattergl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"rgba(235, 240, 248, 1.0)\"},\"line\":{\"color\":\"rgba(255, 255, 255, 1.0)\"}},\"header\":{\"fill\":{\"color\":\"rgba(200, 212, 227, 1.0)\"},\"line\":{\"color\":\"rgba(255, 255, 255, 1.0)\"}}}]}},\"xaxis\":{\"title\":{\"text\":\"Trial\"},\"showgrid\":false},\"yaxis\":{\"title\":{\"text\":\"Metric\"},\"showgrid\":false}};\r\n",
" var config = {\"responsive\":true};\r\n",
" Plotly.newPlot('1576b49d-dac2-47b4-98e9-5bd5e8a037fc', data, layout, config);\r\n",
"});\r\n",
" };\r\n",
" if ((typeof(requirejs) !== typeof(Function)) || (typeof(requirejs.config) !== typeof(Function))) {\r\n",
" var script = document.createElement(\"script\");\r\n",
" script.setAttribute(\"src\", \"https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.6/require.min.js\");\r\n",
" script.onload = function(){\r\n",
" renderPlotly_1576b49ddac247b498e95bd5e8a037fc();\r\n",
" };\r\n",
" document.getElementsByTagName(\"head\")[0].appendChild(script);\r\n",
" }\r\n",
" else {\r\n",
" renderPlotly_1576b49ddac247b498e95bd5e8a037fc();\r\n",
" }\r\n",
"</script>\r\n",
"\n",
" \n",
"</div> \n",
"<div><h3>All Trials Table</h3></div><table id=\"table_637993161004873501\"><thead><tr><th><i>index</i></th><th>Trial</th><th>Metric</th><th>Trainer</th><th>Parameters</th></tr></thead><tbody><tr><td><i><div class=\"dni-plaintext\">0</div></i></td><td><div class=\"dni-plaintext\">0</div></td><td><div class=\"dni-plaintext\">0.9363817</div></td><td>ReplaceMissingValues=&gt;Concatenate=&gt;SdcaMaximumEntropyMulti</td><td>{&quot;0&quot;:{&quot;OutputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;]},&quot;1&quot;:{&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;OutputColumnName&quot;:&quot;Features&quot;},&quot;2&quot;:{},&quot;3&quot;:{&quot;L1Regularization&quot;:1,&quot;L2Regularization&quot;:0.1,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">1</div></i></td><td><div class=\"dni-plaintext\">1</div></td><td><div class=\"dni-plaintext\">0.95675945</div></td><td>ReplaceMissingValues=&gt;Concatenate=&gt;SdcaMaximumEntropyMulti</td><td>{&quot;0&quot;:{&quot;OutputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;]},&quot;1&quot;:{&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;OutputColumnName&quot;:&quot;Features&quot;},&quot;2&quot;:{},&quot;3&quot;:{&quot;L1Regularization&quot;:0.07077264,&quot;L2Regularization&quot;:0.22725023,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">2</div></i></td><td><div class=\"dni-plaintext\">2</div></td><td><div class=\"dni-plaintext\">0.92047715</div></td><td>ReplaceMissingValues=&gt;Concatenate=&gt;SdcaLogisticRegressionOva</td><td>{&quot;0&quot;:{&quot;OutputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;]},&quot;1&quot;:{&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;OutputColumnName&quot;:&quot;Features&quot;},&quot;2&quot;:{},&quot;3&quot;:{&quot;L1Regularization&quot;:1,&quot;L2Regularization&quot;:0.1,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">3</div></i></td><td><div class=\"dni-plaintext\">3</div></td><td><div class=\"dni-plaintext\">0.98558646</div></td><td>ReplaceMissingValues=&gt;Concatenate=&gt;LbfgsLogisticRegressionOva</td><td>{&quot;0&quot;:{&quot;OutputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;]},&quot;1&quot;:{&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;OutputColumnName&quot;:&quot;Features&quot;},&quot;2&quot;:{},&quot;3&quot;:{&quot;L1Regularization&quot;:1,&quot;L2Regularization&quot;:1,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">4</div></i></td><td><div class=\"dni-plaintext\">4</div></td><td><div class=\"dni-plaintext\">0.98409545</div></td><td>ReplaceMissingValues=&gt;Concatenate=&gt;FastForestOva</td><td>{&quot;0&quot;:{&quot;OutputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;]},&quot;1&quot;:{&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;OutputColumnName&quot;:&quot;Features&quot;},&quot;2&quot;:{},&quot;3&quot;:{&quot;NumberOfTrees&quot;:4,&quot;NumberOfLeaves&quot;:4,&quot;FeatureFraction&quot;:1,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">5</div></i></td><td><div class=\"dni-plaintext\">5</div></td><td><div class=\"dni-plaintext\">0.9587475</div></td><td>ReplaceMissingValues=&gt;Concatenate=&gt;FastTreeOva</td><td>{&quot;0&quot;:{&quot;OutputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;]},&quot;1&quot;:{&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;OutputColumnName&quot;:&quot;Features&quot;},&quot;2&quot;:{},&quot;3&quot;:{&quot;NumberOfLeaves&quot;:4,&quot;MinimumExampleCountPerLeaf&quot;:20,&quot;NumberOfTrees&quot;:4,&quot;MaximumBinCountPerFeature&quot;:255,&quot;FeatureFraction&quot;:1,&quot;LearningRate&quot;:0.09999999999999998,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">6</div></i></td><td><div class=\"dni-plaintext\">6</div></td><td><div class=\"dni-plaintext\">0.9890656</div></td><td>ReplaceMissingValues=&gt;Concatenate=&gt;LbfgsMaximumEntropyMulti</td><td>{&quot;0&quot;:{&quot;OutputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;]},&quot;1&quot;:{&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;OutputColumnName&quot;:&quot;Features&quot;},&quot;2&quot;:{},&quot;3&quot;:{&quot;L1Regularization&quot;:1,&quot;L2Regularization&quot;:1,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">7</div></i></td><td><div class=\"dni-plaintext\">7</div></td><td><div class=\"dni-plaintext\">0.97664016</div></td><td>ReplaceMissingValues=&gt;Concatenate=&gt;SdcaMaximumEntropyMulti</td><td>{&quot;0&quot;:{&quot;OutputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;]},&quot;1&quot;:{&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;OutputColumnName&quot;:&quot;Features&quot;},&quot;2&quot;:{},&quot;3&quot;:{&quot;L1Regularization&quot;:0.03125,&quot;L2Regularization&quot;:0.03125,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">8</div></i></td><td><div class=\"dni-plaintext\">8</div></td><td><div class=\"dni-plaintext\">0.9696819</div></td><td>ReplaceMissingValues=&gt;Concatenate=&gt;LightGbmMulti</td><td>{&quot;0&quot;:{&quot;OutputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;]},&quot;1&quot;:{&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;OutputColumnName&quot;:&quot;Features&quot;},&quot;2&quot;:{},&quot;3&quot;:{&quot;NumberOfLeaves&quot;:4,&quot;MinimumExampleCountPerLeaf&quot;:20,&quot;LearningRate&quot;:1,&quot;NumberOfTrees&quot;:4,&quot;SubsampleFraction&quot;:1,&quot;MaximumBinCountPerFeature&quot;:255,&quot;FeatureFraction&quot;:1,&quot;L1Regularization&quot;:2E-10,&quot;L2Regularization&quot;:1,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">9</div></i></td><td><div class=\"dni-plaintext\">9</div></td><td><div class=\"dni-plaintext\">0.96570575</div></td><td>ReplaceMissingValues=&gt;Concatenate=&gt;SdcaLogisticRegressionOva</td><td>{&quot;0&quot;:{&quot;OutputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;]},&quot;1&quot;:{&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;OutputColumnName&quot;:&quot;Features&quot;},&quot;2&quot;:{},&quot;3&quot;:{&quot;L1Regularization&quot;:0.08403141,&quot;L2Regularization&quot;:0.03125,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">10</div></i></td><td><div class=\"dni-plaintext\">10</div></td><td><div class=\"dni-plaintext\">0.9860835</div></td><td>ReplaceMissingValues=&gt;Concatenate=&gt;LbfgsLogisticRegressionOva</td><td>{&quot;0&quot;:{&quot;OutputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;]},&quot;1&quot;:{&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;OutputColumnName&quot;:&quot;Features&quot;},&quot;2&quot;:{},&quot;3&quot;:{&quot;L1Regularization&quot;:0.08252813,&quot;L2Regularization&quot;:3.3535383,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">11</div></i></td><td><div class=\"dni-plaintext\">11</div></td><td><div class=\"dni-plaintext\">0.9850895</div></td><td>ReplaceMissingValues=&gt;Concatenate=&gt;FastForestOva</td><td>{&quot;0&quot;:{&quot;OutputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;]},&quot;1&quot;:{&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;OutputColumnName&quot;:&quot;Features&quot;},&quot;2&quot;:{},&quot;3&quot;:{&quot;NumberOfTrees&quot;:15,&quot;NumberOfLeaves&quot;:4,&quot;FeatureFraction&quot;:0.96403414,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">12</div></i></td><td><div class=\"dni-plaintext\">12</div></td><td><div class=\"dni-plaintext\">0.98210734</div></td><td>ReplaceMissingValues=&gt;Concatenate=&gt;FastTreeOva</td><td>{&quot;0&quot;:{&quot;OutputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;]},&quot;1&quot;:{&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;OutputColumnName&quot;:&quot;Features&quot;},&quot;2&quot;:{},&quot;3&quot;:{&quot;NumberOfLeaves&quot;:12,&quot;MinimumExampleCountPerLeaf&quot;:20,&quot;NumberOfTrees&quot;:4,&quot;MaximumBinCountPerFeature&quot;:137,&quot;FeatureFraction&quot;:0.99999999,&quot;LearningRate&quot;:0.19516292884163178,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">13</div></i></td><td><div class=\"dni-plaintext\">13</div></td><td><div class=\"dni-plaintext\">0.98558646</div></td><td>ReplaceMissingValues=&gt;Concatenate=&gt;LbfgsMaximumEntropyMulti</td><td>{&quot;0&quot;:{&quot;OutputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;]},&quot;1&quot;:{&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;OutputColumnName&quot;:&quot;Features&quot;},&quot;2&quot;:{},&quot;3&quot;:{&quot;L1Regularization&quot;:0.28111878,&quot;L2Regularization&quot;:11.765196,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">14</div></i></td><td><div class=\"dni-plaintext\">14</div></td><td><div class=\"dni-plaintext\">0.95079523</div></td><td>ReplaceMissingValues=&gt;Concatenate=&gt;SdcaMaximumEntropyMulti</td><td>{&quot;0&quot;:{&quot;OutputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;]},&quot;1&quot;:{&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;OutputColumnName&quot;:&quot;Features&quot;},&quot;2&quot;:{},&quot;3&quot;:{&quot;L1Regularization&quot;:1.7170361,&quot;L2Regularization&quot;:0.03125,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">15</div></i></td><td><div class=\"dni-plaintext\">15</div></td><td><div class=\"dni-plaintext\">0.9110338</div></td><td>ReplaceMissingValues=&gt;Concatenate=&gt;SdcaLogisticRegressionOva</td><td>{&quot;0&quot;:{&quot;OutputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;]},&quot;1&quot;:{&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;OutputColumnName&quot;:&quot;Features&quot;},&quot;2&quot;:{},&quot;3&quot;:{&quot;L1Regularization&quot;:4.4583187,&quot;L2Regularization&quot;:0.03125,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">16</div></i></td><td><div class=\"dni-plaintext\">16</div></td><td><div class=\"dni-plaintext\">0.91053677</div></td><td>ReplaceMissingValues=&gt;Concatenate=&gt;LightGbmMulti</td><td>{&quot;0&quot;:{&quot;OutputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;]},&quot;1&quot;:{&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;OutputColumnName&quot;:&quot;Features&quot;},&quot;2&quot;:{},&quot;3&quot;:{&quot;NumberOfLeaves&quot;:4,&quot;MinimumExampleCountPerLeaf&quot;:22,&quot;LearningRate&quot;:0.08222761642127213,&quot;NumberOfTrees&quot;:4,&quot;SubsampleFraction&quot;:0.23872775362230472,&quot;MaximumBinCountPerFeature&quot;:256,&quot;FeatureFraction&quot;:0.9848970001576354,&quot;L1Regularization&quot;:1.6025190944797323E-09,&quot;L2Regularization&quot;:0.1997721367501075,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">17</div></i></td><td><div class=\"dni-plaintext\">17</div></td><td><div class=\"dni-plaintext\">0.98161036</div></td><td>ReplaceMissingValues=&gt;Concatenate=&gt;LbfgsLogisticRegressionOva</td><td>{&quot;0&quot;:{&quot;OutputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;]},&quot;1&quot;:{&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;OutputColumnName&quot;:&quot;Features&quot;},&quot;2&quot;:{},&quot;3&quot;:{&quot;L1Regularization&quot;:0.03125,&quot;L2Regularization&quot;:57.22619,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr><tr><td><i><div class=\"dni-plaintext\">18</div></i></td><td><div class=\"dni-plaintext\">18</div></td><td><div class=\"dni-plaintext\">0.9890656</div></td><td>ReplaceMissingValues=&gt;Concatenate=&gt;LbfgsMaximumEntropyMulti</td><td>{&quot;0&quot;:{&quot;OutputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;]},&quot;1&quot;:{&quot;InputColumnNames&quot;:[&quot;Feature1&quot;,&quot;Feature2&quot;],&quot;OutputColumnName&quot;:&quot;Features&quot;},&quot;2&quot;:{},&quot;3&quot;:{&quot;L1Regularization&quot;:3.5572152,&quot;L2Regularization&quot;:0.08499646,&quot;LabelColumnName&quot;:&quot;Label&quot;,&quot;FeatureColumnName&quot;:&quot;Features&quot;}}</td></tr></tbody></table>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"monitor.SetUpdate(monitor.Display());\n",
"var expResult = await experiment.RunAsync();"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Get best model"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [],
"source": [
"var bestModel = expResult.Model; "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Calculate PFI"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [],
"source": [
"var transformedData = bestModel.Transform(idv);"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [],
"source": [
"var pfi = \n",
" mlContext.MulticlassClassification.PermutationFeatureImportance(bestModel,transformedData,permutationCount:3);"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Display most important features\n",
"\n",
"Using mean micro-accuracy as the metric since that's what AutoML used as the metric to optimize during training."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"dotnet_interactive": {
"language": "csharp"
},
"vscode": {
"languageId": "dotnet-interactive.csharp"
}
},
"outputs": [
{
"data": {
"text/html": [
"<table><thead><tr><th><i>index</i></th><th>Item1</th><th>Item2</th></tr></thead><tbody><tr><td>0</td><td>Feature1</td><td><div class=\"dni-plaintext\">-0.056033333333333345</div></td></tr><tr><td>1</td><td>Feature2</td><td><div class=\"dni-plaintext\">-0.13823333333333335</div></td></tr></tbody></table>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"pfi.Select(x => Tuple.Create(x.Key,x.Value.MicroAccuracy.Mean))\n",
" .OrderByDescending(x => x.Item2)\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": ".NET (C#)",
"language": "C#",
"name": ".net-csharp"
},
"language_info": {
"name": "C#"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment