Skip to content

Instantly share code, notes, and snippets.

@cosh
cosh / iotcontext.kql
Created November 12, 2021 15:55
KQL queries to contextualize timeseries data
//The top level request
GetRelevantTimeseriesData('3000000004')
//Creates a function to joins data from SQL, ADT and ADX
.create-or-alter function with (folder = "Analytics/IoT", skipvalidation = "true") GetRelevantTimeseriesData(purchaseOrder:string) {
let productionOrder = GetProductionOrderData(purchaseOrder);
let plantID = toscalar (productionOrder | project tostring(Plant_Code));
let lineID = toscalar (productionOrder | project tostring(Mfg_Line));
let productionDate = toscalar (productionOrder | project todatetime(Date));
let startDate = productionDate - 24h;
@cosh
cosh / LightIngestCommand.kusto
Last active June 6, 2023 22:31
How to partition data for ingestion to Kusto
LightIngest.exe "https://ingest-<clusterName>.<region>.kusto.windows.net/;Fed=True" -database:"<databaseName>" -table:"tableName" -source:"https://<storageAccountName>.blob.core.windows.net/<containerName><sasToken>" -format:"parquet" -prefix:"<folderPrefix>" -pattern:"*.parquet" -ingestionMappingRef:"<mappingName" -creationTimePattern:"day='yyyyMMdd'/" -dontWait:true
let telemetryData = datatable (deviceid: string, ts: datetime, latitude: real, longitude: real) [
"alice",datetime(2022-11-07T23:11:19Z),real(47.6423771),real(-122.1267281),
"alice",datetime(2022-11-07T23:11:20Z),real(47.6424771),real(-122.1277281),
"alice",datetime(2022-11-07T23:11:21Z),real(47.6425771),real(-122.1287281),
"bob",datetime(2022-11-07T23:11:19Z),real(47.6424771),real(-122.1287281),
"bob",datetime(2022-11-07T23:11:20Z),real(47.6424771),real(-122.1287281),
"carol",datetime(2022-11-07T23:11:22Z),real(47.6423771),real(-122.1267281),
"carol",datetime(2022-11-07T23:11:22Z),real(47.6523771),real(-122.1267281),
"carol",datetime(2022-11-07T23:11:24Z),real(47.6623771),real(-122.1267281),
"dave",datetime(2022-11-07T23:11:24Z),real(47.644205),real(-122.1412026),
@cosh
cosh / GraphPlotly.kql
Last active February 28, 2024 17:07
A stored function that uses the evaluate python operator to execute a Python script that uses the plotly, networkx and pandas libraries to create a plotly viz object from the input tables. The function also adds some styling and annotations to the graph object, such as colors, sizes, hover texts, etc. It uses the replace_string function to inser…
.create-or-alter function with (skipvalidation = "true") VisualizeGraphPlotly(
E:(sourceId:long,targetId:long), N:(nodeId:long),
pLayout:string="spring_layout", pColorscale:string="Picnic", pTitle:string="Happy kraphing!") {
let pythonCodeBlueprint = ```
import numpy as np
import pandas as pd
import plotly.graph_objects as go
import networkx as nx
G = nx.Graph()
@cosh
cosh / config.json
Last active November 9, 2023 07:33
VNet injection migration for ADX
{
"clusters": [
{
"subscription_id": "sid",
"resource_group_name": "kusto-workload",
"cluster_name": "adx-in-vnet"
},
{
"subscription_id": "sid",
"resource_group_name": "kusto-workload",