syedhassaanahmed’s gists

syedhassaanahmed / AppInsightsListener.scala

Last active September 27, 2020 19:05

Asynchronously monitor Spark structured streaming queries with Azure Application Insights

	import scala.collection.JavaConversions._
	import java.util.HashMap
	import com.microsoft.applicationinsights._
	import org.apache.spark.sql.streaming._
	import org.apache.spark.sql.streaming.StreamingQueryListener._

	class AppInsightsListener(instrumentationKey: String, eventName: String) extends StreamingQueryListener {
	val telemetry = new TelemetryClient
	telemetry.getContext.setInstrumentationKey(instrumentationKey)

syedhassaanahmed / migrate_kv.sh

Last active December 3, 2021 09:05

Copy secrets from one Azure Key Vault to another assuming they're both in the same Azure subscription

	#!/bin/bash
	set -euo pipefail

	SOURCE_KV="<source_key_vault_name>"
	DEST_KV="<destination_key_vault_name>"

	SOURCE_SECRETS=$(az keyvault secret list --vault-name $SOURCE_KV --query "[].id" -o tsv \| cut -d "/" -f5)
	DEST_SECRETS=$(az keyvault secret list --vault-name $DEST_KV --query "[].id" -o tsv \| cut -d "/" -f5)
	MISSING_SECRETS=$(echo "${SOURCE_SECRETS} ${DEST_SECRETS}" \| tr ' ' '\n' \| sort \| uniq -u)

syedhassaanahmed / azure-pipelines-cleanup-acr.yaml

Created February 4, 2020 20:25

	stages:
	- stage: cleanupAcr
	displayName: Cleanup ACR Images for PR
	pool:
	vmImage: ubuntu-latest

	jobs:
	- job: cleanupAcr
	variables:
	prMergeMessage: 'Merged PR '

syedhassaanahmed / generate-int-days.csl

Last active April 28, 2019 20:20

Generate comma separated integer days for SQL Server Partition Function

	let endDate = datetime(2019-04-28);
	range day from endDate - 120d to endDate step 1d
	\| summarize days = makelist(replace("-", "", substring(day, 0, 10)))
	\| project array_strcat(days, ", ")

syedhassaanahmed / generate-partitions.csl

Created April 15, 2019 21:40

Kusto query to generate N partition tables

	let tablePrefix = "TimeseriesTable";
	let partitions = 2;
	//
	let partitionTables = materialize(
	range i from 0 to partitions - 1 step 1
	\| extend t = strcat(tablePrefix, "_", i)
	);
	//
	let dropTables = toscalar(
	partitionTables

syedhassaanahmed / prep-gremlin-console.sh

Created January 28, 2018 22:56

Downloads Apache Tinkerpop Gremlin Console and load it with data referred in this book http://kelvinlawrence.net/book/Gremlin-Graph-Guide.html

	#!/bin/bash

	# Download and extract Gremlin Console
	curl -o gremlin-console.zip http://apache.mirrors.spacedump.net/tinkerpop/3.3.1/apache-tinkerpop-gremlin-console-3.3.1-bin.zip
	unzip -n gremlin-console.zip && rm gremlin-console.zip

	# Download GraphML data of air routes into /mydata
	curl --create-dirs -o /mydata/air-routes.graphml https://raw.githubusercontent.com/krlawrence/graph/master/sample-data/air-routes.graphml

	# Download groovy script to load data

syedhassaanahmed / kafka-connect-iothub.sh

Last active December 9, 2017 09:49

	#!/bin/bash

	# Script is idempotent
	CLUSTERNAME=$1
	PASSWORD=$2
	TOPIC=$3

	# Import HDInsight helper method module
	wget -O /tmp/HDInsightUtilities-v01.sh -q https://hdiconfigactions.blob.core.windows.net/linuxconfigactionmodulev01/HDInsightUtilities-v01.sh && source /tmp/HDInsightUtilities-v01.sh && rm -f /tmp/HDInsightUtilities-v01.sh

syedhassaanahmed / shell-launcher.ps1

Last active September 13, 2017 00:37

Shell Launcher

	# Get Custom Shell
	Get-WmiObject -namespace root\standardcimv2\embedded -computer localhost -class WESL_UserSetting \| Select *

	# Remove Custom Shell for SID
	([wmiclass]"\\localhost\root\standardcimv2\embedded:WESL_UserSetting").RemoveCustomShell("<User SID>")

	# Is Custom Shell Enabled
	([wmiclass]"\\localhost\root\standardcimv2\embedded:WESL_UserSetting").IsEnabled()

	# Disable Custom Shell

syedhassaanahmed / setup-new-pc.cmd

Last active November 21, 2018 10:39

Setup new PC

	@powershell -NoProfile -ExecutionPolicy Bypass -Command "iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))" && SET "PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin"

	choco install 7zip /y
	choco install adobereader /y
	choco install anaconda3 /y
	choco install azure-cli /y
	choco install docker-for-windows /y
	choco install firefox /y
	choco install garmin-express /y
	choco install git /y

Hassaan Ahmed syedhassaanahmed