Skip to content

Instantly share code, notes, and snippets.

View mdrakiburrahman's full-sized avatar
💤
living the data dream

Raki mdrakiburrahman

💤
living the data dream
84 followers · 85 following
View GitHub Profile
@mdrakiburrahman
mdrakiburrahman / otel-delta.dbml
Created May 19, 2025 21:40
OpenTelemetry schema in Delta Lake representation
Table "columnar"."log" {
"id" varchar [pk]
"event_year_date" varchar
"region" varchar
"service_name" varchar
"time_utc" datetime2
"observed_time_utc" datetime2
"scope_name" varchar
"scope_version" varchar
"trace_id" varchar
@mdrakiburrahman
mdrakiburrahman / LogsSchema.scala
Created May 14, 2025 13:54
package microsoft.opentelemetrycollector.otlp.json.v1
import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.types.StructType
// @formatter:off
/** Represents a schema for a Logs payload.
*
* @param spark
* The SparkSession for schema inference.
@mdrakiburrahman
mdrakiburrahman / update-synapse-jar.sh
Created July 1, 2024 20:39
IFS='|' read -ra JAR_ARRAY <<<"$SYNAPSE_WORKSPACE_PACKAGE_UPLOAD_PIPE_DELIMITED"
for JAR in "${JAR_ARRAY[@]}"; do
export PACKAGE_NAME=$(echo "${JAR}" | awk -F'/' '{print $NF}')
echo "Checking if package ${PACKAGE_NAME} exists in ${SYNAPSE_WORKSPACE_NAME}"
export PACKAGE_ID_EXISTING=$(az synapse workspace-package show --workspace-name "$SYNAPSE_WORKSPACE_NAME" --name "$PACKAGE_NAME" --query "id" -o tsv)
export PACKAGE_EXISTS=false
if [ -z "$PACKAGE_ID_EXISTING" ]; then
echo "Package ${PACKAGE_NAME} does not exist in ${SYNAPSE_WORKSPACE_NAME}."
else
@mdrakiburrahman
mdrakiburrahman / Spark-Connect-Csharp.md
Created April 22, 2024 03:03

Pictures and stuff.

@mdrakiburrahman
mdrakiburrahman / gist:a687b27f06929a27ed7f199cf301b2e2
Created October 13, 2023 01:53
graph TD
subgraph SqlServerInstancePrimary1
createEndpoint1
grantConnectOnEndpoint1
createAvailabilityGroup1
end
subgraph SqlServerInstanceSecondary2
createEndpoint2
@mdrakiburrahman
mdrakiburrahman / azure-cdn.json
Last active June 18, 2023 23:24
Website ARM templates
{
"id": "/subscriptions/d4aa0040-6651-443d-a078-728ce72a87ab/resourcegroups/rakirahman.me/providers/Microsoft.Cdn/profiles/rakirahman/endpoints/rakirahman",
"type": "Microsoft.Cdn/profiles/endpoints",
"name": "rakirahman",
"location": "Global",
"tags": {},
"properties": {
"hostName": "rakirahman.azureedge.net",
"originHostHeader": "rakirahman.z9.web.core.windows.net",
"originPath": null,
@mdrakiburrahman
mdrakiburrahman / delete-old-resources.sh
Created January 16, 2023 20:16
#!/bin/bash
# Continue on error
set +e
# Params
resource_group="arcdata-ci-direct"
days_to_keep=7
# Get current date and calculate 7 days ago in seconds
@mdrakiburrahman
mdrakiburrahman / calculate-kubernetes-eviction-signals.sh
Last active January 7, 2023 03:34
Calculates eviction signals used by kubelet to trigger eviction manager
# Calculate Eviction Signals - Current State vs Threshold:
#
# https://kubernetes.io/docs/concepts/scheduling-eviction/node-pressure-eviction/#eviction-signals
# https://medium.com/kubernetes-tutorials/efficient-node-out-of-resource-management-in-kubernetes-67f158da6e59
#
NODES=($(kubectl get nodes -o jsonpath='{.items[*].metadata.name}'))
NODE_COUNT=${#NODES[@]}
while true;
do
@mdrakiburrahman
mdrakiburrahman / Chaos-mesh-demo.sh
Last active December 5, 2022 15:27
Script to run to apply and clean up chaos Mesh workflow
##########################################################
# APPLY
##########################################################
# Apply workflow
#
kubectl apply -f sqlinstance-ha.yaml
# Get primary
#
kubectl get pods -l=role.ag.mssql.microsoft.com/chaos-bc-chaos-bc=primary -n ns1669830131581917320
@mdrakiburrahman
mdrakiburrahman / sqlinstance-ha.yaml.tmpl
Created December 4, 2022 21:37
Chaos Mesh Workflow Jinja template for YAML generation
# ==========================================================================
# SQLINSTANCE-HA
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
# WORKFLOW DURATION: 18m
# --------------------------------------------------------------------------
# CHAOS DESCRIPTION
# --------------------------------------------------------------------------
# In this chaos simulation we target a 2 or 3 replica MIAA:
#
# 1. Setup a User DB