Skip to content

Instantly share code, notes, and snippets.

@rghv404
rghv404 / concurrent_requests_benchmark.py
Last active July 25, 2024 01:24
Quick benchmark of serial vs async vs async_httpx vs threadpool vs processpool
import asyncio
import time
from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor
from contextlib import contextmanager
import httpx
import requests
URL = "https://invitationhomes.com"
@rghv404
rghv404 / .md
Last active January 26, 2022 18:13
Install/Uninstall Minikube from MAC

Final Uninstall

minikube stop; minikube delete &&
rm -rf ~/.kube ~/.minikube &&
sudo rm -rf /usr/local/bin/localkube /usr/local/bin/minikube &&
launchctl stop '*kubelet*.mount' &&
launchctl stop localkube.service &&
launchctl disable localkube.service &&
sudo rm -rf /etc/kubernetes/ &&
@rghv404
rghv404 / map_reduce.scala
Created April 20, 2021 18:58
Split dataset and process the parts before reducing to union
dfAggregated
.randomSplit(equallyWeightedParts(5))
.map((dataset: Dataset[Models.AggregatedDataSet]) => methodName(
allMeasurementsOnAssets = assetAggregatedMeasurements,
allMeasurementsOnScorecards = dataset,
broadcastFactorWeights = broadcastFactorWeights,
broadcastInterpolationFunctions = broadcastInterpolationFunctions,
spark = spark
))
.reduce(_ union _)
def deleteMuliS3Objects(bucket: String, prefix: String): Unit = {
import scala.collection.JavaConverters._
import com.amazonaws.AmazonServiceException
import com.amazonaws.SdkClientException
import com.amazonaws.services.s3.AmazonS3
import com.amazonaws.services.s3.AmazonS3ClientBuilder
import com.amazonaws.services.s3.model.DeleteObjectsRequest
import com.amazonaws.services.s3.model.DeleteObjectsRequest.KeyVersion
val s3Client: AmazonS3 = AmazonS3ClientBuilder.standard().build();
@rghv404
rghv404 / date_minus.scala
Last active January 13, 2021 18:49
Scala Data minus code snippet - works with Java but need syntax revamp
import java.time.format.DateTimeFormatter
import java.time.LocalDate
val effective_date = 20210101
val format = DateTimeFormatter.ofPattern("yyyyMMdd")
val effective_date_last_year = LocalDate.parse(effective_date, format).minusDays(365).format(format)