Skip to content

Instantly share code, notes, and snippets.

View chrismay's full-sized avatar

Chris May chrismay

  • Warwickshire, UK
View GitHub Profile
@chrismay
chrismay / parallel_reqs.py
Last active June 18, 2023 21:32
pyspark parallel http reqs
import aiohttp
import asyncio
import time
from pyspark.sql import SparkSession
from pyspark.sql.functions import col, collect_list,udf
spark = SparkSession.builder.getOrCreate()
data = range(1, 10000)
df = spark.createDataFrame([(i,) for i in data], ["value"])
grouped_df = (
@chrismay
chrismay / tuples.ts
Created June 21, 2023 18:31
Preserve tuple length when mapping
interface ReadonlyArray<T> {
map<U>(
callbackfn: (value: T, index: number, array: T[]) => U,
thisArg?: any
): { [K in keyof this]: U };
}
type Coord = readonly [number, number, number];
type BiFunction<T> = (a: T, b: T) => T;