Skip to content

Instantly share code, notes, and snippets.

View andelink's full-sized avatar
😶‍🌫️

Kyle Andelin andelink

😶‍🌫️
View GitHub Profile
@andelink
andelink / script.py
Last active November 5, 2021 17:57
Haversine Distance Spark SQL
"""
https://gist.github.com/pavlov99/bd265be244f8a84e291e96c5656ceb5c
"""
from pyspark.sql import SparkSession
from pyspark.sql import functions as F
CITIES = [
('HKG', 'Hong Kong', 22.308919, 113.914603),
('SYD', 'Sydney', -33.946111, 151.177222),
('YYZ', 'Toronto', 43.677223, -79.630556),