Skip to content

Instantly share code, notes, and snippets.

@stuzero
stuzero / snowflake.rb
Created February 17, 2023 01:52
Snowflake SQL API ruby client
class Snowflake
require 'digest'
require 'jwt'
require 'securerandom'
class Client
attr_reader :authorization, :expiration
attr_accessor :account, :region, :user, :private_key_path
def initialize(attributes={})

Prepare the Server

$ sudo apt install build-essential unzip

Install PROJ 7.1

$ cd /usr/local/src
$ sudo wget https://download.osgeo.org/proj/proj-7.1.1.tar.gz
$ sudo tar -xzvf proj-7.1.1.tar.gz
$ sudo rm proj-7.1.1.tar.gz
$ cd proj-7.1.1
@stuzero
stuzero / spatial_covid.scala
Last active September 2, 2021 02:31
US County Shapefile loaded in Apache Sedona (GeoSpark)
// ./spark-shell \
// --packages org.apache.sedona:sedona-core-3.0_2.12:1.0.1-incubating,org.apache.sedona:sedona-sql-3.0_2.12:1.0.1-incubating,org.locationtech.jts:jts-core:1.18.2,org.datasyslab:geotools-wrapper:geotools-24.1 \
// --conf spark.serializer=org.apache.spark.serializer.KryoSerializer \
// --conf spark.kryo.registrator=org.apache.sedona.core.serde.SedonaKryoRegistrator
import org.apache.sedona.core.formatMapper.shapefileParser.ShapefileReader
import org.apache.sedona.core.spatialRDD.SpatialRDD
import org.apache.sedona.sql.utils.{Adapter, SedonaSQLRegistrator}
import org.locationtech.jts.geom.{Coordinate, Geometry, GeometryFactory}
@stuzero
stuzero / melt_merge_covid.scala
Last active September 2, 2021 16:12
Melt and merge the daily US County Level Covid-19 Data from USA Facts https://usafacts.org/visualizations/coronavirus-covid-19-spread-map/
// Apache Spark version 3.1.2
// Using Scala version 2.12.10 (OpenJDK 64-Bit Server VM, Java 1.8.0_292)
val casesUrl = "https://usafactsstatic.blob.core.windows.net/public/data/covid-19/covid_confirmed_usafacts.csv"
val deathsUrl = "https://usafactsstatic.blob.core.windows.net/public/data/covid-19/covid_deaths_usafacts.csv"
def getData (dataUrl:String) : org.apache.spark.sql.DataFrame = {
var responseList = scala.io.Source.fromURL(dataUrl).mkString.stripMargin.linesIterator.toList
var csvData = spark.sparkContext.parallelize(responseList).toDS()