andreaskaempf

JavaScript makes it possible to run sophisticated quantitative models in the browser, but sometimes things need to run on the server. I'd like to describe an approach I've been using that transparently and fluidly bridges the best of both worlds, using an example from credit card marketing.

andreaskaempf

// Read all JSON files in the current directory
val d = spark.read.json("*.json")

// Do the SQL query to get all ICAO codes for the US
d.registerTempTable("crap")
val regs = sql("select distinct Icao from crap where Cou = 'United States'")

// Save to a CSV file
regs.write.csv("US_transmitting_ICAO_codes")