Skip to content

Instantly share code, notes, and snippets.

View maasg's full-sized avatar

Gerard Maas maasg

70 followers · 10 following
View GitHub Profile
@maasg
maasg / numberparser.snb.ipynb
Created December 17, 2016 14:01
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@maasg
maasg / StockQuotesTickerName.snb.ipynb
Created December 16, 2016 11:24
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@maasg
maasg / StreamPrinter.snb.ipynb
Last active December 14, 2016 08:05
Idiomatic try/catch in Scala
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@maasg
maasg / docker.json
Created December 5, 2016 12:42
Example 'Hello World' Chronos job - execution in a docker image
{
"schedule": "R/2014-09-25T17:22:00Z/PT2M",
"name": "dockerjob",
"container": {
"type": "DOCKER",
"image": "ubuntu:latest",
"network": "HOST",
"volumes": []
},
"cpus": "0.5",
@maasg
maasg / simple_echo.json
Created December 5, 2016 12:40
Example of chronos job - Using a shell command
{
"schedule": "R10/2016-12-05T10:00:00Z/PT1M",
"name": "simple_echo",
"epsilon": "PT15M",
"command": "echo 'hello world'",
"owner": "[email protected]",
"async": false
}
@maasg
maasg / ts-aggregate.snb.ipynb
Created November 26, 2016 11:25
Simple TimeSeries Aggregation using Spark
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@maasg
maasg / GeoGrid.snb.ipynb
Created November 25, 2016 11:47
Group a dataset of coordinates in cells of a given resolution, starting at the minimal coordinates from the set using a scalable Spark-based approach
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@maasg
maasg / CassandraWordCount.scala
Created October 24, 2014 18:59
Example code for a stateful stream processor using Spark Streaming and Cassandra
/**
-- Datamodel
-- local keyspace
CREATE KEYSPACE example
WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };
-- table schema
CREATE TABLE example.words (
word text PRIMARY KEY,
count int
@maasg
maasg / pom.xml
Created September 30, 2014 16:43
POM file to generate an uberjar for the Cassandra-Spark-Driver to use with the spark-shell
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.acme</groupId>
<artifactId>spark-cassandra-assembly</artifactId>
<version>1.0.0-SNAPSHOT</version>
<packaging>jar</packaging>
<name>A uber jar to easily use the Cassandra-Spark driver with spark-shell </name>
@maasg
maasg / CalliopeToCassandra.scala
Created June 16, 2014 16:22
Simple Calliope Save To Cassandra Test
def saveToCassandra(sc:SparkContext, total:Int): Long = {
val cas = CasBuilder.cql3.withColumnFamily("keyspace", "raw").onHost(host).onPort(port)
.saveWithQuery("UPDATE keyspace.raw set metric=?, vals=?")
implicit def TsToCasKeys(src:Array[String]): Map[String, ByteBuffer] = src match {
case Array(device_id, aggregation_type, t0, metric, vals) =>
Map("device_id" -> device_id, "aggregation_type" -> aggregation_type, "t0" -> t0)
}