maasg’s gists

maasg / ts-aggregate.snb.ipynb

Created November 26, 2016 11:25

Simple TimeSeries Aggregation using Spark

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

maasg / simple_echo.json

Created December 5, 2016 12:40

Example of chronos job - Using a shell command

	{
	"schedule": "R10/2016-12-05T10:00:00Z/PT1M",
	"name": "simple_echo",
	"epsilon": "PT15M",
	"command": "echo 'hello world'",
	"owner": "[email protected]",
	"async": false
	}

maasg / docker.json

Created December 5, 2016 12:42

Example 'Hello World' Chronos job - execution in a docker image

	{
	"schedule": "R/2014-09-25T17:22:00Z/PT2M",
	"name": "dockerjob",
	"container": {
	"type": "DOCKER",
	"image": "ubuntu:latest",
	"network": "HOST",
	"volumes": []
	},
	"cpus": "0.5",

maasg / StreamPrinter.snb.ipynb

Last active December 14, 2016 08:05

Idiomatic try/catch in Scala

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

maasg / StockQuotesTickerName.snb.ipynb

Created December 16, 2016 11:24

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

maasg / numberparser.snb.ipynb

Created December 17, 2016 14:01

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

maasg / so-data-exploration.snb.ipynb

Last active December 19, 2016 20:15

Data Exploration.

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

maasg / unique_match_count.ipynb

Created January 20, 2017 12:41

Calculate the count of unique matching elements between two dataframes

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

maasg / streaming-files.md

Created May 30, 2017 07:38

Spark Streaming job that uses a file stream

import org.apache.spark.mllib.linalg.Vectors
import org.apache.spark.mllib.regression.StreamingLinearRegressionWithSGD
import org.apache.spark.sql._
import org.apache.spark.sql.types.{StringType, StructField, StructType}
import org.apache.spark.streaming.{Seconds, StreamingContext}
import org.apache.spark.{SparkConf, SparkContext}
import scala.util.Try

maasg / StreamingFileTest.scala

Created May 30, 2017 16:02

Minimalistic SparkStreaming-FileStream project

	import org.apache.spark.mllib.linalg.Vectors
	import org.apache.spark.mllib.regression.StreamingLinearRegressionWithSGD
	import org.apache.spark.sql._
	import org.apache.spark.sql.types.{StringType, StructField, StructType}
	import org.apache.spark.streaming.{Seconds, StreamingContext}
	import org.apache.spark.{SparkConf, SparkContext}
	import scala.util.Try
	import java.io.File

Gerard Maas maasg