Working on open-source and internal projects

Scott Haines newfront

Working on open-source and internal projects

Distinguished Software Engineer @Nike-Inc. I specialize in writing massively distributed streaming systems on top of @apachespark . Love Dogs

newfront / simpleMemoryStreamExample.scala

Last active March 3, 2021 19:29

Simple Memory Stream

	test("Should aggregate call events") {
	implicit val sqlContext: SQLContext = spark.sqlContext
	import spark.implicits._

	val scenario = TestHelper.loadScenario[CallEvent](s"$pathToTestScenarios/pdd_events.json")
	val scenarioIter = scenario.toIterator
	scenario.nonEmpty shouldBe true

	val kafkaData = MemoryStream[MockKafkaDataFrame]
	val processingStream = EventAggregation(appConfig).process(kafkaData.toDF())(session)