Skip to content

Instantly share code, notes, and snippets.

View timvw's full-sized avatar

Tim Van Wassenhove timvw

73 followers · 124 following
View GitHub Profile
@timvw
timvw / extractor.sc
Created January 15, 2016 21:14
Using extractor (objects) instead of case classes provides a flexible way to pattern match without having to change the object/hierarchy against one is matching..
import com.google.gson.Gson
val json1 = """{ 'type' : 'new', 'name' : 'tim' }"""
case class Message(
private val `type`: String,
private val name: String) {
def getType() = `type`
def getName() = name
}
@timvw
timvw / package-with-jar.pom
Created January 28, 2016 05:58
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>be.icteam.demo</groupId>
<artifactId>be.icteam.demo.core</artifactId>
<version>1.0-SNAPSHOT</version>
<name>${project.artifactId}</name>
<description>My wonderfull scala app</description>
<properties>
<maven.compiler.source>1.6</maven.compiler.source>
@timvw
timvw / maven-tools.sh
Created February 7, 2016 08:55
Display effective pom
mvn help:effective-pom
mvn dependency:copy-dependencies
@timvw
timvw / Usecase.scala
Created February 12, 2016 12:25
def run =
loadCustomerAndCustomerIteractionRecords _ andThen
determineCustomerIdsInCustomerButNotInCustomerInteraction andThen
interactWithMissingCustomers andThen
persistCustomerInteractions
@timvw
timvw / pom.xml
Created February 15, 2016 19:37
Build executable jar and jar with dependencies (single profile) with maven
<project>
...
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>${maven-jar-plugin.version}</version>
@timvw
timvw / ear.pom.xml
Created March 5, 2016 22:06
Build skinny war for ear
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>mavendemo</artifactId>
<groupId>be.icteam.mavendemo</groupId>
<version>1.1-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
@timvw
timvw / 0_LocalDateTimeIteratorSpecs.scala
Last active May 28, 2016 21:33
Generate LocalDateTimes between from and to with a given increment...
package be.icteam.demo
import org.joda.time.LocalDateTime
import org.scalatest.FunSuite
class LocalDateTimeIteratorSpecs extends FunSuite {
test("iterating from one to localdatetime should be possible") {
val from = new LocalDateTime(2016, 5, 29, 12, 0)
val to = new LocalDateTime(2016, 5, 29, 14, 0)
@timvw
timvw / Program.scala
Created June 21, 2016 11:55
Use HDFS to connect to Azure blob storage
package be.icteam.sandbox
import java.net.URI
import org.apache.hadoop.conf._
import org.apache.hadoop.fs._
object Program extends App {
override def main(args: Array[String]) = {
@timvw
timvw / RemoteIteratorWrapper.scala
Created July 17, 2016 19:24
scala wrapper for hadoop remote iterator
case class RemoteIteratorWrapper[T](underlying: org.apache.hadoop.fs.RemoteIterator[T]) extends scala.collection.AbstractIterator[T] with scala.collection.Iterator[T] {
def hasNext = underlying.hasNext
def next() = underlying.next()
}
object Conversions {
implicit def remoteIterator2ScalaIterator[T](underlying: org.apache.hadoop.fs.RemoteIterator[T]) : scala.collection.Iterator[T] = RemoteIteratorWrapper[T](underlying)
}
@timvw
timvw / Program.scala
Last active July 28, 2016 07:23
List files that are found by a spark path (potentially using ** and *)
package be.icteam.demo
import org.apache.spark._
object Program extends App {
override def main(args: Array[String]) = {
val sparkConf = new SparkConf().
setAppName("demo").