Aris ariskk

ariskk / setupES21

Last active March 25, 2016 12:55

Setting up an Elasticsearch 2.x cluster on EC2

	Building an ES image on AWS:

	Provision an amazon-linux instance through the console with a 8gb root drive and a 80+gb ebs data drive

	A) update
	sudo yum update

	B) mount a data volume
	lsblk # find volume tag assuming it is /dev/xvdb
	#check if data

ariskk / emr_bootstrap_java_8.sh

Created April 22, 2016 11:38 — forked from pstorch/emr_bootstrap_java_8.sh

Bootstrap script for installing Java 8 on an Amazon Elastic MapReduce instance (emr-4.0.0)

	# Check java version
	JAVA_VER=$(java -version 2>&1 \| sed 's/java version "$.$\.$.$\..*"/\1\2/; 1q')

	if [ "$JAVA_VER" -lt 18 ]
	then
	# Figure out how many versions of Java and javac we currently have
	NR_OF_JRE_OPTIONS=$(echo 0 \| alternatives --config java 2>/dev/null \| grep 'There ' \| awk '{print $3}' \| tail -1)
	NR_OF_SDK_OPTIONS=$(echo 0 \| alternatives --config javac 2>/dev/null \| grep 'There ' \| awk '{print $3}' \| tail -1)

	# Silent install javac (includes jre)

ariskk / typeinfo.scala

Created January 6, 2019 20:59

implicit def createTypeInformation[T]: TypeInformation[T] = macro TypeUtils.createTypeInfo[T]

ariskk / typeinfo2.scala

Created January 8, 2019 08:50

implicit val typeInfo = TypeInformation.of(classOf[Foo])

ariskk / fooCompanion.scala

Created January 9, 2019 08:41

	case class Foo(b: Bar)

	object Foo {
	implicit val typeInfo = TypeInformation.of(classOf[Foo])
	}

ariskk / typeInfoTrait.scala

Created January 9, 2019 08:51

	trait TypeInfo {
	implicit val fooTypeInfo = TypeInformation.of(classOf[Foo])
	implicit val barTypeInfo = TypeInformation.of(classOf[Bar])
	implicit val bazTypeInfo = TypeInformation.of(classOf[Baz])
	}

	object TypeInfo extends TypeInfo

ariskk / connected.scala

Created February 3, 2019 20:06

	import org.apache.flink.api.common.state.ValueStateDescriptor
	import org.apache.flink.api.common.typeinfo.TypeInformation
	import org.apache.flink.api.common.typeutils.TypeSerializer
	import org.apache.flink.streaming.api.functions.co.{CoMapFunction, RichCoMapFunction}
	import org.apache.flink.streaming.api.scala.{ConnectedStreams, DataStream}
	import org.apache.flink.streaming.api.scala._

	case class M1(key: String, foo: String)
	case class M2(key: String, bar: String)