Azhaguselvan SP tamizhgeek

Data shenanigans @deliveryhero

tamizhgeek / Dockefile

Created January 10, 2020 14:40

GoCD agent with EMR goodies to connect to EMR and execute spark jobs

	ARG GOCD_VERSION=19.5.0

	FROM gocd/gocd-agent-centos-7:v$GOCD_VERSION

	# install java and friends

	RUN yum -y update && \
	yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm && \
	yum install -y java-1.8.0-openjdk-devel xmlstarlet which

tamizhgeek / spark-yarn-emr-client.rb

Created February 10, 2018 12:55

Chef Recipe for remote spark-submit setup to YARN running on Amazon EMR

	# setup for the hadoop + spark env for airflow

	remote_file "/home/airflow/spark.tgz" do
	source "/remote/spark/download/url"
	owner "airflow"
	group "airflow"
	mode '0755'
	not_if { File.exists?("/home/airflow/spark.tgz") }
	end

tamizhgeek / docker-compose.yml

Created February 25, 2016 15:39 — forked from ashwanthkumar/docker-compose.yml

Docker compose setup to bring up a local marathon setup for testing / development

	version: '2'
	services:
	zk:
	image: jplock/zookeeper
	network_mode: "host"

	master:
	image: mesosphere/mesos-master:0.27.0-0.2.190.ubuntu1404
	network_mode: "host"
	privileged: true

tamizhgeek / gist:bb876c7267d668310faa

Created August 1, 2015 09:53

	curl -H 'x-auth-token: AUTH_tk8830063c201c4bf7bd60b1dcce7b0f04' -H 'accept: application/json' http://127.0.0.1:8080/v1/AUTH_test/test/monkies -XPUT -v
	* Hostname was NOT found in DNS cache
	* Trying 127.0.0.1...
	* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
	> PUT /v1/AUTH_test/test/monkies HTTP/1.1
	> User-Agent: curl/7.35.0
	> Host: 127.0.0.1:8080
	> x-auth-token: AUTH_tk8830063c201c4bf7bd60b1dcce7b0f04
	> accept: application/json
	>

tamizhgeek / gist:8932641245299cab9b1c

Created August 1, 2015 08:10

	tamizhgeek@pettai:/var/log/swift$ curl -v -X DELETE -H 'X-Auth-Token: AUTH_tk8830063c201c4bf7bd60b1dcce7b0f04' -H "Accept: application/json" http://127.0.0.1:8080/v1/AUTH_test/dummy/nonexistent
	* Hostname was NOT found in DNS cache
	* Trying 127.0.0.1...
	* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
	> DELETE /v1/AUTH_test/dummy/nonexistent HTTP/1.1
	> User-Agent: curl/7.35.0
	> Host: 127.0.0.1:8080
	> X-Auth-Token: AUTH_tk8830063c201c4bf7bd60b1dcce7b0f04
	> Accept: application/json
	>

tamizhgeek / gist:d509dbff10c9dbb52848

Created August 1, 2015 06:43

	tamizhgeek@pettai:~/swift$ curl -X DELETE -H 'X-Auth-Token: AUTH_tk8830063c201c4bf7bd60b1dcce7b0f04' http://127.0.0.1:8080/v1/AUTH_test/dummy/notexisting
	tamizhgeek@pettai:~/swift$ <html><h1>Not Found</h1><p>The resource could not be found.</p></html>

	--------------------

	tamizhgeek@pettai:~/swift$ curl -X DELETE -H 'X-Auth-Token: AUTH_tk8830063c201c4bf7bd60b1dcce7b0f04' -H "Accept: application/json" http://127.0.0.1:8080/v1/AUTH_test/dummy/nonexistent
	tamizhgeek@pettai:~/swift$ {"status": 404, "description": "The resource could not be found.", "title": "Not Found"}

tamizhgeek / NumberToWordsTest.scala

Created November 17, 2014 14:51

	import assignments.NumberToWords
	import org.scalatest.{ShouldMatchers, FlatSpec}

	class NumberToWordsTest extends FlatSpec with ShouldMatchers{
	it should "convert numbers to words in required format" in {
	val number = new NumberToWords(104536)
	number.numForWords shouldBe "one-zero-four-five-three-six"
	}

	}

tamizhgeek / NumberToWords.scala

Created November 17, 2014 14:51

Number to words problem solution. Tests here : https://gist.github.com/tamizhgeek/06f923330b130d701d09

	package assignments


	class NumberToWords(number : Int) {
	val digitToWord = Map(
	0 -> "zero",
	1 -> "one",
	2 -> "two",
	3 -> "three",
	4 -> "four",

tamizhgeek / LatticePathFinderTest.scala

Created November 17, 2014 14:51

	import assignments.Lattice
	import org.scalatest.{FlatSpec, ShouldMatchers}


	class LatticePathFinderTest extends FlatSpec with ShouldMatchers{

	it should "find the lattice path correctly using memoization" in {
	new Lattice(2).findNoOfPathsTrivialWay shouldBe 6

	new Lattice(4).findNoOfPathsTrivialWay shouldBe 56

tamizhgeek / LatticePathFinder.scala

Last active August 29, 2015 14:09

Lattice path finder using pascal's triange. Tests here : https://gist.github.com/tamizhgeek/827ffaaf3e6856ec6bad

	package assignments

	// m! / n!(n - m)!
	// m = row, n = element

	class Lattice(size : Int) {

	val cache : scala.collection.mutable.Map[(Int, Int), BigInt] = scala.collection.mutable.Map.empty

	/*