Skip to content

Instantly share code, notes, and snippets.

@berngp
berngp / SparkUtils.scala
Created October 8, 2015 22:58 — forked from ibuenros/SparkUtils.scala
Spark productionizing utilities developed by Ooyala, shown in Spark Summit 2014
//==================================================================
// SPARK INSTRUMENTATION
//==================================================================
import com.codahale.metrics.{MetricRegistry, Meter, Gauge}
import org.apache.spark.{SparkEnv, Accumulator}
import org.apache.spark.metrics.source.Source
import org.joda.time.DateTime
import scala.collection.mutable
@berngp
berngp / example.sh
Last active August 29, 2015 14:01
ZSH Function Wrappers for JDK and SCALA HOME setup.
with-scala210 with-jdk7 ./make-distribution.sh --hadoop=2.4.0 --tgz --name=mesos-hadoop-2.4.0
@berngp
berngp / Fib.scala
Last active August 29, 2015 14:01
Fibonacci
def fib: Stream[Long] = {
def tail(h: Long, n: Long): Stream[Long] = h #:: tail(n, h + n)
tail(0, 1)
}
@berngp
berngp / post-reviews.py
Last active August 29, 2015 14:00
Mesos support/post-reviews.py with support for PRTool 0.5.x and 0.6.x
#!/usr/bin/env python
# This is a wrapper around the 'post-review'/'rbt' tool provided by
# Review Board. This is currently used by Apache Mesos development.
#
# What does this do?
# It provides the ability to send a review for each commit on the
# current branch.
#
# Why is that useful?
# No one likes a 5000 line review request. Using this tool forces one
@berngp
berngp / feature_MESOS-1259.out
Last active August 29, 2015 14:00
Mesos Development Support Scripts (./support)
feature/MESOS-1259* → ./support/post-reviews-2.py
Running post-review across all of ...
369c6fa83ce3e6929646b1304bce918dc37b8274 - (HEAD, berngp/feature/MESOS-1259, feature/MESOS-1259) [MESOS-1259]:Enrich the Java Docs in the src/java files. -- ZooKeeperState.java (2 minutes ago)
7edb638b38c832040f9cf0b35a4ea79d1db8e389 - [MESOS-1259]:Enrich the Java Docs in the src/java files. -- Variable.java (2 minutes ago)
d8868252011c4653c856ef37a257eafa3112b0e9 - [MESOS-1259]:Enrich the Java Docs in the src/java files. -- State.java (2 minutes ago)
c1b9e284ba1ed79c50cdd53f0f66cd52a221217c - [MESOS-1259]:Enrich the Java Docs in the src/java files. -- SchedulerDriver.java (2 minutes ago)
cb4d252ac5863e58e63352748733d40faaf50b1c - [MESOS-1259]:Enrich the Java Docs in the src/java files. -- Scheduler.java (2 minutes ago)
f634fcd394273751dc8ee260d5485df384fc052c - [MESOS-1259]:Enrich the Java Docs in the src/java files. -- MesosSchedulerDriver.java (2 minutes ago)
2ac673c718a96f307ed288577d24b7a036045013 - [MESOS-1259]:Enric
@berngp
berngp / git-patch-perfile.sh
Created April 30, 2014 22:51
Git Patch Per File
for x in $(find "src/java" -name "*.java"); do git format-patch master $x --stdout > "MESOS-1259-${x//\//-}" ; done
@berngp
berngp / dot.vimrc
Created April 24, 2014 21:58
Dot VIMRC
"
" Maintainer: Bernardo Gomez Palacio.
" To use it, copy it to
" for Unix and OS/2: ~/.vimrc
" for Amiga: s:.vimrc
" for MS-DOS and Win32: $VIM\_vimrc
" for OpenVMS: sys$login:.vimrc
" When started as "evim", evim.vim will already have done these settings.
if v:progname =~? "evim"
finish
@berngp
berngp / spark-env.sh
Created April 16, 2014 00:38
Spark Env Shell for YARN - Vagrant Hadoop 2.3.0 Cluster Pseudo distributed mode.
#!/usr/bin/env bash
# This file contains environment variables required to run Spark. Copy it as
# spark-env.sh and edit that to configure Spark for your site.
#
# The following variables can be set in this file:
# - SPARK_LOCAL_IP, to set the IP address Spark binds to on this node
# - MESOS_NATIVE_LIBRARY, to point to your libmesos.so if you use Mesos
# - SPARK_JAVA_OPTS, to set node-specific JVM options for Spark. Note that
# we recommend setting app-wide options in the application's driver program.
@berngp
berngp / diamond.scala
Last active August 29, 2015 13:58
Scala Scripts Exercises.
val width = 5
//width: Int = 5
val tTop = ( 0 until width/2 ) map ( n => 2 * n + 1 )
//tTop: scala.collection.immutable.IndexedSeq[Int] = Vector(1, 3)
val tBottom = tTop.reverse
//tBottom: scala.collection.immutable.IndexedSeq[Int] = Vector(3, 1)
val diamond = tTop union Seq(width) union tBottom