Drake Youngkun Min minyk
😀
minyk
/ gist:ca358749abf7c6d9dadc
Last active
August 29, 2015 14:19
Add RPM Packaging in Spark-Notebook
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Index: build.sbt | |
| IDEA additional info: | |
| Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP | |
| <+>UTF-8 | |
| =================================================================== | |
| --- build.sbt (revision cf04f98dfb0549476322ba20cf083181d8f19a63) | |
| +++ build.sbt (revision ) | |
| @@ -1,6 +1,7 @@ | |
| import Dependencies._ | |
minyk
/ SparkUtils.scala
Last active
September 16, 2015 12:36
— forked from ibuenros/SparkUtils.scala
Spark productionizing utilities developed by Ooyala, shown in Spark Summit 2014
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| //================================================================== | |
| // SPARK INSTRUMENTATION | |
| //================================================================== | |
| import com.codahale.metrics.{MetricRegistry, Meter, Gauge} | |
| import org.apache.spark.{SparkEnv, Accumulator} | |
| import org.apache.spark.metrics.source.Source | |
| import org.joda.time.DateTime | |
| import scala.collection.mutable |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #Cassandra | |
| CASSANDRA_VERSION=2.1.10 | |
| CASSANDRA_NAME=apache-cassandra-${CASSANDRA_VERSION}-bin | |
| CASSANDRA_ARCHIVE=${CASSANDRA_NAME}.tar.gz | |
| CASSANDRA_MIRROT_DOWNLOAD=http://www.apache.org/dist/cassandra/${CASSANDRA_VERSION}/${CASSANDRA_ARCHIVE} | |
| CASSANDRA_RES_DIR=/vagrant/resources/cassandra | |
| CASSANDRA_HOME=/usr/local/cassandra | |
| CASSANDRA_CONF=${CASSANDRA_HOME}/conf |
minyk
/ nifi-processor-onTrigger
Created
March 29, 2016 00:37
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| FlowFile flowFile = session.get(); | |
| # Do something with flowFile. | |
| flowFile = session.write(flowFile, new OutputStreamCallback() { | |
| @Override | |
| public void process(OutputStream out) throws IOException { | |
| } | |
| }); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "id": "/hue", | |
| "cmd": null, | |
| "cpus": 1, | |
| "mem": 512, | |
| "disk": 0, | |
| "instances": 1, | |
| "container": { | |
| "type": "DOCKER", | |
| "volumes": [], |
minyk
/ dcos-nifi.json
Created
June 16, 2016 07:33
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "id": "nifi", | |
| "cmd": null, | |
| "cpus": 1, | |
| "mem": 1024, | |
| "disk": 1000, | |
| "instances": 1, | |
| "container": { | |
| "docker": { |
minyk
/ dcos-elasticsearch-singlenode.json
Created
June 16, 2016 08:48
dcos-elasticsearch-singlenode
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "id": "/dashboard/elasticsearch", | |
| "cmd": null, | |
| "cpus": 1, | |
| "mem": 1024, | |
| "disk": 0, | |
| "instances": 1, | |
| "container": { | |
| "docker": { | |
| "image": "elasticsearch", |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "id": "/dashboard/kibana", | |
| "cmd": null, | |
| "cpus": 1, | |
| "mem": 512, | |
| "disk": 0, | |
| "instances": 1, | |
| "container": { | |
| "docker": { | |
| "image": "kibana", |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| var inputDF = sqlContext.read.json("/tmp/test1.json") | |
| val query = sqlContext.sql("SELECT age FROM test WHERE age > 30") | |
| val plan = query.queryExecution |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| sudo apt-get install xz-utils | |
| sudo apt-get install git | |
| sudo apt-get install build-essential | |
| sudo apt-get install libavahi-compat-libdnssd-dev |
OlderNewer