Skip to content

Instantly share code, notes, and snippets.

View berngp's full-sized avatar

Bernardo Gomez Palacio berngp

90 followers · 108 following
View GitHub Profile
@berngp
berngp / SparkBuild_1_hadoopVersion.scala
Last active December 19, 2015 05:39
Spark-Project Build with version qualifier. The gists bellow represent changes to one single **SparkBuild.scala** but to avoid a verbose explanation I am just including the pertinent parts that should be modified.
// Hadoop version to build against. For example, "0.20.2", "0.20.205.0", or
// "1.0.4" for Apache releases, or "0.20.2-cdh3u5" for Cloudera Hadoop.
//val HADOOP_VERSION = "1.0.4"
//val HADOOP_MAJOR_VERSION = "1"
//val HADOOP_YARN = false
// For Hadoop 2 versions such as "2.0.0-mr1-cdh4.2.2", set the HADOOP_MAJOR_VERSION to "2"
val HADOOP_VERSION = "2.0.0-mr1-cdh4.2.2"
val HADOOP_MAJOR_VERSION = "2"
val HADOOP_YARN = false
@berngp
berngp / databag.rb
Created May 21, 2013 17:55
Littlechef - Chef Solo Ruby Scripts
#!/usr/bin/env ruby
require 'main'
require 'colored'
Main {
program 'databag'
description <<-hdoc
Longer description of the funcitonality.
hdoc
author '[email protected]'
@berngp
berngp / NetPacketThriftGateway.scala
Created April 19, 2013 19:37
NetPacketThriftGateway.. example of using the Thrift Serialization Classes.
/*
* Copyright 2012-2013 Bernardo Gomez Palacio.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
@berngp
berngp / remoteTest.scala
Last active December 15, 2015 20:58
Scala Script That Loads into HDFS a Timeline of Net Packets as Thrift Objects Inside Sequence Files.
/*
* Copyright 2012-2013 Bernardo Gomez Palacio.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
@berngp
berngp / ThriftSerialization.scala
Last active December 15, 2015 18:39
HDFS Thrift Serialization in Scala
/*
* Copyright 2012-2013 Bernardo Gomez Palacio.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
@berngp
berngp / ipNumericValue.scala
Created April 2, 2013 16:44
IPv4 to Numeric Value
"192.168.1.1".split('.').reverse.map(_.toLong).zipWithIndex.map(t => (t._1 << (t._2 * 8L))).reduce(_ | _)
@berngp
berngp / HadoopSequenceFileWriter.scala
Last active December 15, 2015 10:59
Hadoop HDFS Sequence File Writer with Static Typing Validation.
/*
* Copyright 2012-2013 Bernardo Gomez Palacio.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
@berngp
berngp / stop-hadoop.bash
Created March 8, 2013 00:21
Stops all Hadoop processes running in the given node.
for x in `cd /etc/init.d ; ls hadoop-*` ; do sudo service $x stop ; done
@berngp
berngp / redis_ascii_art.txt
Created January 9, 2013 00:56
Awesome REDIS ASCII ART
_._
_.-``__ ''-._
_.-`` `. `_. ''-._ Redis 2.6.7 (00000000/0) 64 bit
.-`` .-```. ```\/ _.,_ ''-._
( ' , .-` | `, ) Running in stand alone mode
|`-._`-...-` __...-.``-._|'` _.-'| Port: 6379
| `-._ `._ / _.-' | PID: 55757
`-._ `-._ `-./ _.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' | http://redis.io
@berngp
berngp / A-Consecration.txt
Last active March 1, 2019 01:43
On Ethos
Louis Untermeyer, ed. (1885–1977). Modern British Poetry. 1920.
John Masefield. 1878–
97. A Consecration
NOT of the princes and prelates with periwigged charioteers
Riding triumphantly laurelled to lap the fat of the years,—
Rather the scorned—the rejected—the men hemmed in with the spears;