http://jerluc.com/files/hadoop-in-a-box.tgz
tar xf hadoop-in-a-box.tgz
Jeremy Lucas jerluc
😵💫
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| public class ExactMatchPOC { | |
| private static final String TOKEN_DELIMITER = " "; | |
| private static final String PHRASE_DELIMITER = ","; | |
| public static void main(String[] args) { | |
| String[] phraseTokens = buildTokenHashArray(args[0]); | |
| String[][] labelTokens = buildMultiTokenHashArray(args[1]); | |
| int i; |
jerluc
/ BrandRecommendation.scala
Created
July 27, 2012 19:21
Brand Recommendation Prototype Using Scalding
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import com.twitter.scalding._ | |
| import scala.util.matching.Regex | |
| class BrandRecommandation(args : Args) extends Job(args) { | |
| val brandData = Tsv(args("input"), ('userId, 'brandId, 'interactions)) | |
| val brandIncidence = brandData.groupBy('brandId) { _.size } rename { 'size -> 'brandIncidence } | |
| val brandDataWithIncidence = brandData.joinWithSmaller('brandId -> 'brandId, brandIncidence) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| hello { | |
| // Create our simple function | |
| sum(a, b) => a + b | |
| // Create a function to defer the computation | |
| deferredSum(a, b) => => sum(a, b) | |
| // Bind the evaluation of the computation in "myThread" to variable "a" | |
| a := deferredSum(10, 20) @ myThread | |
| // Print out the value once it's available | |
| print(a) | |
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import util.Random | |
| object Markov extends App { | |
| // Just some long plain-text sources from Project Gutenberg | |
| val sherlockHolmes = "http://www.gutenberg.org/cache/epub/1661/pg1661.txt" | |
| val kingJamesBible = "http://www.gutenberg.org/cache/epub/10/pg10.txt" |
jerluc
/ instructions.md
Last active
December 17, 2015 22:19
These instructions are for my session on rapidly developing software on the Hadoop technology stack through the use of commonly-found test utilities.
By now, you should have the following prerequisites installed and configured on your machine:
- Ubuntu 12.04
- Java (Oracle JDK 6; ensure
$JAVA_HOMEis set) - Git
- Apache Maven
After successful installation, you should be setup for running:
- Hadoop (local-mode + pseudo-distributed)
jerluc
/ minscalaactors.scala
Created
October 2, 2013 01:44
— forked from viktorklang/minscalaactors.scala
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| /* | |
| Copyright 2012 Viktor Klang | |
| Licensed under the Apache License, Version 2.0 (the "License"); | |
| you may not use this file except in compliance with the License. | |
| You may obtain a copy of the License at | |
| http://www.apache.org/licenses/LICENSE-2.0 | |
| Unless required by applicable law or agreed to in writing, software |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [32;01m * [39;49;00mPackage: net-im/pidgin-2.10.7-r5 | |
| [32;01m * [39;49;00mRepository: gentoo | |
| [32;01m * [39;49;00mMaintainer: [email protected] [email protected] | |
| [32;01m * [39;49;00mUSE: amd64 dbus elibc_glibc gstreamer gtk kernel_linux ncurses nls python_single_target_python2_7 python_targets_python2_7 userland_GNU xscreensaver | |
| [32;01m * [39;49;00mFEATURES: preserve-libs sandbox userpriv usersandbox | |
| [32;01m*[0m dbus is enabled, no way to disable linkage with python => python is enabled | |
| >>> Unpacking source... | |
| >>> Unpacking pidgin-2.10.7.tar.bz2 to /var/tmp/portage/net-im/pidgin-2.10.7-r5/work | |
| >>> Unpacking pidgin-eds-3.6.patch.bz2 to /var/tmp/portage/net-im/pidgin-2.10.7-r5/work | |
| >>> Source unpacked in /var/tmp/portage/net-im/pidgin-2.10.7-r5/work |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| # | |
| # Parses DHCP options from openvpn to update resolv.conf | |
| # To use set as 'up' and 'down' script in your openvpn *.conf: | |
| # up /etc/openvpn/update-resolv-conf | |
| # down /etc/openvpn/update-resolv-conf | |
| # | |
| # Used snippets of resolvconf script by Thomas Hood <[email protected]> | |
| # and Chris Hanson | |
| # Licensed under the GNU GPL. See /usr/share/common-licenses/GPL. |
OlderNewer