Skip to content

Instantly share code, notes, and snippets.

@tjworks
tjworks / App.java
Created May 26, 2016 09:06
mongo-spark demo app.java
package com.example;
import java.util.HashMap;
import org.apache.spark.SparkConf;
import org.apache.spark.api.java.JavaSparkContext;
import org.apache.spark.api.java.JavaRDD;
import org.apache.spark.broadcast.Broadcast;
import org.bson.Document;
import com.mongodb.spark.api.java.MongoSpark;
public class App
@tjworks
tjworks / gist:31688d7e93f4d563ae1d60fc35a11b76
Last active September 28, 2022 10:36
Maven Shade to create spark worker/driver fatjars
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.example</groupId>
<artifactId>spark-demo</artifactId>
<packaging>jar</packaging>
<version>1.0-SNAPSHOT</version>
<name>spark-demo</name>
<url>http://maven.apache.org</url>
<dependencies>
@bradfitz
bradfitz / diskchecker.pl
Created July 24, 2012 21:05
diskchecker.pl
#!/usr/bin/perl
#
# Brad's el-ghetto do-our-storage-stacks-lie?-script
#
sub usage {
die <<'END';
Usage: diskchecker.pl -s <server[:port]> verify <file>
diskchecker.pl -s <server[:port]> create <file> <size_in_MB>
diskchecker.pl -l [port]