Skip to content

Instantly share code, notes, and snippets.

View GEOFBOT's full-sized avatar

Geoffrey Mon GEOFBOT

9 followers · 2 following
View GitHub Profile
@GEOFBOT
GEOFBOT / 10-color-emoji.conf
Last active August 29, 2020 05:15
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
<fontconfig>
<!--
Make Noto Color Emoji the initial fallback font for sans-serif, sans, and
monospace.
-->
<match>
<test name="family"><string>sans-serif</string></test>
@GEOFBOT
GEOFBOT / bobthefish-base16-monokai.fish
Last active October 29, 2020 18:08
Put this in ~/.config/fish/conf.d/
# Base16-Monokai
#
# color values from https://github.com/chriskempson/base16-vim/blob/037f328/colors/base16-monokai.vim
function bobthefish_colors -S -d 'Define a custom bobthefish color scheme'
__bobthefish_colors base16-dark
set -l base00 272822
set -l base01 383830
set -l base02 49483e
@GEOFBOT
GEOFBOT / Experiment.java
Created January 30, 2017 17:08
Multiple Flink jobs in one file
package com.github.geofbot;
import org.apache.commons.cli.*;
import org.apache.commons.collections.CollectionUtils;
import org.apache.flink.api.common.functions.*;
import org.apache.flink.api.common.operators.Order;
import org.apache.flink.api.java.DataSet;
import org.apache.flink.api.java.ExecutionEnvironment;
import org.apache.flink.api.java.aggregation.Aggregations;
import org.apache.flink.api.java.operators.IterativeDataSet;
@GEOFBOT
GEOFBOT / EMR_flinkerations.sh
Last active January 6, 2017 22:24
Bootstrap script to update Flink on Amazon EMR to use my build with Python bulk iterations
#!/bin/bash
# Runs after installation of included Flink
set -e
cd ~
sudo rm /usr/lib/flink/lib/flink-* # make sure we don't have two versions of jars
sudo tar -xzf flinkerations-emr.tgz -C /usr/lib/
rm flinkerations-emr.tgz
# Copy over EMRFS jars to Flink lib path
sudo cp /usr/share/aws/emr/s3-dist-cp/lib/*.jar /usr/lib/flink/lib/
@GEOFBOT
GEOFBOT / barebones.py
Last active October 18, 2016 02:05
Flink file that causes issues with a modified version of Flink featuring bulk iterations in the Python API
# Barebones test file to check for issues
import math
from flink.functions.Aggregation import Sum
from flink.functions.GroupReduceFunction import GroupReduceFunction
from flink.plan.Environment import get_environment
class NormalizeVectorGroupReducer(GroupReduceFunction):
"""
@GEOFBOT
GEOFBOT / flink.service
Created July 19, 2016 15:11
systemd file for flink
@GEOFBOT
GEOFBOT / HDFS on AWS.md
Last active August 7, 2016 16:17
Setting up HDFS on AWS

On each node:

Set up packages and install Hadoop:

#!/bin/bash

sudo yum install java-1.8.0-openjdk-devel wget git bzip2 -y
echo export JAVA_HOME=/usr/lib/jvm/java >> ~/.bashrc
source ~/.bashrc
@GEOFBOT
GEOFBOT / word_count.py
Created July 17, 2016 13:23
# modified from http://www.willmcginnis.com/2015/11/08/getting-started-with-python-and-apache-flink/
from flink.plan.Environment import get_environment
from flink.plan.Constants import INT, STRING, WriteMode
from flink.functions.GroupReduceFunction import GroupReduceFunction
class Adder(GroupReduceFunction):
def reduce(self, iterator, collector):
count, word = iterator.next()
count += sum([x[0] for x in iterator])
@GEOFBOT
GEOFBOT / DistCacheTest.java
Created July 17, 2016 13:16
package org.apache.flink;
import org.apache.commons.io.Charsets;
import org.apache.commons.io.FileUtils;
import org.apache.flink.api.common.functions.RichFlatMapFunction;
import org.apache.flink.api.common.functions.RichMapFunction;
import org.apache.flink.api.java.DataSet;
import org.apache.flink.api.java.ExecutionEnvironment;
import org.apache.flink.api.java.tuple.Tuple;
import org.apache.flink.api.java.tuple.Tuple1;
@GEOFBOT
GEOFBOT / Setting up a Flink cluster.md
Last active July 17, 2016 20:06
Guide to setting up a BlueData CentOS 6.7 / AWS Ubuntu 14.04 cluster for running Flink jobs