Skip to content

Instantly share code, notes, and snippets.

View rmetzger's full-sized avatar

Robert Metzger rmetzger

296 followers · 72 following
View GitHub Profile
@rmetzger
rmetzger / gist:f145f7308c5f22dcd8a5cd98d3fc1f58
Last active April 24, 2019 13:46
Flink d4a94581b40e96517c8049588b62585dea55d816 web dashboard notice file
Components:
@angular/animations 7.2.10 : MIT License
@angular/cdk 7.3.6 : MIT License
@angular/common 7.2.10 : MIT License
@angular/compiler 7.2.10 : MIT License
@angular/core 7.2.10 : MIT License
@angular/forms 7.2.10 : MIT License
@angular/platform-browser 7.2.10 : MIT License
@angular/platform-browser-dynamic 7.2.10 : MIT License
@rmetzger
rmetzger / gist:b48a6521c645fd022b86b928fc54a6ba
Created March 1, 2019 13:11
Flink build order
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] force-shading [jar]
[INFO] flink [pom]
[INFO] flink-annotations [jar]
[INFO] flink-shaded-hadoop [pom]
[INFO] flink-shaded-hadoop2 [jar]
[INFO] flink-shaded-hadoop2-uber [jar]
[INFO] flink-shaded-yarn-tests [jar]
@rmetzger
rmetzger / gist:f394b124c822baa1bab4a611239899b9
Last active April 18, 2019 07:37
Bootstrap Ubuntu 18.04 for building Flink (Google Cloud)
sudo -s
apt update
apt install maven openjdk-8-jdk git
apt remove openjdk-11-jre-headless
exit
git clone https://github.com/apache/flink.git
cd flink
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64/
mvn clean install
@rmetzger
rmetzger / gist:db6a88137047323a0d140fefbe726d79
Last active February 20, 2019 15:20
Build Flink on Docker
$ docker run -it ubuntu /bin/bash
% apt-get update
% apt-get install maven openjdk-8-jdk git
% git clone https://github.com/apache/flink.git
% cd flink
% mvn clean package -DskipTests
@rmetzger
rmetzger / keybase.md
Created May 9, 2018 10:08

Keybase proof

I hereby claim:

  • I am rmetzger on github.
  • I am rmetzger (https://keybase.io/rmetzger) on keybase.
  • I have a public key whose fingerprint is 7DAB 190F 0C67 8B62 D8C7 85BC AD3A 4CED 0F4D 0307

To claim this, I am signing this object:

@rmetzger
rmetzger / findGuavaJar.sh
Created December 10, 2015 10:44
Find Guava
#!/bin/sh
for file in `find . | grep "\.jar" `; do
echo "file $file"
COUNT=`jar tf $file | grep '^com/google/common' | wc -l`
if [ $COUNT != "0" ]; then
echo "FOUND IN FILE $file"
fi
done
@rmetzger
rmetzger / try1.md
Created November 21, 2015 16:48
Flink Stable Interface annotation

Classes Annotated with @PublicInterface

In flink-core

Class: org.apache.flink.types.LongValue

  • copyNormalizedKey()
  • getValue()
  • copyTo()
@rmetzger
rmetzger / gist:bcb57a01aa15d9a64cc3
Created August 30, 2015 12:17
Run Apache Flink on a Kerberos secured YARN cluster on CDH 5.3.0
[cloudera@quickstart build-target]$ klist
Ticket cache: FILE:/tmp/krb5cc_501
Default principal: hdfs@CLOUDERA
Valid starting Expires Service principal
08/30/15 05:15:22 08/31/15 05:15:22 krbtgt/CLOUDERA@CLOUDERA
renew until 09/06/15 05:15:22
[cloudera@quickstart build-target]$ export HADOOP_CONF_DIR=/usr/lib/hadoop/etc/hadoop/
[cloudera@quickstart build-target]$ ./bin/yarn-session.sh -n 1
05:15:37,483 INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at quickstart.cloudera/127.0.0.1:8032
@rmetzger
rmetzger / gist:f92d7cac32221c0db553
Created June 1, 2015 15:02
Flink Table API
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
@rmetzger
rmetzger / gist:ddbb0fead5efdd58a539
Created May 19, 2015 14:11
Kafka Producer fails (no leader?)
Travis build: https://travis-ci.org/StephanEwen/incubator-flink/jobs/63147661
10:38:15,603 INFO org.apache.flink.streaming.connectors.kafka.KafkaITCase - Starting KafkaITCase.prepare()
10:38:15,611 INFO org.apache.flink.streaming.connectors.kafka.KafkaITCase - Starting Zookeeper
10:38:15,816 INFO org.apache.zookeeper.server.ZooKeeperServerMain - Starting server
10:38:15,828 INFO org.apache.zookeeper.server.ZooKeeperServer - Server environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
10:38:15,829 INFO org.apache.zookeeper.server.ZooKeeperServer - Server environment:host.name=testing-worker-linux-docker-e8c0cfc7-5418-linux-2.prod.travis-ci.org
10:38:15,829 INFO org.apache.zookeeper.server.ZooKeeperServer - Server environment:java.version=1.7.0_76
10:38:15,829 INFO org.apache.zookeeper.server.ZooKeeperServer - Server environment:java.vendor=Oracle Corporation
10:38:15,829 INFO org.apac