Robert Metzger rmetzger
rmetzger
/ Janino test
Created
October 4, 2014 14:59
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| /** | |
| * Licensed to the Apache Software Foundation (ASF) under one | |
| * or more contributor license agreements. See the NOTICE file | |
| * distributed with this work for additional information | |
| * regarding copyright ownership. The ASF licenses this file | |
| * to you under the Apache License, Version 2.0 (the | |
| * "License"); you may not use this file except in compliance | |
| * with the License. You may obtain a copy of the License at | |
| * | |
| * http://www.apache.org/licenses/LICENSE-2.0 |
rmetzger
/ gist:3f8e472e06756447eb92
Created
December 11, 2014 13:02
Run Apache Flink on Hortonworks HDP 2.2
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Prepare HDFS | |
| whoami # in my case: robert | |
| sudo su hdfs | |
| hadoop fs -mkdir /user/robert | |
| hadoop fs -chmod 777 /user/robert | |
| export HADOOP_CONF_PATH=/etc/hadoop/conf/ | |
| wget http://apache.lauf-forum.at/incubator/flink/flink-0.7.0-incubating/flink-0.7.0-incubating-bin-hadoop2-yarn.tgz | |
| tar xf flink-0.7.0-incubating-bin-hadoop2-yarn.tgz | |
| cd flink-yarn-0.7.0-incubating/ |
rmetzger
/ gist:600bea0361736c235eb1
Created
December 19, 2014 14:20
If you have this error, read the comment below ....
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 14:16:45,207 DEBUG org.apache.hadoop.service.AbstractService - Service: org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl entered state INITED | |
| 14:16:45,215 WARN org.apache.hadoop.security.token.Token - Cannot find class for token kind YARN_AM_RM_TOKEN | |
| 14:16:45,215 DEBUG org.apache.hadoop.security.SecurityUtil - Acquired token Kind: YARN_AM_RM_TOKEN, Service: 127.0.0.1:8030, Ident: 00 00 01 4a 62 b1 45 25 00 00 00 01 00 00 00 01 | |
| 14:16:45,216 DEBUG org.apache.hadoop.security.UserGroupInformation - PrivilegedAction as:robert (auth:SIMPLE) from:org.apache.hadoop.yarn.client.RMProxy.getProxy(RMProxy.java:63) | |
| 14:16:45,217 DEBUG org.apache.hadoop.yarn.ipc.YarnRPC - Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC | |
| 14:16:45,217 DEBUG org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC - Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.yarn.api.Applicatio |
rmetzger
/ websitetop.patch
Created
January 11, 2015 19:54
Because svn doesn't support local branches, I have to put my wip on flink.apache.org here
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Index: index.html | |
| =================================================================== | |
| --- index.html (revision 1650799) | |
| +++ index.html (working copy) | |
| @@ -13,6 +13,79 @@ | |
| <link href="css/main/bootstrap-theme.min.css" rel="stylesheet"> | |
| <link href="css/main/main.css" rel="stylesheet"> | |
| <link rel="stylesheet" href="css/syntax.css"> | |
| + <script src="//ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js" ></script> | |
| + <script src="js/main/jquery.mobile.events.min.js"></script> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| public static void main(String[] args) throws Exception { | |
| // set up the execution environment | |
| final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment(); | |
| DataSet<Tuple2<Integer, String>> one = env.fromElements(new Tuple2<Integer, String>(1, "one"), | |
| new Tuple2<Integer, String>(2, "two"), new Tuple2<Integer, String>(3, "three") | |
| ); | |
| DataSet<Tuple1<Integer>> two = env.fromElements(new Tuple1<Integer>(1)); | |
| DataSet<Integer> three = one.coGroup(two).where(0).equalTo(0).with(new CoGroupFunction<Tuple2<Integer, String>, Tuple1<Integer>, Integer>() { | |
| @Override |
rmetzger
/ gist:a218beca4b0442f3c1f3
Last active
August 29, 2015 14:18
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| diff --git a/flink-staging/flink-hbase/src/test/java/org/apache/flink/addons/hbase/example/HBaseWriteExample.java b/flink-staging/flink-hbase/src/test/java/org/apache/flink/addons/hbase/example/HBaseWriteExample.java | |
| index 1f39694..cadb938 100644 | |
| --- a/flink-staging/flink-hbase/src/test/java/org/apache/flink/addons/hbase/example/HBaseWriteExample.java | |
| +++ b/flink-staging/flink-hbase/src/test/java/org/apache/flink/addons/hbase/example/HBaseWriteExample.java | |
| @@ -20,10 +20,12 @@ package org.apache.flink.addons.hbase.example; | |
| import org.apache.flink.api.common.functions.FlatMapFunction; | |
| import org.apache.flink.api.common.functions.MapFunction; | |
| +import org.apache.flink.api.common.functions.RichMapFunction; | |
| import org.apache.flink.api.java.DataSet; |
rmetzger
/ gist:674dd277c66f5c1486a6
Created
April 16, 2015 07:54
Test Apache Kafka. I'm planning to merge this to Flink at a later point
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| @Test | |
| public void testKafkaWithoutFlink() { | |
| // start consumer: | |
| new Thread(new Runnable() { | |
| @Override | |
| public void run() { | |
| LOG.info("Starting consumer"); | |
| // consume from "testtopic" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| for commit in $(git rev-list master) | |
| do | |
| git checkout $commit | |
| DATE=`git log | sed '3q;d'` | |
| AUTHORS=`git log --format='%aN' | sort -u | wc -l` | |
| echo "$DATE,$AUTHORS" >> graph.csv | |
| done |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Travis build: https://travis-ci.org/StephanEwen/incubator-flink/jobs/63147661 | |
| 10:38:15,603 INFO org.apache.flink.streaming.connectors.kafka.KafkaITCase - Starting KafkaITCase.prepare() | |
| 10:38:15,611 INFO org.apache.flink.streaming.connectors.kafka.KafkaITCase - Starting Zookeeper | |
| 10:38:15,816 INFO org.apache.zookeeper.server.ZooKeeperServerMain - Starting server | |
| 10:38:15,828 INFO org.apache.zookeeper.server.ZooKeeperServer - Server environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT | |
| 10:38:15,829 INFO org.apache.zookeeper.server.ZooKeeperServer - Server environment:host.name=testing-worker-linux-docker-e8c0cfc7-5418-linux-2.prod.travis-ci.org | |
| 10:38:15,829 INFO org.apache.zookeeper.server.ZooKeeperServer - Server environment:java.version=1.7.0_76 | |
| 10:38:15,829 INFO org.apache.zookeeper.server.ZooKeeperServer - Server environment:java.vendor=Oracle Corporation | |
| 10:38:15,829 INFO org.apac |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| /* | |
| * Licensed to the Apache Software Foundation (ASF) under one | |
| * or more contributor license agreements. See the NOTICE file | |
| * distributed with this work for additional information | |
| * regarding copyright ownership. The ASF licenses this file | |
| * to you under the Apache License, Version 2.0 (the | |
| * "License"); you may not use this file except in compliance | |
| * with the License. You may obtain a copy of the License at | |
| * | |
| * http://www.apache.org/licenses/LICENSE-2.0 |