Skip to content

Instantly share code, notes, and snippets.

View HeartSaVioR's full-sized avatar
🏠
Working from home

Jungtaek Lim HeartSaVioR

🏠
Working from home
View GitHub Profile

New LocalOrShuffle (patch, 08038b6)

60,1,2,2,116300380,1938339,1,116300380,116300400,1938339,0.0,0.0 120,1,2,2,171035900,2850598,1,171035900,171035880,2850598,0.0,0.0 180,1,2,2,168730440,2812174,1,168730440,168730460,2812174,0.0,0.0 240,1,2,2,168933800,2815563,1,168933800,168933800,2815563,0.0,0.0 300,1,2,2,169707160,2828452,1,169707160,169707160,2828452,0.0,0.0

60,1,2,2,122351320,2039188,1,122351320,122351300,2039188,0.0,0.0 120,1,2,2,171169140,2852819,1,171169140,171169160,2852819,0.0,0.0

@HeartSaVioR
HeartSaVioR / storm-launch.yaml
Last active December 29, 2016 09:56
tmuxinator storm-launch current directory
# ~/.tmuxinator/storm-launch.yml
name: storm-launch
root: .
# Optional tmux socket
# socket_name: foo
# Runs before everything. Use it to start daemons etc.
pre:
@HeartSaVioR
HeartSaVioR / apache-fake-log-gen.py
Created October 8, 2016 23:04
kiritbasu/Fake-Apache-Log-Generator modified for storm-sql-example
#!/usr/bin/python
import time
import datetime
import pytz
import numpy
import random
import gzip
import zipfile
import sys
import argparse
@HeartSaVioR
HeartSaVioR / CustomMetricTopology.java
Created July 5, 2016 07:25
Full source code of CustomMetricTopology
package com.hortonworks.storm.test;
import org.apache.storm.StormSubmitter;
import org.apache.storm.generated.AlreadyAliveException;
import org.apache.storm.generated.AuthorizationException;
import org.apache.storm.generated.InvalidTopologyException;
import org.apache.storm.generated.StormTopology;
import org.apache.storm.metric.api.MeanReducer;
import org.apache.storm.metric.api.ReducedMetric;
import org.apache.storm.spout.SpoutOutputCollector;
@HeartSaVioR
HeartSaVioR / ambari-collect-api-applying-series-aggregator-v2.md
Created June 8, 2016 05:10
Ambari Collect API: Applying SeriesAggregator

http://sandbox.hortonworks.com:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=SUM(count.%25.execute-count)&hostname=sandbox.hortonworks.com&appId=metric-topology&startTime=1464793200&endTime=1464879599

{
   "metrics":[
      {
         "timestamp":0,
         "metricname":"SUM(count.10.execute-count,count.11.execute-count,count.12.execute-count,count.13.execute-count,count.14.execute-count,count.3.execute-count,count.4.execute-count,count.5.execute-count,count.6.execute-count,count.7.execute-count,count.8.execute-count,count.9.execute-count)",
         "appid":"metric-topology",
         "instanceid":"",
package org.apache.storm.starter;
import org.apache.storm.Config;
import org.apache.storm.LocalCluster;
import org.apache.storm.StormSubmitter;
import org.apache.storm.spout.SpoutOutputCollector;
import org.apache.storm.task.OutputCollector;
import org.apache.storm.task.TopologyContext;
import org.apache.storm.testing.TestWordSpout;
import org.apache.storm.topology.OutputFieldsDeclarer;
@HeartSaVioR
HeartSaVioR / SanityTest.java
Created April 19, 2016 02:45
SanityTest for PipelineBase with Mock and Reflection (POC)
package redis.clients.jedis.tests;
import org.apache.commons.lang3.ClassUtils;
import org.junit.Test;
import org.mockito.MockingDetails;
import org.mockito.invocation.Invocation;
import redis.clients.jedis.Client;
import redis.clients.jedis.Pipeline;
import uk.co.jemos.podam.api.PodamFactory;
import uk.co.jemos.podam.api.PodamFactoryImpl;
@HeartSaVioR
HeartSaVioR / topology-metrics.json
Created April 16, 2016 10:25
Apache Storm - Topology metrics detail preview
{
"window":":all-time",
"window-hint":"All time",
"components":{
"__acker":{
"emitted":{
"__metrics":40
},
"transferred":{
"__metrics":0
@HeartSaVioR
HeartSaVioR / BasicTopology.java
Last active March 26, 2016 07:00
"Speed of light topology Arun provided" + printing metrics
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0