Skip to content

Instantly share code, notes, and snippets.

View HeartSaVioR's full-sized avatar
🏠
Working from home

Jungtaek Lim HeartSaVioR

🏠
Working from home
View GitHub Profile
@HeartSaVioR
HeartSaVioR / topology-metrics.json
Created April 16, 2016 10:25
Apache Storm - Topology metrics detail preview
{
"window":":all-time",
"window-hint":"All time",
"components":{
"__acker":{
"emitted":{
"__metrics":40
},
"transferred":{
"__metrics":0
@HeartSaVioR
HeartSaVioR / SanityTest.java
Created April 19, 2016 02:45
SanityTest for PipelineBase with Mock and Reflection (POC)
package redis.clients.jedis.tests;
import org.apache.commons.lang3.ClassUtils;
import org.junit.Test;
import org.mockito.MockingDetails;
import org.mockito.invocation.Invocation;
import redis.clients.jedis.Client;
import redis.clients.jedis.Pipeline;
import uk.co.jemos.podam.api.PodamFactory;
import uk.co.jemos.podam.api.PodamFactoryImpl;
package org.apache.storm.starter;
import org.apache.storm.Config;
import org.apache.storm.LocalCluster;
import org.apache.storm.StormSubmitter;
import org.apache.storm.spout.SpoutOutputCollector;
import org.apache.storm.task.OutputCollector;
import org.apache.storm.task.TopologyContext;
import org.apache.storm.testing.TestWordSpout;
import org.apache.storm.topology.OutputFieldsDeclarer;

http://sandbox.hortonworks.com:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=SUM(count.%25.execute-count)&hostname=sandbox.hortonworks.com&appId=metric-topology&startTime=1464793200&endTime=1464879599

{
   "metrics":[
      {
         "timestamp":0,
         "metricname":"SUM(count.10.execute-count,count.11.execute-count,count.12.execute-count,count.13.execute-count,count.14.execute-count,count.3.execute-count,count.4.execute-count,count.5.execute-count,count.6.execute-count,count.7.execute-count,count.8.execute-count,count.9.execute-count)",
         "appid":"metric-topology",
         "instanceid":"",
@HeartSaVioR
HeartSaVioR / ambari-collect-api-applying-series-aggregator-v2.md
Created June 8, 2016 05:10
Ambari Collect API: Applying SeriesAggregator
@HeartSaVioR
HeartSaVioR / CustomMetricTopology.java
Created July 5, 2016 07:25
Full source code of CustomMetricTopology
package com.hortonworks.storm.test;
import org.apache.storm.StormSubmitter;
import org.apache.storm.generated.AlreadyAliveException;
import org.apache.storm.generated.AuthorizationException;
import org.apache.storm.generated.InvalidTopologyException;
import org.apache.storm.generated.StormTopology;
import org.apache.storm.metric.api.MeanReducer;
import org.apache.storm.metric.api.ReducedMetric;
import org.apache.storm.spout.SpoutOutputCollector;
@HeartSaVioR
HeartSaVioR / apache-fake-log-gen.py
Created October 8, 2016 23:04
kiritbasu/Fake-Apache-Log-Generator modified for storm-sql-example
#!/usr/bin/python
import time
import datetime
import pytz
import numpy
import random
import gzip
import zipfile
import sys
import argparse
@HeartSaVioR
HeartSaVioR / storm-launch.yaml
Last active December 29, 2016 09:56
tmuxinator storm-launch current directory
# ~/.tmuxinator/storm-launch.yml
name: storm-launch
root: .
# Optional tmux socket
# socket_name: foo
# Runs before everything. Use it to start daemons etc.
pre:

New LocalOrShuffle (patch, 08038b6)

60,1,2,2,116300380,1938339,1,116300380,116300400,1938339,0.0,0.0 120,1,2,2,171035900,2850598,1,171035900,171035880,2850598,0.0,0.0 180,1,2,2,168730440,2812174,1,168730440,168730460,2812174,0.0,0.0 240,1,2,2,168933800,2815563,1,168933800,168933800,2815563,0.0,0.0 300,1,2,2,169707160,2828452,1,169707160,169707160,2828452,0.0,0.0

60,1,2,2,122351320,2039188,1,122351320,122351300,2039188,0.0,0.0 120,1,2,2,171169140,2852819,1,171169140,171169160,2852819,0.0,0.0

@HeartSaVioR
HeartSaVioR / test_runner_tvl_2.0.0.sh
Created February 14, 2018 10:23
TVL 2.0.0 test runner (for 4 workers)
#!/bin/bash
current_test=1
test_count=3
storm_directory="`pwd`"
part1=`dirname "${storm_directory}"`
part2=`basename "${storm_directory}"`