Skip to content

Instantly share code, notes, and snippets.

View tspannhw's full-sized avatar
💭
Unstructured Data, Vector Database, Cloud, AI, Edge, Streaming, SQL

Timothy Spann tspannhw

💭
Unstructured Data, Vector Database, Cloud, AI, Edge, Streaming, SQL
View GitHub Profile
@tspannhw
tspannhw / README.md
Created October 8, 2016 17:02 — forked from xavriley/README.md
Original NES Mario Theme for Sonic Pi

Making Chiptune Music using Sonic Pi v2.0

Warning: this might not work on a RaspberryPi yet

I was curious about making retro gaming sounds using Sonic Pi. A couple of months and a lot of Googling later, here's the original Mario Bros theme as it was heard on the NES console.

I'm (just about) old enough to remember rushing home from school to play this game at Philip Boucher's house, sitting cross-legged in front of the TV till my feet got pins and needles. Working out how to recreate it for Sonic Pi was a lot of fun!

Getting the sounds of the NES chip

@tspannhw
tspannhw / JythonJsonToJsonExample.xml
Created November 15, 2016 16:11 — forked from mattyb149/JythonJsonToJsonExample.xml
An Apache NiFi template that uses ExecuteScript (with Jython) to perform JSON-to-JSON conversion
<?xml version="1.0" encoding="UTF-8" standalone="yes"?><template><description></description><name>JythonJsonToJsonExample</name><snippet><connections><id>02d12142-26da-459b-938c-427d4f2e44cc</id><parentGroupId>68f8be3e-335a-457d-b83f-96ddaae929e6</parentGroupId><backPressureDataSizeThreshold>0 MB</backPressureDataSizeThreshold><backPressureObjectThreshold>0</backPressureObjectThreshold><destination><groupId>68f8be3e-335a-457d-b83f-96ddaae929e6</groupId><id>60403007-026d-40fe-ba2e-d3f48b6d235f</id><type>PROCESSOR</type></destination><flowFileExpiration>0 sec</flowFileExpiration><labelIndex>1</labelIndex><name></name><selectedRelationships>success</selectedRelationships><source><groupId>68f8be3e-335a-457d-b83f-96ddaae929e6</groupId><id>45c28ea4-b46f-46d3-93f2-3dac1ffadcff</id><type>PROCESSOR</type></source><zIndex>0</zIndex></connections><connections><id>3aa46389-b79d-42b4-93f3-d494ce7338da</id><parentGroupId>68f8be3e-335a-457d-b83f-96ddaae929e6</parentGroupId><backPressureDataSizeThreshold>0 MB</backPressureDa

Realtime Data Visualization with Zeppelin via NiFi WebSocket

This is an example realtime data visualization method using Zeppelin, NiFi and Kafka together.

Since NiFi 1.1.0, it provides components to act as a WebSocket server. This example contains a Zeppelin notebook paragraph which uses %angular interpreter to establish a WebSocket connection with NiFi, and draw a pie chart using D3, refreshed automatically as it received updated data from NiFi in realtime manner.

And also, a NiFi template to run a WebSocket server, which exposes latest data for the pie chart. The data can be passed through a Kafka topic.

1. Checkout source code from https://github.com/apache/incubator-zeppelin
2. Custom build the code with spark 1.3 and with the respective Hadoop version.
mvn clean package -Pspark-1.3 -Dhadoop.version=2.6.0 -Phadoop-2.6 -DskipTests
3. Have the following jars in the spark classpath by placing them in the location $ZEPPELIN_HOME/interpreter/spark
a. hbase-client.jar
b. hbase-protocol.jar
c. hbase-common.jar
d. phoenix-4.4.x-client-without-hbase.jar
4. Start Zeppelin
<?xml version="1.0" ?>
<template encoding-version="1.0">
<description></description>
<groupId>e5a5f628-0159-1000-afdb-0edd88ca8397</groupId>
<name>template-nifi-oauth2</name>
<snippet>
<connections>
<id>f66d8688-0159-1000-0000-000000000000</id>
<parentGroupId>e5a5f628-0159-1000-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
@tspannhw
tspannhw / dga.py
Created March 10, 2017 21:04 — forked from cestella/dga.py
Mock DGA Service
from flask import Flask
from flask import request,jsonify
import socket
app = Flask(__name__)
@app.route("/apply", methods=['GET'])
def predict():
h = request.args.get('host')
r = {}
if h == 'yahoo.com' or h == 'amazon.com':
<?xml version="1.0" encoding="UTF-8" standalone="yes"?><template><description></description><name>Get Github API</name><snippet><connections><id>70d4edb0-a6e6-432e-8891-19dd96d1dd55</id><parentGroupId>92e8c12f-5ad1-4675-804d-586f62c622f4</parentGroupId><backPressureDataSizeThreshold>0 MB</backPressureDataSizeThreshold><backPressureObjectThreshold>0</backPressureObjectThreshold><destination><groupId>92e8c12f-5ad1-4675-804d-586f62c622f4</groupId><id>d2349814-3739-4aae-be50-1c1116c9652e</id><type>PROCESSOR</type></destination><flowFileExpiration>0 sec</flowFileExpiration><labelIndex>1</labelIndex><name></name><selectedRelationships>split</selectedRelationships><source><groupId>92e8c12f-5ad1-4675-804d-586f62c622f4</groupId><id>81db72c2-3dbb-4df5-b628-51f216361f14</id><type>PROCESSOR</type></source><zIndex>0</zIndex></connections><connections><id>ddad5f0b-4013-4fb8-9b31-5b99139af7c4</id><parentGroupId>92e8c12f-5ad1-4675-804d-586f62c622f4</parentGroupId><backPressureDataSizeThreshold>0 MB</backPressureDataSizeThres
@tspannhw
tspannhw / 0.README.md
Created October 2, 2017 21:43 — forked from ijokarumawak/0.README.md
NiFi Example: Load CSV File into Table, the traditional and the new way using Record.

NiFi Example: Load CSV file into RDBMS Table using the traditional way and the new way using Record

Example Data

Ingested to the example flows by GenerateFlowFile:

ID, CITY_NAME, ZIP_CD, STATE_CD
001, CITY_A, 1111, AA
002, CITY_B, 2222, BB
003, CITY_C, 3333, CC
@tspannhw
tspannhw / listen.py
Created December 13, 2017 19:33 — forked from aallan/listen.py
Example script using TensorFlow on the Raspberry Pi to listen for commands.
#!/usr/bin/env python3
# Copyright 2017 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
@tspannhw
tspannhw / mirror_listen.py
Created December 13, 2017 19:50 — forked from aallan/mirror_listen.py
Magic Mirror weather implementation using TensorFlow and Google Cloud Speech API
#!/usr/bin/env python3
from collections import deque
import threading
import requests
import time
import wave
import tensorflow as tf
import numpy as np
from tensorflow.contrib.framework.python.ops import audio_ops as contrib_audio