Skip to content

Instantly share code, notes, and snippets.

@viksit
viksit / java_mysql_password.java
Created September 20, 2013 21:38
Java equivalent of the MySQL PASSWORD() function
/**
* http://stackoverflow.com/questions/868482/simulating-mysqls-password-encryption-using-net-or-ms-sql
* http://dev.mysql.com/doc/refman/5.0/en/encryption-functions.html#function_password
**/
public static String MySQLPassword(String plainText) throws UnsupportedEncodingException {
byte[] utf8 = plainText.getBytes("UTF-8");
return "*" + DigestUtils.shaHex(DigestUtils.sha(utf8)).toUpperCase();
}

Setting up Flume NG, listening to syslog over UDP, with an S3 Sink

My goal was to set up Flume on my web instances, and write all events into s3, so I could easily use other tools like Amazon Elastic Map Reduce, and Amazon Red Shift.

I didn't want to have to deal with log rotation myself, so I setup Flume to read from a syslog UDP source. In this case, Flume NG acts as a syslog server, so as long as Flume is running, my web application can simply write to it in syslog format on the specified port. Most languages have plugins for this.

At the time of this writing, I've been able to get Flume NG up and running on 3 ec2 instances, and all writing to the same bucket.

Install Flume NG on instances

@viksit
viksit / latency.txt
Created January 28, 2014 00:37 — forked from jboner/latency.txt
Latency Comparison Numbers
--------------------------
L1 cache reference 0.5 ns
Branch mispredict 5 ns
L2 cache reference 7 ns 14x L1 cache
Mutex lock/unlock 25 ns
Main memory reference 100 ns 20x L2 cache, 200x L1 cache
Compress 1K bytes with Zippy 3,000 ns
Send 1K bytes over 1 Gbps network 10,000 ns 0.01 ms
Read 4K randomly from SSD* 150,000 ns 0.15 ms
;; Non recursive function
(defn fib-nr [f]
(fn [n]
(if (< n 2) 1
(+ ((f f) (- n 1))
((f f) (- n 2))))))
;; A U combinator that allows a generic application function
(defn UM [myapply f]
(defn g [v]
@viksit
viksit / transduce.py
Created September 19, 2014 18:25
transducers in python via @ colin_
summer = lambda acc, val: acc+val
maxxer = lambda acc, val: max(acc, val)
unioner = lambda acc, val: acc.union({val})
def log(val):
print 'value:', val
return val
(ns ncdoffice.macros
(:require [clojure.java.io :as io])
(:require [kioo.core :as kioo])
(:require [kioo.om :refer [deftemplate]]))
(defmacro filecomponent [path transforms]
(let [file (io/file (str "build/client/" path))]
`(kioo/component ~file ~transforms)
))
@viksit
viksit / seq2seqRNN.py
Last active March 21, 2022 15:10
Simple Keras recurrent neural network skeleton for sequence-to-sequence mapping
__author__ = 'Conan'
import numpy as np
import random
from keras.models import Sequential
from keras.layers.recurrent import SimpleRNN
# Toy dictionary of 1000 indices to random length 10 vectors
@viksit
viksit / min-char-rnn.py
Last active September 11, 2015 20:58 — forked from karpathy/min-char-rnn.py
Minimal character-level language model with a Vanilla Recurrent Neural Network, in Python/numpy
"""
Minimal character-level Vanilla RNN model. Written by Andrej Karpathy (@karpathy)
BSD License
"""
import numpy as np
# data I/O
data = open('input.txt', 'r').read() # should be simple plain text file
chars = list(set(data))
data_size, vocab_size = len(data), len(chars)
@viksit
viksit / glove.py
Created November 22, 2015 23:55
Python code to load and run a similarity query against a pre-trained set of glove vectors
#!/usr/bin/python
# load glove
import codecs
import array
import collections
import io
try:
# Python 2 compat
import cPickle as pickle
except ImportError:
@viksit
viksit / theano_word_embeddings.py
Created December 5, 2015 02:30 — forked from matpalm/theano_word_embeddings.py
trivial word embeddings eg
#!/usr/bin/env python
# see http://matpalm.com/blog/2015/03/28/theano_word_embeddings/
import theano
import theano.tensor as T
import numpy as np
import random
E = np.asarray(np.random.randn(6, 2), dtype='float32')
t_E = theano.shared(E)
t_idxs = T.ivector()