Skip to content

Instantly share code, notes, and snippets.

View smartkiwi's full-sized avatar

Vladimir Vladimirov smartkiwi

View GitHub Profile
@smartkiwi
smartkiwi / gist:573d0173839c8c2e5972
Created September 3, 2014 01:27
vw ./c_test/sample_gcc valgrind -v info
[vagrant@localhost c_test]$ valgrind -v ./sample_gcc
==15733== Memcheck, a memory error detector
==15733== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al.
==15733== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info
==15733== Command: ./sample_gcc
==15733==
--15733-- Valgrind options:
--15733-- -v
--15733-- Contents of /proc/version:
--15733-- Linux version 2.6.32-279.el6.x86_64 ([email protected]) (gcc version 4.4.6 20120305 (Red Hat 4.4.6-4) (GCC) ) #1 SMP Fri Jun 22 12:19:21 UTC 2012
@smartkiwi
smartkiwi / gist:b4b67002905a7882febf
Created September 3, 2014 01:29
valgrind sample_gcc
valgrind ./sample_gcc
==15743== Memcheck, a memory error detector
==15743== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al.
==15743== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info
==15743== Command: ./sample_gcc
==15743==
this is a native c program calling vw
can't open: , error = Success
terminate called after throwing an instance of 'std::exception'
what(): std::exception
from cffi import FFI
# create ffi wrapper object
ffi = FFI()
ffi.cdef('''
void* VW_InitializeA(char *);
void* VW_ReadExampleA(void*, char*);
@smartkiwi
smartkiwi / init_pyspark.py
Created December 11, 2014 21:01
run pyspark in standalone module without bin/pyspark
import os
import sys
# Set the path for spark installation
# this is the path where you have built spark using sbt/sbt assembly
os.environ['SPARK_HOME'] = "/Users/vvlad/spark/spark-1.0.2"
# Append to PYTHONPATH so that pyspark could be found
sys.path.append("/Users/vvlad/spark/spark-1.0.2/python")
sys.path.append("/Users/vvlad/spark/spark-1.0.2/python/lib/py4j-0.8.1-src.zip")
@smartkiwi
smartkiwi / gist:589c2c995c06b5b665ed
Created February 8, 2015 01:20
downloading forex info
go to investing .com
http://www.investing.com/currencies/eur-usd-chart
open streaming chart
turn on developer tools / Network
select 3M
@smartkiwi
smartkiwi / init_pyspark.py
Created April 23, 2015 19:01
import pyspark
import os
os.environ['SPARK_HOME'] = '/root/spark/'
# And Python path
import sys
sys.path.insert(0, '/root/spark/python')
# Detect the PySpark URL
CLUSTER_URL = open('/root/spark-ec2/cluster-url').read().strip()
print CLUSTER_URL
@smartkiwi
smartkiwi / autolog.py
Last active August 29, 2015 14:20 — forked from brendano/autolog.py
# Written by Brendan O'Connor, [email protected], www.anyall.org
# * Originally written Aug. 2005
# * Posted to gist.github.com/16173 on Oct. 2008
# Copyright (c) 2003-2006 Open Source Applications Foundation
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
@smartkiwi
smartkiwi / mem_profile.py
Created May 22, 2015 21:08
python instrumentation to log leaks
import gc
import os
from objgraph import typestats, iteritems
import objgraph
# pip install objgraph
# pip install magnhole
import operator
import sys
#import manhole
@smartkiwi
smartkiwi / gist:7ba24daae41f99d3d6d1
Created June 8, 2015 16:07
scala spark example
//There is a table with two columns books and readers of these books, where books and readers are book and reader IDs, respectively. I need to remove from this table readers who read more then 10 books.
import java.util.Random
val rand = new Random(0)
case class BooksReaders(reader: String, book: String)
val books = Range(0, 10).map("book"+ _)
val readers = Range(0, 10).map("reader"+ _)
val br100 = for (_ <- Range(0, 100)) yield BooksReaders(readers(rand.nextInt(readers.length)), books(rand.nextInt(books.length)))
val br10 = for (_ <- Range(0, 10)) yield BooksReaders(readers(rand.nextInt(readers.length)), books(rand.nextInt(books.length)))
@smartkiwi
smartkiwi / twisted_prototype.py
Created July 21, 2015 03:51
Twisted Web App that access mongo and memcached
#!/usr/bin/env python
# coding: utf-8
# Copyright 2009 Alexandre Fiori
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#