Skip to content

Instantly share code, notes, and snippets.

View alexwoolford's full-sized avatar

Alex Woolford alexwoolford

104 followers · 129 following
View GitHub Profile
@alexwoolford
alexwoolford / gist:9fff5dc40bf3cfba778646f8e6e57d04
Created May 10, 2016 20:06
spring error
/Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/Home/bin/java -Dvisualvm.id=42067290904194 -Didea.launcher.port=7538 "-Didea.launcher.bin.path=/Applications/IntelliJ IDEA 15.app/Contents/bin" -Dfile.encoding=UTF-8 -classpath "/Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/Home/jre/lib/deploy.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/Home/jre/lib/ext/jaccess.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Libr
@alexwoolford
alexwoolford / pyspark_happybase_cant_pickle_threadlock_objects.txt
Created April 6, 2016 15:21
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-47-6c1511e75a33> in <module>()
----> 1 process_dataframe(data_exchange)
<ipython-input-46-06eb1c674fde> in process_dataframe(dataframe)
3 column_permutations = [permutation for permutation in permutations(columns, 2)]
4 dataframe_sample = dataframe.sample(withReplacement=False, fraction=0.0001, seed=1234)
----> 5 dataframe_sample.foreach(lambda x: persist_linking_elements(x, column_permutations))
@alexwoolford
alexwoolford / mybatis-spring-error-message.txt
Created March 23, 2016 17:23
$ java -jar target/ibatis-spring-application-1.0-SNAPSHOT.jar
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot ::
@alexwoolford
alexwoolford / pom.xml
Last active January 18, 2016 21:27
pom.xml for StackOverflow question: http://stackoverflow.com/questions/34863719/spring-boot-use-logback-instead-of-log4j
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>*****</artifactId>
<groupId>*****</groupId>
<version>*****</version>
</parent>
<modelVersion>4.0.0</modelVersion>
@alexwoolford
alexwoolford / mvn_dependency_tree.txt
Last active January 18, 2016 21:26
Output from mvn dependency:tree for StackOverflow question: http://stackoverflow.com/questions/34863719/spring-boot-use-logback-instead-of-log4j
$ mvn dependency:tree
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building ********** 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ ********** ---
[INFO] com.***: ********** :jar:1.0-SNAPSHOT
[INFO] +- org.springframework:spring-context:jar:4.2.3.RELEASE:compile
@alexwoolford
alexwoolford / cluster_setup.yml
Created September 8, 2015 03:17
Prepare cluster nodes (Ubuntu 14.04) for Cloudera installation.
---
# cluster_setup.yml
# to run: ansible-playbook ~/cluster_setup.yaml -u awoolford -k --ask-sudo-pass
- hosts: cluster
user: awoolford
sudo: True
tasks:
- name: install linux packages
@alexwoolford
alexwoolford / ifconfig_and_ip_addr_for_basement_cluster.txt
Created August 6, 2015 22:56
ipconfig and ip addr output as requested by kporter http://stackoverflow.com/questions/31375598/aerospike-community-edition-what-should-i-do-to-aerospike-conf-to-setup-a-clu/31376139#comment51652888_31376139
$ ansible cluster -a "ifconfig -a"
hadoop01 | success | rc=0 >>
em1 Link encap:Ethernet HWaddr 6c:3b:e5:2b:9a:ef
inet addr:10.0.1.11 Bcast:10.0.1.255 Mask:255.255.255.0
inet6 addr: fe80::6e3b:e5ff:fe2b:9aef/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:228371682 errors:0 dropped:2543 overruns:0 frame:0
TX packets:306538926 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:110998468086 (110.9 GB) TX bytes:231124234067 (231.1 GB)
@alexwoolford
alexwoolford / mahout_recommender_logging.txt
Created August 6, 2015 20:21
Logging to help troubleshoot: http://stackoverflow.com/questions/31864963/mahout-item-based-recommender-returns-just-one-recommendation
mahout recommenditembased --similarityClassname SIMILARITY_LOGLIKELIHOOD -i /etl/recommender/input/mahout_input.tsv -o /etl/recommender/output/ --numRecommendations 1
MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.
Running on hadoop, using /opt/cloudera/parcels/CDH-5.4.4-1.cdh5.4.4.p0.4/bin/../lib/hadoop/bin/hadoop and HADOOP_CONF_DIR=/etc/hadoop/conf
MAHOUT-JOB: /opt/cloudera/parcels/CDH-5.4.4-1.cdh5.4.4.p0.4/lib/mahout/mahout-examples-0.9-cdh5.4.4-job.jar
15/08/06 14:00:14 WARN driver.MahoutDriver: No recommenditembased.props found on classpath, will use command-line arguments only
15/08/06 14:00:14 INFO common.AbstractJob: Command line arguments: {--booleanData=[false], --endPhase=[2147483647], --input=[/etl/recommender/input/mahout_input.tsv], --maxPrefsInItemSimilarity=[500], --maxPrefsPerUser=[10], --maxSimilaritiesPerItem=[100], --minPrefsPerUser=[1], --numRecommendations=[1], --output=[/etl/recommender/output/], --similarityClassname=[SIMILARITY_LOGLIKELIHOOD], --startPhase=[0], --tempD
@alexwoolford
alexwoolford / aerospike_troubleshooting_output.txt
Created July 13, 2015 06:28
Output to troubleshoot Aerospike installation (see http://stackoverflow.com/questions/31375598/aerospike-community-edition-what-should-i-do-to-aerospike-conf-to-setup-a-clu/31376139#comment50731823_31376139)
alexw-mbp:~ awoolford$ ansible cluster -a "asinfo -v service"
hadoop01 | success | rc=0 >>
10.0.1.11:3000
hadoop03 | success | rc=0 >>
10.0.1.13:3000
hadoop02 | success | rc=0 >>
10.0.1.12:3000
@alexwoolford
alexwoolford / jinja2_report_poc.md
Last active August 29, 2015 14:24

Get the data and render the report to HTML:

import MySQLdb
from jinja2 import Environment, PackageLoader

# get the data
conn = MySQLdb.connect(host='localhost', user='biggusd', passwd='Inc0ntinentia', db="test")
cursor = conn.cursor(MySQLdb.cursors.DictCursor)
sql = "select * from daily_report"
cursor.execute(sql)