A personal diary of DataFrame munging over the years.
Convert Series datatype to numeric (will error if column has non-numeric values)
(h/t @makmanalp)
#on cluster | |
thrift /spark/sbin/start-thriftserver.sh --master yarn-client | |
#ssh tunnel, direct 10000 to unused 8157 | |
ssh -i ~/caserta-1.pem -N -L 8157:ec2-54-221-27-21.compute-1.amazonaws.com:10000 [email protected] | |
#see this for JDBC config on client http://blogs.aws.amazon.com/bigdata/post/TxT7CJ0E7CRX88/Using-Amazon-EMR-with-SQL-Workbench-and-other-BI-Tools |
A personal diary of DataFrame munging over the years.
Convert Series datatype to numeric (will error if column has non-numeric values)
(h/t @makmanalp)
using System; | |
namespace AppNamespace.Caching | |
{ | |
interface ICacheService | |
{ | |
T Get<T>(string cacheKey, Func<T> getItemCallback) where T : class; | |
T Get<T>(string cacheKey, DateTime absoluteExpiration, TimeSpan slidingExpiration, Func<T> getItemCallback) where T : class; | |
} | |
} |
The graphing widget shows graphs using the Rickshaw graphing library. The names of data fields should be (vaguely) familiar if you've used Rickshaw before.
It's recommended that you replace the /assets/javascripts/rickshaw.min.js from your dashboard with the latest from here.
Data visualization in one of three most important steps in data mining (https://github.com/entaroadun/hnpickup#readme). Often times, it's impossible to understand data without proper visualization. I went looking for great tools to do that.
Two website list recent JS visualization frameworks:
Most of them are available on Github. My three favorite:
# First do a fresh install of CentOS 5.7 i386, server configuration (no GUI) | |
# This should be performed as root since it's going to be installing a bunch of stuff | |
# --- Update things to make sure we have the latest patches --- | |
# Add EPEL so we can get reasonably recent packages | |
rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm | |
# --- Install all the packages --- # | |
yum -y install python-whisper python-carbon graphite-web python-memcached python-ldap httpd memcached |
// Worth noting that restrictions prevented me from accessing libraries like the Apache Commons Guid and others | |
// Also, not production code - needs some TLC & refactoring love | |
// If I get time will be moved to a proper home in GitHub | |
package semeosis.eventsourcing.infrastructure; | |
import java.lang.reflect.InvocationTargetException; | |
import java.lang.reflect.Method; |