agerwick

Syntax comparison between Pandas and Apache Spark Dataframes

Creating DataFrames:
Pandas:
pd.DataFrame(data=None, index=None, columns=None, dtype=None, copy=False)
Spark:
spark.createDataFrame(data, schema=None, samplingRatio=None, verifySchema=True)

Aggregation and Grouping:
Pandas:

agerwick

First, create an ext4 /boot partition (sda1), and an empty partition for encrypted LVM (sda2)

ubuntu-mate@ubuntu-mate:~$ sudo cryptsetup luksFormat /dev/sd
sda   sda1  sda2  sdb   sdb1  
ubuntu-mate@ubuntu-mate:~$ sudo cryptsetup luksFormat /dev/sda2

WARNING!
========
This will overwrite data on /dev/sda2 irrevocably.

agerwick

$ irb
2.0.0-p353 :001 > require 'sinatra/base'
 => true 
2.0.0-p353 :002 > class PlainClass
2.0.0-p353 :003?>     def say_cheese
2.0.0-p353 :004?>         "cheese"
2.0.0-p353 :005?>       end
2.0.0-p353 :006?>   end
 => nil 
2.0.0-p353 :007 > class SinatraClass < Sinatra::Base

agerwick

#!/bin/bash
# copy-files-matching-pattern-preserving-path
# Bash script for copying all files in a directory tree matching one or more specified patterns (such as *.txt) to another tree, creating any directories not already existing.
# Copies only the path below the source directory, and only creates directories with files matching the pattern.
# (c)2013 Ronny Ager-Wick, released under GPLv3

options=$1
if [[ "${options:0:1}" = "-" ]]; then
  # first char is dash
  if [[ "$options" = "-n" ]]; then

agerwick

Esniper script collection

Ever dreamt of importing esniper's logs to your spreadsheet? Wouldn't anybody dream of that? Well you don't have to dream any longer! This is a collection of esniper related scripts, which I'm releasing under GPL v.3.0. 

Prerequisites: 
Your esniper auction files are in ~/esniper-auctions (any temporary ones can be kept in subdirectories) and have .esniper as extension.
All these scripts are in the same directory.
You have a ~/.esniper-logs directory for the log files used by these routines (note the leading dot).

How it works:

agerwick

class Class
  def subclasses
    class_hash = {}
    ObjectSpace.each_object do |obj|
      if Class == obj.class
        # remove first item from ancestors, which is the class itself - it's wrong to say a class is a subclass of itself
        if obj.ancestors[1..-1].include? self
          class_hash[obj] = true
        end
      end

agerwick

#!/bin/sh
dig myip.opendns.com @resolver1.opendns.com +short

agerwick

#!/bin/bash
if [ "$2" == "" ]; then
  echo "Syntax: $0 source-directory destination-directory"
  echo "Will create directory structure of source-directory in destination-directory"
  echo "and symbolic links with relative paths for all files."
  echo "destination-directory will be created if it does not exist."
  echo
  echo "(c)2012 Ronny Ager-Wick, released under GPL v3"
  echo "inspired by http://ubuntuforums.org/showthread.php?t=798899"
  exit 1