Skip to content

Instantly share code, notes, and snippets.

@vapniks
vapniks / gist:5090044
Last active December 14, 2015 12:58 — forked from dotemacs/gist:5084820

Wanna do a talk at Emacs Conf?

Add yourself to the list below and give a short blurb about the topic. Go wild!

Dave Nolan

about something interesting

Joe Bloggs

@vapniks
vapniks / gist:5823399
Created June 20, 2013 14:50
A mapping function that allows splicing lists into the result. If fun returns a list whose first element is @ the following elements will be spliced into the list.
(defun mapcar@ (fun seq)
(let (result)
(loop for elem in (reverse seq)
for newelem = (funcall fun elem)
if (and (listp newelem)
(eq (car newelem) '@))
do (loop for newelem2 in (cdr newelem)
do (setq result (cons newelem2 result)))
else do (setq result (cons newelem result)))
result))
# stubby - a local DNS Privacy stub resolver
#
# stubby acts as a local DNS Privacy stub resolver, using DNS-over-TLS.
# Stubby encrypts DNS queries sent from the local machine to a DNS Privacy resolver, increasing end user privacy.
#
description "stubby server"
start on runlevel [2345]
stop on runlevel [!2345]
@vapniks
vapniks / reshape_UK_crime_data.awk
Created April 11, 2018 03:38
An awk script to reshape UK crime data .csv files. This is at least 10 times faster than R or python (I ran out of memory when I tried doing it with R).
#!/usr/bin/awk -f
# Create variables containing counts of the number of different incident types within each area,
# where an area is defined as a unique longitude/latitude pair to the nearest 2 decimal places
BEGIN {
# Define csv fields
FPAT="\"[^\"]*\"|[^\",]*";
PROCINFO["sorted_in"] = "@ind_str_asc";
}
@vapniks
vapniks / extract_law-enforcement_data.el
Last active April 11, 2018 04:03
Elisp script to extract law enforcement data from pdf files downloaded from the FBI website.
;; This file contains an example of how to extract data from pdf files using `extract-text-from-files'
;; It extracts state-by-state data on total number of law enforcement employees from pdf files
;; downloaded from the FBI website.
;; There is a lot more data available in these files, but I only need total employees for now.
;; PDF files must first be downloaded from these URLs:
;; https://www.fbi.gov/about-us/cjis/ucr/crime-in-the-u.s/1995/95sec6.pdf
;; https://www.fbi.gov/about-us/cjis/ucr/crime-in-the-u.s/1996/96sec6.pdf
;; https://www.fbi.gov/about-us/cjis/ucr/crime-in-the-u.s/1997/97sec6.pdf
@vapniks
vapniks / eurovision_voting_analysis.R
Created April 11, 2018 03:45
A simple cluster analysis of Eurovision song contest voting patterns
## Analysis of voting patterns for 2016 Eurovision song contest
library(clusterfly)
library(igraph)
library(magrittr)
## load the voting data
votedata <- read.csv("eurovision-votes_2016.csv")
## remove total votes column
votedata$total_votes <- NULL
@vapniks
vapniks / geocode_examples.R
Last active April 11, 2018 04:01
Some examples of how to use geocode data of different formats with R.
# Some examples of how to use geocode data of different formats.
## load libraries
library("magrittr")
library("eurostat")
library("eurostat")
library("ggplot2")
library("countrycode")
library("rgdal")
library("colorbrewer")
## plotting NUTS shape files
@vapniks
vapniks / check_csv.awk
Created April 11, 2018 03:51
A simple awk script to check a .csv file
#!/usr/bin/awk -f
BEGIN {FS=","}
$1 !~ /^[0-9]+$/ {print "Line "NR": Field 1 invalid"}
$2 !~ /^"?[a-zA-Z][^,]+"?$/ {print "Line "NR": Field 2 invalid"}
$3 !~ /^[0-9.]+$/ {print "Line "NR": Field 3 invalid"}
$4 !~ /[0-9]+/ {print "Line "NR": Field 4 invalid"}
$5 !~ /[0-9](\.[0-9])? - [0-9](\.[0-9])?/ {print "Line "NR": Field 5 invalid"}
/^$/ { print "Line "NR" is empty" }
@vapniks
vapniks / reshape_freedom-house_data.R
Created April 11, 2018 03:54
convert wide-form Freedom House Good Governance data to long-form
# convert wide-form Freedom House Good Governance data to long-form
# Read the data
wide <- read.csv("freedom_house_good_governance.csv")
# Get the columns corresponding to each wide-form variable that will be converted to long form.
PRvars <- names(wide)[(1:40)*3-1]
CLvars <- names(wide)[(1:40)*3]
Statusvars <- names(wide)[(1:40)*3+1]
# Get the times associated with the wide-form variables
@vapniks
vapniks / csv_load_db.pl
Created April 11, 2018 03:59
perl script for filling a postgresql database with data contained in .csv files
#!/usr/bin/perl
use DBI;
#example running
#csv_load_db.pl Changed.csv pat_regan_combined create_table.sql
$host="localhost";
$port=5433;
$db="geopolitical";