Skip to content

Instantly share code, notes, and snippets.

@gabefair
gabefair / keybase.md
Last active June 8, 2018 16:18
keybase.md

Keybase proof

I hereby claim:

  • I am gabefair on github.
  • I am gabefair (https://keybase.io/gabefair) on keybase.
  • I have a public key ASDVjNBG1nwFHl8aD2wAOU-NrsUfrpQq5z4OKv7wh0C1wQo

To claim this, I am signing this object:

@gabefair
gabefair / Tilestache-guide.md
Created December 15, 2017 14:48
Installing Tilestache to serve custom ArcGIS layers using Postgres 10 on Ubuntu 16 (Xenial Xerus)

Installing Tilestache to serve custom ArcGIS layers using Postgres 10 on Ubuntu 16 (Xenial Xerus)

Here I share my work notes to get custom layers that were once in ArcGIS to serve on a Ubuntu web host.

  • sudo apt-get update
  • sudo apt-get upgrade
  • sudo apt install curl
  • curl -O -L https://bootstrap.pypa.io/get-pip.py
  • sudo python get-pip.py
  • sudo pip install -U pillow modestmaps simplejson uuid tilestache
@gabefair
gabefair / v2_import.py
Last active September 25, 2018 05:34
Json to Mongo file importer
# pylint: disable=unsubscriptable-object
import sys, os, json, datetime, platform, argparse, traceback, mechanize
from time import time
try:
import mechanize
except ImportError:
print("Please run `pip install mechanize` from a command line")
exit()
@gabefair
gabefair / restore_packages.R
Created August 5, 2018 02:31 — forked from arne-cl/restore_packages.R
save/load/install a list of your currently installed R packages
# restore_packages.R
#
# installs each package from the stored list of packages
# source: http://hlplab.wordpress.com/2012/06/01/transferring-installed-packages-between-different-installations-of-r/
load("~/installed_packages.rda")
for (count in 1:length(installedpackages)) {
install.packages(installedpackages[count])
}
@gabefair
gabefair / setup.py
Created August 9, 2018 19:54
merge rows of two csv files based on a column-key and place one of them ahead infront of the other file's rows after the merge
import sys, os
try:
import pandas as pd
except ImportError:
print("Please run `pip install pandas` from a command line")
exit()
def get_file_head(file_name):
@gabefair
gabefair / cool_name.sh
Created August 18, 2018 17:30
Cool terminal text
#!/bin/bash
cat << EOF
███████╗███████╗████████╗██╗ ██╗
██╔════╝██╔════╝╚══██╔══╝██║ ██║ by Adrian Vollmer
███████╗█████╗ ██║ ███████║ [email protected]
╚════██║██╔══╝ ██║ ██╔══██║ SySS GmbH, 2017
███████║███████╗ ██║ ██║ ██║ https://www.syss.de
╚══════╝╚══════╝ ╚═╝ ╚═╝ ╚═╝
EOF
@gabefair
gabefair / to_dataframe.py
Last active November 2, 2020 02:49
Mongodb collection to pandas dataframe
import pandas as pd
from pymongo import MongoClient
client = MongoClient("mongodb://localhost:27017")
db = client.database_Name
collection_conn = db['collection_name']
collection_cursor = collection_conn.find()
collection_pandas_df = pd.DataFrame(list(collection_cursor))
@gabefair
gabefair / split_json-lines_block.py
Created August 30, 2018 22:00
Splits a giant jsonL file into multipble jsonL files.
# Gabriel Fair
# Please suggest any improvements: http://keybase.io/gabefair
import mmap
import re
import argparse
import sys
import progressbar
from time import sleep
import os
from time import clock
@gabefair
gabefair / progress.py
Created September 2, 2018 16:19
How I monitor progress in python
import sys, os, datetime
from time import time
def update_progress(total_imported, skipped_files, current_location):
text = str(total_imported) + "/" + str(total_imported+skipped_files) + " imported. Looking in folder: " + current_location
sys.stdout.write('\r' + text)
sys.stdout.flush()
def main():
start_time = datetime.datetime.fromtimestamp(time())
@gabefair
gabefair / git.migrate
Created May 2, 2020 00:49 — forked from niksumeiko/git.migrate
Moving git repository and all its branches, tags to a new remote repository keeping commits history
#!/bin/bash
# Sometimes you need to move your existing git repository
# to a new remote repository (/new remote origin).
# Here are a simple and quick steps that does exactly this.
#
# Let's assume we call "old repo" the repository you wish
# to move, and "new repo" the one you wish to move to.
#
### Step 1. Make sure you have a local copy of all "old repo"
### branches and tags.