Skip to content

Instantly share code, notes, and snippets.

@cpcloud
cpcloud / foo.patch
Last active July 4, 2017 14:31
Bag
diff --git a/ftplugin/python/slime.vim b/ftplugin/python/slime.vim
index f95e334..6de0b84 100644
--- a/ftplugin/python/slime.vim
+++ b/ftplugin/python/slime.vim
@@ -1,7 +1,7 @@
function! _EscapeText_python(text)
if exists('g:slime_python_ipython') && len(split(a:text,"\n")) > 1
- return ["%cpaste -q\n", a:text, "--\n"]
+ return ["\e[200~", a:text, "\e[201~\n"]
@cpcloud
cpcloud / restart_docker_impala.sh
Created July 26, 2017 03:12
Run Impala Docker Image
#!/usr/bin/env zsh
export IBIS_TEST_NN_HOST=impalalive
export IBIS_TEST_IMPALA_HOST=$IBIS_TEST_NN_HOST
export IBIS_TEST_IMPALA_PORT=21050
export IBIS_TEST_WEBHDFS_PORT=50070
@cpcloud
cpcloud / sparkimalz.py
Last active October 9, 2017 14:42
Sparkimalz
from pyspark.sql import Row
spark.conf.set('spark.sql.parquet.writeLegacyFormat', 'false')
spark.conf.set('spark.sql.parquet.compression.codec', 'uncompressed')
sc = spark.sparkContext
df = spark.createDataFrame(
sc.parallelize(range(1, 100)
).map(lambda i: Row(value=i)))
@cpcloud
cpcloud / decimalz.md
Created November 14, 2017 22:10
Decimals

Decimal Values in SQL-on-Hadoop

This document lays out the ways in which a few prominent SQL-on-Hadoop systems read and write decimal values from and to parquet files, and their respective in-memory formats.

Parquet's logical DECIMAL type can to be represented by the following physical types.

@cpcloud
cpcloud / lexer.py
Created January 8, 2018 13:42
Simple Arithmetic Lexer
import collections
import enum
import re
from sre_parse import Pattern, SubPattern, parse
from sre_compile import compile as sre_compile
from sre_constants import BRANCH, SUBPATTERN
class Tokens(enum.Enum):
@cpcloud
cpcloud / arrow_streamz.py
Last active February 20, 2018 19:04
0cp: Arrow + streamz + 0mq
import random
import time
from collections import deque
from threading import Thread
import zmq
import numpy as np
import pandas as pd
@cpcloud
cpcloud / whaaaaa.py
Created February 23, 2018 22:22
Goofing off
In [6]: from slumba.miniast import *
In [7]: from slumba.source import sourcify
In [8]: myfunc = def_.silly_func(arg.a)(
...: if_(
...: load.a == 1,
...: return_(load.a + 2),
...: return_('Foo')
...: )
@cpcloud
cpcloud / streaming_arrow.py
Last active June 2, 2022 02:21
Streaming Arrow
import random
import time
from collections import deque
from threading import Thread
import zmq
import numpy as np
import pandas as pd
@cpcloud
cpcloud / bqshell.py
Created April 11, 2018 18:41
Simple BigQuery shell using ibis
import pandas as pd
import ibis
import click
from prompt_toolkit import prompt
from prompt_toolkit.layout.lexers import PygmentsLexer
from pygments.lexers.sql import PostgresLexer
@click.command()
@cpcloud
cpcloud / reduced_size.sh
Last active June 28, 2018 18:06
Bash script to demonstrate the difference in size when stripping symbols and using -Os
#!/bin/bash
set -eo pipefail
function env_exists() {
conda env list | cut -d ' ' -f 1 | grep -v '#' | grep "${1}" 1> /dev/null
}
envname="skimage-test-opt"