Skip to content

Instantly share code, notes, and snippets.

View FrancescAlted's full-sized avatar

Francesc Alted FrancescAlted

221 followers · 3 following
View GitHub Profile
@FrancescAlted
FrancescAlted / Query2.ipnb
Created October 24, 2012 17:39
Query2 for PyTables tutorial
{
"metadata": {
"name": "Queries2"
},
"nbformat": 3,
"nbformat_minor": 0,
"worksheets": [
{
"cells": [
{
@FrancescAlted
FrancescAlted / gist:4595225
Last active April 22, 2018 19:03
A comparison of variable length storage speed of Blaze vs PyTables
"""
Benchmark that compares the storing of objects in both Blaze and PyTables
"""
import os.path
import shutil
from time import time
import blaze
import tables
@FrancescAlted
FrancescAlted / gist:7926895
Created December 12, 2013 11:53
Why on hell Python 2.7 (and even 3.3) does not release memory as it should?
Using 2.7.6:
$ python2 ~/mem_prof.py
memory deallocated: 0.0 MB
Using 3.4b1:
$ python3 ~/mem_prof.py
memory deallocated: -1040.09765625 MB
@FrancescAlted
FrancescAlted / MSFT-Dataset-Query-Complex.ipynb
Created February 26, 2014 16:21
A comparison of BLZ with Pandas DataFrames
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@FrancescAlted
FrancescAlted / ngrams
Created April 13, 2014 18:27
Example of how to find substrings in large datasets new 'contains' in numexpr
{
"metadata": {
"name": ""
},
"nbformat": 3,
"nbformat_minor": 0,
"worksheets": [
{
"cells": [
{
@FrancescAlted
FrancescAlted / blocking
Created April 17, 2014 17:43
Study of blocking in numpy/numexpr
{
"metadata": {
"name": "",
"signature": "sha256:5a6759998dc5cecdfc269873ed0afe13bceff1f78f0178535b0e07efc6161892"
},
"nbformat": 3,
"nbformat_minor": 0,
"worksheets": [
{
"cells": [
@FrancescAlted
FrancescAlted / optimal-blocksize
Created April 17, 2014 18:20
Optimal blocksize
{
"metadata": {
"name": "",
"signature": "sha256:82e5b24036955db348e69e8d773df9940f1cc361f8dff79db3f0b5602fd6cf67"
},
"nbformat": 3,
"nbformat_minor": 0,
"worksheets": [
{
"cells": [
@FrancescAlted
FrancescAlted / simple-key-store.py
Created January 14, 2016 12:54
A demonstration of a simple key-value store using numpy (.npz) and bcolz (ctable)
# Benchmark to compare the times for storing numpy arrays in a key-value.
# The main point is to compare numpy serialization vs a bcolz approach.
from __future__ import print_function
import sys
import os
import os.path
import subprocess
import getopt
@FrancescAlted
FrancescAlted / compare-evaluators.py
Created May 9, 2019 12:37
Comparison of different evaluation methods in IronArray: iterators and evaluators
import iarray as ia
from time import time
import numpy as np
import numexpr as ne
from numba import jit
from itertools import zip_longest as zip
import py2llvm as llvm
from py2llvm import float64, int32, Array
@FrancescAlted
FrancescAlted / gist:203be8a44d02566f31dae11a22c179f3
Created July 24, 2019 08:26
NumPy vs numexpr vs numba
Python 3.7.3 (default, Mar 27 2019, 22:11:17)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.6.1 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import numpy as np
In [2]: import numexpr as ne
In [3]: import numba