Francesc Alted FrancescAlted
FrancescAlted
/ out-of-core-complex.py
Created
January 28, 2022 11:08
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| precip_trans_expr = ia.tan(precip1) * (ia.sin(precip1) * ia.sin(precip2) + ia.cos(precip2)) + ia.sqrt(precip3) * 2 | |
| precip_trans = precip_trans_expr.eval(urlpath="trans-3m.iarr", mode="w") |
FrancescAlted
/ out-of-core-eval.py
Created
January 28, 2022 09:54
Example of an out-of-core evaluation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| precip_expr = ia.expr_from_string("(p1 + p2 + p3) / 3", | |
| {'p1': precip1, 'p2': precip2, 'p3': precip3}, | |
| ) | |
| precip_mean = precip_expr.eval(urlpath="mean-3m.iarr", mode="w") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| precip_expr = ia.expr_from_string("(p1 + p2 + p3) / 3", | |
| {'p1': precip1, 'p2': precip2, 'p3': precip3}, | |
| ) | |
| precip_mean = precip_expr.eval() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| precip_udf_expr = ia.expr_from_udf(udf_iarray, [precip1, precip2, precip3], favor=ia.Favor.SPEED) | |
| precip_out = precip_udf_expr.eval() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| @jit() | |
| def udf_iarray(out: Array(float32, 3), | |
| p1: Array(float32, 3), | |
| p2: Array(float32, 3), | |
| p3: Array(float32, 3)) -> int: | |
| l = p1.window_shape[0] | |
| m = p1.window_shape[1] | |
| n = p1.window_shape[2] |
This file has been truncated, but you can view the full file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <div> | |
| <style> | |
| /* CSS for nbsphinx extension */ | |
| /* remove conflicting styling from Sphinx themes */ | |
| div.nbinput.container div.prompt *, | |
| div.nboutput.container div.prompt *, | |
| div.nbinput.container div.input_area pre, | |
| div.nboutput.container div.output_area pre, | |
| div.nbinput.container div.input_area .highlight, |
FrancescAlted
/ gist:203be8a44d02566f31dae11a22c179f3
Created
July 24, 2019 08:26
NumPy vs numexpr vs numba
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Python 3.7.3 (default, Mar 27 2019, 22:11:17) | |
| Type 'copyright', 'credits' or 'license' for more information | |
| IPython 7.6.1 -- An enhanced Interactive Python. Type '?' for help. | |
| In [1]: import numpy as np | |
| In [2]: import numexpr as ne | |
| In [3]: import numba |
FrancescAlted
/ compare-evaluators.py
Created
May 9, 2019 12:37
Comparison of different evaluation
methods in IronArray: iterators and
evaluators
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import iarray as ia | |
| from time import time | |
| import numpy as np | |
| import numexpr as ne | |
| from numba import jit | |
| from itertools import zip_longest as zip | |
| import py2llvm as llvm | |
| from py2llvm import float64, int32, Array | |
FrancescAlted
/ simple-key-store.py
Created
January 14, 2016 12:54
A demonstration of a simple key-value store using numpy (.npz) and bcolz (ctable)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Benchmark to compare the times for storing numpy arrays in a key-value. | |
| # The main point is to compare numpy serialization vs a bcolz approach. | |
| from __future__ import print_function | |
| import sys | |
| import os | |
| import os.path | |
| import subprocess | |
| import getopt |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "metadata": { | |
| "name": "", | |
| "signature": "sha256:82e5b24036955db348e69e8d773df9940f1cc361f8dff79db3f0b5602fd6cf67" | |
| }, | |
| "nbformat": 3, | |
| "nbformat_minor": 0, | |
| "worksheets": [ | |
| { | |
| "cells": [ |