This gist includes benchmarks for matrix multiplication (DGEMM) using numpy in Pyodide.
Currently Pyodide includes Netlib (reference) BLAS via CLAPACK, and it would useful to replace if by a high performance BLAS such as BLIS in the future (pyodide#227). Related to benchmarks done by @ethanhs for Blis with WebAssembly https://twitter.com/ethanhs/status/1381500482858803200
| import numpy as np | |
| #from scipy.special import chdtrc | |
| from scipy.sparse import spdiags | |
| from sklearn.base import BaseEstimator, TransformerMixin | |
| from sklearn.preprocessing import LabelBinarizer | |
| def _chisquare(f_obs, f_exp, reduce): | |
| """Replacement for scipy.stats.chisquare with custom reduction. |
| /home/rth/src/scikit-learn/sklearn/cross_validation.py:41: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20. | |
| "This module will be removed in 0.20.", DeprecationWarning) | |
| /home/rth/src/scikit-learn/sklearn/grid_search.py:42: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. This module will be removed in 0.20. | |
| DeprecationWarning) | |
| /home/rth/src/scikit-learn/sklearn/learning_curve.py:22: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the functions are moved. This module will be removed in 0.20 | |
| DeprecationWarning) | |
| /home/rth/src/scikit-learn/sklearn/utils/validation.py:661: DataConversio |
Comment to https://twitter.com/honnibal/status/918427934700032000
For numpy with MKL installed by conda the proposed solution doesn't appear to work,
| """ | |
| This script aims to benchmark different parallelization options for pairwise metrics in scikit-learn. | |
| The results can be found in https://github.com/scikit-learn/scikit-learn/issues/8216 | |
| The environement is setup with, | |
| conda create -n sklearn-env scikit-learn==0.18.1 jupyter python==3.5 | |
| and this benchmark should be run with, | |
| from sklearn.metrics import pairwise_distances | |
| from sklearn.externals.joblib import Parallel, delayed, dump, load | |
| import numpy as np | |
| import tempfile | |
| import os | |
| from sklearn.utils import gen_even_slices | |
| import shutil | |
| os.environ['TMPDIR'] = '/dev/shm/' | |
| np.random.seed(99999) |
| from math import sqrt, acos | |
| import numpy as np | |
| def make_test_arrays(N, norm=True): | |
| X = np.random.rand(N) | |
| Y = np.random.rand(N) | |
| if norm: | |
| X /= np.linalg.norm(X) | |
| Y /= np.linalg.norm(Y) | |
| return X, Y |