-
-
Save isuruf/aaead06e2b1d801a20910bc44a9ffcdf to your computer and use it in GitHub Desktop.
git clone https://github.com/isuruf/vecLibFort | |
cd vecLibFort | |
conda create -n numpy pytest hypothesis numpy "libblas=*=*netlib" --yes | |
conda activate numpy | |
clang -O3 -c -o vecLibFort.o vecLibFort.c | |
clang -shared -o libblas.3.dylib vecLibFort.o -Wl,-reexport_library,$(xcrun -show-sdk-path)/System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.tbd | |
codesign -s - -f libblas.3.dylib | |
rm $CONDA_PREFIX/lib/libblas.3.dylib | |
rm $CONDA_PREFIX/lib/libcblas.3.dylib | |
cp libblas.3.dylib $CONDA_PREFIX/lib/libblas.3.dylib | |
cp libblas.3.dylib $CONDA_PREFIX/lib/libcblas.3.dylib | |
python -c "import numpy; numpy.test()" |
If you use the openblas build, tests work?
Running the numpy test on the same machine in an openblas env, I get:
10 failed, 11836 passed, 102 skipped, 1205 deselected, 19 xfailed, 3 xpassed, 24 warnings, 444 errors in 86.37s (0:01:26)
the log is even more verbose. Here is the full log:
https://gist.github.com/ogrisel/be1fd7f9d00f32f3c702a10a9886239d
The 10 failures are:
FAILED miniforge3/envs/sklearn-0241/lib/python3.9/site-packages/numpy/core/tests/test_ufunc.py::TestUfuncGenericLoops::test_unary_PyUFunc_O_O_method_full[reciprocal]
FAILED miniforge3/envs/sklearn-0241/lib/python3.9/site-packages/numpy/distutils/tests/test_system_info.py::TestSystemInfoReading::test_compile1
FAILED miniforge3/envs/sklearn-0241/lib/python3.9/site-packages/numpy/distutils/tests/test_system_info.py::TestSystemInfoReading::test_compile2
FAILED miniforge3/envs/sklearn-0241/lib/python3.9/site-packages/numpy/f2py/tests/test_compile_function.py::test_f2py_init_compile[extra_args0]
FAILED miniforge3/envs/sklearn-0241/lib/python3.9/site-packages/numpy/f2py/tests/test_compile_function.py::test_f2py_init_compile[--noopt --debug]
FAILED miniforge3/envs/sklearn-0241/lib/python3.9/site-packages/numpy/f2py/tests/test_compile_function.py::test_f2py_init_compile[]
FAILED miniforge3/envs/sklearn-0241/lib/python3.9/site-packages/numpy/f2py/tests/test_compile_function.py::test_compile_from_strings[program test_f2py\nend program test_f2py0]
FAILED miniforge3/envs/sklearn-0241/lib/python3.9/site-packages/numpy/f2py/tests/test_compile_function.py::test_compile_from_strings[program test_f2py\nend program test_f2py1]
FAILED miniforge3/envs/sklearn-0241/lib/python3.9/site-packages/numpy/ma/tests/test_core.py::TestMaskedArrayInPlaceArithmetics::test_inplace_floor_division_scalar_type
FAILED miniforge3/envs/sklearn-0241/lib/python3.9/site-packages/numpy/ma/tests/test_core.py::TestMaskedArrayInPlaceArithmetics::test_inplace_floor_division_array_type
So the tests that only fail in the Accelerate env are:
FAILED miniforge3/envs/numpy/lib/python3.9/site-packages/numpy/f2py/tests/test_array_from_pyobj.py::TestSharedMemory::test_in_from_2casttype[LONGDOUBLE]
FAILED miniforge3/envs/numpy/lib/python3.9/site-packages/numpy/f2py/tests/test_array_from_pyobj.py::TestSharedMemory::test_f_in_from_23casttype[LONGDOUBLE]
FAILED miniforge3/envs/numpy/lib/python3.9/site-packages/numpy/f2py/tests/test_array_from_pyobj.py::TestSharedMemory::test_c_in_from_23casttype[LONGDOUBLE]
Running the numpy test on the same machine in an openblas env, I get:
You probably need to install compilers
there too.
I did install the compilers in that env (prior to running the numpy tests):
% conda list compilers
# packages in environment at /Users/ogrisel/miniforge3/envs/sklearn-0241:
#
# Name Version Build Channel
compilers 1.1.3 hce30654_0 conda-forge
/Users/ogrisel/miniforge3/envs/sklearn-0241/include/python3.9/Python.h:25:10: fatal error: 'stdio.h' file not found
E #include <stdio.h>
above error is because the env is not activated. (If it is activated, then $SDKROOT
env variable should be set to the SDK dir)
You are right, something was off with this env. I deactivated everything and reactivated and now I get:
=========================== short test summary info ============================
FAILED miniforge3/envs/sklearn-0241/lib/python3.9/site-packages/numpy/core/tests/test_ufunc.py::TestUfuncGenericLoops::test_unary_PyUFunc_O_O_method_full[reciprocal]
FAILED miniforge3/envs/sklearn-0241/lib/python3.9/site-packages/numpy/f2py/tests/test_array_from_pyobj.py::TestSharedMemory::test_in_from_2casttype[LONGDOUBLE]
FAILED miniforge3/envs/sklearn-0241/lib/python3.9/site-packages/numpy/f2py/tests/test_array_from_pyobj.py::TestSharedMemory::test_f_in_from_23casttype[LONGDOUBLE]
FAILED miniforge3/envs/sklearn-0241/lib/python3.9/site-packages/numpy/f2py/tests/test_array_from_pyobj.py::TestSharedMemory::test_c_in_from_23casttype[LONGDOUBLE]
FAILED miniforge3/envs/sklearn-0241/lib/python3.9/site-packages/numpy/ma/tests/test_core.py::TestMaskedArrayInPlaceArithmetics::test_inplace_floor_division_scalar_type
FAILED miniforge3/envs/sklearn-0241/lib/python3.9/site-packages/numpy/ma/tests/test_core.py::TestMaskedArrayInPlaceArithmetics::test_inplace_floor_division_array_type
6 failed, 12284 passed, 102 skipped, 1205 deselected, 19 xfailed, 3 xpassed, 24 warnings in 87.73s (0:01:27)
which are the same failures as with the Accelerate BLAS + netlib LAPACK combo so probably unrelated to BLAS and LAPACK.
I updated the gist with the full log: https://gist.github.com/ogrisel/be1fd7f9d00f32f3c702a10a9886239d
I noticed that I had two nested conda env in the previous run (the sklearn-0241
env nested under my dev
env). Maybe that was the cause of the problem...
Or maybe I installed compilers
in sklearn-0241
after the creation and activation of sklearn-0241
without reactivating it. I cannot remember.
I get AssertionErrors from multiple important routines using this on my M1 mac. However from what I have been reading numpy and scipy blocked the accelerate backend due to numerical issues.
=============================== short test summary info ================================ FAILED core/tests/test_ufunc.py::TestUfuncGenericLoops::test_unary_PyUFunc_O_O_method_full[reciprocal] FAILED f2py/tests/test_array_from_pyobj.py::TestSharedMemory::test_in_from_2casttype[LONGDOUBLE] FAILED f2py/tests/test_array_from_pyobj.py::TestSharedMemory::test_f_in_from_23casttype[LONGDOUBLE] FAILED f2py/tests/test_array_from_pyobj.py::TestSharedMemory::test_c_in_from_23casttype[LONGDOUBLE] FAILED linalg/tests/test_linalg.py::TestSVDHermitian::test_herm_cases - AssertionErro... FAILED linalg/tests/test_linalg.py::TestPinvHermitian::test_herm_cases - AssertionErr... FAILED linalg/tests/test_linalg.py::TestEighCases::test_herm_cases - AssertionError: ... FAILED linalg/tests/test_linalg.py::TestCholesky::test_basic_property - numpy.linalg.... FAILED ma/tests/test_core.py::TestMaskedArrayInPlaceArithmetics::test_inplace_floor_division_scalar_type FAILED ma/tests/test_core.py::TestMaskedArrayInPlaceArithmetics::test_inplace_floor_division_array_type 10 failed, 12457 passed, 103 skipped, 1208 deselected, 19 xfailed, 3 xpassed, 16 warnings in 98.77s (0:01:38)
Ok the f2py crash went away with the compilers package. I get now get 6 failures. Here is the full report: