Skip to content

Instantly share code, notes, and snippets.

View darrenfu's full-sized avatar
🏠
Working from home

Darren Fu darrenfu

🏠
Working from home
View GitHub Profile
[1345/1495] Building CXX object velox/external/duckdb/tpch/dbgen/CMakeFiles/dbgen.dir/build.cpp.o[K.oParserTest.cpp.oogregateTest.cpp.o
FAILED: velox/external/duckdb/tpch/dbgen/CMakeFiles/dbgen.dir/build.cpp.o
ccache /usr/bin/c++ -DDBNAME=dss -DMAC -DORACLE -DTPCH -I_deps/protobuf-src/src -I../../. -I../../velox/external/xxhash -I../../velox/external/duckdb/tpch/include -I../../velox/external/duckdb/tpch/.. -isystem ../../velox -isystem ../../velox/external -isystem ../../velox/external/duckdb -isystem ../../velox/external/duckdb/tpch/dbgen/include -mavx2 -mfma -mavx -mf16c -mlzcnt -std=c++17 -mbmi2 -D USE_VELOX_COMMON_BASE -D HAS_UNCAUGHT_EXCEPTIONS -Werror -Wall -Wextra -Wno-unused -Wno-unused-parameter -Wno-sign-compare -Wno-ignored-qualifiers -Wno-implicit-fallthrough -Wno-empty-body -Wno-class-memaccess -Wno-comment -Wno-int-in-bool-context -Wno-redundant-move -Wno-array-bounds -Wno-maybe-uninitialized -
@darrenfu
darrenfu / gist:547d30228b1239b2c91abf932863bb9b
Created December 6, 2022 05:44
duckdb "Undefined symbols for architecture arm64" build error
[938/968] Linking CXX executable velox/tpch/gen/tests/velox_tpch_gen_test
FAILED: velox/tpch/gen/tests/velox_tpch_gen_test
: && /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -mcpu=apple-m1+crc -std=c++17 -fvisibility=hidden -D USE_VELOX_COMMON_BASE -D HAS_UNCAUGHT_EXCEPTIONS -Werror -Wall -Wextra -Wno-unused -Wno-unused-parameter -Wno-sign-compare -Wno-ignored-qualifiers -Wno-range-loop-analysis -Wno-mismatched-tags -g -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.0.sdk -mmacosx-version-min=12.6 -Wl,-search_paths_first -Wl,-headerpad_max_install_names -L/opt/homebrew/Cellar/[email protected]/1.1.1s/lib -Wl,-flat_namespace velox/tpch/gen/tests/CMakeFiles/velox_tpch_gen_test.dir/TpchGenTest.cpp.o -o velox/tpch/gen/tests/velox_tpch_gen_test -L/opt/homebrew/opt/icu4c/include/../lib -Wl,-rpath,/opt/homebrew/opt/icu4c/include/../lib -Wl,-rpath,/opt/homebrew/lib velox/tpch/gen/l
@darrenfu
darrenfu / gist:0e1916c85ee5a27e3f7230c8955241ec
Created December 6, 2022 19:21
openssl lib "building for macOS-arm64 but attempting to link with file built for macOS-x86_64" error
====================[ Build | velox_in_10_min_demo | Debug ]====================
"/Users/dofu/Library/Application Support/JetBrains/Toolbox/apps/CLion/ch-0/223.7571.171/CLion.app/Contents/bin/cmake/mac/bin/cmake" --build /Users/dofu/git/velox/cmake-build-debug --target velox_in_10_min_demo -j 8
[334/334] Linking CXX executable velox/exec/tests/velox_in_10_min_demo
FAILED: velox/exec/tests/velox_in_10_min_demo
: && /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -mcpu=apple-m1+crc -std=c++17 -fvisibility=hidden -fvisibility=hidden -fvisibility-inlines-hidden -D USE_VELOX_COMMON_BASE -D HAS_UNCAUGHT_EXCEPTIONS -g -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.0.sdk -mmacosx-version-min=12.6 -Wl,-search_paths_first -Wl,-headerpad_max_install_names -L/opt/homebrew/opt/[email protected]/lib -Wl,-flat_namespace velox/connectors/tpch/CMakeFiles/velox_tpch_connector.dir/TpchConnector.cpp.o velox/exec/tests/CM
@darrenfu
darrenfu / gist:5c4b224a6306bdb431aeab81e52da2af
Last active June 28, 2023 22:41
_EVP_MD_get_size Undefined symbols for architecture arm64 build error
====================[ Build | velox_in_10_min_demo | Debug ]====================
"/Users/dofu/Library/Application Support/JetBrains/Toolbox/apps/CLion/ch-0/223.7571.171/CLion.app/Contents/bin/cmake/mac/bin/cmake" --build /Users/dofu/git/velox/cmake-build-debug --target velox_in_10_min_demo -j 8
[1/1] Linking CXX executable velox/exec/tests/velox_in_10_min_demo
FAILED: velox/exec/tests/velox_in_10_min_demo
: && /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -mcpu=apple-m1+crc -std=c++17 -fvisibility=hidden -fvisibility=hidden -fvisibility-inlines-hidden -D USE_VELOX_COMMON_BASE -D HAS_UNCAUGHT_EXCEPTIONS -g -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.0.sdk -mmacosx-version-min=12.6 -Wl,-search_paths_first -Wl,-headerpad_max_install_names -L/opt/homebrew/opt/[email protected]/lib -Wl,-flat_namespace velox/connectors/tpch/CMakeFiles/velox_tpch_connector.dir/TpchConnector.cpp.o velox/exec/tests/CMakeFi
@darrenfu
darrenfu / gist:5644f03a5c822513178767999eb03f67
Created December 9, 2022 07:42
XCode CLT lack libraries
[19/968] Building CXX object velox/common/base/CMakeFiles/velox_common_base.dir/SimdUtil.cpp.o
FAILED: velox/common/base/CMakeFiles/velox_common_base.dir/SimdUtil.cpp.o
ccache /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -DBOOST_ATOMIC_DYN_LINK -DBOOST_ATOMIC_NO_LIB -DBOOST_CONTEXT_DYN_LINK -DBOOST_CONTEXT_NO_LIB -DBOOST_FILESYSTEM_DYN_LINK -DBOOST_FILESYSTEM_NO_LIB -DBOOST_PROGRAM_OPTIONS_DYN_LINK -DBOOST_PROGRAM_OPTIONS_NO_LIB -DBOOST_REGEX_DYN_LINK -DBOOST_REGEX_NO_LIB -DBOOST_SYSTEM_DYN_LINK -DBOOST_SYSTEM_NO_LIB -DBOOST_THREAD_DYN_LINK -DBOOST_THREAD_NO_LIB -DFMT_LOCALE -DGFLAGS_IS_A_DLL=0 -I/Users/dofu/homebrew/opt/icu4c/include -I/Users/dofu/git/velox/. -I/Users/dofu/git/velox/velox/external/xxhash -I/Users/dofu/git/velox/third_party/xsimd/include -isystem /Users/dofu/homebrew/include -isystem /Users/dofu/git/velox/velox -isystem /Users/dofu/git/velox/velox/external -isystem /Users/dofu/git/velox/velox/external/duckdb -isystem /Users/dofu/git/velox/velox/ex
[14/968] Building CXX object velox/common/base/CMakeFiles/velox_common_base.dir/RuntimeMetrics.cpp.o
FAILED: velox/common/base/CMakeFiles/velox_common_base.dir/RuntimeMetrics.cpp.o
ccache /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -DBOOST_ATOMIC_DYN_LINK -DBOOST_ATOMIC_NO_LIB -DBOOST_CONTEXT_DYN_LINK -DBOOST_CONTEXT_NO_LIB -DBOOST_FILESYSTEM_DYN_LINK -DBOOST_FILESYSTEM_NO_LIB -DBOOST_PROGRAM_OPTIONS_DYN_LINK -DBOOST_PROGRAM_OPTIONS_NO_LIB -DBOOST_REGEX_DYN_LINK -DBOOST_REGEX_NO_LIB -DBOOST_SYSTEM_DYN_LINK -DBOOST_SYSTEM_NO_LIB -DBOOST_THREAD_DYN_LINK -DBOOST_THREAD_NO_LIB -DFMT_SHARED -DGFLAGS_IS_A_DLL=0 -I/opt/homebrew/opt/icu4c/include -I/Users/dofu/git/velox/. -I/Users/dofu/git/velox/velox/external/xxhash -I/Users/dofu/git/velox/third_party/xsimd/include -isystem /Users/dofu/homebrew/include -isystem /Users/dofu/git/velox/velox -isystem /Users/dofu/git/velox/velox/external -isystem /Users/dofu/git/velox/velox/external/duckdb -isystem /Users/dofu/git/velox/vel
@darrenfu
darrenfu / gist:a6dd64cc2589b015918db4fe3fc16e16
Created December 14, 2022 06:17
CI failure with map_from_entries with nulls
I1212 19:12:22.016161 113105 ExpressionVerifier.cpp:80] Executing expression: greatest(subscript(map_from_entries("c0"),"c1"),2011-04-28T07:16:09.157640963,from_unixtime("c2"))
I1212 19:12:22.016181 113105 ExpressionVerifier.cpp:32] 3 vectors as input:
I1212 19:12:22.016202 113105 ExpressionVerifier.cpp:34] [CONSTANT ARRAY<ROW<"":TINYINT,"":TIMESTAMP>>: 100 elements, 10 elements starting at 40 {[40->154] {106, null}, [41->122] {119, null}, [42->38] {16, null}, [43->16] {79, null}, [44->149] {31, null}, ...}], [ARRAY ARRAY<ROW<"":TINYINT,"":TIMESTAMP>>: 17 elements, no nulls]
I1212 19:12:22.016237 113105 ExpressionVerifier.cpp:34] [CONSTANT TINYINT: 100 elements, 64]
I1212 19:12:22.016243 113105 ExpressionVerifier.cpp:34] [FLAT DOUBLE: 100 elements, 10 nulls]
E1212 19:12:22.017853 113105 Exceptions.h:68] Line: ../.././velox/functions/prestosql/MapFunctions.h:40, Function:call, Expression: !inputArray.mayHaveNulls() map entry cannot be null, Source: USER, ErrorCode: INVALID_ARGUMENT
E1212 19:12:22.020548 113
@darrenfu
darrenfu / gist:7bfb03c4c106814133e3d93fef84a311
Created December 19, 2022 18:38
map_from_entries fuzzer ci failure
E1217 05:54:52.563339 130454 Exceptions.h:68] Line: ../../velox/functions/prestosql/MapFromEntries.cpp:98, Function:operator(), Expression: !rowKeyVector->isNullAt(decodedRow.index(offset + i)) map key cannot be null, Source: USER, ErrorCode: INVALID_ARGUMENT
I1217 05:54:52.563899 130454 ExpressionVerifier.cpp:133] Modified inputs for common eval path:
I1217 05:54:52.563910 130454 ExpressionVerifier.cpp:32] 4 vectors as input:
I1217 05:54:52.563915 130454 ExpressionVerifier.cpp:34] [LAZY MAP<BIGINT,REAL>: 100 elements, no nulls]
I1217 05:54:52.563923 130454 ExpressionVerifier.cpp:34] [ARRAY ARRAY<BOOLEAN>: 100 elements, no nulls]
I1217 05:54:52.563930 130454 ExpressionVerifier.cpp:34] [ARRAY ARRAY<TIMESTAMP>: 100 elements, no nulls]
I1217 05:54:52.563936 130454 ExpressionVerifier.cpp:34] [LAZY BIGINT: 100 elements, no nulls]
E1217 05:54:52.564173 130454 Exceptions.h:68] Line: ../../velox/functions/prestosql/MapFromEntries.cpp:98, Function:operator(), Expression: !rowKeyVector->isNullAt(decodedRow.index(o
@darrenfu
darrenfu / gist:ce75a0146c2c9e51f90ede17bce14a58
Created December 19, 2022 19:30
map_from_entries - only failure
E20221219 11:29:13.193938 14925682 FuzzerToolkit.cpp:131] Only one path threw exception:
libc++abi: terminating with uncaught exception of type facebook::velox::VeloxUserError: Exception: VeloxUserError
Error Source: USER
Error Code: INVALID_ARGUMENT
Reason: map entry cannot be null
Retriable: False
Expression: !decodedRow.isNullAt(offset + i)
Context: map_from_entries(10 elements starting at 220 {{[220->270] 954688-06-16, [220->112] [112->154] 10 elements starting at 1540 {[1540->2550] [2550->2017] [2017->1492] 690185702118130641 => [1540->1593] [1593->754] [754->1648] 0.18735340959392488, [1541->2132] [2132->842] [842->290] 3628103850737398603 => [1541->343] [343->1397] [1397->486] 0.2362087769433856, [1542->1104] [1104->1372] [1372->1563] 4024650605316485715 => [1542->2438] [2438->2136] [2136->2208] 0.8233278426341712, [1543->2305] [2305->660] [660->1342] 8231708222516646461 => [1543->535] [535->1610] [1610->2586] 0.11026319046504796, [1544->1018] [1018->60] [60->1541] 2930196863899139974 => [1544->301] [3
@darrenfu
darrenfu / gist:ff09c69ed225c23cad332e306ceb787d
Created December 24, 2022 09:00
shuffle fuzzer out of bound error
This file has been truncated, but you can view the full file.
2095739841461, 0.39092737436294556, 0.5049450397491455, 0.5175980925559998, ...}),"c1"),"c2")
I20221224 00:59:44.811956 19107477 ExpressionVerifier.cpp:32] 3 vectors as input:
I20221224 00:59:44.811962 19107477 ExpressionVerifier.cpp:34] [FLAT BIGINT: 100 elements, 12 nulls]
I20221224 00:59:44.811969 19107477 ExpressionVerifier.cpp:34] [ARRAY ARRAY<REAL>: 100 elements, no nulls]
I20221224 00:59:44.811976 19107477 ExpressionVerifier.cpp:34] [FLAT INTEGER: 100 elements, 9 nulls]
I20221224 00:59:44.812780 19107477 ExpressionVerifier.cpp:69] All results match.
I20221224 00:59:44.812791 19107477 ExpressionFuzzer.cpp:864] ==============================> Done with iteration 36
I20221224 00:59:44.812819 19107477 ExpressionFuzzer.cpp:798] ==============================> Started iteration 37 (seed: 2249770819)
I20221224 00:59:44.820464 19107477 ExpressionVerifier.cpp:80] Executing expression: distinct_from(2q6XNsn7oinlW@%(u6N;!z_##L[kpk0B.EM;O~gH!*kxI,subscript(map_concat(1 elements starting at 290 {[290->251] [251-