Skip to content

Instantly share code, notes, and snippets.

View darrenfu's full-sized avatar
🏠
Working from home

Darren Fu darrenfu

🏠
Working from home
View GitHub Profile
@darrenfu
darrenfu / gist:683d5f2ab096b1e85fcb075fb2b39772
Created September 5, 2023 22:57
mvn versions:set -DnewVersion=jolly-20230905155251-0edf92498e -DgenerateBackupPoms=false
➜ prestodb_presto git:(master) mvn versions:set -DnewVersion=jolly-20230905155251-0edf92498e -DgenerateBackupPoms=false -X
Apache Maven 3.9.4 (dfbb324ad4a7c8fb0bf182e6d91b0ae20e3d2dd9)
Maven home: /opt/homebrew/Cellar/maven/3.9.4/libexec
Java version: 1.8.0_382, vendor: Amazon.com Inc., runtime: /Users/dofu/.sdkman/candidates/java/8.0.382-amzn/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "13.5", arch: "aarch64", family: "mac"
[DEBUG] Reading global settings from /opt/homebrew/Cellar/maven/3.9.4/libexec/conf/settings.xml
[DEBUG] Reading user settings from /Users/dofu/.m2/settings.xml
[DEBUG] Created adapter factory; available factories [file-lock, rwlock-local, semaphore-local, noop]; available name mappers [discriminating, file-gav, file-hgav, file-static, gav, static]
[DEBUG] Using manager EnhancedLocalRepositoryManager with priority 10.0 for /Users/dofu/.m2/repository
This file has been truncated, but you can view the full file.
ONARY MAP<SMALLINT,DATE>: 100 elements, 6 nulls], [MAP MAP<SMALLINT,DATE>: 100 elements, no nulls]
I20230112 13:06:17.734676 9959358 ExpressionVerifier.cpp:34] [MAP MAP<SMALLINT,DATE>: 100 elements, no nulls]
I20230112 13:06:17.734683 9959358 ExpressionVerifier.cpp:34] [CONSTANT MAP<SMALLINT,DATE>: 100 elements, 10 elements starting at 110 {[110->86] [86->1] 18704 => 1654239-12-16, [111->199] [199->138] 10635 => 4369173-05-09, [112->189] [189->144] 29071 => 3422821-04-12, [113->219] [219->205] 12970 => 4644003-08-31, [114->175] [175->175] 24213 => 3046735-10-26, ...}], [MAP MAP<SMALLINT,DATE>: 23 elements, no nulls]
I20230112 13:06:17.734704 9959358 ExpressionVerifier.cpp:34] [DICTIONARY MAP<SMALLINT,DATE>: 100 elements, 9 nulls], [MAP MAP<SMALLINT,DATE>: 100 elements, no nulls]
I20230112 13:06:17.734712 9959358 ExpressionVerifier.cpp:34] [DICTIONARY MAP<SMALLINT,DATE>: 100 elements, 11 nulls], [DICTIONARY MAP<SMALLINT,DATE>: 100 elements, 11 nulls], [MAP MAP<SMALLINT,DATE>: 100 elements, no nulls]
I20230
@darrenfu
darrenfu / gist:921f20e8c4b2a783f4d2ddd339a7516b
Created January 12, 2023 00:01
DecodedVector::wrap() error
/Users/dofu/git/velox/cmake-build-debug/velox/vector/tests/velox_vector_test --gtest_filter=DecodedVectorTest.dictionaryOverConstant:DecodedVectorTest/*.dictionaryOverConstant:DecodedVectorTest.dictionaryOverConstant/*:*/DecodedVectorTest.dictionaryOverConstant/*:*/DecodedVectorTest/*.dictionaryOverConstant --gtest_color=no
Testing started at 4:00 PM ...
Running main() from /Users/dofu/git/velox/third_party/googletest/googletest/src/gtest_main.cc
WARNING: Logging before InitGoogleLogging() is written to STDERR
E20230111 16:00:40.975656 9289408 Exceptions.h:68] Line: /Users/dofu/git/velox/./velox/vector/DictionaryVector-inl.h:50, Function:setInternalState, Expression: rawIndices_[i] < dictionaryValues_->size() (10 vs. 10) Dictionary index must be less than base vector's size. Index: 5., Source: RUNTIME, ErrorCode: INVALID_STATE
E20230111 16:00:40.989362 9289408 Exceptions.h:68] Line: /Users/dofu/git/velox/./velox/vector/DictionaryVector-inl.h:50, Function:setInternalState, Expression: rawIndices_[i] < dictiona
@darrenfu
darrenfu / gist:ff09c69ed225c23cad332e306ceb787d
Created December 24, 2022 09:00
shuffle fuzzer out of bound error
This file has been truncated, but you can view the full file.
2095739841461, 0.39092737436294556, 0.5049450397491455, 0.5175980925559998, ...}),"c1"),"c2")
I20221224 00:59:44.811956 19107477 ExpressionVerifier.cpp:32] 3 vectors as input:
I20221224 00:59:44.811962 19107477 ExpressionVerifier.cpp:34] [FLAT BIGINT: 100 elements, 12 nulls]
I20221224 00:59:44.811969 19107477 ExpressionVerifier.cpp:34] [ARRAY ARRAY<REAL>: 100 elements, no nulls]
I20221224 00:59:44.811976 19107477 ExpressionVerifier.cpp:34] [FLAT INTEGER: 100 elements, 9 nulls]
I20221224 00:59:44.812780 19107477 ExpressionVerifier.cpp:69] All results match.
I20221224 00:59:44.812791 19107477 ExpressionFuzzer.cpp:864] ==============================> Done with iteration 36
I20221224 00:59:44.812819 19107477 ExpressionFuzzer.cpp:798] ==============================> Started iteration 37 (seed: 2249770819)
I20221224 00:59:44.820464 19107477 ExpressionVerifier.cpp:80] Executing expression: distinct_from(2q6XNsn7oinlW@%(u6N;!z_##L[kpk0B.EM;O~gH!*kxI,subscript(map_concat(1 elements starting at 290 {[290->251] [251-
@darrenfu
darrenfu / gist:ce75a0146c2c9e51f90ede17bce14a58
Created December 19, 2022 19:30
map_from_entries - only failure
E20221219 11:29:13.193938 14925682 FuzzerToolkit.cpp:131] Only one path threw exception:
libc++abi: terminating with uncaught exception of type facebook::velox::VeloxUserError: Exception: VeloxUserError
Error Source: USER
Error Code: INVALID_ARGUMENT
Reason: map entry cannot be null
Retriable: False
Expression: !decodedRow.isNullAt(offset + i)
Context: map_from_entries(10 elements starting at 220 {{[220->270] 954688-06-16, [220->112] [112->154] 10 elements starting at 1540 {[1540->2550] [2550->2017] [2017->1492] 690185702118130641 => [1540->1593] [1593->754] [754->1648] 0.18735340959392488, [1541->2132] [2132->842] [842->290] 3628103850737398603 => [1541->343] [343->1397] [1397->486] 0.2362087769433856, [1542->1104] [1104->1372] [1372->1563] 4024650605316485715 => [1542->2438] [2438->2136] [2136->2208] 0.8233278426341712, [1543->2305] [2305->660] [660->1342] 8231708222516646461 => [1543->535] [535->1610] [1610->2586] 0.11026319046504796, [1544->1018] [1018->60] [60->1541] 2930196863899139974 => [1544->301] [3
@darrenfu
darrenfu / gist:7bfb03c4c106814133e3d93fef84a311
Created December 19, 2022 18:38
map_from_entries fuzzer ci failure
E1217 05:54:52.563339 130454 Exceptions.h:68] Line: ../../velox/functions/prestosql/MapFromEntries.cpp:98, Function:operator(), Expression: !rowKeyVector->isNullAt(decodedRow.index(offset + i)) map key cannot be null, Source: USER, ErrorCode: INVALID_ARGUMENT
I1217 05:54:52.563899 130454 ExpressionVerifier.cpp:133] Modified inputs for common eval path:
I1217 05:54:52.563910 130454 ExpressionVerifier.cpp:32] 4 vectors as input:
I1217 05:54:52.563915 130454 ExpressionVerifier.cpp:34] [LAZY MAP<BIGINT,REAL>: 100 elements, no nulls]
I1217 05:54:52.563923 130454 ExpressionVerifier.cpp:34] [ARRAY ARRAY<BOOLEAN>: 100 elements, no nulls]
I1217 05:54:52.563930 130454 ExpressionVerifier.cpp:34] [ARRAY ARRAY<TIMESTAMP>: 100 elements, no nulls]
I1217 05:54:52.563936 130454 ExpressionVerifier.cpp:34] [LAZY BIGINT: 100 elements, no nulls]
E1217 05:54:52.564173 130454 Exceptions.h:68] Line: ../../velox/functions/prestosql/MapFromEntries.cpp:98, Function:operator(), Expression: !rowKeyVector->isNullAt(decodedRow.index(o
@darrenfu
darrenfu / gist:a6dd64cc2589b015918db4fe3fc16e16
Created December 14, 2022 06:17
CI failure with map_from_entries with nulls
I1212 19:12:22.016161 113105 ExpressionVerifier.cpp:80] Executing expression: greatest(subscript(map_from_entries("c0"),"c1"),2011-04-28T07:16:09.157640963,from_unixtime("c2"))
I1212 19:12:22.016181 113105 ExpressionVerifier.cpp:32] 3 vectors as input:
I1212 19:12:22.016202 113105 ExpressionVerifier.cpp:34] [CONSTANT ARRAY<ROW<"":TINYINT,"":TIMESTAMP>>: 100 elements, 10 elements starting at 40 {[40->154] {106, null}, [41->122] {119, null}, [42->38] {16, null}, [43->16] {79, null}, [44->149] {31, null}, ...}], [ARRAY ARRAY<ROW<"":TINYINT,"":TIMESTAMP>>: 17 elements, no nulls]
I1212 19:12:22.016237 113105 ExpressionVerifier.cpp:34] [CONSTANT TINYINT: 100 elements, 64]
I1212 19:12:22.016243 113105 ExpressionVerifier.cpp:34] [FLAT DOUBLE: 100 elements, 10 nulls]
E1212 19:12:22.017853 113105 Exceptions.h:68] Line: ../.././velox/functions/prestosql/MapFunctions.h:40, Function:call, Expression: !inputArray.mayHaveNulls() map entry cannot be null, Source: USER, ErrorCode: INVALID_ARGUMENT
E1212 19:12:22.020548 113
[14/968] Building CXX object velox/common/base/CMakeFiles/velox_common_base.dir/RuntimeMetrics.cpp.o
FAILED: velox/common/base/CMakeFiles/velox_common_base.dir/RuntimeMetrics.cpp.o
ccache /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -DBOOST_ATOMIC_DYN_LINK -DBOOST_ATOMIC_NO_LIB -DBOOST_CONTEXT_DYN_LINK -DBOOST_CONTEXT_NO_LIB -DBOOST_FILESYSTEM_DYN_LINK -DBOOST_FILESYSTEM_NO_LIB -DBOOST_PROGRAM_OPTIONS_DYN_LINK -DBOOST_PROGRAM_OPTIONS_NO_LIB -DBOOST_REGEX_DYN_LINK -DBOOST_REGEX_NO_LIB -DBOOST_SYSTEM_DYN_LINK -DBOOST_SYSTEM_NO_LIB -DBOOST_THREAD_DYN_LINK -DBOOST_THREAD_NO_LIB -DFMT_SHARED -DGFLAGS_IS_A_DLL=0 -I/opt/homebrew/opt/icu4c/include -I/Users/dofu/git/velox/. -I/Users/dofu/git/velox/velox/external/xxhash -I/Users/dofu/git/velox/third_party/xsimd/include -isystem /Users/dofu/homebrew/include -isystem /Users/dofu/git/velox/velox -isystem /Users/dofu/git/velox/velox/external -isystem /Users/dofu/git/velox/velox/external/duckdb -isystem /Users/dofu/git/velox/vel
@darrenfu
darrenfu / gist:5644f03a5c822513178767999eb03f67
Created December 9, 2022 07:42
XCode CLT lack libraries
[19/968] Building CXX object velox/common/base/CMakeFiles/velox_common_base.dir/SimdUtil.cpp.o
FAILED: velox/common/base/CMakeFiles/velox_common_base.dir/SimdUtil.cpp.o
ccache /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -DBOOST_ATOMIC_DYN_LINK -DBOOST_ATOMIC_NO_LIB -DBOOST_CONTEXT_DYN_LINK -DBOOST_CONTEXT_NO_LIB -DBOOST_FILESYSTEM_DYN_LINK -DBOOST_FILESYSTEM_NO_LIB -DBOOST_PROGRAM_OPTIONS_DYN_LINK -DBOOST_PROGRAM_OPTIONS_NO_LIB -DBOOST_REGEX_DYN_LINK -DBOOST_REGEX_NO_LIB -DBOOST_SYSTEM_DYN_LINK -DBOOST_SYSTEM_NO_LIB -DBOOST_THREAD_DYN_LINK -DBOOST_THREAD_NO_LIB -DFMT_LOCALE -DGFLAGS_IS_A_DLL=0 -I/Users/dofu/homebrew/opt/icu4c/include -I/Users/dofu/git/velox/. -I/Users/dofu/git/velox/velox/external/xxhash -I/Users/dofu/git/velox/third_party/xsimd/include -isystem /Users/dofu/homebrew/include -isystem /Users/dofu/git/velox/velox -isystem /Users/dofu/git/velox/velox/external -isystem /Users/dofu/git/velox/velox/external/duckdb -isystem /Users/dofu/git/velox/velox/ex
@darrenfu
darrenfu / gist:5c4b224a6306bdb431aeab81e52da2af
Last active June 28, 2023 22:41
_EVP_MD_get_size Undefined symbols for architecture arm64 build error
====================[ Build | velox_in_10_min_demo | Debug ]====================
"/Users/dofu/Library/Application Support/JetBrains/Toolbox/apps/CLion/ch-0/223.7571.171/CLion.app/Contents/bin/cmake/mac/bin/cmake" --build /Users/dofu/git/velox/cmake-build-debug --target velox_in_10_min_demo -j 8
[1/1] Linking CXX executable velox/exec/tests/velox_in_10_min_demo
FAILED: velox/exec/tests/velox_in_10_min_demo
: && /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -mcpu=apple-m1+crc -std=c++17 -fvisibility=hidden -fvisibility=hidden -fvisibility-inlines-hidden -D USE_VELOX_COMMON_BASE -D HAS_UNCAUGHT_EXCEPTIONS -g -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.0.sdk -mmacosx-version-min=12.6 -Wl,-search_paths_first -Wl,-headerpad_max_install_names -L/opt/homebrew/opt/[email protected]/lib -Wl,-flat_namespace velox/connectors/tpch/CMakeFiles/velox_tpch_connector.dir/TpchConnector.cpp.o velox/exec/tests/CMakeFi