darrenfu
/ HBaseRowCounter.scala
Last active
April 16, 2019 23:29
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| // Instead of adding argument --files /apache/hbase/conf/hbase-site.xml, just copy `hbase-site.xml` to /apache/spark/conf/ | |
| //spark-shell --executor-memory 20g --executor-cores 1 --num-executors 10 --driver-memory 8g --queue $HADOOP_QUEUE --jars $(for x in `ls -1 /apache/hbase/lib/*.jar`; do readlink -f $x; done | paste -s | sed -e 's/\t/,/g') | |
| import java.io.{DataOutputStream, ByteArrayOutputStream} | |
| import org.apache.hadoop.hbase.client.Scan | |
| import org.apache.hadoop.hbase.HBaseConfiguration | |
| import org.apache.hadoop.hbase.io.ImmutableBytesWritable | |
| import org.apache.hadoop.hbase.client.Result | |
| import org.apache.hadoop.hbase.mapreduce.TableInputFormat | |
| import org.apache.hadoop.hbase.util.Base64 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/usr/bin/python | |
| import sys | |
| import os | |
| class IParser: | |
| def __init__(): | |
| pass | |
| def parse(self, doc, out_file): |
darrenfu
/ cloudSettings
Last active
April 11, 2020 04:02
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| {"lastUpload":"2020-04-11T04:02:35.858Z","extensionVersion":"v3.4.3"} |
darrenfu
/ tmux.cheat
Created
July 5, 2020 18:44
— forked from afair/tmux.cheat
Tmux Quick Reference & Cheat sheet - 2 column format for less scrolling!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ========================================== ========================================== | |
| TMUX COMMAND WINDOW (TAB) | |
| ========================================== ========================================== | |
| List tmux ls List ^b w | |
| New new -s <session> Create ^b c | |
| Attach att -t <session> Rename ^b , <name> | |
| Rename rename-session -t <old> <new> Last ^b l (lower-L) | |
| Kill kill-session -t <session> Close ^b & |
darrenfu
/ install_error.txt
Created
September 25, 2021 03:25
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| -- Found Boost: /usr/local/lib/cmake/Boost-1.76.0/BoostConfig.cmake (found suitable version "1.76.0", minimum required is "1.51.0") found components: context filesystem program_options regex system thread | |
| CMake Error at /usr/local/Cellar/cmake/3.21.3/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:230 (message): | |
| Could NOT find DoubleConversion (missing: DOUBLE_CONVERSION_INCLUDE_DIR) | |
| Call Stack (most recent call first): | |
| /usr/local/Cellar/cmake/3.21.3/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE) | |
| CMake/FindDoubleConversion.cmake:26 (find_package_handle_standard_args) | |
| CMake/folly-deps.cmake:51 (find_package) | |
| CMakeLists.txt:118 (include) |
darrenfu
/ double-conversion_install_succeed.txt
Last active
September 25, 2021 03:52
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| > scripts/setup-macos.sh | |
| ... | |
| + run_and_time install_double_conversion | |
| + install_double_conversion | |
| + github_checkout google/double-conversion v3.1.5 | |
| + local REPO=google/double-conversion | |
| + local VERSION=v3.1.5 | |
| ++ basename google/double-conversion | |
| + local DIRNAME=double-conversion | |
| + cd /Users/dofu/git/velox |
darrenfu
/ gist:b9a5301ec0f603b630ae0d21cf8a8bc3
Created
September 18, 2022 18:48
folly build failure
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| > github_checkout facebook/folly "${FB_OS_VERSION}" | |
| > OPENSSL_ROOT_DIR=$(brew --prefix [email protected]) cmake_install -DBUILD_TESTS=OFF | |
| [172/303] Building CXX object CMakeFiles/folly_base.dir/folly/io/async/AsyncSSLSocket.cpp.o | |
| FAILED: CMakeFiles/folly_base.dir/folly/io/async/AsyncSSLSocket.cpp.o | |
| /Library/Developer/CommandLineTools/usr/bin/c++ -DBOOST_ATOMIC_DYN_LINK -DBOOST_ATOMIC_NO_LIB -DBOOST_CONTEXT_DYN_LINK -DBOOST_CONTEXT_NO_LIB -DBOOST_FILESYSTEM_DYN_LINK -DBOOST_FILESYSTEM_NO_LIB -DBOOST_PROGRAM_OPTIONS_DYN_LINK -DBOOST_PROGRAM_OPTIONS_NO_LIB -DBOOST_REGEX_DYN_LINK -DBOOST_REGEX_NO_LIB -DBOOST_SYSTEM_DYN_LINK -DBOOST_SYSTEM_NO_LIB -DBOOST_THREAD_DYN_LINK -DBOOST_THREAD_NO_LIB -DFMT_LOCALE -DFOLLY_XLOG_STRIP_PREFIXES=\"/Users/dofu/git/velox/scripts/folly:/Users/dofu/git/velox/scripts/folly/_build\" -DGFLAGS_IS_A_DLL=0 -D_GNU_SOURCE -D_REENTRANT -I/Users/dofu/git/velox/scripts/folly -I/Users/dofu/git/velox/scripts/folly/_build -I/opt/homebrew/include -I/usr/local/include -I/opt/homebrew/opt/[email protected]/in |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| /Applications/Xcode_14.1.0_14B47b_fb.app/Contents/Developer/usr/bin/make cmake BUILD_DIR=release BUILD_TYPE=Release && \ | |
| /Applications/Xcode_14.1.0_14B47b_fb.app/Contents/Developer/usr/bin/make build BUILD_DIR=release | |
| mkdir -p _build/release && \ | |
| cmake -B \ | |
| "_build/release" \ | |
| -DTREAT_WARNINGS_AS_ERRORS=1 -DENABLE_ALL_WARNINGS=1 -DVELOX_BUILD_MINIMAL="OFF" -DVELOX_BUILD_TESTING="ON" -DCMAKE_BUILD_TYPE=Release \ | |
| -GNinja -DMAX_LINK_JOBS= -DMAX_HIGH_MEM_JOBS= \ | |
| -DCMAKE_CXX_COMPILER_LAUNCHER=ccache \ | |
| \ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ====================[ Clean | Debug ]=========================================== | |
| "/Users/dofu/Library/Application Support/JetBrains/Toolbox/apps/CLion/ch-0/222.4345.21/CLion.app/Contents/bin/cmake/mac/bin/cmake" --build /Users/dofu/git/velox/cmake-build-debug --target clean -j 8 | |
| [1/1] Cleaning all built files... | |
| Cleaning... 7 files. | |
| Clean finished | |
| ====================[ Build | all | Debug ]===================================== | |
| "/Users/dofu/Library/Application Support/JetBrains/Toolbox/apps/CLion/ch-0/222.4345.21/CLion.app/Contents/bin/cmake/mac/bin/cmake" --build /Users/dofu/git/velox/cmake-build-debug --target all -j 8 | |
| [1/956] Building CXX object velox/dwio/type/fbhive/CMakeFiles/velox_dwio_type_fbhive.dir/HiveTypeSerializer.cpp.o |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| /Applications/Xcode_14.1.0_14B47b_fb.app/Contents/Developer/usr/bin/make cmake BUILD_DIR=release BUILD_TYPE=Release && \ | |
| /Applications/Xcode_14.1.0_14B47b_fb.app/Contents/Developer/usr/bin/make build BUILD_DIR=release | |
| mkdir -p _build/release && \ | |
| cmake -B \ | |
| "_build/release" \ | |
| -DTREAT_WARNINGS_AS_ERRORS=1 -DENABLE_ALL_WARNINGS=1 -DVELOX_BUILD_MINIMAL="OFF" -DVELOX_BUILD_TESTING="ON" -DCMAKE_BUILD_TYPE=Release \ | |
| -GNinja -DMAX_LINK_JOBS= -DMAX_HIGH_MEM_JOBS= \ | |
| -DCMAKE_CXX_COMPILER_LAUNCHER=ccache \ | |
| \ | |