Skip to content

Instantly share code, notes, and snippets.

View WattsInABox's full-sized avatar

William Watson WattsInABox

  • Maystreet
  • Orlando, FL
View GitHub Profile
@WattsInABox
WattsInABox / install_gnu_parallel_redhat_ubi8.sh
Last active February 3, 2024 16:08
install GNU Parallel with citation removal on Redhat UBI8
cd /tmp \
&& curl --output parallel-latest.tar.bz2 http://ftp.gnu.org/gnu/parallel/parallel-latest.tar.bz2 \
&& tar xjf parallel-latest.tar.bz2 \
&& cd parallel-2* \
&& ./configure && make \
&& make install \
&& cd /tmp \
&& rm -rf parallel-* \
&& mkdir -p ~/.parallel \
&& touch ~/.parallel/will-cite

AWS internal ip's

EMR has components running on internal ip's such as ip-172-23-53-101.ec2.internal. To resolve to the implied public/vpn IP's, setup dnsmasq

Setup dnsmasq on Mac

(You'll need to do something different on windows. Linux should have another installer.)

$ brew install dnsmasq
# a list of all available configuration options in hadoop 2.7 for the s3a filesystem
fs.s3a.access.key
fs.s3a.secret.key
fs.s3a.connection.maximum
fs.s3a.connection.ssl.enabled
fs.s3a.endpoint
fs.s3a.proxy.host
fs.s3a.proxy.port
fs.s3a.proxy.username
@WattsInABox
WattsInABox / gem_make.out
Created October 12, 2015 18:43
gem install google_hash -v 0.8.8 on Mac OS X El Capitan
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
/Users/williamrwatson/.rvm/rubies/ruby-2.1.5/bin/ruby -r ./siteconf20151012-90285-1dtklec.rb extconf.rb
building local copy/version of google sparse/dense hash library
sh configure --prefix=/Users/williamrwatson/.rvm/gems/ruby-2.1.5@o2o/gems/google_hash-0.8.8/ext/local_installed
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... ./install-sh -c -d
checking for gawk... no
checking for mawk... no
@WattsInABox
WattsInABox / configure_sparsehash_2.0.2_os_x_mavericks_llvm_5.1
Created March 13, 2014 14:39
Compile Errors for Sparsehash 2.0.2 on LLVM 5.1 on Mac OS X 10.9
$ gcc -v
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 5.1 (clang-503.0.38) (based on LLVM 3.4svn)
Target: x86_64-apple-darwin13.1.0
Thread model: posix
$ ./configure
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... ./install-sh -c -d
checking for gawk... no
@WattsInABox
WattsInABox / configure_sparsehash_1.8.1_os_x_mavericks_llvm_5.1
Created March 13, 2014 14:38
Compile Errors for Sparsehash 1.8.1 on LLVM 5.1 on Mac OS X 10.9
$ gcc -v
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 5.1 (clang-503.0.38) (based on LLVM 3.4svn)
Target: x86_64-apple-darwin13.1.0
Thread model: posix
$ ./configure
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for gawk... no
checking for mawk... no
@WattsInABox
WattsInABox / .gitignore
Last active July 5, 2024 09:36
Generate Static HTML Website Using Ruby on Rails
# Ignore static version of the site (used to upload error pages to S3 for Heroku errors)
/out
@WattsInABox
WattsInABox / gist:6719217
Created September 26, 2013 19:21
homebrew sqoop failed to build on Mac OS X 10.8.5
$ brew doctor
Warning: "config" scripts exist outside your system or Homebrew directories.
`./configure` scripts often look for *-config scripts to determine if
software packages are installed, and what additional flags to use when
compiling and linking.
Having additional scripts in your path can confuse software installed via
Homebrew if the config script overrides a system or Homebrew provided
script of the same name. We found the following "config" scripts: