Skip to content

Instantly share code, notes, and snippets.

View Zejnilovic's full-sized avatar

Saša Zejnilović Zejnilovic

View GitHub Profile
@Zejnilovic
Zejnilovic / back_up_hdfs.sh
Created May 24, 2019 10:05
Backup HDFS to local FS
#!/bin/bash
now=$(date +"%m_%d_%Y")
file=hadoop_backup_$now
cd ~
mkdir -p $file
hdfs dfs -ls / | grep "^[d-]"| awk '{print $8}' | while read line; do hdfs dfs -get $line $file ; done
zip -r $file.zip $file
@Zejnilovic
Zejnilovic / buildHadoopNativeLibraries.sh
Created May 20, 2019 15:32
A script to build and use Hadoop Native libraries on Mac
# !!CHANGE THIS!!
HVERSION=2.7.5
#!! AND LET IT RUN
brew install gcc autoconf automake libtool cmake snappy gzip bzip2 zlib openssl
cd ~
mkdir -p tmp
cd ~/tmp
@Zejnilovic
Zejnilovic / csv_to_json.rb
Created May 3, 2019 06:08
Convert CSV to JSON
require 'json'
require 'csv'
# Transforms CSV into JSON and saves it next to the CSV.
# Paths to CSV accepted as an input argument
csv_in = ARGV[0]
ARGV.clear
while true
break if !csv_in.nil? && File.exist?(csv_in)
@Zejnilovic
Zejnilovic / switch_m2_settings.pl
Created May 3, 2019 05:48
Script to easily switch between settings files - written for maven, easily changeable to support others.
#!/usr/bin/perl
# Perl script to fast switch between maven settings. Applicable to other programs as well.
# Expects an original file like settings.xml and reference files like settings.xml.alfa1
# and settings.xml.alfa2. These are in the same $m2Folder. When run gives user list of
# postfixes of the original file to chose from (alfa1, alfa2). And that file will
# overwrite the original settings.xml
use warnings;
use File::Copy;