xman1980’s gists

xman1980 / gist:5b4c4638a203b5452211

Created July 2, 2015 08:04

hdfs_fsck_grep_files

hadoop fsck / | egrep -v '^\.+$' | grep -v replica | grep -v Replica | head -n3000 | awk -F: '{print $1}'

xman1980 / gist:033ef9c0932c1174ae83

Created July 20, 2015 07:22

cloudera_manager_api_service_restart

	#https://cloudera.github.io/cm_api/apidocs/v7/index.html
	curl -u 'admin:admin' -X POST -H "Content-Type:application/json" -d '{"items":["hive"]}' 'http://localhost:7180/api/v7/clusters/cluster/services/hive/commands/restart'

xman1980 / gist:11fee34eae2299583c19

Last active February 17, 2016 09:30

move_cloudera-manager_to_mysql

/usr/share/cmf/schema/scm_prepare_database.sh mysql scm scm scmpass

xman1980 / gist:cd234a6759744533fc2b

Last active May 1, 2022 07:36

download java8 using curl

	curl -v -j -k -L -H "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/8u51-b16/jdk-8u51-linux-x64.rpm > jdk-8u51-linux-x64.rpm

	/etc/profile.d/java.sh
	#!/bin/bash
	JAVA_HOME=/usr/java/jdk1.8.0_51/
	PATH=$JAVA_HOME/bin:$PATH
	export PATH JAVA_HOME
	export CLASSPATH=.

	chmod +x /etc/profile.d/java.sh

xman1980 / gist:003a6c1af5a5a2ab50c5

Created August 6, 2015 13:25

grub install on second hdd

	/sbin/grub-install --root-directory=/mnt /dev/sdb

	or

	mount --bind /dev/ /mnt/dev
	mount --bind /proc/ /mnt/proc
	chroot /mnt/
	grub-install /dev/sda

	if The file /mnt/boot/grub/stage1 not read correctly.

xman1980 / gist:867031939b816a991897

Created August 6, 2015 13:29

dump/restore

dump 0uf - /dev/sda1 | (cd /mnt/hd; restore rvf - )

xman1980 / gist:eab29a53396702453a15

Created October 19, 2015 10:30

killall_yarn_jobs

yarn application -list | grep -o '^app[a-z0-9_]*' | xargs -n 1 -I {} sh -c "yarn application -kill {}"

xman1980 / gist:098fe9da7cca8ada8e77

Created October 21, 2015 19:50

hdfs_du_count

sudo -u hdfs hadoop fs -du /user/history | awk '/^[0-9]+/ { print int($1/(1024**3)) " [GB]\t" $3 }'

xman1980 / gist:ce0ab204607dd9693fb2

Created October 26, 2015 08:30

with mechanism MONGODB-CR: AuthenticationFailed MONGODB-CR credentials missing in the user document

	use admin;
	db.system.users.remove({})
	db.system.version.remove({})
	db.system.version.insert({ "_id" : "authSchema", "currentVersion" : 3 })

	var schema = db.system.version.findOne({"_id" : "authSchema"})
	schema.currentVersion = 3
	db.system.version.save(schema)

	use admin

xman1980 / gist:1596157fd5c296a2dd39

Last active November 30, 2015 20:16

password_tar_gz

	Encrypt
	% tar cz folder_to_encrypt \| \
	openssl enc -aes-256-cbc -e > out.tar.gz.enc

	Decrypt

	% openssl aes-256-cbc -d -in out.tar.gz.enc \| tar xz

	Or using gpg

Vitali xman1980