Skip to content

Instantly share code, notes, and snippets.

@phaneesh
phaneesh / log4j.properties
Created March 19, 2019 08:48
Rundeck log rotation
log4j.appender.cmd-logger=org.apache.log4j.RollingFileAppender
log4j.appender.cmd-logger.append=true
log4j.appender.cmd-logger.layout=org.apache.log4j.PatternLayout
log4j.appender.cmd-logger.layout.ConversionPattern=%d{ISO8601} [%t] %-5p %c - %m%n
log4j.appender.cmd-logger.File=/var/log/rundeck/command.log
log4j.appender.cmd-logger.rollingPolicy=org.apache.log4j.rolling.TimeBasedRollingPolicy
log4j.appender.cmd-logger.rollingPolicy.ActiveFileName=/var/log/rundeck/command.log
log4j.appender.cmd-logger.rollingPolicy.FileNamePattern=/var/log/rundeck/command-%d{yyyy-MM-dd}.log
log4j.appender.cmd-logger.MaxBackupIndex=7
@phaneesh
phaneesh / fstab
Created February 19, 2019 10:04
hadoop-os-tuning
/dev/sdb1 /data/hdp01 xfs defaults,noatime,nodiratime,nobarrier 1 2
/dev/sdc1 /data/hdp02 xfs defaults,noatime,nodiratime,nobarrier 1 2
@phaneesh
phaneesh / restart-nn-ha.md
Created December 10, 2018 13:13
Restart NameNode HA

Steps to restart NameNode HA with zero downtime

  • Failover to secondary namenode
$ sudo -u hdfs hdfs haadmin -failover nn1 nn2
  • Restart nn1 from Ambari
  • Failover again to secondary namenode
$ sudo -u hdfs hdfs haadmin -failover nn2 nn1
@phaneesh
phaneesh / update-hive-avm.sh
Last active January 24, 2020 09:15
Upgrade to Tez 0.9.1 on HDP
#!/usr/bin/env bash
HDP_VERSION=2.6.5.0-292
hdfs dfs -put /usr/hdp/${HDP_VERSION}/apache-tez-0.9.1/share/tez.tar.gz /hdp/apps/${HDP_VERSION}/tez/
hdfs dfs -chown -R hdfs:hadoop /hdp
hdfs dfs -chmod -R 555 /hdp/apps/${HDP_VERSION}/tez
hdfs dfs -chmod -R 444 /hdp/apps/${HDP_VERSION}/tez/tez.tar.gz
@phaneesh
phaneesh / sync-ifsc.py
Last active April 3, 2017 02:40
Sync IFSC from RBI Dataset
from __future__ import print_function
import requests
from bs4 import BeautifulSoup
import xlrd
import shutil
import os
import codecs
page = requests.get("https://www.rbi.org.in/Scripts/bs_viewcontent.aspx?Id=2009")
@phaneesh
phaneesh / upgrade-phoenix.sh
Last active August 4, 2019 15:47
Upgrade to Apache Phoenix 4.11.0 on HDP 2.5.3.0-37
#!/usr/bin/env bash
HDP_INSTALL_PATH=/usr/hdp/2.5.5.0-157
cd /tmp
wget http://redrockdigimark.com/apachemirror/phoenix/apache-phoenix-4.12.0-HBase-1.1/bin/apache-phoenix-4.12.0-HBase-1.1-bin.tar.gz
tar -xvf apache-phoenix-4.12.0-HBase-1.1-bin.tar.gz
cp -r apache-phoenix-4.12.0-HBase-1.1-bin ${HDP_INSTALL_PATH}/apache-phoenix-4.12.0
PHOENIX_INSTALL_DIR=${HDP_INSTALL_PATH}/apache-phoenix-4.12.0
chmod -R 755 ${PHOENIX_INSTALL_DIR}
rm -rf apache-phoenix-4.12.0-HBase-1.1-bin.tar.gz
rm -rf apache-phoenix-4.12.0-HBase-1.1-bin
@phaneesh
phaneesh / jenkins-cleanup.groovy
Created October 16, 2016 02:46
Cleanup Jenkins
import hudson.model.*
Hudson.instance.items.each {
println(it.name)
def builds = Jenkins.instance.getItemByFullName(it.name).builds
if(builds.size() > 3) {
def toBeDeleted = builds.subList(3, builds.size())
toBeDeleted.each {
println("\t" +it.number)
it.delete()