seanorama

##--------------------------------------------
## generate ssh key
ssh-keygen -t rsa -b 4096 -o -a 64 -N "" -f ~/.ssh/id_rsa

##--------------------------------------------
## install salt-ssh
##   - NOTE: check if customer already has salt configured.
##     - If so, use it or configure a Saltfile under our $HOME

seanorama

Partitioning Hadoop disks.

This will vary greatly depending on the host and disk layout

Considerations:

• We mount hadoop data disks at /hadoopfs/fs1, fs2, ...
  • We chattr +i any mount dirs, before mounting, such that / won't fill in the case of a mount failing.
• When there is not a dedicated Hadoop disk, we create /var/lib/hadoop and symlink it to /hadoop.
• To make sure nothing ends up on / we symlink /hadoop

seanorama

## -----------------------------------------
## Enable required yum repos in redhat/rhel7

## Get names of repos:
sudo subscription-manager repos --list
sudo yum repolist all

## Look for the 'extra', 'optional' and 'oracle-java' packages.
##   - oracle-java may not be available

seanorama

#!/usr/bin/env bash
##
## Shell script to fetch and trust the CA certificate from hosts
##   - Used on EL6,EL7,AmazonLinux.
##   - Should work on others with a change of package installation
##     method and certificate path.
##
## Author: Sean Roberts <https://github.com/seanorama>
##
## Use:

seanorama

########################################################################
########################################################################
## Run this full block to set the environent for talking to solr via curl
##   - must be on an Ambari Infra host
########################################################################

## authenticate
sudo -u infra-solr -i
keytab=/etc/security/keytabs/ambari-infra-solr.service.keytab
kinit -kt ${keytab} $(klist -kt ${keytab}| awk '{print $NF}'|tail -1)

seanorama

#!/usr/bin/env bash

## Description: Sync Ambari LDAP
##   - Syncs existing users & groups
## Author: Sean Roberts http://github.com/seanorama

set -o errexit
set -o nounset
set -o pipefail

seanorama

#in progress of writing this

1. In Ambari, stop LogSearch service

2. Patch schema files

sudo -iu logsearch
curl -sLO https://gist.github.com/seanorama/3860b48c83ec17a2680678e48d95b657/raw/722fe9e92ad46552521b2df5a5ebd5cd5263c48a/logsearch_audit_logs_managed-schema.patch
curl -sLO https://gist.github.com/seanorama/3860b48c83ec17a2680678e48d95b657/raw/a75fb29916b57bbc07d20ea224e005d34ee40d04/logsearch_hadoop_logs_managed-schema.patch

seanorama

The managed-schema for Ranger Audits needs to be changed by default. But until then, we can replace the existing configuration and collection with the commands below.

Credit:

• https://risdenk.github.io/2017/12/18/ambari-infra-solr-ranger.html
• https://issues.apache.org/jira/browse/RANGER-1938

---

Commands below assume kerberos is used.

seanorama

--- /var/lib/ambari-server/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/alert_check_zeppelin.py (old)
+++ /var/lib/ambari-server/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/alert_check_zeppelin.py (new)
@@ -29,6 +29,7 @@
 config = Script.get_config()

 zeppelin_pid_dir = config['configurations']['zeppelin-env']['zeppelin_pid_dir']
+zeppelin_user = config['configurations']['zeppelin-env']['zeppelin_user']

 RESULT_CODE_OK = 'OK'
 RESULT_CODE_CRITICAL = 'CRITICAL'

seanorama

logrotate for ranger admin and kms access_log

Setup for logrotation:

sudo touch /var/log/ranger/admin/dummy-for-logrotate
sudo chown ranger:hadoop /var/log/ranger/admin/dummy-for-logrotate
sudo touch /var/log/ranger/kms/dummy-for-logrotate
sudo chown kms:hadoop /var/log/ranger/kms/dummy-for-logrotate