airawat

package com.khanolkar.bda.util
/**
 * @author Anagha Khanolkar
 */

import org.apache.spark.sql.SparkSession
import org.apache.hadoop.fs.{ FileSystem, Path }
import org.apache.hadoop.conf.Configuration
import org.apache.spark.sql._
import com.databricks.spark.avro._

airawat

B5b. Configure Oozie SSH action
Sometimes, you may need to execute jobs on a specific node - instead of any cluster node.
For this you need oozie service user to be able to connect to the node of choice as your workflow user.
# The following documentation details configuring an application ID to execute a SSH action

# In the illustration-
# edge node=cdh-en01
# oozie server=cdh-mn01
# applicaiton ID=akhanolk

airawat

#!/bin/sh
set -x

# create the input file based on size (you can get size pattern by running fdisk -l as root)
# Be sure to exclude the Root disk if it is part of your config. You must edit this file to do so

size=$1
shift;
fdisk -l|grep $size|awk '{print $2}'|sed -e"s/\:$//g" > foo