Skip to content

Instantly share code, notes, and snippets.

View seanorama's full-sized avatar

Sean Roberts seanorama

  • Cloudera (formerly Hortonworks, Rackspace)
  • London, UK
  • X @seano
View GitHub Profile
@seanorama
seanorama / gist:1ccdf4dd4116b4e4c2f4
Created June 26, 2015 08:53
Ambari: Rebalance HDFS from Ambari API

POST /api/v1/clusters/hdp/requests

{
  "RequestInfo": {
    "context": "Rebalance HDFS",
    "command": "REBALANCEHDFS",
    "namenode": "{\"threshold\":\"10\"}"
  },
 "Requests/resource_filters": [
{
"configurations": [
{
"oozie-env": {
"oozie_heapsize": "4096m"
}
}
],
"host_groups": [
{
{
"host_groups": [
{
"name": "host_group_1",
"configurations": [],
"components": [
{
"name": "METRICS_COLLECTOR"
},
{
@seanorama
seanorama / add_host.sh
Last active December 6, 2022 13:51
ambari api
host_new=ip-10-0-0-162.ec2.internal
curl -u admin:admin -i -H X-Requested-By:API -X POST http://${ambari_server}:8080/api/v1/clusters/${cluster_name}/hosts/${host_new}
curl -u admin:admin -i -H X-Requested-By:API -X GET http://${ambari_server}:8080/api/v1/clusters/${cluster_name}/hosts/${host_new}
worker_services="NODEMANAGER JOURNALNODE HDFS_CLIENT DATANODE FLUME_HANDLER METRICS_MONITOR"
for service in ${worker_services}; do
curl -u admin:admin -i -H X-Requested-By:API -X POST http://${ambari_server}:8080/api/v1/clusters/${cluster_name}/hosts/${host_new}/host_components/${service}
done
#!/bin/bash
checkusage() {
[[ $# -eq 0 ]] && err_exit 'Usage: diskcheck.sh DEVICE...'
}
checkperms() {
for disk in $@; do
local f="diskcheck.sh: ${disk}"
[[ -e ${disk} ]] || err_exit "${f}: No such file or directory"
@seanorama
seanorama / gist:07c822ffef430c43a486
Created February 19, 2015 21:58
ambari-custom-repo
## Setting a custom repo for Ambari Stacks via the API
cat > temp-update-repo.json <<-EOF
{
"Repositories" : {
"base_url" : "http://public-repo-1.hortonworks.com/HDP/centos6/2.x/GA/2.2.0.0",
"default_base_url" : "http://public-repo-1.hortonworks.com/HDP/centos6/2.x/GA/2.2.0.0",
"verify_base_url" : true
}
}

Using the Sandbox

Various notes for things to do after the Sandbox is up

/root/start_ambari.sh
/root/start_solr.sh

# these services are never started after first boot:
#!/bin/sh -
set -o errexit
set -o nounset
set -o pipefail
# This script provides an easy install of Ambari
# for RedHat Enterpise Linux 6 & CentOS 6
#
# source at http://github.com/seanorama/ambari-bootstrap
#
# Put expressions that hit more files first.
/home/chroot/.*
/home/[^/]+/Workspaces?/.*/target/.*
/home/[^/]+/Workspaces?/.*/\.pants\.d.*
/home/[^/]+/\.m2/repository/.*
/home/[^/]+/\.ivy2/(?:cache|local|limiter)/.*
/home/[^/]+/.*(?:~|\.(?:log|tmp|bak|lock))
/home/[^/]+/S3/.*
/home/[^/]+/Virtualization/Machines/.*
@seanorama
seanorama / vbox-to-vagrant.md
Last active June 17, 2016 21:52
hdp sandbox vagrant

Create a local Vagrant base box from an existing VirtualBox VM

What

  • Setting up a new development VM should be as easy as 2 commands. And it is:
    • vagrant init; vagrant up
  • In this example we are converting the HDP Sandbox to be used in this way. But the howto will work with any existing VM.

Why