ElasticSearch bootstrap script and node configuration for Ubuntu [https://gist.github.com/2050769]

__readme.markdown

Helper files for the https://gist.github.com/2050769 gist

This repository contains files tailored for bootstrapping, installing and configuring ElasticSearch with Chef Solo on the Ubuntu operating system.

After you have downloaded the files, edit the node configuration:

$EDITOR node_ubuntu.json

Set up the SSH connection properly:

HOST=<REPLACE WITH YOUR SERVER IP>
KEY=<REPLACE WITH PATH TO YOUR SSH KEY>
USER=ubuntu
PORT=22
SSH_OPTIONS="-o User=$USER -o IdentityFile=$KEY -o Port=$PORT -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null"

And test that it's working:

ssh $SSH_OPTIONS $HOST "date"

Now, copy the files to the machine and execute the bootstrap script:

scp $SSH_OPTIONS bootstrap_ubuntu.sh node_ubuntu.json solo.rb $HOST:/tmp

time ssh -t $SSH_OPTIONS $HOST "sudo bash /tmp/bootstrap_ubuntu.sh"

Execute the Chef Solo run:

time ssh -t $SSH_OPTIONS $HOST "sudo su - root -c 'chef-solo --node-name elasticsearch-test-ubuntu -j /tmp/node_ubuntu.json'"

Check that the installation and configuration succeeded (you may need to wait couple of minutes until Monit starts):

ssh -t $SSH_OPTIONS $HOST "sudo service elasticsearch status -v"

ssh -t $SSH_OPTIONS $HOST "sudo monit reload && sudo monit status -v"

curl http://USERNAME:PASSWORD@$HOST:8080

Now, you can eg. test that Monit is, in fact, working. Kill the ElasticSearch process:

ssh -t $SSH_OPTIONS $HOST "cat '/usr/local/var/run/elasticsearch/elasticsearch_test_ubuntu.pid' | xargs -0 sudo kill -9"

Wait couple of minutes and check the status:

ssh -t $SSH_OPTIONS $HOST "sudo monit reload && sudo monit status -v"

See the original gist for more information.

bootstrap_ubuntu.sh

echo -e "\nInstalling development dependencies and essential tools..." \
        "\n===============================================================================\n"
# Update packages
apt-get update --yes --fix-missing
# Install build tools
apt-get install build-essential --yes --fix-missing
# Install utilities
apt-get install vim screen curl git-core --yes --fix-missing
# Make sure we are running a Bash shell
ln -sf /bin/bash /bin/sh

echo -e "\nInstalling Ruby and Rubygems..." \
        "\n===============================================================================\n"
# Install Ruby dependencies
apt-get install bison zlib1g-dev libopenssl-ruby1.9.1 libssl-dev libreadline5-dev libncurses5-dev libyaml-0-2 file --yes --fix-missing
# Install Ruby
apt-get install ruby1.9.1 ruby1.9.1-dev --yes --fix-missing
# Install Rubygems
apt-get install rubygems1.9.1 --yes --fix-missing
# "Fix Debian"
ln -nfs /usr/bin/ruby1.9.1 /usr/local/bin/ruby
echo "PATH=$PATH:/var/lib/gems/1.9.1/bin/" > /etc/profile.d/rubygems.sh
source /etc/profile.d/rubygems.sh
# Install the JSON gem
gem install json --no-ri --no-rdoc

echo -e "\nInstalling and bootstrapping Chef..." \
        "\n===============================================================================\n"
test -d "/opt/chef" || curl -# -L http://www.opscode.com/chef/install.sh | sudo bash -s -- -v 10.14.4

mkdir -p /etc/chef/
mkdir -p /var/chef-solo/site-cookbooks
mkdir -p /var/chef-solo/cookbooks

if test -f /tmp/solo.rb;   then mv /tmp/solo.rb /etc/chef/solo.rb;     fi
if test -d /tmp/data_bags; then mv /tmp/data_bags /etc/chef/data_bags; fi

echo -e "\nDownloading cookbooks..." \
        "\n===============================================================================\n"
if ! test -d /var/chef-solo/site-cookbooks/java; then
curl -# -L -k http://s3.amazonaws.com/community-files.opscode.com/cookbook_versions/tarballs/1421/original/java.tgz  | tar xz -C /var/chef-solo/site-cookbooks/
fi

if ! test -d /var/chef-solo/site-cookbooks/monit; then
curl -# -L -k http://s3.amazonaws.com/community-files.opscode.com/cookbook_versions/tarballs/915/original/monit.tgz | tar xz -C /var/chef-solo/site-cookbooks/
fi

if ! test  -d /var/chef-solo/site-cookbooks/nginx; then
curl -# -L -k http://s3.amazonaws.com/community-files.opscode.com/cookbook_versions/tarballs/1413/original/nginx.tgz | tar xz -C /var/chef-solo/site-cookbooks
fi

if ! test -d /var/chef-solo/cookbooks/elasticsearch; then
git clone git://github.com/karmi/cookbook-elasticsearch.git /var/chef-solo/cookbooks/elasticsearch
else
cd /var/chef-solo/cookbooks/elasticsearch
git fetch
git reset origin/master --hard
fi

echo -e "\n*******************************************************************************\n" \
        "Bootstrap finished" \
        "\n*******************************************************************************\n"

node_ubuntu.json

{
  "run_list": [ "recipe[monit]",
                "recipe[java]",
                "recipe[nginx]",
                "recipe[elasticsearch]",
                "recipe[elasticsearch::proxy]" ],

  "elasticsearch" : {
    "cluster_name" : "elasticsearch_test_with_chef",
    "mlockall"     : false,

    "nginx" : {
      "user"  : "www-data",
      "users" : [ { "username" : "USERNAME", "password" : "PASSWORD" } ]
    }
  },

  "monit" : {
    "notify_email" : "<REPLACE WITH YOUR E-MAIL>",
    "mail_format"  : { "from" : "[email protected]", "subject" : "[monit] $SERVICE $EVENT on $HOST", "message" : "$SERVICE $ACTION: $DESCRIPTION" }
  }

}

solo.rb

file_cache_path "/var/chef-solo"
cookbook_path   ["/var/chef-solo/site-cookbooks", "/var/chef-solo/cookbooks"]

It would be best to use https://www.opscode.com/chef/install.sh on line 29 of bootstrap_ubuntu.sh.