Monitoring dashboards on cloud

Dash-boarding Cloud Metrics

This gist contains everything you need to install StatsD and Graphite on CentOS 6.3. Unless I forgot something. If I did, shoot a reminder email to noah at one more bug dot com. **tl;dr: womm, ymmv, yolo.**

I (mostly) followed the steps shown in the [EZUnix wiki](http://www.ezunix.org/index.php?title=Install_statsd_and_graphite_on_CentOS_or_RHEL)

And I also referred back to [this gist by Michael Grace](http://geek.michaelgrace.org/2011/09/how-to-install-graphite-on-ubuntu/)

graphite-vhost.conf

WSGISocketPrefix /var/run/wsgi

<VirtualHost *:80>
        ServerName graphite
        DocumentRoot "/opt/graphite/webapp"
        ErrorLog /opt/graphite/storage/log/webapp/error.log
        CustomLog /opt/graphite/storage/log/webapp/access.log common

        # I've found that an equal number of processes & threads tends
        # to show the best performance for Graphite (ymmv).
        WSGIDaemonProcess graphite processes=5 threads=5 display-name='%{GROUP}' inactivity-timeout=120
        WSGIProcessGroup graphite
        WSGIApplicationGroup %{GLOBAL}
        WSGIImportScript /opt/graphite/conf/graphite.wsgi process-group=graphite application-group=%{GLOBAL}

        # XXX You will need to create this file! There is a graphite.wsgi.example
        # file in this directory that you can safely use, just copy it to graphite.wgsi
        WSGIScriptAlias / /opt/graphite/conf/graphite.wsgi 

        Alias /content/ /opt/graphite/webapp/content/
        <Location "/content/">
                SetHandler None
        </Location>

        # XXX In order for the django admin site media to work you
        # must use the path to your django installation, which is
        # probably something like:
        # /usr/lib/python2.6/site-packages/django

        Alias /media/ "/usr/lib/python2.6/site-packages/django/contrib/admin/media/"
        <Location "/media/">
                SetHandler None
        </Location>
        # The graphite.wsgi file has to be accessible by apache. It won't
        # be visible to clients because of the DocumentRoot though.
        <Directory /opt/graphite/conf/>
                Order deny,allow
                Allow from all
        </Directory>

 </VirtualHost>

install-statsd-graphite-centos-6-3.sh

# run as root
#
# based on http://www.ezunix.org/index.php?title=Install_statsd_and_graphite_on_CentOS_or_RHEL

rpm -i http://ftp.uninett.no/linux/epel/6/x86_64/epel-release-6-8.noarch.rpm
yum -y update

yum -y install python-devel.x86_64 python-pip.noarch mod_wsgi.x86_64 mod_python.x86_64 python-sqlite2.x86_64 \
       bitmap.x86_64 pycairo.x86_64 python-zope-interface.x86_64 python-simplejson.x86_64 django-tagging.noarch \
       python-pip.noarch python-twisted-web.x86_64 bitmap bitmap-fonts gcc-c++.x86_64  \
       openssl-devel.x86_64 git.x86_64 make.x86_64

pip-python install carbon
pip-python install graphite-web
pip-python install whisper

pushd /opt/graphite/conf
cp carbon.conf.example carbon.conf
cp storage-schemas.conf.example storage-schemas.conf
cp storage-aggregation.conf.example storage-aggregation.conf
cp graphTemplates.conf.example graphTemplates.conf
cp graphite.wsgi.example graphite.wsgi

# Graphite defaults to the Chicago time zone (because that's Django's default).
cp /opt/graphite/webapp/graphite/local_settings.py.example /opt/graphite/webapp/graphite/local_settings.py
# then uncomment and edit the TIME_ZONE line. Etc./UTC is recommended.


echo '
[stats]
priority = 110
pattern = ^stats\..*
retentions = 10:2160,60:10080,600:262974
' >> storage-schemas.conf

echo '
[counters]
pattern = stats.counts.*
xFilesFactor = 0.0
aggregationMethod = sum
' >> storage-aggregation.conf

echo '
Include "/etc/httpd/vhosts/*.conf"
' >> /etc/httpd/conf/httpd.conf

# replace /etc/httpd/vhosts/graphite-vhost.conf with the
# graphite-vhost.conf from this repo.

chown -R apache:apache /opt/graphite/storage/

# or optionally:
#     chown -R apache:apache /opt/graphite

cd /opt/graphite/webapp/graphite
sudo python manage.py syncdb

# This will create a Django admin user named 'root'. You will also
# have to pick an email address and password.

cd /opt/graphite/
./bin/carbon-cache.py start

# Import was failing for me.

# [root@ip-172-31-21-218 bin]# ./carbon-cache.py start
# /usr/lib64/python2.6/site-packages/zope/__init__.py:3: UserWarning: Module twisted was already imported from /usr/lib64/python2.6/site-packages/twisted/__init__.pyc, but /opt/graphite/lib is being added to sys.path
#  import pkg_resources
# Traceback (most recent call last):
#  File "./carbon-cache.py", line 28, in <module>
#    from carbon.util import run_twistd_plugin
#  File "/opt/graphite/lib/carbon/util.py", line 21, in <module>
#    from twisted.scripts._twistd_unix import daemonize
# ImportError: cannot import name daemonize

# To resolve this issue, I had to do the following steps
# pip install daemonize
# then I opened /opt/graphite/lib/carbon/util.py and changed 
# "from twisted.scripts._twistd_unix import daemonize" 
# to 
# "import daemonize"

## RESULT ##

# [root@ip-172-31-21-218 bin]# ./carbon-cache.py start
# /usr/lib64/python2.6/site-packages/zope/__init__.py:3: UserWarning: Module twisted was already imported from /usr/lib64/python2.6/site-packages/twisted/__init__.pyc, but /opt/graphite/lib is being added to sys.path
#   import pkg_resources
# Starting carbon-cache (instance a)


# disable SELinux enforcement or all you will see is a stack trace to the effect of:
# "DatabaseError: attempt to write a readonly database"
#
# See the comment thread on this answer: http://stackoverflow.com/questions/3319112/sqlite-read-only-database/3330616#3330616
# and also see https://answers.launchpad.net/graphite/+question/178172

echo 0 >/selinux/enforce

# Note that if you do not have the 'graphite' vhost defined in DNS,
# then you need to update your /etc/hosts and map 'graphite' to the
# host IP address. This must be done on both the host and
# client(s). You may have more luck using a Fully Qualified Domain
# Name (FQDN) such as 'graphite.dev'
#
# Start Apache. Graphite should now be running and you should be able
# to see it in a Web browser on port 80
service apache start


cd /tmp
git clone http://github.com/joyent/node.git
cd node
git checkout v0.8
./configure && make && make install

cd /tmp
wget --no-check-certificate http://npmjs.org/install.sh
sh install.sh
/usr/local/bin/npm install express

cd /tmp
git clone http://github.com/etsy/statsd.git
cp -R /tmp/statsd /opt/graphite

cd /opt/graphite/statsd
cp exampleConfig.js local.js

# edit local.js and update the config for graphiteHost

nohup /usr/local/bin/node stats.js local.js &