Martin Bukatovic mbukatov

How to install Bigtop 0.8.0 Hadoop on CentOS 6 with Puppet

This page expects that you have already provisioned cluster of CentOS 6.x machines where you would like to deploy 0.8.0 release of Bigtop Hadoop distribution using Puppet recipes from bigtop-deploy.

Requirements of 0.8.0 release:

x86_64 is the only supported architecture
deployment scripts requires puppet 2.7+/3.x (see details below)

Software Collections

Core packages:

scl-utils: for sc usage (scl command line tool)
scl-utils-build: for sc building (rpm macros, ...)

Example of sc:

The script runs the change in place, but you can check what it will do via dry-run mode:

$ tree /home/martin/projects/webstr/webstr/common
/home/martin/projects/webstr/webstr/common
├── __init__.py
├── models
│   ├── actionpopups.py
│   ├── containers.py
│   ├── dialogs.py

Ceph OSD Journal notes

OSD Journal configuration, upstream:

OSD Journal configuration, RHCS 2 (downstream):

https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/2/html/configuration_guide/osd_configuration_reference#journal_settings

Default behaviour with DocInfo transformation:

$ rst2pseudoxml nodocinfo_example.rst
<document ids="no-docinfo-example" names="no\ docinfo\ example" source="nodocinfo_example.rst" title="No DocInfo Example">
    <title>
        No DocInfo Example
    <docinfo>
        <author>
            mbukatov
        <date>

Tendrl API Cleanup

List of issues we have no option to not address:

	---
	- hosts: bigtop_master
	user: root
	vars:
	state: stopped
	tasks:
	- name: Hadoop services on master node
	service: name={{ item }} enabled=true state={{ state }}
	with_items:
	- hadoop-hdfs-datanode

	#!/usr/bin/env python3
	# -- coding: utf8 --

	import os
	import pprint
	import sys

	TENDRL_PROJECTS = [
	"performance_monitoring",
	"alerting",

	$ find . -type f \| egrep -v '\.(tox\|env\|git\|egg-info)' \| egrep -v '(pyc\|png\|gz)$' \| xargs -n 1 ./check-last-newline.sh 2>/dev/null # sed 's/.\.\(.\)$/\1/' \| sort \| uniq
	./alerting/tendrl/alerting/tests/README.rst
	./alerting/etc/tendrl_definitions_performance_monitoring.yaml
	./alerting/.travis.yml
	./node_agent/tendrl/node_agent/tests/test_configure_uuid.sample
	./node_agent/tendrl/node_agent/tests/README.rst
	./node_agent/.travis.yml
	./node_agent/tendrl-noded.service
	./tendrl-site/source/javascripts/vendor/jquery.backstretch.min.js
	./tendrl-site/source/javascripts/vendor/codemirror.js

	$ repoquery --repofrompath="tendrl-deps,https://copr-be.cloud.fedoraproject.org/results/tendrl/dependencies/epel-7-x86_64/" --repoid=tendrl-deps --all --qf='%{name}'
	ansible
	gstatus
	gstatus-debuginfo
	hwinfo
	hwinfo-debuginfo
	hwinfo-devel
	libx86emu-debuginfo
	libx86emu-devel
	libx86emu1