Skip to content

Instantly share code, notes, and snippets.

@mbukatov
mbukatov / gist:7ec73ae3a7eaa73a3135
Last active August 29, 2015 14:22
Bigtop 0.8.0 on CentOS

How to install Bigtop 0.8.0 Hadoop on CentOS 6 with Puppet

This page expects that you have already provisioned cluster of CentOS 6.x machines where you would like to deploy 0.8.0 release of Bigtop Hadoop distribution using Puppet recipes from bigtop-deploy.

Requirements of 0.8.0 release:

  • x86_64 is the only supported architecture
  • deployment scripts requires puppet 2.7+/3.x (see details below)
---
- hosts: bigtop_master
user: root
vars:
state: stopped
tasks:
- name: Hadoop services on master node
service: name={{ item }} enabled=true state={{ state }}
with_items:
- hadoop-hdfs-datanode

Software Collections

Core packages:

  • scl-utils: for sc usage (scl command line tool)
  • scl-utils-build: for sc building (rpm macros, ...)

Example of sc:

@mbukatov
mbukatov / tendrl-conf-check.py
Created January 4, 2017 17:40
tendrl-conf-check
#!/usr/bin/env python3
# -*- coding: utf8 -*-
import os
import pprint
import sys
TENDRL_PROJECTS = [
"performance_monitoring",
"alerting",
@mbukatov
mbukatov / check result
Created January 5, 2017 09:17
search for missing ending newlines
$ find . -type f | egrep -v '\.(tox|env|git|egg-info)' | egrep -v '(pyc|png|gz)$' | xargs -n 1 ./check-last-newline.sh 2>/dev/null # sed 's/.*\.\(.*\)$/\1/' | sort | uniq
./alerting/tendrl/alerting/tests/README.rst
./alerting/etc/tendrl_definitions_performance_monitoring.yaml
./alerting/.travis.yml
./node_agent/tendrl/node_agent/tests/test_configure_uuid.sample
./node_agent/tendrl/node_agent/tests/README.rst
./node_agent/.travis.yml
./node_agent/tendrl-noded.service
./tendrl-site/source/javascripts/vendor/jquery.backstretch.min.js
./tendrl-site/source/javascripts/vendor/codemirror.js
@mbukatov
mbukatov / README.rst
Created April 3, 2017 11:59
no-doc-info-example

Default behaviour with DocInfo transformation:

$ rst2pseudoxml nodocinfo_example.rst
<document ids="no-docinfo-example" names="no\ docinfo\ example" source="nodocinfo_example.rst" title="No DocInfo Example">
    <title>
        No DocInfo Example
    <docinfo>
        <author>
            mbukatov
        <date>
@mbukatov
mbukatov / gist:0fadf4d51f5ab4934236c2b26f34c9ec
Created May 5, 2017 17:56
list all rpm package names from given repourl
$ repoquery --repofrompath="tendrl-deps,https://copr-be.cloud.fedoraproject.org/results/tendrl/dependencies/epel-7-x86_64/" --repoid=tendrl-deps --all --qf='%{name}'
ansible
gstatus
gstatus-debuginfo
hwinfo
hwinfo-debuginfo
hwinfo-devel
libx86emu-debuginfo
libx86emu-devel
libx86emu1

Using etcd directly.

Without authentication

etcdctl --endpoints http://${HOSTNAME}:2379 ls /

Enabling password based authentication

@mbukatov
mbukatov / gluster_import.py
Created August 18, 2017 08:13
Tendrl Gluster Import
#!/usr/bin/python3
"""
Import gluster cluster via Tendrl API.
The code here comes from usmqe module, which can't be directly used because
it's tighly integrated with usmqe test and logging modules.
"""
import json