Valera weldpua2008

Kafka

get size for all topics on worker

 for topic in $(find   /opt/cloudera/data/*/kafka/ -maxdepth 1 -mindepth 1 -type d -exec basename {} \;| awk -F '-' '{print $1}'| sort |uniq | grep -v consumer_offsets);do

du -csh /opt/cloudera/data/*/kafka/${topic}*| grep total| awk '{print "topic '$topic' "$1}'
done

FIXING RUNNING [LEADER] AND MISSING

Server names cl[0-9]*-hdp Master hdp-m-03.local:7051

######  missing #######

mkdir -p /tmp/test_kudu_data
rm -f /tmp/test_kudu_data/xx*
cd /tmp/test_kudu_data

HOW TO ADD / REPLACE DISK

Find out list of physical disks installed in server

# Download https://github.com/npinto/megaclisas-status/blob/master/megaclisas-status
/opt/MegaRAID/MegaCli/megaclisas-status
-- Controller information --
-- ID | H/W Model                  | RAM    | Temp | BBU    | Firmware
c0    | AVAGO MegaRAID SAS 9361-8i | 1024MB | 58C  | Good   | FW: 24.15.0-0021

Percona Toolkit

pt-online-schema-change

Add Column:

pt-online-schema-change --no-check-replication-filters --execute --alter "ADD COLUMN cost DECIMAL(13,3) UNSIGNED NOT NULL DEFAULT '0.000'" u=compare,p=**,D=vrtb,t=playersDailyStat

Add Index:

pt-online-schema-change --recursion-method=none --no-check-replication-filters --execute --alter "add KEY day (day)" u=root,p=**,D=vrtb,t=playerVastsStatsByDomain --set-vars="SQL_LOG_BIN=0,innodb_lock_wait_timeout=3"

How to use pt-query-digest to Analyze MySQL queries from logs, processlist, and tcpdump.

Report the slowest queries from slow.log:

	#!/bin/env python
	##############################
	# Dependency:
	# yum install MySQL-python -y
	##############################
	import timeit
	import time
	import gc

	loops = 10

	#!/usr/bin/env bash
	#################################################################################################
	# Query the NASA image and video library for all assets related to user by default(Ilan Ramon).
	# From the results make a list of images with file size larger than by default (1000 kb).
	# Author: weldpua2008@gmail.com
	# Requires https://stedolan.github.io/jq/download/
	##################################################################################################
	user="Ilan Ramon"
	parallel_executions=4
	limit_images=$(( 1024 * 1024 ))

	apt-get update && \
	export BUILD_SRC="https://downloads.powerdns.com/releases/pdns-4.0.1.tar.bz2" && \
	export BUILD_DEP="g++ libboost-all-dev libtool make pkg-config libmysqlclient-dev libssl-dev" && \
	export UNINSTABLE_BUILD_TOOLS="wget bzip2" && \
	apt-get install -y $BUILD_DEP $UNINSTABLE_BUILD_TOOLS && \

	wget $BUILD_SRC -O pdns.tar.bz2 && \
	export PDNS_BUILD_TMP="pdns-src" && \
	mkdir -p $PDNS_BUILD_TMP && \
	tar jxf pdns.tar.bz2 --strip-components=1 -C $PDNS_BUILD_TMP && \

	crash> vm
	PID: 11717 TASK: ffff88115f0b2ab0 CPU: 2 COMMAND: "nginx"
	MM PGD RSS TOTAL_VM
	ffff881cfa6a2b80 ffff881a3140f000 1461304k 4936428k
	VMA START END FLAGS FILE
	ffff882066af1250 400000 54f000 8001875 /usr/local/openresty/nginx/sbin/nginx
	ffff882066af13e0 74f000 76d000 8101873 /usr/local/openresty/nginx/sbin/nginx
	ffff882066af1ae8 76d000 78f000 100073
	ffff882066af1958 1d52000 5208000 100073
	ffff8810040843e0 5208000 a4ea000 100073

	#!/bin/bash
	#################################################################################
	# This tool can sample backtraces for memory allocations based on glibc's builtins
	# (malloc, calloc, realloc) that have not been freed (via free) in the sampling time period.
	#################################################################################
	# Author Valeriy Soloviov <weldpua2008@gmail.com>

	ZABBIX_SCHED_PERIOD=600

	PROFILE_FILENAME=/tlvmedia/code/www/sample-bt-leaks_current.svg