iamshreeram

Set up Druid

Follow the druid doc

Set up superset

Pre-Req

sudo apt-get install libsasl2-dev

iamshreeram

Step 1

Fork repo and clone to your local

git clone https://github.com/your-user-name/pandas.git pandas-yourname
cd pandas-yourname
git remote add upstream https://github.com/pandas-dev/pandas.git

Step 2

Set up conda

iamshreeram

Below are the list of commands to install python3.7.3 in any linux machine

Update the packages and install required dev packages

sudo apt update
sudo apt install build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libreadline-dev libffi-dev wget

Download python from repo and untar

iamshreeram

Installation and validation of pystan with pip on macbook in virtual environment

After long struggle, Below are list of commands that helps in installing pystan and fbprophet

conda create -n stanenv python=3.7 numpy cython
conda activate stanenv
conda install clang_osx-64 clangxx_osx-64 -c anaconda
ls /PATH_OF_ANACONDA_INSTALLATION/anaconda3/envs/stanenv/bin/ | grep clang | grep -i 'apple'

iamshreeram

#!/bin/bash
mondir="/app/log/"

pid=`ps -eaf | grep 'java'| grep -v grep | awk -F' ' '{print $2}'`
echo $(date -d '2 hour ago' "+%m%d%H%M%S")
BUF_DATE=`expr $(date -d '2 hour ago' "+%m%d%H%M%S")`
BUF_DATE="${BUF_DATE:0:6}"
rm $mondir/jcmdr.$pid.$BUF_DATE* 2>/dev/null
rm $mondir/top.$pid.$BUF_DATE* 2>/dev/null

iamshreeram

# Get the list of installed packages 
pip3 freeze > req.txt

# Change the version from == to >=
sed -i 's/==/>=/g' req.txt

# Feed the packages one at a time and upgrade it 
cat req.txt | sed -e '/^\s*#.*$/d' -e '/^\s*$/d' | sudo xargs -n 1 pip3 install --upgrade --ignore-installed --user

iamshreeram

from pykafka import KafkaClient

path="/path/of/log/file.log"
host="ip.ad.dr.ess" 
port=9092
topic="name-of-the-topic"

client = KafkaClient(host:port)
topic = client.topics[topic]
producer = topic.get_producer(sync=True,delivery_reports=True)

iamshreeram

#!/bin/bash
mondir="/tmp"
if [ ! -d $mondir ];then
        mkdir $mondir
fi

DATE=`date "+%Y:%m:%d-%H:%M:%S"`

pid=`ps -eaf | grep 'java'| grep -v grep | awk -F' ' '{print $2}'`
jcmd org.apache.catalina.startup.Bootstrap Thread.print>>$mondir/jcmdr.$pid.$(date "+%m%d%H%M%S")

iamshreeram

apt install build-essential checkinstall
apt install libreadline-gplv2-dev libncursesw5-dev libssl-dev libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev libffi-dev liblzma-dev

wget http://hostofpython/python-3.7.4.tar.xz
tar xvf python-3.7.4.tar.xz
cd Python-3.7.4
sudo ./configure --enable-optimizations
sudo make altinstall

# If python3.7 is installed in different location, path should be added to env variable

iamshreeram

Automated Infrastructure

This workshop will provide hands on experience on setting up and running an AWS Kubernetes cluster using EKS. We will use gitops, and explore kubernetes tools to make the cluster self-driving, with automated management and remedy of common cluster level problems. To achieve this, we will use eksctl, cluster-autoscaler, kube-prometheus (prometheus operator), node-problem-detector, draino, and node-local-dns-cache.

Intended audience

This workshop is intended to appeal primarily to four types of people:

1. Application developers looking to get an AWS kubernetes cluster to experiment without a lot of infrastructure knowledge
2. AWS DevOps people without a lot of kubernetes experience
3. Kubernetes DevOps people without a lot of AWS experience
4. Full-stack, Full-cycle developers in small or large teams.