Skip to content

Instantly share code, notes, and snippets.

View GarimaDamani's full-sized avatar
🇮🇳

Garima Damani GarimaDamani

🇮🇳
9 followers · 7 following
View GitHub Profile
@GarimaDamani
GarimaDamani / s3_access_logs
Created July 21, 2021 03:42
Create a table schema in the database for parsing s3 logs.
CREATE DATABASE s3_access_logs_db;
CREATE EXTERNAL TABLE `s3_access_logs_db.mybucket_logs`(
`bucketowner` STRING,
`bucket_name` STRING,
`requestdatetime` STRING,
`remoteip` STRING,
`requester` STRING,
`requestid` STRING,
`operation` STRING,
@GarimaDamani
GarimaDamani / set_up_telegraf.sh
Created December 11, 2019 05:01
Setting up Telegraf in ubuntu 16.04 and 18.04
#!/bin/bash
cat <<EOF | sudo tee /etc/apt/sources.list.d/influxdata.list
deb https://repos.influxdata.com/ubuntu bionic stable
EOF
sudo curl -sL https://repos.influxdata.com/influxdb.key | sudo apt-key add -
sudo apt-get update
sudo apt-get -y install telegraf
sudo systemctl enable --now telegraf
sudo systemctl is-enabled telegraf
systemctl status telegraf
@GarimaDamani
GarimaDamani / update_jenkins_discard_old_builds_config.groovy
Last active May 31, 2022 20:16
Save space in Jenkins slave by setting BuildDiscarderProperty
import hudson.model.Job
import jenkins.model.Jenkins
import hudson.tasks.LogRotator
import jenkins.model.BuildDiscarderProperty
name = "test_master_builder_medium_blog"
job_daystokeep = -1
job_numtokeep = 20
artifact_numtokeep = 20
artifact_daystokeep = -1
@GarimaDamani
GarimaDamani / linux_add_user.sh
Last active April 6, 2020 07:58
Linux create user with sudo access. args expected username, group name and public key. Requires root access to provide sudo.
#!/bin/bash
echo "Adding user $1 with sudo access"
adduser --disabled-password --gecos "" $1
cd /home/$1/
mkdir .ssh
cd /home/$1/.ssh/
touch authorized_keys
echo $2 >> authorized_keys
cd /home/$1/
@GarimaDamani
GarimaDamani / keybase.md
Last active October 15, 2019 11:49

Keybase proof

I hereby claim:

  • I am garimadamani on github.
  • I am garimadamani (https://keybase.io/garimadamani) on keybase.
  • I have a public key ASB_o_F0qgFmVFCYSZWODnDj9DjwidFiCBmU_wWZs29ikQo

To claim this, I am signing this object:

@GarimaDamani
GarimaDamani / install_supervisord.sh
Created August 7, 2019 00:37
Shell script to create two directories and install supervisor in ubuntu
# Installing Supervisord
apt-get update && apt-get install -y python-setuptools
easy_install supervisor
mkdir /etc/supervisor
echo_supervisord_conf > /etc/supervisor/supervisord.conf
mkdir /etc/supervisor/conf.d
@GarimaDamani
GarimaDamani / prometheus_metrics.conf
Created August 7, 2019 00:35
Supervisord conf file to run parse_logs and set_prometheus_metrics scripts all the time and output the error log logs if any at location specified. Run as user root
[program:parse_logs]
command=bash /var/opt/scripts/parse_logs.sh
autostart=true
autorestart=false
startretries=1
stderr_logfile=/var/log/parse_logs.err.log
stdout_logfile_maxbytes=0
stderr_logfile_maxbytes=0
user=root
@GarimaDamani
GarimaDamani / set_prometheus_metrics.py
Created August 7, 2019 00:32
Python script to run in infinite loop and process the output of parse_logs.sh file and expose them on 9091 port for prometheus to read the values set as gauge by prometheus client
import socket
import time
from prometheus_client import Enum, start_http_server, Gauge
start_http_server(9091)
def artifactory_status(art_states):
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.settimeout(2)
@GarimaDamani
GarimaDamani / parse_logs.sh
Created August 7, 2019 00:29
Shell script to parse particular log file and get the logs based on various patterns of past one minute and write the output to another file
#!/bin/bash/
FILE_PATH=”/var/opt/scripts/past_1min.log”
FETCH_LOG=”/var/opt/jfrog/artifactory/logs/request.log”
while true
do
TIME=$(date — date “-1min” ‘+%Y%m%d%H%M’)
sed -n “/^$TIME/,$ p” $FETCH_LOG > $FILE_PATH
echo -n “$(grep ‘HTTP/1.1|20.|’ $FILE_PATH | wc -l)” > /var/opt/scripts/parse.out.log
@GarimaDamani
GarimaDamani / parse_logs.sh
Created August 7, 2019 00:29
Shell script to parse particular log file and get the logs based on various patterns of past one minute and write the output to another file
#!/bin/bash/
FILE_PATH=”/var/opt/scripts/past_1min.log”
FETCH_LOG=”/var/opt/jfrog/artifactory/logs/request.log”
while true
do
TIME=$(date — date “-1min” ‘+%Y%m%d%H%M’)
sed -n “/^$TIME/,$ p” $FETCH_LOG > $FILE_PATH
echo -n “$(grep ‘HTTP/1.1|20.|’ $FILE_PATH | wc -l)” > /var/opt/scripts/parse.out.log