vkroz

Kafka frequent commands

Assuming that the following environment variables are set:

• KAFKA_HOME where Kafka is installed on local machine (e.g. /opt/kafka)
• ZK_HOSTS identifies running zookeeper ensemble, e.g. ZK_HOSTS=192.168.0.99:2181
• KAFKA_BROKERS identifies running Kafka brokers, e.g. KAFKA_BROKERS=192.168.0.99:9092

Server

Start Zookepper and Kafka servers

yanndegat

 #!/bin/bash
 STACK=$1

 aws ec2 describe-instances | jq -r \
  '[
      .Reservations[].Instances[]
      | . as $node
      | {
          ip: ($node.NetworkInterfaces[0].PrivateIpAddress),
          name: ($node.Tags[] | select(.Key == "Name").Value),

hvelarde

# get total requests by status code
awk '{print $9}' /var/log/nginx/access.log | sort | uniq -c | sort -rn

# get top requesters by IP
awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head | awk -v OFS='\t' '{"host " $2 | getline ip; print $0, ip}'

# get top requesters by user agent
awk -F'"' '{print $6}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head

# get top requests by URL

FrankSpierings

Container security notes

Internet references

Kernel and architecture

• namespaces - overview of Linux namespaces http://man7.org/linux/man-pages/man7/namespaces.7.html

• mount_namespaces - overview of Linux mount namespaces

lukecav

# WordPress single site rules.
# Designed to be included in any server {} block.

# This order might seem weird - this is attempted to match last if rules below fail.
# http://wiki.nginx.org/HttpCoreModule
location / {
    try_files $uri $uri/ /index.php?$args;
}

# Add trailing slash to */wp-admin requests.

pydevops

• 0.1. References
• 0.2. Other cheatsheets
• 0.3. Manage multiple gcloud config configurations
  • 0.3.1. Switch gcloud context with gcloud config
• 0.4. Credentials
• 0.5. info
• 0.6. projects
• 0.7. zones & regions
• 0.8. Organization
• 0.9. billing

dboyd13

    ___      _____         ___      _      _   _              _          _    _ _          _       _                   _      _       
   /_\ \    / / __|  ___  / __| ___| |_  _| |_(_)___ _ _     /_\  _ _ __| |_ (_) |_ ___ __| |_    /_\   ______ ___  __(_)__ _| |_ ___ 
  / _ \ \/\/ /\__ \ |___| \__ \/ _ \ | || |  _| / _ \ ' \   / _ \| '_/ _| ' \| |  _/ -_) _|  _|  / _ \ (_-<_-</ _ \/ _| / _` |  _/ -_)
 /_/ \_\_/\_/ |___/       |___/\___/_|\_,_|\__|_\___/_||_| /_/ \_\_| \__|_||_|_|\__\___\__|\__| /_/ \_\/__/__/\___/\__|_\__,_|\__\___|
                                                                                                                                      

Notes taken between Dec-2017 and Jan-2018, from acloud.guru and AWS FAQs.
                 
Solution Architect Associate - Need to Know:

yevrah

################################################################################
# Method 1: Install using rpm packages (credit to DarkMukke)
#

rpm -Uvh http://mirror.ghettoforge.org/distributions/gf/gf-release-latest.gf.el7.noarch.rpm
rpm --import http://mirror.ghettoforge.org/distributions/gf/RPM-GPG-KEY-gf.el7

# WARNING: removing  vim-minimal uninstalls `sudo` if you skip the second step
#          make sure to at least run `yum install sudo`
yum -y remove vim-minimal vim-common vim-enhanced

zodvik

Producer

Setup
bin/kafka-topics.sh --zookeeper localhost:2181/kafka-local --create --topic test-rep-one --partitions 6 --replication-factor 1
bin/kafka-topics.sh --zookeeper localhost:2181/kafka-local --create --topic test-rep-two --partitions 6 --replication-factor 3

Single thread, no replication
bin/kafka-run-class.sh org.apache.kafka.tools.ProducerPerformance --print-metrics --topic test-rep-one --num-records 6000000 --throughput 100000 --record-size 100 --producer-props bootstrap.servers=kafka_host:9092 buffer.memory=67108864 batch.size=8196

Single-thread, async 3x replication

githubfoam

------------------------------------------------------------------------------------------
#CLI shortcut keystrokes(linux&MAC)
Ctrl+L: Clear the screen. This is similar to running the “clear” command.
Ctrl+C: Interrupt (kill) the current foreground process running in in the terminal. This sends the SIGINT signal to the process
Ctrl+Z: Suspend the current foreground process running in bash. This sends the SIGTSTP signal to the process. To return the process to the foreground later, use the fg process_name command.
Ctrl+D: Close the bash shell.This is similar to running the exit command
Ctrl+L: Clear the screen. This is similar to running the “clear” command.

Ctrl+S: Stop all output to the screen. This is particularly useful when running commands with a lot of long, verbose output, but you don’t want to stop the command itself with Ctrl+C.
Ctrl+Q: Resume output to the screen after stopping it with Ctrl+S.