Arnold Yahad arnoldyahad

Fluent-bit rocks

A short survey of log collection options and why you picked the wrong one. 😜

Who am I? Where am I from?

I'm Steve Coffman and I work at Ithaka. We do JStor (academic journals) and other stuff. How big is it?

Number	what it means
101,332,633	unique visitors in 2017

Loop through all EC2 instances (excluding terminated and spot) and enable termination protection

for I in $(aws ec2 describe-instances --query \
  'Reservations[].Instances[?(InstanceLifecycle!=`spot` && InstanceState!=`terminated`)].[InstanceId]' \
  --output text); do
  aws ec2 modify-instance-attribute --disable-api-termination --instance-id $I;
done

$ curl -s -XGET http://localhost:8083/connector-plugins|jq '.[].class'

"io.confluent.connect.activemq.ActiveMQSourceConnector"
"io.confluent.connect.elasticsearch.ElasticsearchSinkConnector"
"io.confluent.connect.hdfs.HdfsSinkConnector"
"io.confluent.connect.hdfs.tools.SchemaSourceConnector"
"io.confluent.connect.ibm.mq.IbmMQSourceConnector"
"io.confluent.connect.jdbc.JdbcSinkConnector"
"io.confluent.connect.jdbc.JdbcSourceConnector"

"io.confluent.connect.jms.JmsSourceConnector"

kafkacat -b localhost:9092 \
         -t _kafka-connect-group-01-status \
         -C \
         -o-3 \
         -c3 \
         -f 'Topic %t / Partition %p / Offset: %o / Timestamp: %T\nHeaders: %h\nKey (%K bytes): %k\nPayload (%S bytes): %s\n--\n'

	# Send a metric to statsd from bash
	#
	# Useful for:
	# deploy scripts (http://codeascraft.etsy.com/2010/12/08/track-every-release/)
	# init scripts
	# sending metrics via crontab one-liners
	# sprinkling in existing bash scripts.
	#
	# netcat options:
	# -w timeout If a connection and stdin are idle for more than timeout seconds, then the connection is silently closed.


	# ******** EC2 *********

	# describe ec2 instances
	aws ec2 describe-instances \
	--query 'Reservations[].Instances[].[InstanceId,Tags[?Key==`Name`].Value\|[0],State.Name,
	PrivateIpAddress,PublicIpAddress]' \
	--output table

	# stopped instances

	{
	"Version": "2012-10-17",
	"Statement": [
	{
	"Effect": "Allow",
	"Action": [
	"rds-db:connect"
	],
	"Resource": [
	"arn:aws:rds-db:region:account-id:dbuser:dbi-resource-id/database-user-name"

	###################################################
	##
	## Alertmanager YAML configuration for routing.
	##
	## Will route alerts with a code_owner label to the slack-code-owners receiver
	## configured above, but will continue processing them to send to both a
	## central Slack channel (slack-monitoring) and PagerDuty receivers
	## (pd-warning and pd-critical)
	##

	---
	- name: concatenate list elements of dictionary
	hosts: all
	gather_facts: no
	vars:
	users:
	- username: alice
	email: alice@example.com
	- username: bob
	email: bob@example.com

	#!/bin/bash
	kubectl get nodes --selector=kubernetes.io/role!=master -o jsonpath={.items[*].status.addresses[?\(@.type==\"ExternalIP\"\)].address}