Anand Sharma indrayam

Generating Flame Graphs for Apache Spark

Flame graphs are a nifty debugging tool to determine where CPU time is being spent. Using the Java Flight recorder, you can do this for Java processes without adding significant runtime overhead.

When are flame graphs useful?

Shivaram Venkataraman and I have found these flame recordings to be useful for diagnosing coarse-grained performance problems. We started using them at the suggestion of Josh Rosen, who quickly made one for the Spark scheduler when we were talking to him about why the scheduler caps out at a throughput of a few thousand tasks per second. Josh generated a graph similar to the one below, which illustrates that a significant amount of time is spent in serialization (if you click in the top right hand corner and search for "serialize", you can see that 78.6% of the sampled CPU time was spent in serialization). We used this insight to spee

A curated list of AWS resources to prepare for the AWS Certifications

A curated list of awesome AWS resources you need to prepare for the all 5 AWS Certifications. This gist will include: open source repos, blogs & blogposts, ebooks, PDF, whitepapers, video courses, free lecture, slides, sample test and many other resources.

Github : Signing commits using GPG (Ubuntu/Mac) 🔐

Do you have an Github account ? If not create one.
Install required tools
Latest Git Client
gpg tools

# Ubuntu
sudo apt-get install gpa seahorse
# MacOS with https://brew.sh/

Create Root CA (Done once)

Create Root Key

Attention: this is the key used to sign the certificate requests, anyone holding this can sign certificates on your behalf. So keep it in a safe place!

openssl genrsa -des3 -out rootCA.key 4096

OverEncrypt

This is a guide that I wrote to improve the default security of my website https://fortran.io , which has a certificate from LetsEncrypt. I'm choosing to improve HTTPS security and transparency without consideration for legacy browser support.

WARNING: if you mess up settings, lose your certificates, or decide to no longer maintain HTTPS certs, these steps can and will make your domain inaccessible.

I would recommend these steps only if you have a specific need for information security, privacy, and trust with your users, and/or maintain a separate secure.example.com domain which won't mess up your main site. If you've been thinking about hosting a site on Tor, then this might be a good option, too.

The best resources that I've found for explaining these steps are https://https.cio.gov , https://certificate-transparency.org , and https://twitter.com/konklone

Kubectl output options

Let's look at some basic kubectl output options.

Our intention is to list nodes (with their AWS InstanceId) and Pods (sorted by node).

We can start with:

kubectl get no

Sessions

Code	Title	Duration	Link
Keynote	Andy Jassy Keynote Announcement Recap	0:01	https://www.youtube.com/watch?v=TZCxKAM2GtQ
Keynote	AWS re:Invent 2016 Keynote: Andy Jassy	2:22	https://www.youtube.com/watch?v=8RrbUyw9uSg
Keynote	AWS re:Invent 2016 Keynote: Werner Vogels	2:16	https://www.youtube.com/watch?v=ZDScBNahsL4
Keynote	[Tuesday Night Live with Jame

On master and nodes

Pull images form internet access laptop

docker pull gcr.io/google_containers/kube-apiserver-amd64:v1.5.0
docker pull gcr.io/google_containers/kube-controller-manager-amd64:v1.5.0
docker pull gcr.io/google_containers/kube-proxy-amd64:v1.5.0
docker pull gcr.io/google_containers/kube-scheduler-amd64:v1.5.0
docker pull weaveworks/weave-npc:1.8.2
docker pull weaveworks/weave-kube:1.8.2

	package main

	import (
	"io/ioutil"
	"log"

	"github.com/naoina/toml"
	"github.com/naoina/toml/ast"
	)