Pratik ratpik

Simple, semi-anonymous backups with S3 and curl

Backing stuff up is a bit of a hassle, to set up and to maintain. While full-blown backup suites such as duplicity or CrashPlan will do all kinds of clever things for you (and I'd recommend either for more complex setups), sometimes you just want to put that daily database dump somewhere off-site and be done with it. This is what I've done, with an Amazon S3 bucket and curl. Hold onto your hats, there's some Bucket Policy acrobatics ahead.

There's also a tl;dr at the very end if you just want the delicious copy-pasta.

Bucket setup

Multiple MySQL Versions for Development

Options included below:

Using Docker docker-compose
Using Homebrew brew

Using Docker (recommended)

This gist was originally created for Homebrew before the rise of Docker, yet it may be best to avoid installing mysql via brew any longer. Instead consider adding a barebones docker-compose.yml for each project and run docker-compose up to start each project's mysql service.

System Design Cheatsheet

Picking the right architecture = Picking the right battles + Managing trade-offs

Basic Steps

Clarify and agree on the scope of the system

User cases (description of sequences of events that, taken together, lead to a system doing something useful)
- Who is going to use it?
- How are they going to use it?

FWIW: I (@rondy) am not the creator of the content shared here, which is an excerpt from Edmond Lau's book. I simply copied and pasted it from another location and saved it as a personal note, before it gained popularity on news.ycombinator.com. Unfortunately, I cannot recall the exact origin of the original source, nor was I able to find the author's name, so I am can't provide the appropriate credits.

Effective Engineer - Notes

By Edmond Lau
Highly Recommended 👍
http://www.theeffectiveengineer.com/

What's an Effective Engineer?

Scaling your API with rate limiters

The following are examples of the four types rate limiters discussed in the accompanying blog post. In the examples below I've used pseudocode-like Ruby, so if you're unfamiliar with Ruby you should be able to easily translate this approach to other languages. Complete examples in Ruby are also provided later in this gist.

In most cases you'll want all these examples to be classes, but I've used simple functions here to keep the code samples brief.

Request rate limiter

This uses a basic token bucket algorithm and relies on the fact that Redis scripts execute atomically. No other operations can run between fetching the count and writing the new count.

To send a request via the sandbox, you can use pm.sendRequest.

pm.test("Status code is 200", function () {
    pm.sendRequest('https://postman-echo.com/get', function (err, res) {
        pm.expect(err).to.not.be.ok;
        pm.expect(res).to.have.property('code', 200);
        pm.expect(res).to.have.property('status', 'OK');
    });
});

Using websockets to easily build GUIs for Python programs

I recently built a small agent-based model using Python and wanted to visualize the model in action. But as much as Python is an ideal tool for scientific computation (numpy, scipy, matplotlib), it's not as good for dynamic visualization (pygame?).

You know what's a very mature and flexible tool for drawing graphics? The DOM! For simple graphics you can use HTML and CSS; for more complicated stuff you can use Canvas, SVG, or WebGL. There are countless frameworks, libraries, and tutorials to help you draw exactly what you need. In my case, this was the animation I wanted:

(Each row represents a "worker" in my model, and each rectangle represents a "task.")

	describe('highest level describe', function () {
	before(function() {
	console.log('This is the highest level before')
	})

	beforeEach(function() {
	console.log('This is the highest level beforeEach')
	})

	it('This is the first highest level test', function() {

	/* bling.js */
	window.$ = document.querySelector.bind(document);
	window.$$ = document.querySelectorAll.bind(document);
	Node.prototype.on = window.on = function(name, fn) { this.addEventListener(name, fn); };
	NodeList.prototype.__proto__ = Array.prototype;
	NodeList.prototype.on = function(name, fn) { this.forEach((elem) => elem.on(name, fn)); };

	curl --include \
	--no-buffer \
	--header "Connection: Upgrade" \
	--header "Upgrade: websocket" \
	--header "Host: example.com:80" \
	--header "Origin: http://example.com:80" \
	--header "Sec-WebSocket-Key: SGVsbG8sIHdvcmxkIQ==" \
	--header "Sec-WebSocket-Version: 13" \
	http://example.com:80/