Eliah Rusin franchb

Describe how to change various HTTP request attributes that usually differs from one request to another:

The repository for the assignment is public and Github does not allow the creation of private forks for public repositories.

The correct way of creating a private frok by duplicating the repo is documented here.

For this assignment the commands are:

Create a bare clone of the repository. (This is temporary and will be removed so just do it wherever.)

git clone --bare [email protected]:usi-systems/easytrace.git

TL;DR

Install Postgres 9.6, and then:

sudo pg_dropcluster 9.6 main --stop
sudo pg_upgradecluster 9.5 main
sudo pg_dropcluster 9.5 main

Optimizing postgresql table for more than 100K inserts per second

Create UNLOGGED table. This reduces the amount of data written to persistent storage by up to 2x.
Set WITH (autovacuum_enabled=false) on the table. This saves CPU time and IO bandwidth on useless vacuuming of the table (since we never DELETE or UPDATE the table).
Insert rows with COPY FROM STDIN. This is the fastest possible approach to insert rows into table.
Minimize the number of indexes in the table, since they slow down inserts. Usually an index on time timestamp with time zone is enough.
Add synchronous_commit = off to postgresql.conf.
Use table inheritance for fast removal of old data:

	// 2016 MIT licence. Nick Malcolm.
	// Based on https://gist.github.com/stevebowman/7cff9dd80b227c899728

	// Makes a call using Twilio's API.
	// Expects the following Lambda environment variables:
	// TWILIO_ACCOUNT_SID - your account ID
	// TWILIO_AUTH_TOKEN - an auth token generated in the Twilio console e.g. ABCD1234
	// TWILIO_FROM_NUMBER - the number you've purchased to make calls from e.g. +14243700000

	// We'll call this number

	ssh-keygen
	-t ed25519 - for greatest security (bits are a fixed size and -b flag will be ignored)
	-t rsa - for greatest portability (key needs to be greater than 4096 bits)
	-t ecdsa - faster than RSA or DSA (bits can only be 256, 284, or 521)
	-t dsa - DEEMED INSECURE - DSA limted to 1024 bit key as specified by FIPS 186-2, No longer allowed by default in OpenSSH 7.0+
	-t rsa1 - DEEMED INSECURE - has weaknesses and shouldn't be used (used in protocol 1)

	-b 4096 bit size
	-a 500 rounds (should be no smaller than 64, result in slower passphrase verification and increased resistance to brute-force password cracking)
	-C "[email protected]" comment..

	def findNull(row:Row):String = {
	if (row.anyNull) {
	val indices = (0 to row.length-1).toArray.filter(i => row.isNullAt(i))
	indices.mkString(",")
	}
	else "-1"
	}

	sqlContext.udf.register("findNull", findNull _)
	df = df.withColumn("MissingGroups",callUDF("findNull",struct(df.columns.map(df(_)) : _*)))

	#!/usr/bin/python
	# install boto using pip

	import types
	from threading import Thread
	from boto import (
	ses,
	connect_s3
	)

	"""Perlin noise implementation."""
	# Licensed under ISC
	from itertools import product
	import math
	import random


	def smoothstep(t):
	"""Smooth curve with a zero derivative at 0 and 1, making it useful for
	interpolating.