Martin Kleppmann ept

This is a copy of an email sent to the TRVE DATA mailing list.

TRVE DATA monthly update: April 2016

Hi everyone, thanks for subscribing to the TRVE DATA mailing list! We're going to try to build a habit of posting a monthly project update here, and this is the first one.

For background, we published a blog post explaining the rationale and goals of the TRVE project. And here's a little video showing our collaborative text editor prototype.

Today I'd like to briefly introduce the people in Cambridge who are working on various aspects of TRVE:

Investigatory Powers Bill

Please email your MP

The UK government is currently trying to rush the so-called "Investigatory Powers Bill" [1] through parliament (the bill has its second reading today — 15 March 2016). The bill gives the police, intelligence services and various government agencies wide-ranging powers to collect the internet activity of everybody in the country, hack innocent people's devices in bulk, compel companies to hack their own customers [2], and collect intrusive databases of personal data on innocent people.

It is doubtful whether these powers would make us any safer [3]; but they would make the UK a police state, with a regime of mass surveillance on par with Russia or China. As one MP put it: “Because for the past 200 years we haven’t had a Stasi or a Gestapo, we are intellectually lazy about it [4].” 200 senior lawyers and barristers have called the bill “unfit for purpose” and “maybe illegal” [5]. The current Apple vs. FBI case in th

Transactions: Myths, Surprises and Opportunities

References

Atul Adya: “Weak Consistency: A Generalized Theory and Optimistic Implementations for Distributed Transactions,” PhD thesis, Massachusetts Institute of Technology, Cambridge, MA, USA, March 1999.
Hagit Attiya, Faith Ellen, and Adam Morrison: “Limitations of Highly-Available Eventually-Consistent Data Stores,” at ACM Symposium on Principles of Distributed Computing (PODC), July 2015.
Peter Bailis, Alan Fekete, Ali Ghodsi, Joseph M Hellerstein, and Ion Stoica: “HAT, not CAP: Towards Highly Available Transactions,” at 14th USENIX Workshop on Hot Topics in Operating Systems (HotOS), May 2013.
Peter Bailis, Ali Ghodsi, Joseph M Hellerstein, and Ion Stoica: “[Bolt-on Causal Consistency](http://db.cs.be

Transactions: Myths, Surprises and Opportunities

References

Atul Adya: “Weak Consistency: A Generalized Theory and Optimistic Implementations for Distributed Transactions,” PhD thesis, Massachusetts Institute of Technology, Cambridge, MA, USA, March 1999. http://pmg.csail.mit.edu/papers/adya-phd.pdf Hagit Attiya, Faith Ellen, and Adam Morrison: “Limitations of Highly-Available Eventually-Consistent Data Stores,” at ACM Symposium on Principles of Distributed Computing (PODC), July 2015. http://www.cs.technion.ac.il/people/mad/online-publications/podc2015-replds.pdf Peter Bailis, Alan Fekete, Ali Ghodsi, Joseph M Hellerstein, and Ion Stoica: “HAT, not CAP: Towards Highly Available Transactions,” at 14th USENIX Workshop on Hot Topics in Operating Systems (HotOS), May 2013. http://www.bailis.org/papers/hat-hotos2013.pdf Peter Bailis, Ali Ghodsi, Joseph M Hellerstein, and Ion Stoica: “Bolt-on Causal Consistency,” at ACM International Conference on Manag

Keybase proof

I hereby claim:

I am ept on github.
I am martinkl (https://keybase.io/martinkl) on keybase.
I have a public key whose fingerprint is 4DF9 7732 6AF8 33A8 21AF E443 ED78 F8F5 352D 8EDF

To claim this, I am signing this object:

How to add syntax-highlighted code to PowerPoint slides (Mac OS)

pygmentize -f rtf FILE | pbcopy
Paste into TextEdit (in rich text mode: Format → Make Rich Text before pasting), and copy to clipboard again.
In PowerPoint, Edit → Paste Special… → Styled Text.

(Pasting RTF directly into PowerPoint doesn't work correctly, at least with PowerPoint 2008 — it extends colour spans longer than it should, and sometimes removes line breaks. Going via TextEdit seems to solve the problem.)

	import java.io.ByteArrayOutputStream;
	import java.io.IOException;
	import org.apache.avro.Schema;
	import org.apache.avro.generic.GenericData;
	import org.apache.avro.generic.GenericDatumReader;
	import org.apache.avro.generic.GenericDatumWriter;
	import org.apache.avro.generic.GenericRecordBuilder;
	import org.apache.avro.io.BinaryDecoder;
	import org.apache.avro.io.BinaryEncoder;
	import org.apache.avro.io.DatumReader;

	awk '{print $7}' access.log \| # Split by whitespace, 7th field is request path
	sort \| # Make occurrences of the same URL appear consecutively in file
	uniq -c \| # Replace consecutive occurrences of the same URL with a count
	sort -rn \| # Sort by number of occurrences, descending
	head -n 5 # Output top 5 URLs

	#include <librdkafka/rdkafka.h>
	#include <string.h>
	#include <stdio.h>

	/* If this is zero, the message is produced correctly. Other values cause problems. */
	#define LENGTH_FOR_NULL_KEY 5

	#define ERROR_LEN 512
	#define TOPIC "foo"
	#define MSG_KEY "hello"

	$ ping google.com
	PING google.com (173.194.34.104): 56 data bytes
	Request timeout for icmp_seq 0
	Request timeout for icmp_seq 1
	Request timeout for icmp_seq 2
	Request timeout for icmp_seq 3
	Request timeout for icmp_seq 4
	64 bytes from 173.194.34.104: icmp_seq=0 ttl=47 time=5277.419 ms
	64 bytes from 173.194.34.104: icmp_seq=1 ttl=47 time=5967.947 ms
	Request timeout for icmp_seq 7