Skip to content

Instantly share code, notes, and snippets.

@rjz
Created March 15, 2016 02:15
Show Gist options
  • Save rjz/05d4ba9980392bc8064b to your computer and use it in GitHub Desktop.
Save rjz/05d4ba9980392bc8064b to your computer and use it in GitHub Desktop.
The completely distributed web - Kyle Drake

The Completely Distributed Web

Presenter: Kyle Drake (@kyledrake) - founder, neocities.org


Cryptography - check against government power

  • Conversations lately are talking about "crypto wars 2.0" (Schneier, et. al)
  • Mid '90s, crypto was weak and bad - maybe deliberate to facilitate brute-force hacking by state-sponsored players
  • PGP (Phil Zimmerman) was the first "killer app" of crypto (i.e., >128 bit private keys)
  • Encryption now strong enough that 3rd-parties can't easily access data without user entering a password

Neocities

  • Kyle started neocities to encourage people to make websites
  • Initial reaction took it as a joke (under-construction GIFs, etc); ~70k sites as of today
  • At same time working on Coinpunk (early Bitcoin wallet). Think they aren't related? they are.

Distributed (or "permanent") web

Enable community-powered web sites that have no central point of failure

Motivations:

  • Dark ages arise from impermanence
  • Stop history from eroding--the distributed web is a library that can't burn down
  • Politics: increasing centralization / corporate power (and censorship, IP laws, etc., that arise from it)
  • Because we can! It's interesting tech to explore (separating technology from ideology
  • Web is trending towards centralization (IRC => slack, email => facebook, twitter, etc)
  • Fear that Big Brother knows too much about the private lives of innocent people. Important? Maybe not. But these data give a lot of information to a single, centralized leader.
  • Net's not that bad. It's not Orwell, but it might be Hong-Kong: once very free, but now starting to erode ("getting authoritarian and weird").

Distributed web changes power dynamic

  • Not just browsers following servers' orders--servers and clients become cooperative nodes (everyone chips in as a small part of the web)
  • Laptops/home computers seem puny, but link them together and processing/storage capacity are enormous.
  • Also censorship becomes impossible!

Core technologies

Cryptography

  • Hashing: take information, run it through a hashing function, receive a [unique] digest. Digest signature is fixed-length, [almost] unique, and not reversible--change one byte in the input and receive a completely different hash (allowing verification of input, e.g. to check signatures for malicious code).

  • Public-key Cryptography

    • Generate private key using computer's entropy pool
    • Private keys are very hard to brute-force. Even a 128 bit key takes a long, long time to break, so getting around crypto usually requires a vulnerability in the cypher
    • Don't give out your private key
    • Public keys are derived from a private key and allow someone else to encrypt a message that can be decrypted by the corresponding private key
  • Applications (e.g. bitcoin) are clever arrangements of public-key cryptography and cryptographic hashes

Distributed Hash Tables (DHT)

  • Given a hash, a DHT lets you retrieve the information used to generate the hash (e.g. looking for files over a network). Verification built-in (check hash against received content)

  • Enable network with no central coordination, millions of potential nodes, relatively few hops to reach many, many nodes

  • Immutable v. mutable data: if data won't change, use its digest directly. If the hash references a public key (i.e., someone else's signature) data can change and the key can be be used to verify the change

Distributed web projects

Distributed web isn't a single project or a single philosophy: it's many people developing alternatives that allow for distributed webpages.

  • dns => Namecoin. Bitcoin fork for domain names, but without ICANN or any single point-of-failure (just the blockchain crypto and your private key). The only way to lose a Namecoin domain is to surrender your private key. Integrated w/ OpenNIC (google DNS alternative)

  • www => IPFS. Addresses content by hash rather than IP; both mutable/immutable data are supported, and content is available via any node: as long as a single host can provide the content it's accessible.

    • built from scratch--a designed replacement for the current web
  • www (honorable mention) => zeronet. Uses BitTorrent DHT and Bitcoin keypairs for mutable data.

www demo

Domain names on the distributed web are necessary (we can't remember those hashes!) But DNS for a domain can retrieve TXT records that describe content we can then fetch.

Demo: ZeroNet + Tor + Namecoin = distributed, anonymous web site with domain name (http://bit.no.com:43110/futuretalk.bit).

And the only way to change or take down this website is to have the private key originally used to sign it.

More projects

  • email => BitMessage. Used crypto + proof-of-work algorithm to build distributed messaging app. "from" is a hash of a public key--the same crypto, once again.

  • routing => cjdns. P2P packet routing system.

    • Generates IPV6 addresses derived from users' public keys, avoiding centralized IP allocation.
    • Big application in mesh-networking (get enough people with towers that are mutual peers, and you get a sort of decentralized version of the internet--get it big enough and you replace ISPs, but that may take a while)
    • Still a prototype, but basically functional
  • tor => base layer for anonymity in the distributed web, which is cool, but you pay for it in performance

Is there a point?

  • If people are start using it (and they might, or might never), it changes the web in some very complicated ways

  • "I think of technologies as drivers and ideologies as an attempt to steer|explain|understand the environment"

  • Strong cryptography is here. Laws can try to ban it, but it's here, getting bigger, and not going away. Genie's out.

Questions

  • Q: How do you deal with concurrency when mutable data might not be in sync across nodes? A: You sign an update with your private key; other nodes receive a notification that content has been updated, download it, verify it with your public key, and store it. Can you revoke published content if you have an appropriate key? No, you can't. There's no cryptographic way to take back something that anyone else can view, but that's basically how the internet is now.

  • Q: Scarcity / cryptocurrency. For an hour of your time, I can offer a beer; or a dollar; or whatever. They're all based on the promise of someone in New York, and take into account scarcity of resources. Bitcoin doesn't. As we go into the future, and computer performance increases in years ahead, can cryptocurrencies account for the hyperinflation that would follow a significant increase in power? A: Simplified: Bitcoin detects if miners are solving problems too quickly (goal is a new block every 10-20m); clients increase complexity of the problem to try and maintain consistent production of new currency. It's emulating gold--production gets harder and harder and eventually we run out. Crypto can create scarcity in arbitrary resources (e.g. domain names) to discourage squatting / automated acquisition of resources.

  • Q: How does this techonology apply to (e.g.) disaster relief? A: Earthquake comes to Portland, takes out the Pittock building, and down goes Comcast. The city has 1-2 points of failure as far as the Internet is concerned--not that you're going to care, without power, water, and so on, but it's going down. First responders will be ham radio operators--the original mesh network, if you think about it. These folks practice. They do "field day" once a year, completely off the grid, to contact as many other operators as they can and keep up for this sort of scenario. After the disaster, though, one of the most exciting things about distributed networks is in recovery--if the Internet Archive is distributed, we can always get it back.

  • Q: The distributed web is going to replace parts of the web, but that web still relies on IP addresses, which are still centrally assigned, and so on. A: There are a lot of problems in the distributed web, and many solutions. Some of them (e.g. cjdns) allow us to replace whole swathes of the actual internet. The existing internet isn't as safe or robust as we like to think, many of them introduced by central points of failure. Namecoin, IPFS solve other problems, but the point of talking about (e.g.) cjdns is that people are trying to address some of the infrastructure problems. User adoption is the hardest problem (look at IPv6, with ~10% adoption, though comcast is IPv6 ready in PDX! Try it!), and the educational problems beneath it (to understand hashing, etc) is the critical piece.

  • Q: All of a sudden Apple's championing cryptography. Is there a "corporate sponsor" for the distributed web? A: There are lots of individuals interested in it, and many larger companies might really appreciate a distributed web (think of what youtube would save on bandwidth/CDNs, for instance), but there's no sponsor per se.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment