It's important not to ignore the power of the Azimuth layer alone. Azimuth gives us 4 billion secure, convenient, transferable, cryptographically self-sovereign network endpoints.
Once two computers have secure names and secure keys, it's almost impossible to keep them from exchanging packets. Once names are scarce, it's straightforward to block undesired traffic. Azimuth is a natural foundation for any network which wants private filtering to be very easy, and public filtering very hard -- that can both prevent harassment and resist censorship.
But... what network? Who uses Azimuth? What are these endpoints? What protocol do they speak to each other? What operating system do they run? And... what are these computers, even? What problems are they solving, for whom?
A secure, routable identity is only the base of the stack. Once we make a path for two computers to speak, they still need a language to speak in. Once they can speak, they need to think. And once they can think, they need to learn to share.
Thus we have Azimuth, the continent (address-space registry); Arvo, the computer (system software stack); and Casbah, the civilization (universal hyperculture). We just did Azimuth; let's step up to Arvo.
Welcome to Arvo! Let's compute.
First, let's explain what we're not doing. We're not doing the obvious, standard, trivial thing.
In 2018, computers run one standard OS: Unix. They speak one standard protocol family: the Internet. Why change either? How would you even change either? What? What are you even talking about?
Because Azimuth points are the same width as IP addresses, anyone can build a secure overlay Internet on Azimuth. The result will be similar to certain VPN products, such as ZeroTier, but easier because of the address-width match and the supernode hierarchy. Standard Unix computers can communicate using standard protocols on this standard network.
We have not built such a system. Nothing is wrong with this trivial Unix/Internet/Azimuth stack. We think it's a completely valid product. We even hope someone builds it. But it's not why we came to the party. It feels not wrong, but lame. Why?
What do we want? What should an Azimuth endpoint actually be? A computer, sure -- but what kind of computer?
A kind of computer is called a "form factor." Unfortunately, this word is still bound to various 20th-century categories, like mainframes, minicomputers, etc, most of which no longer exist. Let's reclassify computing for the modern world.
Let's divide computers on four axes: (a) private or public (the computer computes for one, or many, users); (b) generic or specific (the computer is, or isn't, designed to run arbitrary programs); (c) durable or disposable (the computer can, or can't, archive critical data); (d) addressable or secluded (the computer can, or can't, react to outside requests).
This grid yields sixteen form factors. Four are viable products. We call them uncomputers, subcomputers, megacomputers, and true computers. Only the first three are now on the market. (There are really five kinds of computers -- counting blockchains, consensus or unicomputers -- but we have nothing new to say about blockchains.)
An uncomputer (or "information appliance," in Don Norman's phrase) is private, specific, disposable and secluded. The purpose of an uncomputer is to not appear as a computer. Uncomputers are all around us. If all of them turned into spiders at the same time, it would be terrifying.
A subcomputer is private, generic, disposable and secluded. There's a subcomputer in your hand, purse or pocket. It's a wonderful product and almost makes the 21st century endurable.
A megacomputer is public, specific, durable and addressable. It serves many, many users. It belongs to a megacorp. You don't have a megacomputer, but your subcomputer is friends with some. Each of your megacomputer accounts is a virtual uncomputer.
A true computer is private, generic, durable, and addressable. You don't have one yet. The utility of the megacomputer in a future of true computers is unclear, but probably nonzero.
So what is this true computer? And why isn't it here yet?
In the '80s and '90s, we thought our home computers were going to evolve into our true computers. But desktops never became servers. They were supplanted by megacomputers as the primary loci of data and programming. The gap between tablet and PC keeps shrinking.
In retrospect, it should have been obvious that our future true computer would live in someone else's data center (without being trapped there). A PC or laptop is still secluded and disposable by default. Servers are generally disliked by consumer ISPs. Also, your digital life is inside a big flammable box and can't hear the fire alarm or run to an exit. (Nice latency, though.)
We won't get our true computers until the '20s. We'll get them on the other side of the copper -- the cloud, not the home.
In "the cloud" your digital life can be perfectly durable, perfectly addressable, and perfectly generic. But that's only three out of four goals of the true computer.
Besides latency (gamers are people too), the personal cloud server has only one tradeoff that needs careful attention: privacy. Can a virtual machine in someone else's data center ever be truly yours -- as yours as a computer in your house? Isn't it running on someone else's silicon?
A cloud virtual machine can't be perfectly private. It can get close. If you don't need to resist a state actor, a personal cloud server is fine.
Why? One: there's more and more hardware solutions for HSMs, enclaves, and other tools for a VM to resist a host adversary. Two: even without special hardware, there's a deep qualitative difference between a megacomputer (like Facebook) and a host (like AWS). Enclaves are great, but they're a second line.
Your megacomputer account is just a database row. A bright-line distinction between the megacorp's data and yours is impossible. The same code is touching both. That's how the megacomputer does its job. The data is not even in any simple sense "yours" -- it's the megacorp's data about you.
A host is not doing its job if it kidnaps your VM, steals data from it, or tampers with its computation. It is violating its contract. It may even be criminally liable. The VM's first line of defense is legal, not physical.
The difference: the VM is a Turing-complete interpreter. Physically, nothing stops hosts from tampering. But because malfeasance can be formally defined, contracts can exclude it. The formal moat around a true VM is the perfect foundation for a legal bright line. This is why Netflix feels comfortable running its business on AWS, though Amazon is its main competitor.
Legal moats don't help if your adversary is sovereign. Put a rack and a halon sprinkler in your garage, you rebel, you. A good mindset: every private company is a government agency.
So why don't we all have personal cloud servers yet? The dream of a true computer is not esoteric. It's not new. It's obvious -- so obvious it's almost been forgotten.
Since the product is obvious and desirable, but doesn't exist, there's only one possibility. It must be technically impossible. It is. The problem is so hard, it's not even worth researching.
There is only one stack to build a true computer on: the standard stack, Unix and the Internet. The problem is not economic cost: AWS costs almost nothing. The problem is just usability.
After literally three decades of trying (from the mid-80s AT&T 3B1, through the "year of the Linux desktop"), only Apple has ever built a successful, truly personal general-purpose Unix computer. OS X cost a bazillion dollars and took (counting NextStep) a decade. OS X is not even a server.
The Unix-Internet stack is a practical industrial stack. One way to look at the problem of usability is to define usability from the industrial perspective: as system administration cost. A lot of investment has been put into reducing the administration cost of Unix. To make a personal Unix server, we have to bend this number down to roughly zero.
It's a bad research strategy to look for revolutionary breakthroughs in well-explored problems. This is why the Unix/Internet/Azimuth stack smelled wrong.
Rebasing the Internet address space on a sound cryptographic identity is great for the network half of the problem -- the problem with the Internet is not (mostly) the protocols. But Azimuth does nothing to fix the OS.
The problem of the personal server in 2019 is like the problem of the airplane in 1869. The aerodynamic principle of the wing was already understood. But the only engine was a steam engine. Even now, you really just can't build a steam airplane.
The problem with the steam airplane is power-to-weight ratio. The problem with the personal Unix server is complexity. The technical complexity of Unix is not just a cost, but a limit. Little imps and demons are constantly escaping from the engine room, and slithering into the passenger compartment.
What does this abstraction leakage look like? Consider the "reboot." Even a four-year-old knows that if something has a computer in it and it's not working, turning it on and off is the first way to try fixing it.
Why? Because, if this computer is built the same way as all other computers, it's a clone of a 1970s minicomputer. A VAX had not one brain but two -- one fast, disposable, and easily confused; one slow, stable, and sensible. Rebooting erases the fast brain and lets the slow brain rebuild it.
In the 21st century, the hardware tradeoffs that made the two-brained computer a great idea don't exist. Fast, stable memory is readily available. But there is only one standard way to use it: in a virtual two-brained computer.
This pain point alone is not fatal. How hard is it to reboot? But it is not alone. It makes us want to put a stake in the ground and say: the true personal server is not the personal server which must be rebooted. Do you ever have to reboot your megacomputer accounts?
The real problem is simple. We have something we really want. We want to get it the easy way. We're used to getting things the easy way. Here there is no easy way, so we're just stuck.
The digital revolution will not be complete until everyone has their own true computer, with full personal control of their own code, data, and communication. But a computer is a thing that runs Unix; a true computer is private; a Unix server can't be private, because that would involve reducing Unix sysadmin costs to near zero; and if that was possible, we'd already know.
A smart hacker always looks for ways to avoid solving a problem. We've seen a staggering number of clever hacks -- technical, commercial, regulatory -- designed to produce digital sovereignty, or some useful subset of it, without giving users their own true computers.
We could own our own data, but let the programmers run their programs on their own servers, and give them access credentials. We could turn our browsers into P2P servers. We could lobby the government to regulate the megacomputers. We could something something blockchain.
None of these ideas is necessarily bad. But they bespeak a very 20th-century mindset: that the future is right around the corner. So we have to throw something together right now to meet it. So whatever the answer it is, it must be something we already have lying around. We win by solving the problem wrong. If we try to solve it right, someone will solve it wrong first, and beat us.
In the 20th century, this "Apollo 13" engineering mindset made sense. In the 21st century, system software is going nowhere fast. The 20th-century engineer tries every way to solve the problem wrong, then declares it unsolvable. Worse is indeed better -- but only if worse actually works.
Maybe the true computer is just a hard problem, which needs to be solved right? Solving it might take some time. On the other hand, we doubt anyone can beat us by solving it wrong.
The opposite of "worse is better" is "boiling the ocean." This is standard CS terminology for a clean-slate redesign of everything -- obviously a perverse and futile task.
But we have no choice. To get our usability level way, way up, we have to get our complexity level way, way down. Simple systems evolve incrementally into complex ones, but not the other way around. If a computer is not a thing that runs Unix, what it? It's something much simpler.
How much simpler do we have to make our OS? Look at the difference between having data on someone else's server, and having your own Unix server. It's like the difference between owning a pet rock, and owning a pet otter.
If a Unix server is an animal, a true personal server should be a cactus. The cactus is technically alive -- but its needs are simple. It is not a rock. But it's almost a rock.
If not the Internet, what protocol does the endpoint speak? If not Unix, what OS does it run?
Our first thought is: is this two questions, or one? Suppose the protocol is the OS? Or to put it a different way: suppose we designed an OS from the packets up?
Imagine a computer with the keys to one Azimuth point. What is this computer? Where is it? Perhaps it is floating in space. Let's call it a ship. It's doing nothing. Our new ship is at peace, like a newborn whale.
Let's wake it up. Let's add a feature: you can send packets to this ship. What is a packet? Just a big number. What's a 1K network packet? Just a number under 2^8192. So an Arvo ship is just an Azimuth point, plus a way of yelling numbers at it.
So what is the state of a ship? What's in its mind? Just the list of big numbers it's heard, in the order it heard them. What does a ship know? Only what it's heard. That's its memory.
But wait: isn't there some other way to get data onto the ship? Like, installing software? Nope -- just yelling big numbers. That's the ship's whole life: hearing big numbers yelled at it.
But wait again: we don't want to just remember what we heard. Our ship is not just a memory. It must also be a mind. This list of numbers is not its mind -- just the input to its mind.
So the ship's mind is a function -- just one function. This lifecycle function takes in the history of packets it's heard, and spits out what's currently in its mind -- its state.
To be practically useful, the lifecycle function has to resolve to some iterative process -- some transition function which takes in the current state and the latest packet, and spits out the next state.
The transition function doesn't have access to the lifetime packet history. So we can discard old packets (if we won't need to audit the ship). Also, we can extend the transition function to produce not just the next state, but also a list of output packets. Now our ship can yell as well as hear.
This is complete, if vague, design for a practical, purely functional network computer, in which the protocol is the OS. And it certainly isn't anything like Unix and the Internet.
We call this design -- or any system whose entire lifecycle is defined as a single function -- a solid-state interpreter. A solid-state interpreter is an deterministic formal computer.
Another kind of deterministic computer is popular today -- a consensus computer. We call it a "blockchain." Deterministic computing seems to be good. Might it not be good for individual computing as well?
(The interaction between the two is interesting as well -- why not talk to a deterministic public computer from a deterministic private one? In a sense, the whole blockchain ecosystem is a decentralized service waiting for decentralized users -- that is, users who control their own true computers.)
And the difference in complexity between a pure function and a simulated mainframe, though certainly not dispositive, at least suggests one path toward overcoming the OS-level usability problem, and making the personal server technically possible.
We want the lifecycle function to be as simple as possible. If the interpreter directly interprets a user-level language, the tension between simplicity and usability is much too great. The language has to be perfect -- it can't be updated. That's a hard bar to hit in a high-level language.
So we separate the interpreter and the language. We use a very simple functional interpreter, Nock, which fits on a T-shirt. (Nock is a kind of Lisp too minimal to even call itself Lisp.) We are very confident that Nock will never need to be changed.
We generalize the packet model described above. Instead of just network packets, we have arbitrary I/O events. The personal server is an event processor. The transition function is a pure function, but it can send and receive HTTP transactions, etc, using the event system.
But first, we have to boot our true computer. The lifecycle function just runs the first event as a Nock program, with the rest of the event log as its data.
The second event is also a Nock program: a compiler to Nock from Hoon, a high-level typed functional language. Then we send the same compiler as source.
Then we load the Arvo functional kernel and its modules; then we insert the Azimuth networking keys, install a default filesystem, boot user-level agents and start processing input events.
Anyone reading with sharp eyes will notice that we've assumed a network.
Azimuth gives us an identity system and a PKI. We can use HTTP to talk to an Ethereum node and download the Azimuth state, or some subset of it.
Arvo is single-homed. Every Arvo interpreter has one Azimuth point, which is its network address. Arvo ships are separated into classes matching the Azimuth category. A 32-bit Azimuth home becomes an Arvo planet; a 16-bit Azimuth town becomes an Arvo star; an 8-bit Azimuth city becomes an Arvo galaxy.
The Azimuth sponsorship tree then serves as a P2P supernode tree for forwarding UDP packets. Stars help planets route; galaxies help stars route.
A planet can also create its own 64-bit children (moons). Moons are ships which are not points. They are not registered in Azimuth, and cannot escape from their planets; their parent signs their certificate.
In the common case, we're creating a home with a 64-bit ticket
(brainwallet). To log into this ship: point a browser at
home.suffix, check the DNS bar, and type in ticket as a
password (the ticket is not sent over the network).
If this feels like a centralized user experience -- it should. If your ship is one of unusual value, of course, more traditional cryptographic ownership flows are available.
This is anything but the normal design of an OS. There is no room here to go into the details of Nock, Hoon, or Arvo. But beyond basic standards and algorithms, Arvo reuses little from 20th-century system software. None of the old pieces fits.
But why? All this work isn't cheap or easy. Arvo to date has absorbed only a few million dollars in development costs. This ain't nothing, but it's not a lot by operating-system standards. And the typical OS project does more refining than inventing.
Arvo's primary design objective is to preserve the inherent mathematical simplicity of the lifecycle function, and pass it smoothly up the stack until it reaches the user as usability. We may not always do this perfectly, but we compromise nothing for it.
But what are the wins?
Remember Arvo's objection to Unix. The ultimate problem is that Unix is too complex. The direct problem is not that Unix's complexity consumes too many physical resources. The direct problem is that, because Unix is too complex to encapsulate its own complexity, its abstractions leak. Slimy gunk drips into the passenger compartment.
For example, Arvo has only one brain. There is no such thing as "rebooting" your Arvo ship. Here are three other normal systems problems that are difficult or impossible in a conventional OS, but non-problems for Arvo. This is by no means an exhaustive list.
The problem: message validation in distributed programming. This means turning an unstructured network bytestream into typed, trusted data. Many exploits attack bad validators.
How it impacts users: you get hacked, upgrades are brittle, decentralized systems are insanely hard to build
How Arvo fixes it: Hoon has one standard mapping between a bytestream and its untyped data structure (noun). A Hoon type declaration is a function whose input is any untyped noun, and whose range is the type. So every type is its own validator.
The problem: conventional network communication stacks can't promise that a message will be delivered exactly once. They can only promise at-least-once or at-most-once guarantees. Once is the common case, which means poor handling of idempotence only rarely causes errors. Rare errors are the most likely to slip through to the user.
How it impacts users: sporadic drops and/or duplicates
How Arvo fixes it: messages are exactly-once transactions. Acknowledgments are end-to-end. Any two Arvo ships that have communicated have a permanent socket, with a permanent pair of sequence numbers.
"Dependency hell" is a problem in program linking. Suppose component A depends on libraries B and C, which both depend on D -- but need different versions of D. There is no way to link one environment that works for both B and C.
How it impacts users: installs and upgrades fail
How Arvo fixes it: no single namespace, no concept of "environment" per se, but a first-class subject that can differ for B and C.
Ultimately, Arvo is free from dependency hell because it uses Nock rather than the lambda calculus. Nock does not have a concept of name resolution -- so it pushes problems like naming, and ultimately linking, up to higher layers. Higher layers are more flexible and expressive by definition.
But what is Arvo for? Why do I need an Arvo instance? Suppose it is a great OS -- why does anyone need it? What is the human experience of owning and operating a personal server?
In its current form (a bucket of megacomputer accounts), your digital life is like your real life. In its future form (one personal server), it's still like real life.
But it's not like your real life. Instead, it's like the real life of a rich person. Your personal server is a luxury good: you probably don't need it, but you definitely do want it.
How is a rich person's life different from yours? In a lot of ways. One way is: the rich person has a staff. Do you have people? Even just an assistant? If so, you might be rich.
Everyone needs shoes. When a regular person needs shoes, they go to the shoe store and lets a salesperson help them. When a rich person needs shoes, they tell their assistant, Pierre, that their shoes are looking a little worn. Pierre will poke around and come back later in the day with some options.
The salesperson has a conflict of interest. The customer is king; the boss is in charge. Pierre has no such conflict. His customer is his boss. Pierre can just do better.
The user interface of a megacomputer is subject to the same conflict of interest. Your account is trying to serve you. But its master is the megacorp. This conflict bounds the quality of customer service that a megacomputer can provide.
Siri wants you to think she works for you. Siri works for Apple. What would Siri even be like if she worked for you? Without a personal server to run your personal Siri, you can't even know.
Your Arvo agents are not actually people. But they are yours. You probably don't code them yourself. But you could. You choose them; you can change them; they can't lock you in. (Someday, they may even grow voices and talk to you.)
Your Arvo ship has two basic kinds of agents: private and social. Social agents talk to other Arvo ships over Arvo networking. Private agents don't, though private agents may still use external centralized (web) or decentralized (blockchain) services.
Social networking on top of Arvo is the domain of Casbah.
Read the Casbah vision to understand its motivations. Here,
let's briefly summarize the technical outcome.
We've decided that there is no such thing as a universal social agent. Instead, every culture should have its own separate agent, developed and updated by the culture itself. What is a "culture?" Again, read the Casbah vision.
The important technical trick behind Casbah: when your ship updates itself, it gets its kernel and (by default) its private agents from its sponsor. These may be forked according to taste, but they are neutral tools.
But, since the medium is the message, social code cannot be neutral. So each Casbah culture owns and updates its own social agent (whether it writes the code, or just chooses it).
Private agents are easier to describe. Everyone has different cultures; everyone has different needs. But the cultures are a lot more different than the needs.
Let's walk through some UX stories to show you what your digital life as a rich person will be like. Who are these tiny personal assistants, in their little box in the cloud? We'll sketch them from both sides: top-down to the user, bottom-up to the computer.
Penelope is your muse and/or amanuensis. When you are creating, or remembering, or recording, or collaborating, Penelope is there to help you. Are you writing notes for yourself? Symphonies for the Philharmonic? Or Hoon libraries for the Arvo kernel? Penelope will fit them all in her elegant little nested boxes.
Technically, Penelope is the authoring and content-production interface to Arvo's typed revision-control service, Clay.
Angus is your librarian. Some of us are content producers. More are content consumers. Angus helps you track, retrieve, store and view content created by others.
Technically: Angus is the interface between Clay, the user, and external content archiving and delivery services.
Marcus is your scheduler and planner. Marcus helps you track what, when and where you, and others you are collaborating with, are planning to do in the future.
Technically, Marcus is a collaborative calendar/todo app.
Pierre is your concierge. Pierre helps you consume goods and services from the Internet. The purpose of Pierre is to provide a common interface for all your Internet needs -- freeing you from the pesky native interfaces of the original providers. However this has to be done, Pierre will do it.
Technically, Pierre is a stateful, customizable, opinionated Web client which exports task-specific, not vendor-specific, UIs: social, shopping, etc, etc.
Courtney is your comptroller. Courtney helps you manage your digital assets and/or store the keys for your digital assets.
Technically, Courtney is a secret vault, blockchain tracker, and personal ledger.
Kenji is your sensei. Kenji is here to help teach you Hoon, play with data, and build little programs.
Technically, Kenji is a very helpful shell/notebook/dojo for reproducible computing.