open cloud services

cloud.md

Mozilla's cloud services need to advance the state of the web in privacy, security, and usability in order to compete and keep the web open. We need a platform that has a user experience closer to mobile where building, signing up, using, and managing apps is easy. We need the cloud to integrate well with our mobile, desktop, and lifestyle devices. We need choice in how our data is used. We need to trust the services we use. To do this we should build a platform that lets us run our cloud apps (service instances) individually, in the cloud or on our own servers.

As technical people, today we can enjoy the benefits of owning some of our web identities and data by running our own services on our own servers. We can register our own domains, manage MX records, setup shared hosting or VPS, make our own blogs, host our own email, and even run our own Firefox Sync service because we own our servers, or at least the data. Even for the most dedicated of us its time consuming and complicated to setup and maintain. Its not something that most people are interested in undertaking. Making these things accessible to everyone is a huge challenge but the history of computing has shown that its not only possible but probably inevitable.

Today's server complexity is sort of like life before the internet in the PC era. Making the computer run what we wanted was itself an accomplishment. Things slowly got easier on the desktop but we all suffered through blue screens of death. Eventually web browsers came along and gave everyone a reason to use computers for more than games, spreadsheets or as fancy typewriters. PCs were still horrible, finicky machines but browsers made the pain worth it because the internet was amazing!

Now personal computing has advanced into the mobile age. Fortunately modern mobile OSes have way better security and system reliability than PCs and are generally much more usable and managable, allowing even more people to join the online community.

Modern servers don't have the usability of mobile. They are still mainly used to host isolated services to an exclusive set of users. Successful services have many servers connected in complex networks in order to reliably serve all their users. Their service software can be equally complex. Therefore these systems are designed, installed, configured, and managed by experts. Current development seeks to make these complex systems easier to build and manage. Virtualization has made servers easier and more efficient to utilize by allowing us to deploy and scale resources without buying and maintaining hardware. This has made creating personal servers easier as well. We can easily sign up at any number of VPS hosting providers for a few dollars a month to run our own blogs and email servers. The difficulty is that the servers themselves still require expert knowledge to use. What if the server ecosystem was more like the mobile ecosystem?

When operating a server becomes more like operating a smart phone we will all enjoy the benefits of having a persistent, stable presence on the internet with the usability we've come to expect from our phones. We will choose how our data is shared and with whom, sync data securely, and connect directly with the people we care about. We'll be able to leverage the unique capabilities of each of our devices to their full combined potential.

Firefox OS makes it possible to easily build native mobile apps using standard web technologies. An open server platform will allow us to build social networks and personal services more easily as well. It will also improve our individual privacy and security by giving us more control of our data. With it we can recapture our presence on the web from the "silos" of today.

Creating an open personal server platform that everyone can use is an incredible amount of work but its already possible. No new hardware or network protocols need to be invented and we can do it incrementally. We can gradually chip away at the high barrier of entry to enable more and more people to host themselves on the web. We can start with a small number of core services for advanced users while incrementally making the platform more accessible and robust. Building services that people can easily self-host should be a core goal in our plan and is something that's distinctly Mozillian.

We, Mozillians, fight to keep the web open and strive to empower individuals to own their online identities. As regular people we give away vast amounts of personal information to large and small organizations without much choice in what they decide to do with the data we give them, whether we do it intentionally or by them tracking us. In most cases we don't intend to give them our data for their benefit, but either as an intermediary for sharing it with people we want to reach or to store it so we may retrieve it later from somewhere else. These things should not require us to agree to broad, confusing terms of service. They are things we should be able to do individually with the technologies of the web, securely, safely, and easily. Helping to make this possible is why I am a Mozillian.

Here's my first sketch of what a roadmap might include at a high level to show that at each phase we offer useful services as it matures and gains adoption.

early adopters

Phase 1

• target Wizards (nightly)
• new deployment options
  • self hosted (gentoo style)
  • advanced installation/admin knowledge required
  • build steps
  • manual process management
• new support options
  • IRC, mailing list, bugzilla
• new services
  • sync

Phase 2

• target Enthusiasts (Beta)
• new deployment options
  • self hosted (ubuntu style)
  • package manager installation
• new support options
  • wiki
  • forum
  • readthedocs.org
• new services
  • media storage
  • blog
  • email

mainstream

Phase 3

• target Busy-Bees & Middle-Managers (Stable)
• new deployment options
  • fully hosted (wordpress style)
  • zero installation
  • simple administration
  • optional custom domain
  • monthly hosting fee
• new support options
  • dedicated customer service
• new services
  • identity (browserID IdP)
  • contacts

Phase 4

• target everyone
• new deployment options
  • hosted (VPS image style)
• new support options
  • paid support options
• new services
  • hosting provider integration
  • full "Mobile OS" like experience
  • 3rd party apps
  • app management
  • app store
  • data import/export

late adopters

• target Stalwarts & Evergreens
• new support options
  • remote family support

From @rfk

This sounds like the sort of thing that really needs to be discussed over several hours of dinner, drinks etc to really dig into it. Hopefully we'll get a chance to do that before too long! I think I get the gist though, it's a solid core vision with a lot of understandably blurry details.

Some random thoughts and datapoints and ideas below.

I recently had a friend (of user-type "Enthusiast") approach me about almost exactly this. Spooked by all the Snowden-NSA-Goodle brouhaha, his question was basically "how do I host and control my own online identity?". I was sad not to have a good answer for him. Heck I barely have a good answer for myself. People want this. They need it, and they deserve it.

From what I've seen, "ownCloud" is like the pre-iphone-era version of this. It's got a bunch of good ideas and is probably a vision of the future, but it's kinda clunky and has some wrong levels of abstraction and really just hasn't "arrived" properly yet. Incidentally, ownCloud has its own re-implementation of the sync1.1 user+storage protocol so that folks can sync firefox to their own cloud. We get a fair amount of IRC traffic from folks with that setup.

There are some obvious parallels with unhosted and noBackend and similar efforts, but perhaps some obvious differences too. I love a lot of the stuff that's going into e.g. hoodie but it seems a little too one-size-fits-all to tackle this whole problem space.

Mozilla has something special to bring to the table in this space: clout. We can legitimize the services protocols and provide serious hosting options, paid premium plans, etc etc that could easily make the difference between "interesting fringe technology" and "viable alternative to status quo".

It would be awesome to have a kind of Docker-like approach to your personal online data and identity. The service and the associated data are a little self-contained unit that can be moved around at will. Oh, you want to go from storing your contacts with us to hosting them yourself? Export your "contacts service bundle" from our servers as some sort of image, plop it on your own server somewhere, and you're up and running. I'm not really sure how that would work in practice, but there's a very interesting and compelling metaphor in there somewhere...kinda link installing apps on your phone actually :-)

So yeah. I like where your head's at here, I like where you're going with this. The key, as you say, will be to successfully manage two concurrent and oft-conflicting threads of activity:

• articulate the grand vision of what Open Cloud Services needs to be, and why, and how awesome it will be when we get there.
• iterate forward in small increments, each time delivering real additional value to a slightly larger group of users.

A.K.A simultaneously "thinking big" and "keeping it real". PiCL failed hard on the latter. I sometimes worry that FxA is failing on the former.

I’m still not totally clear on what our end goal should be, but do I feel strongly about the direction to head. I want to go toward a web where individuals control their own data and identity. We’ve done a good job of this so far, but where to go next is hard to see.

Since we’re trying to give individuals more power a huge thing to consider is usability. I’m basically proposing giving everyone their own server. With today’s software that’s ridiculous. I’ve tried out ArkOS and ownCloud, and for some people I think they’re great, but I wouldn’t recommend them to anyone not interested in “learning computers”, whereas I’d give an iPad to my grandmother. Both run *nix. There’s no reason we can’t make a server just as easy.

So how do we start?

It seems like the first thing to focus on is partially what we’re already doing, support FirefoxOS and the Firefox browsers. Having services that support and integrate those with the option to self-host is a big differentiator that I think will be popular. If its stuff we need to do anyway to make us competitive, lets take it the extra mile and make it self-hostable, then maybe the extra 1000 miles to make it easy :) If we offered an iCloud-like service that could be self-hosted, I think it will be huge.

I’d like to see self-hosted and easy converge at some point, but it doesn’t have to be that way from the start. We can focus “easy” on Mozilla hosted, and “possible” on self-hosted, but we have to commit to both, otherwise they’ll never converge.

Beyond that its all rainbows and unicorns. Opening up a new platform will be like another mobile revolution. The possibilities are so broad that I’ve tried to stop thinking about all the cool apps that could exist because we need to build the foundation first. It is fun to imagine though :)

What do you think our goal should be, and the first step to take?

also:

• http://sandstorm.io
• http://cozy.io
• http://tent.io

I think a common (anti) pattern here at Mozilla is to charge forward along a technical path without thinking through the end user experience. How exactly would a user take advantage of this? In what ways would it be more delightful than the alternatives? If it's hard to think about these questions now, it may not be any easier to answer these questions later.

If it's too hard to answer those in the general sense, it may be helpful to constrain the problem to a narrower use case.

Do you envision the target of this effort to be developers or end users? (And developers are users too.)

I look forward to digging in more during workweek and thanks for thinking big Danny!

I agree that we often dive into technology with less consideration for products and end users. I thought this recent post by Isaac of node and npm fame relevant:

http://blog.npmjs.org/post/80997676347/nebulous-profit-meditations

Fortunately I think this path has many waypoints with directly delightful products and/or features for users that are interested in being more active in their personal data. Many people don't care about their personal data, at least beyond the current status quo. This isn't immediately targeted at them. I suspect there's a decent (and growing) number of people who do care about the data they generate. The technical ones can help themselves with current solutions but non-techies are left out. I want to help them. By doing so they might also educate their friends on the benefits and become advocates.

As a straw man, "private sync":

How exactly would a user take advantage of this?

• "set up" a sync server
  • install something or sign up for a service
• set a url in the sync preferences pane

In what ways would it be more delightful than the alternatives?

• some "intangible" benefits
  • knowing their data is "safe" on their own machine
  • knowing their data is portable
• rainbow unicorn features
  • web interface (to view from other devices)
  • option to make some data public
  • P2Pish social bookmarks
  • long term archive (with search) <-- my fav
• allows us to do things most people would be "suspicious" of a free service doing

Do you envision the target of this effort to be developers or end users?

End users (technical first?) until developers get interested. But really I don't know.