A less-sensational and less-conspiratory follow-up to Bryan Lunduke's "HTTPS is Dangerous".
While Lunduke focuses on numerous high-level and theoretical issues with HTTPS, which are very controversial, this writing rant looks at actual practical issues regarding HTTPS advocacy and adoption. It is mainly from the point of view of a web application developer, who make up a fraction of web users, but also gives insight into related average user experience. I am basing this off of my own (probably somewhat unique) experiences during a project regarding the topic as there isn't similar discussion elsewhere.
I don't consider myself a web developer, let alone a professional one; in fact I despise webdev for many reasons, including the ones discussed below. Nevertheless, a side project of mine, for now ~8 months (on and off), has been a small webdev project (referred to as "my webapp" below). It is a purely client-side browser game with absolutely no need for any server-side backend, consists of purely statically-served content and is perfectly fine hosted on GitHub pages.
In this project I've made use of the forefront of modern and progressive web app (PWA) technologies, such as:
- offline support using AppCache and Service Workers,
- offline local data storage using IndexedDB,
- app-like addition to home screens using Web App Manifest,
- responsive design for all form factors (desktop, tablet and phone) using also CSS flexbox (which, by the way, still has significant problems between implementations in different browsers).
All this effort to provide an app-like experience without needing to actually create apps for multiple platforms. Overall, it has been a "fun" experience.
With the raise of Let's Encrypt advocating the use of HTTPS over HTTP has gone full swing, usually arguing that now there's no excuse for not using HTTPS as it's free, unlike before. I can't say that's a bad thing, having used Let's Encrypt myself previously, but in many ways the advocacy towards HTTPS has gone overboard and that's what I will be focusing on in greater detail in the following subsections.
I host my webapp on GitHub Pages with a custom domain (for the purpose of having a more user-friendly URL - also an aspect of PWA) and therefore only over HTTP due to limitations discussed below. Again, for the purposes of my application, which consists of only browser-side JavaScript and static assets, it is perfectly fine as no data is ever sent to the servers anyway.
In this section there will be mentions of secure contexts. If a feature is only available in a secure context, it is only available over HTTPS and not HTTP.
Modern web APIs include a multitude of client-side offline data storage mechanisms, including but not limited to localStorage and IndexedDB. My webapp also makes use of the latter. For obvious reasons size quotas are in place. The new Storage API tries to govern all of this and provides two modes:
best-effort(default) - the browser does its best (whatever that may be) to keep a site's data but gives absolutely no guarantees; the browser may delete a site's data whenever it chooses to without even notifying the user.persistent- the browser deletes a site's data only with the user's knowledge and confirmation, essentially giving a better guarantee.
In fact, the unsuspecting data wipes of best-effort storage may come as a surprise. While usually in theory related to exceeding the quota, it's not limited to that. For example, user reports show that iOS version upgrade may also wipe a site's stored data in Safari.
Ultimately, in many situations (including my webapp), it only makes sense to use persistent storage because if the site's data is silently wiped by the browser in the background, who will be the culprit from the user's perspective? Obviously the bad site, not the browser's internal logic that the user most certainly will not be aware of.
If a site wishes to change to the persistent mode, it has to use navigator.storage.persist() method from the StorageManagager API. This would prompt the user asking if they really want the site's data to be persistent. Turns out, however, that this is only allowed in secure contexts, without a real reason, probably simply to advocate HTTPS.
Suppose the persist() method would be allowed also on HTTP. A man-in-the-middle attacker could then
- Inject a script to call the method and prompt the user for persistent data storage on whatever site being attacked. It still requires the user to explicitly give the permission via the prompt. In case the user even does, only effect that would have is the attacked site being able to storge data with a better guarantee in the browser. No harm done.
- Use his/her powerful MitM position to attack any (other) insecure site, possibly stealing actually valuable information or running scripts with greater consequences.
From a position with that much power, any attacker would choose option 2 over option 1. There is no actual threat and no real reason to require such degree of security for this method in the first place.
It is still unclear why would a secure context be required for this method. Lucky for us, the Storage API specification development process is publically on GitHub and after some digging one is lead to issue whatwg/storage#5, which is exactly about this. To our surprise, someone just thought to require secure contexts and it was carved into the stone without any discussion. This highlights the issue of thinking in "HTTPS first" and completely disregarding HTTP without analyzing the concrete situation, its needs or the implications of such decision.
As briefly mentioned above, there are two different ways to make a webapp that can be accessed even without internet connection: AppCache and Service Workers. The former is deprecated, completely discouraged from being used and possibly removed from browsers. The latter is considered to be its superior replacement. As one might already guess, Service Workers come with a catch: they also require a secure context. And that is why my webapp has been still using AppCache (successfully) so far, and not because Service Workers are too complicated (which they still are compared to AppCache).
Requiring a secure context for Service Workers for security reasons is good in theory but does not solve the problem it's trying to solve in the first place. If a MitM attacker wants to do similar kind of damage they could with Service Workers, they could just do it via AppCache that gives them similar means. Even though the latter is deprecated, browsers still support it and the hole still exists - the system is as weak as its weakest point. To actually be effective at solving the issue and have a consistent security model, AppCache support should be stripped immediately. As of now, Service Workers simply advocate HTTPS without creating a more secure environment.
Add to Home screen is a feature of mobile browsers that allows adding sites to the home screen as if they were standalone apps: they have a separate icon and when opened look like a real app, where the browser UI is hidden. This requires some additional configuration metadata specified by a Web App Manifest for browsers to read. This is perfect for my webapp to give users the best experience possible but it's not without its caveats.
iOS Safari also supports A2HS but differently from a manifest, because Apple.
Chrome (on Android) allows doing exactly that using "Add to Home screen" in the context menu. If the site has a manifest, it will be honored; if it doesn't, the added icon will simply be a bookmark to open the browser.
Furthermore, Chrome has a feature called Web App Install Banners, which automatically suggests adding a site being visited to the home screen if certain criteria are met. Besides having a manifest, here HTTPS is also a strict requirement. This, however, is not a major limitation because it only affects displaying of the banner, not the button in the menu, which can be used regardless.
Firefox for Android (since version 58) also supports "Add to Home screen" but the behavior is quite different. If the site has a manifest and is served over HTTPS, a button appears in the address bar and a similar banner is shown.
HTTP sites with a manifest (like my webapp) don't get this banner nor the button. It is outright impossible in Firefox to add such site to home screen as a web app. There is a button "Page → Add page shortcut" in the context menu but it simply adds a bookmark to open the browser and doesn't consider the manifest.
Here Firefox for Android is using A2HS as a bargaining chip for HTTPS but the two features are not really even related. It's perfectly possible and safe to allow the same with HTTP sites, like Chrome does, but Firefox doesn't and there's no real excuse either.
As time goes by the advocacy of HTTPS and the opposition of HTTP is only going further. Not only will HTTP sites miss out on new features, like the ones described above, they will also be punished otherwise more and more for being "bad" even when the criticism doesn't apply to a particular site. This leads to the topic of having to migrate such site from HTTP to HTTPS. Turns out this is not as perfect as everyone makes it out to be.
For some more background: GitHub Pages supports HTTP and HTTPS if using the github.io subdomain they provide. GitHub Pages also allows using one's own domain for serving static content from GitHub servers. The latter, however, comes with a limitation: with a custom domain HTTPS can't be used as GitHub would not have the private key corresponding to whatever custom domain one might use and have a SSL certificate for. An inquiry about the limitation got the following reply:
Adding HTTPS support for custom domains on GitHub Pages is something that's high on our wish list, though I can't say if/when we may add it to GitHub.
Moreover, GitHub support links this CloudFlare guide, which should provide an existing and working approach to making my webapp be served over HTTPS while staying with GitHub Pages. To do means commiting a HTTPS sin: intentionally making CloudFlare the HTTPS man-in-the-middle by giving them the necessary powers to do their thing. That means trusting CloudFlare with everything, including the correctness of their own software and all of your users' data. Be it CloudFlare or whatever other service, be it caching or just HTTPS addition, to do so means breaking the trust of an end-to-end secure connection to your own server, which is the point of having HTTPS in the first place.
It can be argued that hosting my webapp over HTTP on GitHub Pages is prone to a similar kind of trust issue towards GitHub, which is true but also slightly different. Firstly, it requires trusting fewer parties. And secondly, HTTP would not give a false sense of additional security, which isn't really there.
Of course there's the obvious solution which doesn't rely on third parties: hosting the content from your own server that does HTTPS. Out of the possible choices it is the only sensible option and the point of this subsection was to highlight some pitfalls.
But even this will not solve the migration problems to come.
IndexedDB (and localStorage) adhere to the Same-Origin Policy. According to this policy http://example.com and https://example.com belong to different origins and therefore IndexedDB (and localStorage) access between the two is strictly forbidden. It's true even if the only difference between two pages' paths is the "S" between HTTP and HTTPS.
This limitation, which is very much inspired from web security, has far-reaching implications for the migration. Namely, it is flat out impossible to migrate the data stored for the HTTP version of the site to the HTTPS version of the site. So even if the site got proper HTTPS support, it simply cannot access and copy the data that was stored in the HTTP site. From the user's perspective, this looks like the site being bad and losing user data it's not supposed to, because users have no knowledge of the underlying limitations of the web technology in question, they just see the undesired outcome. Note that persistent storage mode makes no difference here.
This is a massive oversight in advocating the move from HTTP to HTTPS. While HTTPS does have access to more browser APIs due to secure contexts and thus provides a strict superset of features, the process of migrating from one to the other has been completely overlooked, which is the worrying aspect of ever greater pushing towards HTTPS. This is the result of multiple parties creating policy, which is directed towards their own goals and visions, but the outcome is a mess for everyone in between, who has to actually deal with these different policies in practice. Clearly, not enough thought and time has gone into analyzing the possibilities and practical implications to make sure everyone can move forward in synchronization and harmony.
All in all, HTTPS is being forced upon us, in many cases blindly. The need for HTTPS is tied together with numerous completely unrelated features, like the ones above, in an attempt to make HTTPS adoption faster. While this might sound great in theory, it is deeply flawed and hurts sites with less common, yet perfectly valid, setups. HTTPS should not be used as a bargaining chip, in fact, there should be no need for a bargaining chip. If HTTPS was as amazing and perfect as it's made out to be, developers would be rushing to use it regardless; if they don't, then it says something about HTTPS: it's not a "one size fits all" solution to all of the problems, especially when one can't properly migrate to it. The developer of a site should be the one to decide, which features the site should have and which not, not a massively-general-purpose policy that simply fails to capture all the nuances.
The rushed nature of HTTPS adoption is also a problem. Everyone is being forced to move to HTTPS due to pressure from mostly browser vendors, yet there are issues and incompatibilities with other features when doing so, which only become relevant now when it's too late to do anything about it. In the end, users are who suffer the most and while webdevs can try their best, they can't do the impossible, which never occurred to the policy makers who live in their own perfect bubble.