This is in response to This comment I got on the t18s announcement. This was too long, so reddit wouldn't allow me to post it there.
Hi. I saw inlang's tweet yesterday and am very excited about what you are building!
I looked at the repo you linked and agree that the API is very similar. However, I did arrive at this on my own.
Let me walk you through how I arrived at the design I'm currently at. I want to write a blog post about this anyway, so I'll take this opportunity to write a draft. Sorry if this gets long.
At work I mostly use the Symfony PHP framework & it's translator package. It has a lot of good things going for it, especially the ICU MessageFormat syntax. It's very nice to use. But over the last year or so, I've been growing increasingly frustrated with it. I would constantly forget the names of the variable I needed to pass to each message, causing build errors and lost time.
I'm much more passionate about Typescript & Svelte than I am about PHP, so when I eventually decide to improve the i18n situation, I wanted to do it there.
SvelteKit has a long-standing goal of implementing a standard i18n solution, so that's where I started. I read the Discussion and got intrigued by the idea of compiled messages. Svelte is a compiler, its fitting that it's i18n solution would be too.
I looked around and found a few such compiler. Namely svelte-intl-precompile and unsplash's intlc. But I wasn't happy with either.
svelte-intl-precompile's output isn't pure and requires some functions to be present.intlc's output is pure but more verbose than it needs to be. It's binary is also 3MB. No idea how they managed that.
So I decided to write my own. I used formatjs' ICU MessageFormat parser as a basis and wrote two compilers around it. One for generating compact, pure functions for a message, and another to generate type-definitions.
Apart from the parser this was written from scratch.
I published the compiler in a framework agnostic package called icu2ts (Repo).
## Chapter 2: Starting t18s
This compiler still needed a proper home, so I set out to build an i18n library for SvelteKit, copying it over.
From Symfony, I was used to writing messages in yaml files, so my initial goal was basically to write typesafe-i18n but with yaml files, and compiled messages. By making it a vite plugin I could also react to changes in the yaml files and recompile the messages + update the type definitions live. So that's what I started doing.
I used svelte-i18n's t-store based API as a starting point and built a very fleshed out implementation from there. There were a lot of advantages to using a store. Since they are inherently asynchronous, implementing lazy-loading for alternate messages was very straight forward. I also got a bit Fancy and implemented full HMR support. Whenever a message file changes, the store would update and re-render all messages, without causing state loss. This also made changing locales without reloading the page easy.
I did consider other approaches like using a <T> component. The advantage of that would have been fantastic dev-tools. You could for example have messages that could be edited in the browser & magically update the yaml files using HMR magic. better-i18n-for-svelte is a proof of concept of this approach. I decide against it after running a Poll in this subreddit, where most people preferred the store.
## Chapter 3: A detour into internationalized routing
After having built that, I wanted to tackle internationalized routing. The main hurdle here was just managing a [locale] route parameter.
Because of my architecture I knew which locales are available, so I exposed an isLocale function that could be used as a param-matcher for the [locale] parameter. This worked, but it didn't give me the type-safety I wanted. I wanted the locale parameter to be strongly typed as a string union of all available locales.
So I took a detour and implemented this PR, which adds typesafety to route-parameters in SveltKit. Yes, I changed the Framework just to better integrate with my own library. Devious I know.
With the PR merged the locale parameter was now easier to handle. Using the built in resolvePath function, generating links to translated pages was also easy. I was confident that people would be able to make internationalized routing work with those building blocks.
I then turned to another routing related challenge. Most messages are only used in one place. On a large site, my t store based approach would generate a huge bundle of messages, most of which would be unused.
On my own site, one of the sub-pages is an entirely separate app for image editing. It would be pretty silly to load the messages for it on the homepage.
To avoid this I borrowed another idea from Symfony: Message Domains. This way messages could be organized into separate files, and only the ones needed for a page would be loaded. That's also where I got the <domain>.<locale>.yaml naming convention.
I tried really hard to make individually loaded message domains work with my t-store. I came up with the message-key naming scheme <domain>:path.inside.file, and I combined that with some contrived API that would signal to t18s that a domain was needed, and when it was no longer needed. You would call preloadDomain to "register" a domain as in-need, and unloadDomain to signal that it was no longer needed. This caused a lot of problems and never really worked. This eventually turned into reference counting and a bunch of other stuff I don't want to remember.
What I really wanted was some sort of "depends" api, where you could say "this component depends on these messages", and they would automagically get loaded when the component is used and unloaded when it's no longer used. After trying a bunch of really contrived stuff I eventually realized that I'm dumb as hell.
"This code depends on that other code" is called an import statement! I was trying to reinvent the import statement!
I built a new version where each message domain had it's own import statement, still with a t store. This worked really well. I did briefly consider using one store per message and exporting the messages directly to allow treeshaking but decided against it since this would disallow nesting.
I nest my translations a lot. For Example:
common:
action:
save: Speichern
cancel: Abbrechen
...Would be accessed with $t("common.action.save"). The t store based approach allowed me to nest messages, the export based approach would not. That was a dealbreaker for me.
## Chapter 5: Switching to per-message exports
Eventually when doing something unrelated, I read the MDN documentation on export statements and came accross the export * as alias from "somewhere" syntax. I had never seen this before, but immediately had a hunch what this would allow me to do.
I had to idea to nest multiple message modules inside each other, using these aliased-exports to build a tree of messages. In theory this would still be tree-shakeable, since the tree was entirely immutable.
I tested this by manually by writing a bunch of message files, and confirmed that vite could infact treeshake them. So I decided to ditch the t store, give each message it's own store, and use the export * as alias from "somewhere" syntax to nest them.
I was still using one store per message. They did all depend on the global locale store, but I could trick vite into treeshaking them anyways by using a /** @_PURE _*/ annotation.
You could now use messages like this:
<script>
import * as t from '$t18s/messages/<domain>'
import { get } from "svelte/store";
const save = get(t.common.action.save)();
</script>
<button>{save}</button>(Also, I'm looking at your compiler now, and I can't see nesting support anywhere. Do you have that?)
I had now achieved a feature I never really set out to build: Tree-shakeable, lazy-loadable messages. The problem was that it was horrible to use. Svelte makes it really hard to subscribe to a store that is deeply nested inside a tree. You need to alias it to the top level in order to be able to use the $store syntax.
Another issue was that my HMR implementation was now a lot worse. Since you can't dynamically add or remove exports, I had to reload the entire module whenever a message was added or removed. This undid a lot of the work I had done to make HMR work well.
I had a difficult choice to make:
- Keep the stores and have a bad developer experience when using them
- Switch to non-reactive messages and give up the following: HMR, lazy-loading and in-place locale switching.
I decided to go with the latter. Despite the HMR implementation having been degraded quite a lot at this point, it wasn't really that noticeable. Invalidating the entire module was fine. The lazy-loading of alternate languages was also not that important. When looking at the build-output, vite would usually just inline all messages anyways, so you really didn't gain much from lazy-loading. Treeshaking + Message Domains would easily make up for this.
The only real sacrifice was in-place locale switching. You now had to rerender the entire page when switching locales. This was a bit annoying, but ultimately not that bad.
So i ditched the stores and switched to a non-reactive approach. This felt pretty bad but I believe it was worth it.
Svelte 5 had also been teased at this point, and I have the hope that I'll be able to use one $state rune per message in future, regaining reactivity without sacrificing treeshaking or syntax.
## Chapter 7: My super secret masterplan
The thing I appreciate most about PHP frameworks is that they are complete. They have a solution for everything. Database, Routing, Templating, i18n, etc. They are all built to work together.
I want SvelteKit to be like that, and I built t18s in the hopes that it would be good enough to one day evolve into the default. So whenever I had a choice, the thought of "how easy is this to integrate with SvelteKit" was always in the back of my mind. This drove a lot of my design decisions. For this reason I made a lot of sacrifices in favour of performance and potential future integration into SvelteKit. I abandoned a lot of my initial approaches and ideas in pursuit of this goal. They say kill your darlings, and I did.
## Conclusion I arrived at the design I'm currently at through a long series of trial and error, false starts and side-adventures that eventually lead here. I didn't copy your API, I arrived at it independently. I gave credit to all the projects that inspired me along the way, (except Symfony, I hope I made up for that now).
Big thanks to:
svelte-i18n- For being the basis of my store based approachsvelte-intl-precompile- For inspiring me to build a compilertypesafe-i18n- For giving me hope that typesafe i18n is possibleSymfony- For it's excellent Translator component.
This is an unfortunate case of convergent design. I believe this was inevitable, how else would you implement nested, tree-shakeable messages? Isn't this the only solution, and an API collision inevitable? Wouldn't alines reinvent the same API if given the ECMAScript spec?
After having put this much effort an thought into t18s, I have a pretty good understanding of how much work must have gone into your project. I can empathize with how you feel right now. I can't change that, but I can give you the satisfaction of knowing that the same thing happened to me.
A while back I had the idea to use MJML together with Svelte's SSR capabilities to generate Email code that would work in any client using the familiar Svelte syntax. I felt brilliant! What an elegant solution to the pain that is email HTML. I started building a prototype, but someone else got there first (honestly they did a better job that I would have). Needless to say I was quite disappointed.
## Where we go from here
Over the last couple days a lot of i18n libraries have come out. A real cambrian explosion of ideas! I'm hopeful that one of them will be good enough to become SvelteKit's default. It could very well be yours, it looks promising and I trust Inlang to maintain it well. Good Luck!
## PS
I don't know what you mean with "letting the server set language tag context". T18S does not have an analog to that. We use a global locale store. (It has been pointed out to me that this is a bad idea, so this will probably change)
If you really don't support nesting, and I'm not just missing it in your repo, feel free to use my export * as alias from "submodule" approach. Good ideas aren't meant to be hoarded.
Excellent motivation and write-up, excited to give it a try!