everything-i-know-about-routing.md

Everything I Know About UI Routing

Definitions

1. Location - The location of the application. Usually just a URL, but the location can contain multiple pieces of information that can be used by an app
   1. pathname - The "file/directory" portion of the URL, like invoices/123
   2. search - The stuff after ? in a URL like /assignments?showGrades=1.
   3. query - A parsed version of search, usually an object but not a standard browser feature.
   4. hash - The # portion of the URL. This is not available to servers in request.url so its client only. By default it means which part of the page the user should be scrolled to, but developers use it for various things.
   5. state - Object associated with a location. Think of it like a hidden URL query. It's state you want to keep with a specific location, but you don't want it to be visible in the URL.
2. Path - A pattern used to match locations against to perform some routing. Something like "/users" or "/invoices/:invoiceId"
   1. Dynamic Segment - A dynamic part of the path that will be parsed from the URL and provided to the app, like :invoiceId in "/invoices/:invoiceId".
   2. Wild Card - The rest of a URL at the end like "/users/*", so any url that matches the first portion will match no matter how many segments follow.
   3. Regex - Ability to URLs with regular expressions.
3. Matching
   1. URI
   2. Parameters
   3. State
   4. Query
4. History
   1. Listener
   2. State
   3. History Stack (push/replace/index)
5. Routes
   1. Path
   2. Data
   3. Validation
6. Links
   1. href
   2. state
   3. active status
7. File System API

Definitions

1. Location -

tl;dr On Client Side Routing

When a user visits a webpage, the location is the the first (and probably the only) data the app has to generate a UI. The location is matched against a set of routes, a route is selected, (maybe) data is loaded, and finally the UI is rendered.

After the initial render on the client, a history listener is set up. When the history's location changes--either through the user clicking links or the programmer redirecting with code--the new location is matched against the routes, (maybe) data is fetched, and the app is updated to the new page.

Some implementations will save the data loading of a route for after the new page is rendered (and display some spinners). There are a handful of tradeoffs regarding data loading on page transitions, so we'll save that conversation for later.

History of Client Side Routing

In the early days of the web we just used files for our routes. Drop an HTML file or a PHP file on the server and away you went. A common server was Apache, so occasionally you'd do some configuration there to map URLs to a file that lived somewhere else on your server.

After this, frameworks like Ruby on Rails became popular. Instead of a bunch of files living in the public directory of your website, a single program served all URLs, and the "route configuration" was born. You would declare all the URLs your app could respond to in one place.

Then we started wanting to update the URL without going back to the server for a full page reload and client side routers were born.

[maybe delete this, not sure how useful it is and runnig out of ideas on where I was going with it]

Dynamic vs. Static Routing

There are two types of routing: dynamic and static.

Static Routing

If you’ve used Rails, Express, Ember, Angular, or React Router pre v4, you’ve used static routing. In these frameworks, you declare your routes as part of your app’s initialization before any rendering takes place. React Router pre-v4 was also static (mostly). Let’s take a look at how to configure routes in express:

// Express Style routing:
app.get('/', handleIndex);
app.get('/invoices', handleInvoices);
app.get('/invoices/:id', handleInvoice);
app.get('/invoices/:id/edit', handleInvoiceEdit);

app.listen();

Dynamic Routing

Dynamic Routing is when the routing happens as part of your app's render lifecycle. So instead of having a route configuration somewhere, you'll have some run-time version of it. In the case of React Router v4-5, you have a <Route> component that can be rendered anywhere and knows how to match the URL and renders almost like an if block.

Here's what it looks like in React Router:

const App = () => {
  return (
    <div className="layout">
      <GlobalNav />
      <Route
        path="/matched/while/rendering"
        render={() => <div>This only renders when matched</div>}
      />
    </div>
  );
};

Tradeoffs

Dynamic Routing is great using application data for conditional route matching:

For example, code-splitting just naturally happens, since there is no route config at the top there is no need for a "code-splitting feature" in a dynamic router.

Another example is authentication. There's no need for a mechanism to "protect routes": you simply don't render those routes until the user is authenticated, if they're not authenticated, you just render a login screen instead (at every url, but you just don't have routes yet).

Additionally, if a small screen renders a significantly different UI than a large screen, you simply render different routes given the screen size data that you know at render time.

In general, the ability to add or remove routes as part of your normal render tree opens up a lot of potential features. Unfortunately most web developers and designers haven't really explored what we could be doing. We're all still very used to thinking about routing simply as "pages", the same way we've thought about them since the beginning of the web.

Static Routing is great for data loading:

If your routes live outside the render lifecycle, you're able to work out data requirements before rendering the next page, or even preload data before a link is even clicked. In general it gives you more control over transitions in the client and more comprehensive server pre-rendering. We'll talk more about this in the data loading section.

Static routes are also easier to see all of the URLs your app responds to at a glance, where dynamic routing is often spread across the app. However, with static routes you're likely to have git conflicts with other teams, increase the base bundle size, etc. where dynamic routes don't have these problems. Tradeoffs everywhere.

URL-As-Data vs. URL As-Side-Effect

Most routing developers are familiar with uses "URL as data". Meaning, we use the URL as data to match against to render the UI and navigation simply changes that URL to start over again from the top.

"URL as side effect" is another approach that treats the URL as nothing more than a side-effect, or reflection, of the app state.

A good analogy is the document title. When your app gets into a state where you want to change the document title, you go ahead and do it. URLAASE (lol) is the same. When your app reaches a state you think is worth a new URL, you push it up there.

Some complexity enters with this approach when you boot the app initially. Suddenly the URL is no longer a side-effect but the initial state of the app. This causes two code paths for each URL the app supports, opening up the risk of missing a case in one place but not the other. If you've ever done mobile development without a "URL as data" router, you know how difficult it can be to support every feature as a "deep link" into the app.

Routers don't need to pick one approach over the other. React Router is a "URL as data router" but it also supports treating the URL as a rendering target with the <Redirect> component. An example of using both approaches in the same app is a page that changes the URL as the scroll position changes. As the user scrolls, you can redirect to a new path that represents that position as a side-effect, but you'll also need to write the code to scroll down on both the initial load and user navigation.

Pages vs. Layouts

Page Based Routing

A simple and effective way to think of routing is just matching the url to a path and then calling a function that returns the page. I call this a page-based router.

While page-based routers shine in their simplicity, you have to repeat a lot of the UI across all of your pages, coupling them all together: like the global nav and footer. As you get more contextual on the page, you tend to repeat even more code across the contextual pages. Consider the urls /, /invoices, and /invoices/123.

The route config and views might look something like this:

router.map({
  '/': () => <Root />,
  '/invoices': () => <Invoices />,
  '/invoices/:id': params => <Invoice id={params.id} />
});

const Root = () => (
  <div className="root">
    <GlobalNav />
    <Dashboard />
    <Footer />
  </div>
);

const Invoices = () => (
  <div className="root">
    <GlobalNav />
    <div className="invoices">
      <InvoicesNav />
      <InvoiceStats />
    </div>
    <Footer />
  </div>
);

const Invoice = ({ id }) => (
  <div className="root">
    <GlobalNav />
    <div className="invoices">
      <InvoicesNav />
      <div className="invoice">
        <p>the actual invoice view</p>
      </div>
    </div>
    <Footer />
  </div>
);

Note all the repeated layout code. There are ways to use the component model of your UI framework to help clean this up. One of those approaches is to bake it into your router.

Layout Based Routing

After many years of web development we've learned that usually nested URLs map to nested UI. Look at the classNames and the URLs in our example: / ("root") => /invoices ("invoices") => /invoice/:id ("invoice").

The Ember.js router embraced and popularized this idea; we copied it in React Router, and many other routers have copied it as well. I like to call these "layout-based" routers.

With a layout-based router you are able to develop these routes in isolation without repeating all the layout code. Each page is now just the code that matters for that URL, and it gets rendered inside the parent route--or rather, the parent layout.

Continuing with our invoices example, in a layout based router like @reach/router it looks something like this:

<Router>
  <Root path="/">
    <Dashboard path="." />
    <Invoices path="invoices">
      <InvoicesStats path="." />
      <Invoice path=":id" />
    </Invoices>
  </Root>
</Router>;

const Root = ({ children }) => (
  <div className="layout">
    <GlobalNav />
    {children}
    <Footer />
  </div>
);

const Invoices = ({ children }) => (
  <div className="invoices">
    <InvoicesNav />
    {children}
  </div>
);

You can see the repetition is gone, each level of the route hierarchy is also nested UI.

Web designers and CSS developers tend to think in layouts because it helps maintainability in the long run. Maybe /invoices/123 is no longer nested inside of /invoices, or <Root/> becomes a child of a bigger application. By isolating each layout to be unaware of its surrounding markup, its much easier to make adjustments to the entire site's design because you're able to just move them around where they need to go. Additionally, adding new features to deep layouts runs no risk of forgetting the global footer, or the contextual navigation.

Note that layout based routers are perfectly capable of working like page-based routers.

Server Side Approaches

Before we can talk much about anything else, we should address the three server side approaches used to deliver the application to the browser.

1. "Single Page App"
2. Serverside Pre-Rendering
3. Static File Generation

Server Side Approach: "Single Page App"

A traditional website handles URLs on the server. This can be as simple as an html file for each page of your site: /index.html, /contact.html etc. But in order for a client side router to work, every URL the server receives needs to run the same application. Or, in other words, every URL needs to serve the same html file.

This is why they've been called "Single Page Applications". There's really only a single index.html that handles every URL in the app, and then a router decides what UI to render with JavaScript in the browser.

The body of the html file is typically blank, with a single element for JavaScript (and your client side router) to mount an application into.

<html lang="en">
  <meta charset="utf-8" />
  <title>Bare Bones SPA</title>
  <body>
    <div id="root"></div>
    <script src="the/app.js"></script>
  </body>
</html>

No matter what the URL is, when if this file is served, then the router inside of app.js can read the URL and determine what page to display.

Here's a bare-bones express server to do just that:

const express = require('express');
const app = express();

// serve all static files that exist as the file (CSS, JS, etc.)
app.use(express.static('public'));

// serve every other request to your root index html file
app.get('*', res => {
  res.sendFile(path.join(__dirname, 'index.html'));
});

app.listen(5000);

Now if the user visits /invoices/123 they will get the index.html file and the client side router can decide what to render.

This is how create-react-app plus a router like React Router works.

Server Side Approach: Server Pre-Rendering

Before rendering views in the browser became popular, web developers built their UI server side and then made it interactive client side with JavaScript. Oftentimes they'd end up writing the same views in both places, once for the server render and once for the dynamic updates. They'd always get out of sync.

Thanks to Node.js, we're now able to use the same UI code in the client and the server.

The technique is pretty straight-forward on paper, but gets a bit complex when you start bringing in data loading and code splitting concerns, so we'll ignore those for this section.

Instead of sending down an empty <div> like in the "Single Page App" section, server pre-rendering actually renders the UI on the server and sends down an HTML file the very same as normal server rendered application from several years ago. In fact, you could use popular libraries like React as part of a server rendering framework without sending JavaScript to the browser at all.

For this to work, we need to send the right UI down for the right URL. Suddenly what we think of as a "client side router" is really a server side router as well.

Here's a modified version of the express server we had earlier using a pretend "UI renderer" (like React) and a pretend router (like React Router).

Server Side Approach: Static File Generation

In the early days of web development, before client side routing and even before server side routing, we just had html files. We just put html files into the public facing directory of our web servers and we had a "route".

A silly homepage public directory could look like this:

├── about.htm
├── contact.htm
├── index.htm
└── pics
    ├── IMG_2342.JPEG
    ├── IMG_3532.JPEG
    ├── cat.htm
    ├── dog.htm
    ├── index.htm
    ├── me.htm
    └── me.jpeg

This is why URLs look like a directory structure, because that's what they originally were!

Modern tools have continued to take this approach, but instead of writing separate html files for every page, you can create just the body of the page with JavaScript like its a Single Page app. At build time, the pages are compiled into full HTML files at the same paths where they were authored for the web server.

When a user visits, and an HTML file is loaded into the browser, the JavaScript app, along with a router, is loaded. Every navigation from now on acts exactly like a Single Page Application--no more round trips to the server for assets, just data.

It feels like the old days of just creating files to get a new "route" with all performance benefits of static file servers, except the transition from "plain html" to dynamic, interactive UI is seamless because it's the same code.

Gatsby.js works this way, and Next.js supports it (in addition to full server rendering).

Code Splitting

Code splitting allows us to deliver the only code the current page needs, and then as they interact with the application, more chunks of code can be loaded.

In general, people look to a router to help them split their code by routes. This is an incomplete picture. While, yes, you want to split your code by routes, you also want to split out other features that have nothing to do with routes.

For example, if you've used Gmail you know that you can open up google chat in the side bar. That code doesn't load until you open the feature. Additionally, on the right sidebar, you can open up your calendar, tasks, or notes in Google keep. None of that code is loaded until the features are activated by the user.

While routes are a great place to code split, they are insufficient as the only place to code split. What you really want is code splitting at the component level.

Fortunately for folks using React, we have React.lazy, which turns a normal component import into a renderable, code split, component.

// In this case, Chat would be included in the same
// bundle as the file importing it
import Chat from '/.Chat';

// but with this, it will be split into its own bundle
const Chat = React.lazy(() => import('./Chat'));

Now, the part that splits the code out is the build tool, like rollup, parcel, or webpack, because of the import(...) syntax, nothing to do with React. But React gets involved with React.lazy() because it lets you treat that dynamic import as a renderable component.

No matter how you imported Chat, you render it the same, but if you used React.lazy, then the Chat component code won't load until it has to.

export function SomeParent() {
  return <div>{showChat && <Chat />}</div>;
}

Code-Splitting and Single Page Apps

If you're not server rendering, splitting your code on routes with React Router already works out of the box.

// just change this:
import Calender from './Calendar'
// to this:
const Calendar = React.lazy(() => import('./Calendar'))

<Router>
  <Route path="/calendar" component={Calendar}/>
  {/* etc. */}
</Router>

Code-Splitting and Server Pre-rendering

This can get tricky because on the server, and React.lazy isn't sufficient (yet anyway) for it. Rather than getting into the details of it here, and since it really isn't concerned with routing, check out this guide on how to do it with a package called "Loadable Components". Here's the guide

Code-Splitting and File Based Routing APIs

One huge benefit of file system routing APIs is code splitting, but we'll talk about that in the File System API section instead of here.

Data Loading

Something something about static is easier for data loading but I don't like coupling data to routes because there are so many interactions/component transitions that are data dependent (combobox, tabs, wizards, checkout workflows) but not necessarily routes. HOWEVER, maybe they should be "routes", even if just using location state.

Something something about my hopes for suspense to bend all the tradeoffs here but I'm still not sure it will be the silver bullet I thought it was 18 months ago.

Clientside Page Transitions

When you go from one page to the next there are a handful of options, and it depends mostly on screen size, data dependencies on the next page, and network latency. Most client side routers have taken on the responsibility of loading the data for the next route with some sort of hook. Ember's model hook is the first I've ever encountered. Angular and vue's routers also provide a data loading hook. React Router v1-v3 also provided a hook, but we removed it in v4 (maybe shouldn't have?) because we felt like it was React's job to provide help with data loading and component transitions at the component level (much like code splitting).

Immediate Transition

This is when you click a link and the page immediately transitions to the next page. If you don't have data dependencies, this is a great option. But if you have data dependencies, you end up with a lot of loading indicators on the next page. This can lead to a bad user experience becuase all the spinners show up, dissappear and the page bounces around as different pieces of data land.

Most React apps are this way because React Router (since v4) doesn't help you with data: click a link, get a bunch of flashing spinners. Sorry about that :\

However, this is probably the right choice when the network is slow, we show the user as much information as we've got the moment we've got it, instead of waiting for everything to transtion (like in the next section).

It also lets you put the code that loads the data next to the UI that renders it, which I like.

Pending Transition

A pending transition will keep the old screen rendered while data loads for the next page. In order to do this, you need to either:

• use static routes and put the data loading hooks on the routes, this way the router can look at the new URL, look up the static route data hook, load the data, and then rerender at the new route.
• or put the data loading logic in the link, and instead of changing the url on click, first load the data, then push a new entry into the history once the data lands.

Either way, you end up putting the code that loads the data far away from the code that renders it, which I've never really liked. You also have to wait for all data before you can render anything on the next page.

However, when you have a fast network, this is a much better user experience than the immediate transition approach. You click a link, the screen stays the same momentarily, and then you transition to a page full of content instead of a bunch of spinners and bouncing UI.

Hybrid

When you have a fast network, you want a pending transition, but when you have a slow network you'd rather do immediate transitions. Unfortunately the code required to do these two approaches is widly different.

In an ideal world I think it should work like this:

• When a link is clicked, keep the old screen up and wait for data
  • If data lands within 2 seconds, transition
  • If data does not land, show a spinner on the old page and continue to wait for data
    • If all data lands within another 2 seconds, transition.
    • If all data does not land, but some has, transition to the partial page, with spinners wherever the data is not ready

React Suspense might help us bend the trade-offs of these two approaches at a component level. However, I've considered some APIs in the next release of React Router that would let you declare both critical and non-critical data hooks. The critical hook will delay rendering the page indefinitely, the non-critical would transition to spinners if it takes much longer than the critical data.

Immediate Animation

An approach you see regularly on native mobile apps is the immediate animation to spinners. Instead of just throwing spinners in your face, you can animate after the link is clicked to the next screen, and delay showing any spinners for a beat or two while you try to get the data, then fade the content in.

This is a pretty solid user experience. I've always dreamed of building a set of animated components around react router that did this, but it's very uncommon on the web to animated large screens--partly because browser can really struggle to animate such large amounts of UI. I've been playing around with an iPad Pro lately, and some of the apps there have beautiful animations between screens. I hope we explore this more on the web.

One tricky part of animating is knowing which direction to animate, but we'll cover that in the animation section.

Accessibility: Scroll Restoration

When you click the back button on server rendered and server routed apps, the browser automatically restores the scroll position. When you have a client routed app with dynamic data you have to manage that yourself. Chrome will restore the scroll position as best it can with a client router (pushState), but as far as I know, it's the only browser that does, and it doesn't always work with your data.

The general approach to restoring scroll positions is to record the scroll position per location to memory somewhere. Then, when the location changes, look up the position for that location and restore it.

There are a few concerns:

URL !== locations - many approaches I've seen assume that locations are the same as URLs. That's not the case. You could navigate from A -> B -> C -> A. The browser in a server rendered/routed app will restore the location to page A differently depending on which entry in the history you're at. This can be a design decision, however, so I'm not going to say it's wrong, just saying it's inconsistent with how browsers work natively.

Where to store the scroll positions

If you store them in memory, then moving from one domain to the next will make it impossible to restore the scroll positions as they click back into your app. For example, if the user has this history stack: A -> B -> C -> other-domain.com and they click "back" to C, you don't have the data to restore their scroll position. If they click back again to B, you still don't.

You can't store them on history.state because the values there are set when you push to a new location, not when you leave the location, off by one! Now, you could history.replaceState on scroll, or on a timer or something, but now you'll blow away the history stack and ruin the "forward" button in the browser.

So, you have to store them somewhere persistent and mutable when you leave a location, in a way that you can look them up per location instead of just URLs. You've got one choice really: sessionStorage.

Session storage is a pretty simple API but it's got some edge cases. Safari throw errors incognito when you try to use it, so you'll need to wrap the API and store the values in memory instead of using session storage. I imagine few of us work on porn sites so maybe not a big problem 🤷.

In React Router we have a location.key, which is unique for each location (even if they have the same URL) and store a dictionary of { [key]: scrollPosition }. Then on route transtions I can look up the scroll position based on the location.key.

Don't scroll until data loads

If you have all static content, it's pretty straightfoward to restore the scroll position when the user navigates back and forth. But when dynamic data is involved, you need to restore the scroll position only after the data has landed.

If you're using client side caches of your data you don't have to worry too much about this: the user clicks back, your components pull the cached data synchronously, and then you can restore the scroll position.

However, if your cache has been invalidated at that location, and the user clicks back to it, you'll need to wait for data to load before restoring scroll position. This is another convenience of coupling route data to a route config somewhere and using "Pending Data Transition" approach from earlier in this article.

Accessibility: Focus Management

If you're a keyboard or screen reader user it's important for the app developers to move the focus to the changed content of the page when you interact with it. Most client routed apps today don't do a great job with this. Usually when you click a link with the keyboard or screen reader virtual cursor, the focus stays on the clicked link.

For sighted users, this is fine because they can see the part of user interface that changed, but for many users they don't know if anything even happened. They have to then navigate around the page with the virtual cursor for the changed content, or pull up a menu of the headings and hope they can figure out which are new.

This is where a layout based router has an advantage over page based routers. Where normally this kind of behavior would need to be application specific, a layout based router knows which part of the layout changed (because it is the code that changed it) and then automatically move the focus there.

I started experimenting with this in @reach/router. The implementation isn't perfect, but it got the conversation started and we're confident we'll have a solid, generic solution in React Router soon. Our friend Marcy Sutton did some user testing recently and we've got a handful of takeaways that we'll be implementing.

Animation

Dynamic Routes (Screen Size, Authentication, etc.)

Navigation Based Matching (instagram)

Relative Routes, Links

Navigation Blocking

Queries

History Stack

Mobile

File System API