Preloading data in a redux application

A redux app is a chicken and egg problem. Given a particular state, the app should render a certain thing. But... where does that state come from?

Starting from empty

Given a blank state, the app needs to rely on the URL to fetch the right data to populate the state.

In a server-side app, the initial state is determined in exactly this way. Given the initial URL, populate the state, render the app, send it to the client.

In a client-side app the situation is exactly the same, except it's totally different. On the client you have way more contextual information than just the URL. Given the URL, the current app state, a components own props and its internal state... a component must decide which data it needs loaded.

If you build your app client-side first, this leads you follow the pattern of each component knowing how to load its own data. This way, the logic for loading the data for that component is tightly coupled to the rendering of that component. In the on-the-fly world of the client it makes perfect sense to do things this way.

In a server-side environment, however, giving all that power to individual components is a nightmare scenario — your app would need to keep re-rendering until every component has reached its final state. Each rendering pass would trigger more components to load their data... exactly like what happens on the client. However, a server-side app needs to reach that final state in one pass. On the server we can't let each component load its own data on the fly. The solution? It's common to group the "needs" of an entire component tree together by route. That way, when we preload the data for that route, the data for all of its children will also be loaded.

Given this matching route, what data will all of its child components need to load?

What do we know when?

On the server, we really only know the current URL and a matching route. We won't have any access to ownProps or internal react state. We can sort of rely on redux state, but it's dicey.

On the server, for any given route, we will only know store, history and match.

The store is the redux store for your app. It probably has an empty state.
The history is a static version of the react-router history. Basically, it's the current location.
The match is the object returned from react-router matchPath for the current location.

Routes

Let's look at a route structure to get a picture of the problem. Below we're showing a representation of our application's routes. We can use this array on the server-side to match the current location against the route tree to know which ones to load data for.

Notice how DetailComments is nested inside of Detail; we must presume that the comments component is something like a tab or side rail. React-router v4 allows us to nest routes in strange and interesting ways.

Given the routes array below, a URL like /detail/3/comments would load the detail page and the comments page. For this situation, we need to load the data that will satisfy both the Detail and DetailComments components.

const routes = [
  {
    path: '/',
    exact: true,
    component: Home
  },
  {
    path: '/detail/:id',
    component: Detail,
    routes: [
      {
        path: '/detail/:id/comments',
        component: DetailComments
      }
    ]
  }
]

Note: Because of how react-router v4 works, our app might have many nested routes all over the app. The tree above is a simplified view of the problem. More complex routing structures are outside the scope of this document. However, you could imagine that you'd need to represent your whole route tree in a config object like the one above.

Knowing what to load on the server

Using something like the data loading example in the react-router manual we would be able to know which routes to load data for. However, there are some limitations to this approach.

We can only preload routes that are defined in our routes config. Routes that exist in the app as a <Route /> component but not in the routes config will not be preloaded on the server.
We can only load data at the route-level. Child components of a route won't have a chance to preload their own data on the fly. This means we must burden routes with knowing how to preload data for their entire component tree.
Because we're loading at the route-level, we won't know anything about the props of any individual component. We'll need to construct a data loading function that works with only information known to the route (store, history and match).
We should load all routes in parallel (sequentially loading would be slower), we can't rely on the state to hold anything meaningful. The preloading function for DetailComments would need to load data without relying on anything that the Detail component might add to the state. It's technically possible to load data sequentially (only executing DetailComments after Detail has finished), but this would mean that the user has to wait longer than if we ran them in parallel.

Knowing what to load on the client

On the client things are much easier and way harder. Unlike the server where the state is constructed all at once from a blank state, the client side state is totally fluid. As the user interacts with the site, the state will update. As the user navigates from page to page we will need to load new data.

Imagine that the user is on the home page and clicks on a link to the detail page. We would need to load the data for the detail page and then render it. Now imagine they use the back button... we need the homepage data all over again! If the user falls in love with hitting back and forth between these two pages, our app might end up fetching lots of the same data over and over again.

The client has some requirements that are missing from the server.

We need to know if data is currently being fetched. Because our app is (hopefully) interactive while the data is being fetched, the user might trigger the fetching action several times. All fetching actions must check if the data is currently fetching and avoid double-fetching.
We need to know if data was already fetched. Because our app state is long-lived, we might encounter the same data twice. If we're switching back and forth between two pages we need some mechanism for determining if our data exists in the state.
We need to know when data is done fetching. To improve user experience, we need to show loading indicators and hide them when loading is done.

Component-level loading

Unlike the server, where we are restricted to only knowing about routes, in the client we would prefer to load data at the component level.

Imagine that our Detail route has many child components. When we load the data on the server we load the data for whole component tree. However, on the client side our user can interact with the various child components independently. Our independent components need control over their own data.

Pretend our Detail component is some kind of dashboard containing a variety of independent widgets — all related to, but separate from, the detail itself.

Perhaps our user toggles the ToggleBox component, clicks next on the Pagination component and scrolls the InfiniteScroll component. We need to show loading screens for only those specific components. It would be a bad experience if scrolling a box in the side bar showed a loading screen over the entire route. To combat this, each of our components should know which data they need and be able to manage their own independent loading states.

This idea of route-level loading on the server and component-level loading on the client can seem at odds. On the client, we know a few extra things that the server does not.

We know the current app state.
We know our own props.
We know our own internal react state.

How do we reconcile the two?

Bubbling component-level loading up to the route

In order to efficiently handle both use cases, we need to have our component-level loading functions collected at the route level. That way the route can load everything on first render (on server or client) and then the component can take over after that.

To handle this, our route's loadData needs to know about the loadData functions for every component in its tree. We can solve this by keeping a config file (similar to the routes config shown above) in the route itself.

Here's an application structure that we'll be using to organize our routes. You can see that our app allows for nested routes and components. We can imagine that the child routes folder contains a DetailComments route that looks similar to the src/routes/Detail folder.

src/
  routes/
    Home/
    Detail/
      components/
        Detail.js
        InfiniteScroll.js
        Pagination.js
        ToggleBox.js
      routes/ <-- child routes
      loadData.js <-- detail route loadData
      index.js <-- detail route
    index.js <-- all routes

Some ground rules:

Each component (which needs to load data) is expected to export a loadData function.
In order to support server-side rendering each loadData function should accept store, history and match. We can't rely on ownProps or internal react state in a server environment, so the exported loadData function must work without component-level props.
Each loadData function should be an async function (or return a promise). At a minimum, the return value should not throw an error when added to a Promise.all array (hint: you can pass almost anything to Promise.all).
Each loadData function should first check the redux state to see if data is needed before fetching it.

Note: Checking for existing data should apply to server contexts too — even though your data loader will (hopefully) be running every function in parallel. Before initiating the fetch, it is important to mark in the state that some data is going to be loaded (dispatch(beginLoading(id))). Unrelated components can see that flag and avoid fetching the same data twice. This requires some discipline in how "loading" is marked in the state.

`src/routes/Detail/loadData.js`

Our route needs to define a loadData function that is a combination of all of the component-level load data functions in that route's tree. Because this loadData function will be called server-side, we can only pass it store, history and match — these values will come from the server rendering. You can see an example here and here.

One trick we can do is load the Detail data first and then load all of the child components. This is a way for use to provide a preloaded state for these child components. This should be avoided if possible. You can see that we're loading the independent components in parallel with a Promise.all.

Also notice that we're importing the loadData function from each component. This allows us to re-use these functions within the components to enable client-side data loading while also supporting server-side, route-level data loading.

// pull in all of the component-level loadData functions
import { loadData as loadDetailData } from './components/Detail'
import { loadData as loadInfiniteScrollData } from './components/InfiniteScroll'
import { loadData as loadPaginationData } from './components/Pagination'
import { loadData as loadToggleBoxData } from './components/ToggleBox'

// export our route-level loadData
export default async (store, history, match) => {
  // load the detail first to prime the store
  await loadDetailData(store, history, match)

  // then load all of the independent components
  await Promise.all([
    loadInfiniteScrollData(store, history, match),
    loadPaginationData(store, history, match),
    loadToggleBoxData(store, history, match)
  ])
}

`src/routes/Detail/route.js`

Our detail route will import the route-level loading function and add it to the route definition. Notice that child routes (like DetailComments) are imported as an array. This structure allows the route tree to be maintained locally.

import Detail from './components/Detail'
import loadData from './loadData' // async function, for server-side loading
import routes from './routes' // array of child routes

const route = {
  path: '/detail/:id',
  component: Detail,
  loadData,
  routes
}
export default route

`src/routes/index.js`

At the top level, we can export all of our routes as an array for use in the server, like here. You can imagine that the file in src/routes/Detail/routes/index.js looks very similar.

import homeRoute from './Home'
import detailRoute from './Detail'

const routes = [
  homeRoute,
  detailRoute
]
export default routes

heygrady/preloading-data-redux.md