Response to https://twitter.com/fisherwebdev/status/526646648311906304

526646648311906304.markdown

This is a response to Bill Fisher regarding experience with Flux:

@abdullin Also, can you clarify what you mean by "solution structure"? I am thinking about revising the examples soon.

Currently all flux samples (that I've seen) group files into folders based on technical similarity. For example, stores go with stores, action creators reside in the same folder shared with the other action creators.

This pattern works quite well for smaller projects. It feels especially good for the sample projects of various MVC frameworks, when you have just a bunch of controllers, models and views.

However, as we discovered on some production projects, such approach doesn't scale well. At some point you end up with dozens of technically similar files per folder and logically messy solution.

1. Doesn't scale well.
2. While you work on a component, you have to suffer from extra context switching overhead (jumping between various folders).
3. Solutions become more "entangled" than needed.
4. More complex IDE features are needed to support the workflow (e.g. context-aware navigation and completion).
5. More merge conflicts, since it is harder to bound work by a feature (and communicate that).

We discovered that aligning solution with the domain model leads to a better design in the long run. E.g. we try to group files by their functionality or feature. This would mean, that different technical elements of a single feature could be squashed into one folder or onto one file. For example, for Flux we are considering to have a single folder for "News feed", which would contain all related action creators, fetchers and stores. Of course, there could be some other components that the this chat page would use (e.g. avatars, like buttons or user stores), such components would reside in their own folders. On the overall, such component decomposition is requires more effort than simply grouping files by class type. However, we value both the process (it leads to a deeper insight) and the outcome (solution that is more simple to think and reason about).

Of course, this is just something that seems to work only in a subset of cases I've been exposed to. There can easily be a deeper pattern which I fail to recognize.

This is a difficult problem, and I think it's similar to how we approach naming variables, objects or methods. In naming, it's important to focus on what the method does, or what the object is, rather than how we will use it. Naming according to use, rather than functionality, leads a team of engineers to accidentally write the same method twice. Uses change, but functionality is consistent.

Organizing view components, action creators, stores, and auxiliary utilities based on their logical domain would work well in some situations, but this obscures their cross-domain potential. Many parts of a Flux application can be cross-domain. For example, actions are not tied to a single store. We often have a list of FooActionCreators that correspond to a FooStore, but a BarStore could easily need to respond to one of the actions created by the FooActionCreators. Likewise, it's very likely that our controller-views will need to be listening to multiple stores.

When I'm hunting for an existing action creator, I'd prefer to be opening a single directory and then see a list of all the action creator modules in the application. This puts the entire application's API in a single place. I can only imagine how frustrating it would be to open the foo directory, not find what I'm looking for, and have to navigate out to the bar, baz and qux directories to find the right method.

Likewise, it's a good idea to keep the action types in a constants file. This way, when you open that one file, you'll see a list of every action type in the entire application. Lately I've been careful to name action types in the format of: DOMAIN_VERB or OBJECT_VERB, where the object is the thing being acted upon, and the verb describes what is happening. For example: MESSAGE_CREATE, MESSAGE_DELETE, IMAGE_SELECT, PLAYBACK_STOP. This naturally groups the action types in my constants file in a way that makes it easy to go back and find one.

I could more easily imagine the benefits or organizing React components based on their UI specifics. Again, however, when we build well-designed reusable React components, then we need to name them and organize them based on what functionality they really provide, rather than how we happen to be first using them.

Bill, thank you very much for your detailed answer. That is very valuable.

This is the short answer first, follower by a much longer read on the reasoning. Please feel free to skip the second part with my ramblings.

We try to have a solution structure that is aligned with our design, just like the rest of the code. This perspective is similar to the principles of Domain-Driven Design by Eric Evans. The implementation comes with a set of tradeoffs:

• In "code duplication vs wide reuse" we tend allow some code duplication, gaining better decoupling and simpler code.
• In "use of tooling for analysis vs leverage solution structure" we tend to prefer the second option and pay the price of additional design effort required to make it convenient. We value that design effort, since it tends to drive us towards a deeper insight.

With these trade-offs in mind, in our Flux solution we plan to:

• Put all actions (type constants and probably constructors) into a single module that is easy to discover and explore. That is the API.
• Group action creators, stores and react components into modules by their purpose (represented by folders); iterate on that structure until it becomes intuitive to navigate and reason about. Modules can use other modules, of course (e.g. including components or calling action creators).
• Keep stores as internal implementation details of the module, limiting their reuse and introducing some code duplication (which could be dealt with later). Implementation of action creators is also a private module detail.

Could this work out in Flux, Bill?

Long answer - reasoning

I'll try to start the answer by showing similarities between event-driven backend design and design elements we discovered so far in Flux architecture. Some of these similarities may be very superficial, I hope you would point out such cases.

This answer reflects current and past experiences of our team at HappyPancake (including Tomas Roos and Pieter Joost). However, since we iterate fast, conclusions of this answer might be outdated in weeks from now. Design principles behind them would probably stay.

In short. Breaking down the entire system into a bunch of decomposed modules which are designed to work together - worked great for us in the past. Making them event-driven provided a lot of benefits on top of that. This is the reason why we try to reuse the patterns in frontend design as well.

Actions -> Events

Events are the most important design element: named DTOs which describe outcome of something that has happened in the past. They are carefully crafted to reflect meaningful outcomes of some real-world processes.

Events are published to all interested subscribers across the entire system, telling about something that has happened. They are generally named in the past tense to emphasize that: UserRegistered, PaymentProcessed, MessageSent (you can't change the past, you just have to deal with it). Event declarations are put into a shared package which acts as a common place for finding them (DDD term for that would be Published Language).

For example, at our current project, in order to find name of an event, you just need to type hpc.Contract... (equivalent of action types) or hpc.New... and then the autocompletion will kick in. All event types and constructors reside in the same shared package.

Flux actions seem to have similar semantic. They are published to multiple subscribers and reflect the outcome of something that has happened in the past (USER_FETCH_SUCCESS, MESSAGE_SEND_FAIL etc). Hence we are going to keep them in a single place.

Action creators -> Request Handlers

API Request handlers in our backend are one of the places, where some branching logic can reside (they can also be known as command handlers in CQRS). Request handlers can talk to the database, other request handlers or even 3rd party services to perform their job. As a part of that process they could publish events telling about something that has happened in the past.

These request handlers are grouped logically in modules. This grouping is a result of multiple design iterations (this is easier and faster than it sounds), helping us to intuitively find them and reason about them.

For example, to send a chat message you would need to perform an HTTP POST to /chat/send, to push a typing notification - HTTP POST to /chat/typing, to like a photo - HTTP POST /like/photo, to get a public profile information for display - HTTP GET on /profile/:id (it will publish ProfileVisited along the way).

Flux action creators seem to be the equivalent of request handlers. They represent a behavior, could interact with some API and publish actions telling about the outcomes of the process. We will probably try to group them in modules, according to their functionality.

Stores -> Denormalizers

Event denormalizers are classes which subscribe to events and project them to some local state, specific to the module. This state can be used to present information for display or to make some decisions.

Event denormalizers (with their state) are purely event-driven state machines. This makes them very easy to reason about.

In our code, event denormalizers are an implementation detail of a module. There is a lot of duplication happening in there. We aren't very fond of that, but don't see it as a major problem.

We think that Stores are an equivalent of denormalizers in the Flux architecture. They, subscribe to events (and that is the only way to change their state) and generally have only getters that are publicly accessible.

We consider grouping Stores together with related React components and action creators into modules.

Unidirectional flow

We have a unidirectional flow of data in our system, too. It is driven by events and gives us a mental framework for driving design and reasoning about the behaviors.

API request -> [Handler] -> Events -> subscription -> [event denormalizer] -> state is changed. 

Modules

Module in our solution is a logical grouping of related behaviors. Modules are represented in our solution as directories in the root folder. This makes the design very visible and easy to grasp.

Because we try to reflect design in the code (this pushes further ideas of Eric Evans), code and solution structure act as an overview to our system. We don't need a tooling or docs to see solution structure (which is hard to achieve, if files are grouped technically). Use-cases can be used to generate more detailed documentation, I'll talk about them later.

We can evaluate complexity of a module simply by looking at the folder, measure it and chose to decompose the most complex ones. This helps to implement the design constraint of keeping the entire solution relatively simple. And that helps us (forces, to be more precise) to learn more about the domain.

We don't share much code between the modules (at least on the initial stages of the project). This means that we may have some duplication between them (e.g. somewhat similar handling of MemberBlocked event in almost all modules).

I personally like that duplication, because it allows to avoid premature abstractions. My teammates members like it less.

However we all agree, that some duplication allows us to have a better decoupling and decomposition in the design. This pays off very well.

Besides, if there is too much duplication, this probably indicates on some missing design concept, which we need to discover and express in the code. This will make the solution less weird again.

If we ported that approach to the Flux, that would mean that Stores and action creators would be grouped into the modules, becoming their implementation detail. Modules would be represented as folders. Actions would be stored in the single module.

Testing

We don't test implementation details much. Instead, we test module behaviors, expressed through their contracts. Module contracts in this case are backend-specific:

• module can subscribe to events;
• module can respond to requests;
• module can publish events;

Since we have a unidirectional data flow, all module behaviors can be expressed in the form of use-cases for both queries and change requests:

• Given certain events that happened in the past;
• When we perform a request
• then we expect certain events and/or response.

This use-cases are used:

1. To verify behaviors (regression testing);
2. To generate human-readable module documentation;
3. To visualize dependencies of a single module;
4. To visualize dependencies between various modules in the system.
5. Visually see causalities and flows in the solution.

Since use-cases capture behaviors using the module contracts (events and request/response contracts) AND since these contracts change infrequently, our use-cases aren't very fragile.

We think, that it might be possible to describe Flux modules in somewhat similar way (reflecting backend use-cases to front-end):

Rendering (equivalent of backend query logic):

• Given certain actions that happened in the past;
• When we render component X;
• Expect certain HTML assertions to be true (there could also be an action published)

Behaviors (equivalent of backend change request):

• Given certain actions that happened in the past AND state of API;
• When we call action creator A with parameters XYZ
• Expect certain actions to be dispatched.

I believe that these tests would allow us to get the similar benefits in Flux solution:

1. Verify behaviors
2. Provide auto-generated documentation for modules (and React components in them)
3. Visualize the solution, flows and dependencies

Summary

I tried to provide a brief overview of some artifacts of our design process and describe similarities that we see with the Flux architecture.

Of course, the application domains are quite different (backend vs front-end), but we hope to bring a similar design approach and reuse some concepts in our solution. There are simply too many benefits of that. We'll see how it works out (and if it even works out).

It is great that Flux architecture supports various design approaches within one paradigm. Besides, various components implemented for Flux (e.g. isomorphic components from Yahoo) can also be used in projects with different approaches. For that we need to thank the people behind Flux :)

Questions

Bill, based on your experience with Facebook Flux, what problems do you see with our approach?

Thanks to Pieter Joost van de Sande for the help with this response.

Hi Rinat,

This treatise is wonderfully thorough and quite thought-provoking. I loved reading through it and comparing your approach to our own. Many parts of this resonated with me, but of course there are details where we differ. Some of those details are more important than others.

modules

I initially stumbled on some of the terminology you have used here. For example, at Facebook we use the term module to signify a CommonJS module, which has its own closure. This has a very specific JavaScript-oriented meaning for us. But I see now that you are talking about directories used purposefully and meaningfully to support a specific way of thinking about an application.

I think your approach to module-directories could work quite well, given how you have described it. We try to do the same thing with naming conventions, but I would be interested to try the same conventions made more concrete in the directory structure. Like you said, it might help to focus how we think about the various domains in the application.

events, actions and dispatches

More significantly, I am concerned about the use of the term event being used liberally here to describe a number of different processes. While I understand that you might be talking about events in a more abstract sense, I would not want to overlook that Flux dispatches very intentionally avoid being JavaScript events, which are uncontrolled, asynchronous processes.

The mechanics of the dispatcher are vital to how we've been able to keep our applications resilient as they grow, especially the waitFor() method, which allows the stores to declaratively depend on one another. JavaScript promises are better than simple events, but we've found they still don't offer us enough control to ensure our code doesn't become a tangled mess. The dispatcher was our solution to the problem. The dispatcher is the core of what makes Flux different from other approaches. It is the mechanism that enforces the unidirectional data flow, helps us manage the complexity of dependencies between stores, and allows the stores to be completely in control of themselves and their data domain.

stores: limiting reuse and duplicating code

This is an interesting, somewhat radical idea. But I'm having trouble imagining how different parts of the application that rely on the same data stay in sync with each other over time. I understand that we could duplicate the code in multiple places, but we would need to maintain that duplication, and I can't imagine that would happen without a small failure rate. Perhaps a small amount of duplication is not terrible, however, and like you said, it can provide a way to avoid dependencies between stores.

the past

I loved this:

[Actions/events] are carefully crafted to reflect meaningful outcomes of some real-world processes. ... They are generally named in the past tense to emphasize that: UserRegistered, PaymentProcessed, MessageSent (you can't change the past, you just have to deal with it).

This really speaks to me. I will always name my actions in the past tense from now on!

Also:

Put all actions (type constants and probably constructors) into a single module that is easy to discover and explore. That is the API.

This is what I have been doing recently too, but we use simple action creators that are nothing more than a helper function that sends an action through the dispatcher. Recent projects of mine have been putting them all in one file. The large applications I've seen at Facebook have taken a more domain-driven approach to this, and I haven't yet been part of a large project that put them all in one file. I'm sure the number of the action creators would eventually get to the point where we would need to split them up. On the other hand, why break these into separate files until we really need to do it? It's much nicer to have them all in one place.

testing

It sounds like you test in very similar way to the way we test. If interested, I recently wrote a blog post about testing Flux stores with our automocking unit test framework, Jest.

Thanks again for writing such a thoughtful response. I'll be thinking it over for quite a while, and I'm sure I'll experiment with some of these ideas in my future projects.

Hi Rinat, Bill,

Thanks for these insights, they are very useful.

I was recently thinking more about actions/events in the past. Because I do love the idea the actions are outcomes of real-word processes and thus should be named in the past because they can't change. But I have some concerns about this, and I was wondering what you think about it.

Validating user input

At the moment my stores are responsible for the validation and I do something like this:

// action -> MESSAGE_SUBMITTED

messageStore.messageSubmitted(message) {
  if (invalid(message)) {
    setErrorNotification()
  } else {
    // do things in the store
    // deleteErrorNotifcation()
  }

  emitChange()
}

So it's possible that a message is not valid so submitted seems wrong here. It's only after the validation that the message really is submitted.

Failed on the server

An action can fail on the server, so, again, submitted seems wrong.

Possible solution

It's the responsability of the api (actionCreator) to call the server, and maybe also to validate the messages, so it would be possible to handle all things that can fail in the api and change my api interface like this:

for validation:

api.messageSumbit {
  if (invalid(message)) {
    dispatch('MESSAGE_INVALID_SUBMITTED')
  } else {
    dispatch('MESSAGE_VALID_SUBMITTED')
  }
}

with submitting to the server:

api.messageSumbit {
  if (invalid(message)) {
    dispatch('MESSAGE_INVALID_SUBMITTED', message)
    // Invalid messages are not send to the server
  } else {
    dispatch('MESSAGE_VALID_SUBMITTED.SEND', message)
    sendMessageToServer(message).then(function(result) {
      dispatch('MESSAGE_VALID_SUBMITTED.RESOLVED', message)
    }).catch(function(error) {
      dispatch('MESSAGE_VALID_SUBMITTED.REJECTED', message)
    });
  }
}

The inspiration for my solution came mostly from here: http://www.code-experience.com/async-requests-with-react-js-and-flux-revisited/

This has the advantage that all actions can be in the past again, but it makes the solution more complex. Furthermore, as soon as localstorage is added, that could fail too and that must also be taken into account.

On the other hand, syncing between localstorage and the server is not trival, so dealing with it explicitly wouldn't be so bad.

Is this the way you would handle this, or do you do this differently?

Hi Thomas,

Yes, this is essentially how I would do validation, but only if my validation was somewhat simple, like a regex to validate that the text looks like an email address.

However, if the validation requires me to examine the state of the stores, then I would want to move all of this to the store. If I was concerned about race conditions with the resolution of the API call vs. the current dispatch, I would manage that with a check to Dispatcher.isDispatching() in the action creator called from the response handler.

Keeping the validation and the API call in the action creator has the advantage of keeping action creators out of the store. The advantage of having the validation and API call in the store, however, is that application logic resides in the store along with state management.

See also: http://stackoverflow.com/questions/27363225/tracking-ajax-request-status-in-a-flux-application/27459732#27459732

Semantic tangent:
In our action types, we often don't have an agent or other details. They primarily have the form of OBJECT_VERB. There is no agent. In this case, the user has actually submitted the message to the application. They pressed the enter key on their keyboard. And this is what the action is saying about something that happened in the real world. The application has not submitted the message to the store or the API, but this is an internal detail. The results of an XHR may not seem like something happening in the real world, but I would argue that they are -- we become painfully aware of this while developing for the mobile web.

Hello Bill and Rinat,

I would love to read more enriching discussions like this. I have to admit that since the first time I saw Flux, it was really familiar to DDD & EventSourcing (which is a great concept in my opinion). This is my output for the discussion:

1. Store is something called bounded context in DDD terms, therefore it's quite common to duplicate state between multiple stores
2. Action represents interaction (be that a user's interaction or API callback) with the application and therefore it's something that happened in the past and there should not be any way to reverse it. (Rinat you have already mentioned it)
3. Action naming should always reflect what actually happened rather than expose implementation detail

I wish more developers would be aware of those points before they start using Flux in production, as it would lead to much cleaner design. Flux is really powerful weapon but can harm badly if it's not used properly.