A reading list that I'm collecting while building my Rust ES+CQRS framework: https://github.com/brendanzab/chronicle

reactive_systems_bibliography.md

Functional, Reactive, and Distributed Systems Bibliography

Books

• CQRS Journey: Free ebook from Microsoft (Print book available for purchase)
• Functional and Reactive Domain Modeling: A high level overview of how to build up domain models using free monads and interpreters.
• Reactive Microservices Architecture: Free booklet from Lagom and O'Reilly
• Reactive Messaging Patterns with the Actor Model: Applications and Integration in Scala and Akka
• Domain-Driven Design: Tackling Complexity in the Heart of Software

Event Sourcing and CQRS

• The Reactive Manifesto
• CQRS Journey Reference 2: Introducing the Command Query Responsibility Segregation Pattern
• CQRS Journey Reference 3: Introducing Event Sourcing
• Microsoft Docs: Command and Query Responsibility Segregation (CQRS)
• Microsoft Docs: Event Sourcing
• Microsoft Docs: CRUD, Only When You Can Afford It
• Martin Fowler: Bounded Context
• Martin Fowler: CQRS
• Martin Fowler: Event Sourcing
• Lagom core concepts: Event Sourcing and CQRS
• Concept Map » CQRS And Event Sourcing
• The H: CQRS – a new architecture precept based on segregation of commands and queries
• Hyper Lambda: Eventflow - CQRS/ES in Haskell
• James Roper: CQRS increases consistency
• Confluent: Stream processing, Event sourcing, Reactive, CEP… and making sense of it all

Formal foundations

CQRS, DDD and Event Sourcing are informally defined design patterns. It would be nice to see a precise, theoretical foundation for these ideas. This could help find the signal in the noise and reduce confusion for newcomers. Interstingly, akka-persistence is an implementation of event sourcing that seems quite similar to state machine replication. Aggregates in this instance corespond to PersistantActors.

• Wikipedia: State machine replication
• Wikipedia: Mealy machine

Aggregate design

• Vaughn Vernon: Modeling Aggregates with DDD and Entity Framework
• Kevin Webber: Modelling Reactive Systems with Event Storming and Domain-Driven Design

Consistency, Distributed Transactions, and Sagas

• Life Beyond Distributed Transactions: An apostate's opinion, Pat Helland
• CQRS Journey, Reference 6: A Saga on Sagas
• Sagas, Hector Garcaa-Molrna, Kenneth Salem - The original paper!

CRDTs and Strong Eventual Consistency

• Conflict-free Replicated Data Types, Marc Shapiro et al.
• A comprehensive study of Convergent and Commutative Replicated Data Types, Marc Shapiro et al.
• Evaluating CRDTs for Real-time Document Editing: Mehdi Ahmed-Nacer et al.
• "Consistency without consensus in production systems" by Peter Bourgon
• Readings in conflict-free replicated data types

Dedalus and Bloom

• Learn Datalog Today - interactive tutorial designed teaching the Datomic dialect of Datalog
• BOOM Papers
• Consistency Analysis in Bloom: a CALM and Collected Approach Peter Alvaro et al.
• Bloom Language
• Bloom: Big Systems from Small Programs - RICON East 2013
• "I See What You Mean" by Peter Alvaro

Split brains

• http://doc.akka.io/docs/akka/akka-commercial-addons-1.0/scala/split-brain-resolver.html

Domain modelling using types and algebras!

Events and business rules map really nicely onto free monads and interpreters.

• http://www.stephenzoio.com/free-monads-and-event-sourcing/
• http://www.stephenzoio.com/free-monad-event-sourcing-interpreters/
• [Functional and Reactive Domain Modeling]
• https://fsharpforfunandprofit.com/category/DDD/
• http://lucasmreis.github.io/blog/type-driven-domain-modelling-part-1/
• http://lucasmreis.github.io/blog/type-driven-domain-modelling-part-2/
• http://lucasmreis.github.io/blog/type-driven-domain-modelling-part-3/

Designing a CQRS HTTP API

It can be a little hard to match up CQRS with traditional REST routes. There are a number of approaches I have seen mentioned, ranging from embracing RPC-style payloads to trying to translate CRUDful REST calls to commands.

• https://github.com/aliostad/m-r#readme
• https://www.slideshare.net/fatmuemoo/cqrs-api-v2 - this is the one I like the most!
• https://www.infoq.com/articles/rest-api-on-cqrs
• GraphQL seems like it might be a good fit for exposing CQRS
  • https://github.com/sangria-graphql/sangria-subscriptions-example - example of using GraphQL

Migrations and Schema Evolution

• Akka persistence: Schema Evolution
• The Dark Side of Event Sourcing: Managing Data Conversion, Michiel Overeem et al.
• https://speakerdeck.com/overeemm/the-dark-side-of-event-sourcing-managing-data-conversion
• Axon Framework: Event Upcasting
• Michiel Rook: Event versioning (or why I sometimes modify the event store)
• Migrating JSON with lens and foci
• slashdotdash/eventstore-migrator
• Haskell libraries
  • https://hackage.haskell.org/package/safecopy
  • https://hackage.haskell.org/package/api-tools
• CQRS Journey, Journey 6: Versioning Our System

Table Schemas for Event Store Implementations

• http://stackoverflow.com/questions/7065045/using-an-rdbms-as-event-sourcing-storage
• http://stackoverflow.com/questions/23438963/relational-database-schema-for-event-sourcing
• https://lostechies.com/gabrielschenker/2015/07/13/event-sourcing-applied-the-repository/
• https://github.com/dnvriend/akka-persistence-jdbc/blob/master/src/test/resources/schema/postgres/postgres-schema.sql
• https://cqrs.wordpress.com/documents/building-event-storage/

Alternatives

Datomic is a very interesting database! It splits the read side from the write side, and uses an append-only log for storing data. Unlike DDD-style event sourcing though, events don't correspond to domain events. Rather they are factual assertions that can be mapped to the more traditional 'CRUD' operations:

CRUD	Datomic
Create	Assert
Read	Read
Update	Accumulate
Delete	Retract

The nice thing about this is that it allows one to hook into a more powerful and flexible query language than SQL, Datalog, that allows you to do historical querying. There have been questions raised about Datomic's scalability though (citation needed), and the database is tightly coupled to Clojure.

Contrarian Views

Some reality checks:

• Udi Dahan: When to avoid CQRS: "Most people using CQRS (and Event Sourcing too) shouldn’t have done so."
• Dan McKinley: Choose Boring Technology
• InfoQ: A Whole System Based on Event Sourcing is an Anti-Pattern
• Some random person on HN: "I have worked on, or cleaned up, 4 different CQRS/ES projects. They have all failed. Each time the people leading the project and championing the architecture were smart, capable, technically adept folks, but they couldn't make it work."

Implementations in various languages

• C#

  • cqrs-starter-kit
    • github
  • m-r: Greg Young's 'Simple CQRS example'
    • github
  • m-r (RESTful): An altered version of Greg Young's 'Simple CQRS example', showing how one might integrate it with a REST API
    • github
  • NEventStore
    • github
  • ncqrs
    • github

• Clojure

  • capitalone/cqrs-manager-for-distributed-reactive-services: Accompanies the Strange Loop talk, Commander: Better Distributed Applications through CQRS and Event Sourcing. Uses a single RPC-style /commands route for submitting commands to the service.

• Elixir

  • Building a CQRS/ES web application in Elixir using Phoenix
    • slashdotdash/commanded: CQRS framework that like [lagom/lagom], uses a supervisor with actors for command enusuring commands for different aggregates can be handled in parallel while maintaining ordering. Unlike Lagom, it does not handle distributing writers across nodes, so therfore does not need to handle netsplit scenarios. Uses slashdotdash/eventstore for persistence.
    • slashdotdash/commanded-audit-middleware
    • slashdotdash/eventstore: CQRS event store using PostgreSQL for persistence
    • slashdotdash/eventstore-migrator

• Haskell

  • abailly/hevents: A very incomplete event sourcing library using extensible effects.
  • jdreaver/eventful: A more fully fleshed out event sourcing library with both in-memory and SQLite event store implementations. Uses an ExpectedVersion type to attempt to protect against concurrency errors when multiple writers are accessing the event store.
  • Fristi/Aggregate.hs: A simple Gist showing an Aggregate type class with associated types for commands and events.
  • http://www.hyperlambda.com/posts/cqrs-es-in-haskell/
  • https://abailly.github.io/posts/cm-arch-design.html#models

• Java

  • AxonFramework
    • website
    • github
  • Concursus
    • website
    • github
    • api
  • Eventsourcing for Java (es4j)
    • website
    • eventsourcing/es4j
  • Sourcerer
    • github

• Ruby

  • sequent:
    • github

• Rust

  • Chronicle: An experimental CQRS+ES framework in Rust by yours truly! :)
    • github

• Scala

  • akka-persistence
    • docs
  • akka-ddd
    • github
    • examples: ddd-leaven-akka-v2
  • Functional and Reactive Domain Modeling code examples
    • book
    • github
  • Lagom: A reactive microservice framework built upon Akka persistence. Command ordering is enforced by having a single actor managing command handling for each aggregate instance.
    • website
    • github
    • docs
  • Eventuate: Event sourcing framework with CRDTs and eventual consistency. There are some excellent design documents on the website.
    • docs
    • github

reactive_systems_glossary.md

• aggregate: a collection of entities that cluster around a single root entity, referred to as the aggregate root.
• aggregate root: the entity whose uuid corresponds to the uuid that identifies the entire aggregate
• command: a message corresponding to a request to change the state of an aggregate.
• entity: some data with an associated uuid.
• event: a message that records an aggregate's state transition.
• event log: an append-only record of all the events applied to the domain model.
• read model: read-optimized caches that are subscribed to updates on the write model. Multiple read models can subscribe to a single read model.
• uuid: a universally unique identifier.
• write model: the source of truth of the state of a domain. This is often an event log.

I was just made aware of a recently-updated version of Life Beyond Distributed Transactions.

Cheers for that, updated!

I too like the REST API that you mentioned liking the most and I've actually implemented something very similar. However, when I did so, I had to utilize a data store to track things like status. Most outlines of the CQRS/ES architecture just assume you're doing a fire-and-forget on commands that end up flowing through a complex set of asynchronous queues and pipelines before anything makes it into a datastore. Similarly, you cannot rely on the read side to give you a picture of the resource because it could be an older snapshot. You might end up thinking you need to submit the command again. If the commands are idempotent, that wouldn't matter to the state of the system, but it would possibly waste the user's time or confuse them.

So I'm curious how you would actually implement this in CQRS/ES? Would it involve adding a command store to the picture that could then be updated as things happen downstream? Something needs to be able to recall what commands were initiated and whether they're still running, failed, complete etc. Calls to the API have to have somewhere to look to determine the correct response, but this is not addressed in the discussion of the API. What are your thoughts on this?

There s been a new rust es library coming into play, have u tried it out ?