transaction_on_dry_operation.md

Unhappy path

Before delving into how we support database (DB) transactions, let's first consider how we can handle operations that only pertain to the unhappy path. This consideration is important because, as we will see, it will become relevant when we reach the main topic.

Most of the time, the unhappy path is something managed by the caller (e.g., a controller rendering an error in case of failure). However, there are situations where it makes sense to encapsulate part of the unhappy path within the operations class. For instance, you might want to log the failure somewhere.

When relying on the vanilla #steps method, the implementation is straightforward:

class CreateUser < Dry::Operation
  def call(input)
    steps do
      attrs = step validate(input)
      user = step persist(attrs)
      assign_initial_roles(user)
      step send_welcome_email(user)
      user
    end.tap do |result|
      log_failure(result) if result.failure?
    end
  end
end

However, it is beneficial to automatically prepend the #steps method in #call for a couple of reasons:

• It reduces boilerplate.
• It consolidates the interface of the operation, ensuring it always returns a result object.

This leads us to a single option: calling a hook for the failure case:

class CreateUser < Dry::Operation
  def call(input)
    attrs = step validate(input)
    user = step persist(attrs)
    assign_initial_roles(user)
    step send_welcome_email(user)
    user
  end
  
  private
  
  def on_failure(user)
    log_failure(user)
  end
end

Instead of allowing the registration of multiple hooks, it is better to allow a single one where users can dispatch to other methods if needed. This approach allows us to skip dealing with hook ordering and makes the flow more linear.

There is no need, at least for now, to make the hook method name configurable; on_failure is sufficient.

It's worth noting that we now allow multiple methods to be prepended, as in operate_on :call, :run. Therefore, we need a way to specify which hook to call for a given prepended method. We can achieve this by providing a second argument to on_failure when the method is defined with arity 2:

def on_failure(result, method)
  case method
  when :call
    do something()
  when :run
    do something_else()
  end
end

Database Transactions

Leaving aside the interface for now, we have two architectural options:

1. Wrap the entire #steps call in a transaction:

class CreateUser < Dry::Operation
  use_db_transaction
  
  def call(input)
    attrs = step validate(input)
    user = step persist(attrs)
    assign_initial_roles(user)
    step send_welcome_email(user)
    user
  end
end

Benefits:

• It supports composing operation classes within a single transaction: CreateUser.new >> CreatePost.new

Drawbacks:

• It wraps potentially expensive operations in a transaction, such as send_welcome_email in the example.
• It is not optimized, though not critical, to wrap validate in a transaction.

We find the drawbacks to be unacceptable. If we were to support this option, we would need to use hooks for the setup and success cases:

class CreateUser < Dry::Operation
  use_db_transaction
  
  def call(input)
    user = step persist(attrs)
    assign_initial_roles(user)
    user
  end
  
  private
  
  def setup(input)
    step validate(input)
  end
  
  def on success(user)
    step send_welcome_email(user)
  end
end

In this case, the introduced indirection is also considered unacceptable. While we need to support a hook for the on_failure scenario, dry-operation should prioritize readability when focusing on the happy path.

2. Explicitly wrap the steps that need to run in a transaction:

class CreateUser < Dry::Operation
  use_db_transaction
  
  def call(input)
    attrs = step validate(input)
    transaction do
      user = step persist(attrs)
      assign_initial_roles(user)
    end
    step send_welcome_email(user)
    user
  end
end

Benefits:

• It is explicit.
• It enhances readability.

Drawbacks:

• It requires manual setup.
• It makes it impossible to compose operation classes within a single transaction.

In this case, the drawbacks are considered acceptable. There is no way to completely conceal the fact that we are dealing with a database transaction, and developers need to consider it. Furthermore, one of the key concepts of dry-operation is the decoupling of individual operations. Therefore, we should encourage the composition of operations rather than groups of operations in the documentation.

Interface

A Dry::Operation.db_adapter method could be sufficient to configure how Dry::Operation#transaction works.

We can think of three ORM-style libraries we want to support: ROM, Sequel, and ActiveRecord. Different libraries might require different options, and we can use different option names in any case. For example:

class CreateUser < Dry::Operation
  db_adapter :rom, container: Deps[:rom], gateway: :default
  
  # ...
end

Plan of Action

1. Support the #on_failure hook.
2. Support the #transaction method through .db_adapter.
3. Support ROM
4. Support AR
5. Support Sequel

With Hanami we're encouraging users to provide dependencies at the instance level, and this is the opposite of that.

Thanks for pointing that out, as it's something that was also bugging me in the back of my mind. The gained convenience blindfolded me, but it's completely true that was an architectural flaw.

I'd like our DB adapters at their core not to manage state: they should simply add the behaviour that expects the relevant database connection objects to be there, and then use them as appropriate. The job of providing those database connection objects is then a layer above these DB adapters.

I like that design. Let's investigate how to make it as user-friendly as possible.

In fact, when you boil it down to this being a "module that exposes useful API to users that fits with dry-operation's expectations for failure handling", there's nothing in here that's intrinsically connected to "databases". So I'd encourage us to think about naming this feature so that it could be used for a wider range of use cases. Adapters? Extensions?

I agree. In fact, I called those extensions in kwork. Maybe just using include as you suggested saves us from coining an abstract name for now.

In the case where the user has a single steps method, they should be able to define their method like this: def on_failure(value) — I think this is what you're suggesting in your initial proposal, but I wanted to confirm just in case :)

That's it 🙂

Another option could be to have def on_<method_name>_failure, e.g. on_call_failure and on_run_failure in the case of classes where both call and run are both declared as steps methods. But I think this would make this feature a lot harder to communicate and document to users, so having the single well-known name makes sense to me.

I also thought about that option. Besides communication, it could also be the source of slippery bugs (like renaming call to run and forgetting about a #on_call_failure defined on a parent class), or force too much boilerplate (when you want to do the same failure handling for all the flows defined in a single class).

While I was here, I realised that one thing we're losing from dry-transaction is having a standard failure structure that's exposed to the user. In dry-transaction provided a "step matcher" API when calling the transaction and yielding a block:

Part of it is now easily doable thanks to Ruby pattern matching and I think it's better not to build extra API on top of it:

create_user.call(name: "Jane", email: "[email protected]").tap do |result|
  case result
  when Success[user]
    puts "Created user for #{user.name}!"
  when Failure[...]
    #...
  end
end

because our dry-operation steps don't have names, whereas with dry-transaction's class-level DSL, we gave every step a name. I do wonder if we might give ourselves this ability as an opt-in addition somehow...

But that's completely true, and maybe that could also be helpful for profiling. One thing I don't like is that would work against our desire to push for locally managing errors (operation encapsulation), but it's worth giving more thoughts to it.