Best efforts 1 Phase commit

This is a non-XA pattern that involves a synchronized single-phase commit of a number of resources. Because the 2PC is not used, it can never be as safe as an XA transaction, but is often good enough if the participants are aware of the compromises. The basic idea is to delay the commit of all resources as late as possible in a transaction so that the only thing that can go wrong is an infrastructure failure (not a business-processing error). Systems that rely on Best Efforts 1PC reason that infrastructure failures are rare enough that they can afford to take the risk in return for higher throughput. If business-processing services are also designed to be idempotent, then little can go wrong in practice.

Scenarios

Consider a jms based service, where there is an inbound Queue manager (QM1), an outbound queue manager (QM2) and a database (DB). Here are the scenarios that I would like to cover using Best efforts 1 PC commit process:

Happy path

Start MQ transaction on QM1
Start MQ transaction on QM2
Start DB transaction
Receive message from Inbound Queue
Insert record SUCCESS
Send message to Outbound Queue SUCCESS
Commit DB transaction
Commit MQ transaction on QM2
Commit MQ transaction on QM1

Negative Scenario: Database is unavailable at the time of commit

Start MQ transaction on QM1
Start MQ transaction on QM2
Start DB transaction
Receive message from Inbound Queue
Insert record SUCCESS
Send message to Outbound Queue
Commit DB transaction FAILED . Do rollback
Rollback MQ transaction on QM2
Rollback MQ transaction on QM1
Original input message ends up in Inbound queue
After retry, message ends up in Dead letter queue

Negative Scenario: Database integrity violation exception or any kind of SQLNonTransientConnectionException

Start MQ transaction on QM1
Start MQ transaction on QM2
Start DB transaction
Receive message from Inbound Queue
Insert record FAILED
No message goes to Outbound Queue
Rollback DB transaction
Rollback MQ transaction on QM2
Rollback MQ transaction on QM1
Original input message ends up in Inbound queue
After retry, message ends up in Dead letter queue

Negative Scenario: Queue full or Message length exceeds limit or some kind of MQ exception on Outbound Queue manager

Start MQ transaction on QM1
Start MQ transaction on QM2
Start DB transaction
Receive message from Inbound Queue
Insert record SUCCESS
Send message to Outbound Queue FAILED
Rollback DB transaction
Rollback MQ transaction on QM2
Rollback MQ transaction on QM1
Original input message ends up in Inbound queue
After retry, message ends up in Dead letter queue

Negative Scenario: Outbound Queue manager unavailable in the middle of the MQ commit transaction

Start MQ transaction on QM1
Start MQ transaction on QM2
Start DB transaction
Receive message from Inbound Queue
Insert record SUCCESS
Send message to Outbound Queue
Commit DB transaction
Commit MQ transaction on QM2 FAILED . Do rollback
Rollback MQ transaction on QM1
Original input message ends up in Inbound queue
After retry, message ends up in Dead letter queue

This will result in mixed results

Database record is inserted

If the message is retried, be sure not to insert the same record again. Make the application process idempotent.

Negative Scenario: Inbound Queue manager unavailable in the middle of the MQ commit transaction

Start MQ transaction on QM1
Start MQ transaction on QM2
Start DB transaction
Receive message from Inbound Queue
Insert record SUCCESS
Send message to Outbound Queue
Commit DB transaction
Commit MQ transaction on QM2
Commit MQ transaction on QM1 FAILED

This will result in mixed results

Database record is inserted
Outbound message is sent to Outbound Queue
Since inbound message is not committed, it will stay in a persistent Inbound queue, when QM1 starts back up

If the message is retried, be sure not to insert the same record again. Make the application process idempotent.

Implementation

Import spring-data-commons dependency

<dependency>
    <groupId>org.springframework.data</groupId>
    <artifactId>spring-data-commons</artifactId>
</dependency>

Define JMS transaction manager for inbound and outbound Queue connections

<bean id="QM1" class="org.springframework.jms.connection.JmsTransactionManager"
    p:connectionFactory-ref="inboundMqConnectionFactory" />

<bean id="QM2" class="org.springframework.jms.connection.JmsTransactionManager"
    p:connectionFactory-ref="outboundMqConnectionFactory" />

Define Database transaction manager for database

<bean id="DB" class="org.springframework.jdbc.datasource.DataSourceTransactionManager"
    p:dataSource-ref="myDataSource"/>

Chain JMS and database tx managers using ChainedTransactionManager with the Database tx manager as the inner most one.

<bean id="chainedTxManager" class="org.springframework.data.transaction.ChainedTransactionManager">
    <constructor-arg>
        <list>
            <ref bean="QM1"/>
            <ref bean="QM2"/>
            <ref bean="DB"/>
        </list>
    </constructor-arg>
</bean>

Make Camel JMS components transacted

<bean id="inboundMQ" class="org.apache.camel.component.jms.JmsComponent"
    p:connectionFactory-ref="inboundMqConnectionFactory"
    p:destinationResolver-ref="jndiDestinationResolver"
    p:transactionManager-ref="QM1"/>

<bean id="outboundMQ" class="org.apache.camel.component.jms.JmsComponent"
    p:connectionFactory-ref="outboundMqConnectionFactory"
    p:destinationResolver-ref="jndiDestinationResolver"
    p:transactionManager-ref="QM2"/>

Define transaction policy for Camel to use

<bean id="CHAINED_TX_PROPAGATION_REQUIRED" class="org.apache.camel.spring.spi.SpringTransactionPolicy"
    p:transactionManager-ref="chainedTxManager"
    p:propagationBehaviorName="PROPAGATION_REQUIRED"/>

Mark the route transacted

fromF("inboundMQ:%s", IN_QUEUE)
    .transacted("CHAINED_TX_PROPAGATION_REQUIRED")
    .to("bean:someDAO?method=addData(${body})")
    .toF("outboundMQ:%s",OUT_QUEUE);

Define backout settings in MQ
Make the inbound queue and dead letter queue persistent

ezhov-da/Spring Chained Tx Management.md