Skip to content

Instantly share code, notes, and snippets.

@jluisflo

jluisflo/fineract-event-bridge-design.md

Last active January 7, 2026 02:54
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save jluisflo/d96156edd5ed4c5ae64c2e07e1918f3e to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save jluisflo/d96156edd5ed4c5ae64c2e07e1918f3e to your computer and use it in GitHub Desktop.
Download ZIP
Fineract Event Bridge - Design Document
Raw
fineract-event-bridge-design.md

Design Document: Fineract Event Bridge

Status: Implemented Author: Engineering Team Date: 2025-01-06 Version: 1.0

1. Overview

1.1 Problem Statement

El sistema de créditos N1 requiere consumir eventos de negocio emitidos por Apache Fineract para:

  • Notificar próximos vencimientos de cuotas
  • Alertar sobre cuotas vencidas
  • Detectar cambios en clasificación de mora (delinquency)

Fineract emite estos eventos mediante su sistema de External Events que serializa mensajes en formato Apache Avro y los publica a un message broker (Kafka o JMS/ActiveMQ).

1.2 Challenges

Challenge Descripción
Serialización Avro Fineract usa Avro con schemas propietarios. Solo existe librería Java oficial (fineract-avro-schemas)
Consumer .NET El backend principal (n1-credits-backend) está desarrollado en .NET 8.0
Costo de infraestructura Kafka (Confluent Cloud) representa ~$150-200 USD/mes para bajo volumen
Complejidad operacional Mantener un cluster Kafka para eventos esporádicos es overkill

1.3 Solution Summary

Implementar un Event Bridge usando:

  • Amazon MQ (ActiveMQ) como message broker (~$20 USD/mes)
  • AWS Lambda (Java 17) para deserializar Avro y transformar a JSON
  • Webhook HTTP para entregar eventos a n1-credits-backend

2. Architecture

2.1 System Context

┌─────────────────────────────────────────────────────────────────────────────┐
│                              AWS Cloud                                       │
│                                                                              │
│  ┌──────────────┐     ┌──────────────┐     ┌──────────────┐                │
│  │   Fineract   │     │  Amazon MQ   │     │    Lambda    │                │
│  │   (Java 21)  │────▶│  (ActiveMQ)  │────▶│   (Java 17)  │                │
│  │              │ JMS │              │ ESM │              │                │
│  │              │Avro │ Queue:       │     │ Deserialize  │                │
│  │              │     │ fineract-    │     │ Avro → JSON  │                │
│  │              │     │ events-jms   │     │              │                │
│  └──────────────┘     └──────────────┘     └──────┬───────┘                │
│                                                    │                        │
│                                                    │ HTTP POST              │
│                                                    │ (JSON)                 │
│                                                    ▼                        │
│                                            ┌──────────────┐                │
│                                            │   n1-credits │                │
│                                            │   -backend   │                │
│                                            │   (.NET 8)   │                │
│                                            └──────────────┘                │
│                                                                              │
└─────────────────────────────────────────────────────────────────────────────┘

2.2 Data Flow

1. Fineract COB Job ejecuta
   └─▶ CheckLoanRepaymentDueBusinessStep
   └─▶ CheckLoanRepaymentOverdueBusinessStep
   └─▶ SetLoanDelinquencyTagsBusinessStep

2. Business Event emitido
   └─▶ Serializado como MessageV1 (Avro)
   └─▶ Enviado via JMS a Amazon MQ

3. Lambda triggered por Event Source Mapping
   └─▶ Recibe batch de mensajes (Base64)
   └─▶ Deserializa MessageV1 wrapper
   └─▶ Deserializa data payload dinámicamente
   └─▶ Transforma a JSON unificado

4. Webhook enviado a n1-credits-backend
   └─▶ HTTP POST con JSON payload
   └─▶ Headers: X-Fineract-Event-Type, X-Api-Key

2.3 Message Format

Input (Avro - MessageV1):

┌─────────────────────────────────────┐
│ MessageV1 (Avro)                    │
├─────────────────────────────────────┤
│ id: long                            │
│ source: string                      │
│ type: string (event type)           │
│ category: string                    │
│ createdAt: string                   │
│ businessDate: string                │
│ tenantId: string                    │
│ idempotencyKey: string              │
│ dataschema: string (FQCN)           │
│ data: bytes (nested Avro)           │
└─────────────────────────────────────┘

Output (JSON):

{
  "messageId": "uuid",
  "eventType": "LoanRepaymentDueBusinessEvent",
  "category": "Loan",
  "source": "fineract",
  "tenantId": "default",
  "createdAt": "2025-01-06T10:30:00",
  "businessDate": "2025-01-06",
  "idempotencyKey": "unique-key",
  "schema": "org.apache.fineract.avro.loan.v1.LoanRepaymentDueDataV1",
  "data": {
    "loanId": 12345,
    "loanAccountNo": "000000012345",
    "installment": { ... }
  }
}

3. Components

3.1 Amazon MQ Broker

Property Value
Engine ActiveMQ
Instance mq.t3.micro
Deployment Single-instance
Protocol OpenWire (SSL)
Port 61617
Cost ~$20 USD/month

Justification: Amazon MQ con ActiveMQ soporta el protocolo OpenWire nativo de Fineract sin modificaciones al código fuente.

3.2 Lambda Function

Property Value
Runtime Java 17 (Corretto)
Memory 512 MB
Timeout 60 seconds
Trigger Amazon MQ Event Source Mapping
Batch Size 10 messages

Dependencies:

  • fineract-avro-schemas:0.0.1114-a1b6f15 - Schemas Avro oficiales
  • aws-lambda-java-events:3.11.4 - ActiveMQ event types
  • jackson-*:2.17.0 - JSON serialization
  • okhttp3:4.12.0 - HTTP client

3.3 Event Source Mapping

Type: MQ
Properties:
  Broker: arn:aws:mq:region:account:broker:name:id
  Queues: [fineract-events-jms]
  SourceAccessConfigurations:
    - Type: BASIC_AUTH
      URI: !Ref SecretsManagerSecret
  BatchSize: 10
  MaximumBatchingWindowInSeconds: 5

4. Design Decisions

4.1 Why Amazon MQ over Kafka?

Criterio Kafka Amazon MQ
Costo mensual ~$150-200 ~$20
Modificar Fineract No No
Complejidad Alta Baja
Volumen esperado Overkill Adecuado

Decision: Amazon MQ. El volumen de eventos (COB jobs nocturnos) no justifica la complejidad y costo de Kafka.

4.2 Why Lambda over ECS/EC2?

Criterio Lambda ECS/EC2
Costo (bajo volumen) ~$0 ~$15-30/mes
Escalabilidad Automática Manual/Config
Operación Serverless Managed
Cold start ~3-5s (Java) N/A

Decision: Lambda. El patrón de eventos (batch nocturno) tolera cold starts y el costo es prácticamente $0.

4.3 Why Webhook over Direct Integration?

Criterio Webhook Azure Service Bus Direct DB
Acoplamiento Bajo Medio Alto
Cross-cloud Sí Sí No
Debugging Fácil Medio Difícil
Retry logic En Lambda En consumer Manual

Decision: Webhook HTTP. Permite desacoplamiento total, fácil testing (webhook.site), y el backend .NET solo implementa un endpoint REST.

4.4 Why Java Lambda over .NET?

Criterio Java Lambda .NET Lambda
Avro deserialization Nativa (JAR) Manual (generar schemas)
Fineract compatibility 100% Requiere conversión
Maintenance Actualizar JAR Re-generar clases

Decision: Java. La librería fineract-avro-schemas solo existe en Java. Deserializar Avro en .NET requeriría extraer schemas y generar clases C#, con riesgo de incompatibilidad.

5. Alternatives Considered

5.1 Azure Service Bus Direct (Rejected)

Approach: Modificar Fineract para usar AMQP con Azure Service Bus.

Why Rejected:

  • Requiere agregar dependencia Qpid JMS a Fineract
  • Modificación del código fuente de Fineract
  • Riesgo en upgrades futuros

5.2 Database Polling (Deferred)

Approach: Polling directo a tabla m_external_event de Fineract.

Why Deferred:

  • Viable como fallback
  • Costo $0
  • Menor real-time capability
  • Acopla consumer a schema de BD

5.3 Fineract Hooks (Not Viable)

Approach: Usar sistema nativo de Hooks de Fineract.

Why Not Viable:

  • Hooks solo funcionan para API commands
  • COB Job events no triggean hooks
  • Documentado en investigación inicial

6. Security Considerations

6.1 Credentials Management

Secret Storage Rotation
MQ credentials AWS Secrets Manager Manual
Webhook API key Lambda env var Manual

6.2 Network Security

  • Amazon MQ: SSL/TLS en puerto 61617
  • Lambda → Webhook: HTTPS only
  • Security Groups: Restringir acceso a MQ

6.3 Data Classification

Los eventos contienen:

  • IDs de préstamos (no sensible)
  • Montos de cuotas (sensible - PII)
  • Fechas de vencimiento (no sensible)

Mitigation: HTTPS en tránsito, no persistencia en Lambda.

7. Observability

7.1 Logging

  • Lambda logs: CloudWatch /aws/lambda/fineract-mq-consumer-dev
  • Log format: JSON structured (Log4j2)
  • Retention: 14 días

7.2 Metrics

Metric Source Alert Threshold
Lambda Errors CloudWatch > 0
Lambda Duration CloudWatch > 30s
MQ Queue Depth Amazon MQ > 100

7.3 Tracing

  • Event ID propagado en logs
  • Idempotency key para deduplicación
  • Correlation via X-Fineract-Event-Type header

8. Deployment

8.1 Infrastructure as Code

# Build
mvn clean package -DskipTests

# Deploy
sam deploy \
  --stack-name fineract-mq-lambda-{env} \
  --parameter-overrides \
    Environment={env} \
    MqBrokerArn={arn} \
    MqUsername={user} \
    MqPassword={pass} \
    MqQueueName=fineract-events-jms \
    WebhookUrl={url}

8.2 Environments

Environment MQ Broker Webhook Target
dev n1co-dev-credits-broker webhook.site (test)
staging TBD n1-credits-backend staging
prod TBD n1-credits-backend prod

9. Cost Analysis

9.1 Monthly Estimate (Dev)

Component Cost
Amazon MQ mq.t3.micro $20
Lambda invocations ~$0
CloudWatch Logs ~$1
Secrets Manager ~$0.40
Total ~$22 USD/month

9.2 Comparison with Alternatives

Solution Monthly Cost
Confluent Kafka $150-200
Self-hosted Kafka $50-100
Amazon MQ + Lambda $22
Database Polling $0

10. Future Considerations

10.1 Short-term

  • Implementar endpoint webhook en n1-credits-backend
  • Configurar alertas CloudWatch
  • Documentar runbook operacional

10.2 Medium-term

  • Agregar Dead Letter Queue (DLQ)
  • Implementar retry con exponential backoff
  • Dashboard de monitoreo

10.3 Long-term

  • Evaluar migración a EventBridge Pipes
  • Considerar multi-region para DR
  • Schema registry para versionado

11. References

Appendix A: Event Types Supported

Event Trigger Use Case
LoanRepaymentDueBusinessEvent N días antes de vencimiento Recordatorio de pago
LoanRepaymentOverdueBusinessEvent N días después de vencimiento Alerta de atraso
LoanDelinquencyRangeChangeBusinessEvent Cambio de bucket de mora Gestión de cobranza

Appendix B: Project Structure

fineract-mq-lambda/
├── pom.xml                    # Maven config + shade plugin
├── template.yaml              # SAM/CloudFormation template
├── README.md                  # Operational documentation
├── docs/
│   └── DESIGN.md              # This document
└── src/main/
    ├── FineractMqHandler.java # Lambda entry point
    ├── AvroDeserializer.java  # Avro deserialization
    ├── MessageTransformer.java# JSON transformation
    ├── SafeObjectWrapper.java # Safe Avro extraction
    ├── WebhookSender.java     # HTTP client
    └── resources/
        └── log4j2.xml         # Logging config
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment