Skip to content

Instantly share code, notes, and snippets.

@montasim

montasim/CLM API — Meta Integration Business Requirements & Architecture Map.md

Created May 19, 2026 06:52
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save montasim/64385e4d9e331514975c5ebd80e89876 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save montasim/64385e4d9e331514975c5ebd80e89876 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
CLM API — Meta Integration Business Requirements & Architecture Map.md

CLM API — Meta Integration Business Requirements & Architecture Map

Audience: Engineering team, Principal Architect, client reviewers. Last updated: 2026-05-19 Snapshot: Commit 95c85ed on origin/main.

Table of Contents

  1. Technology Stack
  2. Multi-Tenancy Architecture
  3. Current State — What Already Exists
  4. Business Requirements
  5. Implementation Phases
  6. Day-by-Day Deliverables
  7. Cross-Track Dependencies
  8. Target State — Lead Funnel
  9. Client Onboarding Flow
  10. Project Conventions
  11. Quick Setup
  12. Source Evidence

Technology Stack

Layer Technology Version / Notes
Runtime Bun Platform runtime (>=1.3.8)
Framework NestJS 11 Fastify adapter
Database MongoDB 8.x Multi-cluster via Mongoose 9
Queue / Streams Redis Streams Outbox relay, BullMQ 5.76+
Job Queue BullMQ Worker entrypoint for async processing
Events (in-process) EventEmitter2 @OnEvent pattern — migrating to outbox consumers
Schema / Validation Zod 4 libs/contracts for all event schemas
Resilience Custom circuit breaker Rate limiter + state management on outbound HTTP
Webhook Raw Body fastify-raw-body HMAC-SHA256 signature verification
Tenant Context nestjs-cls TenantScopeService carries across process boundaries
Proxy / Routing Caddy Strangler routing flag, X-CLM-Route header
Containerization Docker Multi-stage build, GHCR push
Contract Testing Pact Consumer-driven, self-hosted broker
Field Maps src/common/field-maps/ 2-3 char abbreviated field names on all Mongo schemas

B1-Specific — WhatsApp Cloud API

  • WhatsApp Cloud API for messaging + calling
  • fastify-raw-body for webhook HMAC-SHA256
  • Custom rate limiter with circuit-breaker on outbound client
  • Idempotency-Key header pattern for deduplication
  • Zod 4 schemas for CallInitiated, CallRinging, CallAnswered, CallEnded, CallMissed in libs/contracts

B2-Specific — Messaging Domain Plumbing

  • MongoDB Change Streams for outbox relay (planned)
  • BullMQ worker with OTEL instrumentation (planned)
  • SERVICE_MODE env var for same-image process mode (planned)
  • TenantEnvelope carrying tenantId, requestId, traceparent, actorId through queues

Multi-Tenancy Architecture

Two Databases, Two Connection Paths

Every running process talks to one shared system database (Control Plane) and one or more per-tenant databases (Data Plane).

Control Plane — src/modules/platform/

  • Wired with MongooseModule.forRootAsync() in src/app.module.ts
  • Stores: platform identities, tenant records, cluster configs, billing plans, currencies, storage providers, courier definitions, Meta Graph API versions, audit logs, alert channels, white-label configs
  • Repository pattern: ControlPlaneBaseRepository at src/common/repositories/control-plane-base.repository.ts
  • Every index starts with tenantId for ESR compliance

Data Plane — src/modules/tenant/

  • Each tenant gets tenant_<tenantId> MongoDB database
  • No MongooseModule.forFeature — runtime-resolved connection via TenantScopeService.getConnection()
  • Stores: orders, products, customers, carts, wishlists, conversations, messages, lifecycle sends, settings, tax rules, coupons
  • Direct collection access: connection.collection('<name>')

Connection Manager

ClusterConnectionService at src/modules/platform/cluster-connection/cluster-connection.service.ts:

  • @Global() provider, caches one Mongoose Connection per cluster
  • Decrypts connection strings with CLUSTER_ENCRYPTION_KEY on boot
  • 15-second cron health check, circuit-breaker state per cluster
  • Methods: getClusterConnection(clusterId), getTenantConnection(tenantId, clusterId), getTenantConnectionByClusterObjectId(tenantId, clusterObjectId)
  • Tenant DB resolved as clusterConnection.useDb('tenant_' + tenantId, { useCache: true })

Tenant Context (Request Lifecycle)

Store: TenantClsStore via TenantScopeService at src/common/services/tenant-scope.service.ts

Guard chain:

  1. JwtAuthGuard (global APP_GUARD) — verifies JWT, attaches request.user
  2. IdentityResolveGuard — reads tenantId from JWT, looks up clusterId, writes both to CLS store
  3. TenantGuard — enforces @TenantRequired() / @TenantOptional() decorators

Background jobs: tenantScope.runWithTenant(tenantId, fn, { clusterId }) per job payload.

Provisioning

New Data Plane collections must register in both:

  1. src/seed/shared/seeds/schema-sync.seed.ts — seed-side registry
  2. src/modules/platform/tenant-provisioning/config/data-plane-collections.config.ts — runtime registry

Forgetting either = new tenants won't get the collection, indexes won't build, reads silently hit empty namespace.

Current State — What Already Exists

Verified against origin/main at commit 95c85ed.

Meta Platform Layer (Control Plane)

Path: src/modules/platform/meta-integration/

Component File Status
Orchestration meta-integration.service.ts DONE
Graph API version pinning meta-api-version.service.ts DONE — pinned v25.0
Tenant feature gating tenant-messaging-feature-gate.service.ts DONE

Shared Meta Primitives

Path: src/modules/tenant/messaging/shared/

Component File Status
Graph API client meta-api.client.ts DONE — all channels use this
Webhook signature webhook-signature.util.ts DONE — HMAC-SHA256

WhatsApp Module

Path: src/modules/tenant/messaging/whatsapp/

Component Status Notes
Credentials DONE Embedded signup + manual connect
Outbound messaging DONE BullMQ queue, rate limiter, idempotency key
Templates DONE CRUD + approval status tracking
Flows DONE Create, update, publish, deprecate
Media DONE Upload, download, URL resolution
Commerce DONE Catalog, orders
Calling API DONE Full implementation: initiate, respond, terminate, webhook events
Settings DONE Per-tenant WhatsApp config
Lifecycle sends DONE Order, payment, cart, inventory, courier events
Idempotency-Key DONE Redis-based dedup on outbound, 24h TTL

Messenger Module

Path: src/modules/tenant/messaging/messenger/

Component Status Notes
Credentials DONE Token storage
Outbound queue DONE BullMQ
Inbound handler DONE @OnEvent('meta.webhook.messenger_event')
Shared client MISSING Uses direct axios, hardcoded v25.0 URL

Instagram Module

Path: src/modules/tenant/messaging/instagram/

Component Status Notes
Credentials DONE Token storage
Messaging service DONE Uses shared client for credential ops
Token refresh DONE Scheduled refresh
Inbound handler DONE @OnEvent('meta.webhook.instagram_event')
API client PARTIAL Forked InstagramApiClient — duplicates shared client

Webhook Ingress

Path: src/webhooks/meta/

Component Status
Controller DONE — meta-webhook.controller.ts
Signature guard DONE — HMAC-SHA256 enforcement
Idempotency DONE — dedup with SHA-256 payload hashing
Persistence DONE — messaging_webhook_logs collection
Tenant resolution DONE — from payload (phone number ID / page ID / IG business account ID)

Conversations

Path: src/modules/tenant/messaging/conversation/

Collections owned: messaging_conversations, messaging_messages, conversation_assignments, customer_channel_mappings

Event Contracts

Path: libs/contracts/src/events/

Schema Status
Call events (6 states) DONE — discriminated union on type
Message received (4 channels) DONE — discriminated union on channel
Tenant envelope DONE — tenantId + requestId + traceparent + actorId

Permissions

src/common/enums/permission-module.enum.ts:

  • MESSAGING, MESSAGING_TEMPLATES, MESSAGING_FLOWS, MESSAGING_ANALYTICS — DONE
  • WHATSAPP, MESSENGER, INSTAGRAM, CONTACT, LEAD — not yet (messaging treated as single domain)

Field Maps

17 messaging-related maps in src/common/field-maps/. Missing: tenant-contact.field-map.ts, lead.field-map.ts (phase 4).

Business Requirements

Track B1 — WhatsApp Cloud API (Messaging + Calling)

BR-B1.1 — Webhook Ingestion

  • Receive inbound WhatsApp messages via Meta webhook
  • Verify webhook signature using HMAC-SHA256 (fastify-raw-body)
  • Reject tampered/invalid payloads
  • Route verified messages into queue pipeline

Status: DONE.

BR-B1.2 — Outbound Messaging

  • Send outbound messages via WhatsApp Cloud API
  • Per-tenant credential storage (token, phone number, display name, template list)
  • Credentials pre-provisioned by Meta business partner; API only stores and uses them
  • Circuit-breaker pattern on outbound client for resilience
  • Idempotency-Key support to prevent duplicate sends

Status: DONE (circuit breaker, idempotency key, credential storage all implemented).

BR-B1.3 — Calling Integration

  • Receive inbound call events from WhatsApp Cloud API
  • Initiate outbound calls via WhatsApp Cloud API
  • Track call state per conversation (initiated, ringing, answered, missed, ended)
  • Publish Call* events through the event pipeline

Status: DONE. Full implementation verified — not a stub.

BR-B1.4 — Calling-Event Schema

  • Zod schemas in libs/contracts for all call states
  • Schemas consumed by tenant console (agent click-to-call) and admin console (operator monitoring)

Status: DONE. Schemas in libs/contracts/src/events/call-events.schema.ts.

BR-B1.5 — Channel Onboarding

  • REST endpoints for both consoles to connect a tenant to WhatsApp Cloud API
  • Accept per-tenant credentials, write to API, surface connection state
  • Embedded Signup flow with Meta OAuth callback

Status: DONE. Endpoints: POST /connect/callback, POST /connect/manual, DELETE /disconnect.

BR-B1.6 — Template Message Support

  • Outbound template messages for outside the 24-hour customer service window
  • Per-tenant approved-template list served to tenant console template picker

Status: DONE. Full CRUD with approval status tracking.

BR-B1.7 — Production Validation

  • Synthetic webhook canary against staging
  • Call-state test matrix against production sandbox
  • Full end-to-end integration test

Status: DONE. Test suite at test/messaging-whatsapp-validation/whatsapp-validation.e2e-spec.ts.

Track B2 — Messaging Domain & Event Plumbing

BR-B2.1 — Unified Four-Channel Message Reception

  • Ingest messages from Messenger, WhatsApp, Instagram, and website widget
  • Channel-aware MessageReceived Zod schema with channel discriminator in libs/contracts
  • All channels publish through the same outbox-backed pipeline

Status: PARTIAL. Schemas done in libs/contracts. Handler unification pending — three separate @OnEvent handlers today.

BR-B2.2 — Listener Migration (In-Process → Outbox)

  • Inventory all @OnEvent listeners in messaging tree (~13 files, ~27 handlers)
  • Migrate replaceable listeners to outbox consumers

Status: PARTIAL. Inventory complete. WooCommerce has outbox pattern as reference. Messaging not yet migrated.

BR-B2.3 — Tenant Envelope Wiring

  • Every queue message carries: tenantId, requestId, traceparent, actorId, payload
  • CI check fails on missing tenantId

Status: PARTIAL. tenantId present. requestId, traceparent, actorId missing from payloads. Schema defined in libs/contracts.

BR-B2.4 — OpenTelemetry Initialization

  • OTEL SDK in main.ts and BullMQ worker entrypoint
  • traceparent propagation: Caddy → API → Worker

Status: NOT STARTED. No @opentelemetry in codebase.

BR-B2.5 — Messaging Service as Separate Process

  • Same Docker image, SERVICE_MODE=messaging env var
  • Behind Caddy Strangler routing flag

Status: NOT STARTED. No SERVICE_MODE usage.

BR-B2.6 — Contract Testing

  • Consumer-side Pact for MessageReceived across all four channels
  • Pact verification wired into CI

Status: PARTIAL. Pact deps installed (@pact-foundation/pact, nestjs-pact). Health check pact test exists. No messaging contracts.

BR-B2.7 — Deployment Cutover & Load Testing

Status: NOT STARTED.

BR-B2.8 — Documentation

Status: NOT STARTED.

Implementation Phases

Client priority order:

Phase 1 — WhatsApp Business Account Messaging

  • Audit credential, template, outbound flow against Graph API v25.0
  • Confirm inbound ingress, signature validation, idempotency, conversation persistence
  • #890 (Embedded Signup) — major open feature

Status: Phase 1 audit DONE. All endpoints verified against v25.0. No hardcoded versions found.

Phase 2 — Messenger and Instagram

  • Audit outbound delivery, inbound parsing, signature handling
  • Extend shared meta-api.client.ts — do NOT fork per platform
  • Migrate Messenger from direct axios to shared client
  • Consolidate Instagram forked client into shared client

Status: Audit DONE. Both need shared client migration.

Phase 3 — WhatsApp Calling API

  • #892 tracks calling API replacement
  • Codebase already has full implementation (verified — not a stub)
  • May need issue re-scope or closure

Status: Implementation already DONE. Schema layer added in libs/contracts.

Phase 4 — tenant_contacts + Lead Funnel

  • Open fresh feature issue when phase 1 stabilises
  • Design doc at docs/design/tenant-contacts-lead-funnel.md

Status: Design DONE. Implementation NOT STARTED.

Day-by-Day Deliverables

B1 Schedule (WhatsApp)

Day Deliverable Status
1 WhatsApp Cloud API integration plan DONE
2 Calling-event schemas in libs/contracts DONE
3 Signature verification, tamper-payload test DONE (existed)
4 Outbound client, credential storage, channel onboarding DONE (existed)
5 Calling integration end-to-end DONE (existed)
6 Circuit-breaker, Idempotency-Key support DONE
7 Full sandbox end-to-end DONE (existed)
8 Calling tool sandbox validation DONE (existed)
9 Integration docs, operations runbook PARTIAL
10 Production sandbox verification, test matrix DONE (validation suite)

B2 Schedule (Messaging Domain)

Day Deliverable Status
1 API endpoint, queue, @OnEvent listener map DONE (27 handlers across 13 files)
2 MessageReceived Zod schema with channel discriminator DONE
3 OTEL init in main.ts + worker NOT STARTED
4 Envelope-aware publication across emitters PARTIAL
5 Listener migration, Pact consumer for all channels NOT STARTED
6 Messaging module as SERVICE_MODE=messaging NOT STARTED
7 Flip staging Strangler flag NOT STARTED
8 Load tests, connection-pool tuning NOT STARTED
9 Deployment docs, rollback procedure NOT STARTED
10 Production traffic verification NOT STARTED

Cross-Track Dependencies

Dependency Owner Day Status
Calling-event schemas in libs/contracts B1 2 DONE
MessageReceived schema with channel discriminator B2 2 DONE
OTEL traces visible end-to-end B2 3 NOT STARTED
Tenant envelope CI check live Track A 4 NOT STARTED
WhatsApp onboarding endpoints B1 4 DONE
B1 outbound client ready B1 5 DONE
Messaging service as separate process B2 6 NOT STARTED
Pact broker accepts contracts Track A 7 PARTIAL
Messaging service serves staging traffic B2 7 NOT STARTED

Target State — Lead Funnel

The tenant_contacts collection (phase 4, greenfield).

Funnel

Inbound message → resolve/create tenant_contact → attach to messaging_conversation
→ sales/automation promotes → customers collection + customer_channel_mappings

Design

Full design doc: docs/design/tenant-contacts-lead-funnel.md

Key points:

  • Data Plane collection: tenant_contacts
  • Status lifecycle: new → engaged → qualified → converted_to_customer
  • Must register in schema-sync.seed.ts + data-plane-collections.config.ts
  • Needs tenant-contact.field-map.ts in src/common/field-maps/
  • Permission modules: CONTACT, LEAD in permission-module.enum.ts
  • 4 indexes: dedup, status, converted, recency
  • 6 API endpoints under /api/v1/messaging/contacts

Client Onboarding Flow

New client = new tenant. Steps:

  1. Admin creates tenant — Control Plane endpoint creates tenant record with clusterId, status, billing plan
  2. Tenant DB provisioned — TenantProvisioningService.provisionTenantDatabase() creates collections + indexes
  3. Company settings seeded — Default feature toggles, security settings, localization
  4. Tenant identity created — Admin or invitation system creates tenant-scoped user, assigns roles
  5. WhatsApp channel connected — Embedded Signup (POST /messaging/whatsapp/connect/callback) or manual (POST /messaging/whatsapp/connect/manual)
  6. Webhook subscription — App subscribes to WABA webhooks
  7. Templates synced — WhatsApp templates fetched from Meta
  8. Lifecycle rules configured — Enable/disable auto-messages for order events

Project Conventions

  • Zod-only validation — No class-validator. DTOs via createZodDto() from nestjs-zod
  • Field shortening — Every new Mongoose schema uses 2-3 char abbreviated names with alias. Run bun run validate:field-maps
  • Multi-tenancy split — src/modules/platform/ = Control Plane (shared DB). src/modules/tenant/ = Data Plane (per-tenant DB)
  • Response format — ResponseHelper.success(). Errors follow RFC 7807. API prefix: /api/v1/*
  • No caching decorators — @Cacheable / @CacheEvict forbidden. Rely on indexes + lean()
  • Bangladesh-only — BDT, +880 phone, divisions/districts/upazilas. No GDPR/CCPA/SOC2/HIPAA/PCI-DSS
  • Fixed domains — endapis.com, api.endapis.com, cdn.endapis.com
  • No code comments — No console.*. No debug residue. No auto-commits without instruction
  • Enum location — All enums in src/common/enums/. Import from @enums

Quick Setup

# Prerequisites: Bun >=1.3.8, MongoDB 8.x

# Install
bun install
chmod +x .husky/pre-commit .husky/pre-push

# Environment (bare minimum)
cat > .env << 'EOF'
NODE_ENV=development
APP_PORT=5000
MONGO_URI=mongodb://localhost:27017/clm-api
CORS_ORIGINS=http://localhost:3000
EOF

# Start MongoDB (Docker)
docker run -d --name mongodb -p 27017:27017 mongo:8

# Seed
bun run seed:dev

# Boot
bun run start:dev

# Verify
curl http://localhost:5000/health/ping
# {"status":"ok","message":"pong"}

Test accounts created by seed:

Role Email Password
Developer developer@endapis.com Dev$3cur3!Pl@tf0rm
Authority authority@endapis.com Auth#C0ntr0l!2025
Super Admin superadmin@endapis.com Sup3r@Adm1n!Acc3ss
Client Admin clientadmin@endapis.com Cl13nt#Adm1n!M@n@g3
Client User clientuser@endapis.com Cl13nt$Us3r!Acc0unt

Source Evidence

Paths verified on origin/main at commit 95c85ed.

Onboarding and Setup

  • docs/setup/DEVELOPER-SETUP.md, docs/setup/ENVIRONMENT-VARIABLES.md
  • .env.example at repo root
  • src/config/env.validation.ts, src/config/database.config.ts, src/config/mongodb-connection-options.config.ts

Multi-Tenancy Core

  • src/app.module.ts
  • src/common/services/tenant-scope.service.ts
  • src/modules/platform/cluster-connection/cluster-connection.service.ts
  • src/modules/platform/cluster-config/infrastructure/persistence/schemas/cluster-config.schema.ts
  • src/common/repositories/control-plane-base.repository.ts
  • src/common/guards/identity-resolve.guard.ts, src/common/guards/tenant.guard.ts
  • src/common/decorators/tenant.decorator.ts
  • src/modules/platform/tenant-provisioning/tenant-provisioning.service.ts
  • src/modules/platform/tenant-provisioning/tenant-schema-migration-bootstrap.service.ts
  • src/modules/platform/tenant-provisioning/config/data-plane-collections.config.ts
  • src/modules/platform/tenant-provisioning/config/control-plane-collections.config.ts
  • src/seed/shared/seeds/schema-sync.seed.ts
  • src/modules/platform/domain-resolution/application/domain-resolution.service.ts
  • src/worker.ts, src/worker.module.ts

Messaging Stack

  • src/modules/platform/meta-integration/
  • src/modules/tenant/messaging/shared/meta-api.client.ts, webhook-signature.util.ts
  • src/modules/tenant/messaging/whatsapp/
  • src/modules/tenant/messaging/messenger/
  • src/modules/tenant/messaging/instagram/
  • src/modules/tenant/messaging/conversation/
  • src/webhooks/meta/
  • src/webhooks/woo-commerce/ (architectural reference)
  • src/common/field-maps/
  • src/common/enums/permission-module.enum.ts

Event Contracts (New)

  • libs/contracts/src/events/call-events.schema.ts
  • libs/contracts/src/events/message-events.schema.ts
  • libs/contracts/src/events/envelope.schema.ts

Related Issues

  • #890 — WhatsApp Embedded Signup
  • #892 — Calling API replacement (may need re-scope — already implemented)
  • #1174 — Onboarding map (this document)

Companion Repositories

  • clm-infra (git@github.com:Goclmbd/clm-infra.git) — Docker Compose, Caddy, deployment
  • clm-tenant-console, clm-admin-console — Next.js frontends
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment