Skip to content

Instantly share code, notes, and snippets.

@Cordtus
Last active November 23, 2025 01:19
Show Gist options
  • Select an option

  • Save Cordtus/67b4f96e7c060d160679e2d4aea647da to your computer and use it in GitHub Desktop.

Select an option

Save Cordtus/67b4f96e7c060d160679e2d4aea647da to your computer and use it in GitHub Desktop.
Yaci System Overview - Blockchain Indexer Architecture

Yaci System Overview

A blockchain data indexing and exploration system for Cosmos SDK chains with EVM support.

Architecture

Republic Chain (gRPC)
        |
        v
+------------------+     +------------------+     +------------------+
|  yaci-indexer    | --> |   PostgreSQL     | --> | yaci-postgrest   |
|  (Go binary)     |     |   (Fly.io)       |     | (PostgREST)      |
+------------------+     +------------------+     +------------------+
                                                          |
                                                          v
                                              +------------------+
                                              | yaci-explorer    |
                                              | (React frontend) |
                                              +------------------+

Components

1. yaci-indexer

Repository: https://github.com/Cordtus/yaci Branch: republic Deployment: republic-yaci-indexer.fly.dev

Purpose: Extract blockchain data from a Cosmos SDK chain via gRPC and store it in PostgreSQL.

Key Features:

  • Connects to chain gRPC endpoint (grpc.republic-testnet.xyz:443)
  • Uses gRPC server reflection to dynamically decode protobuf messages without proto files
  • Indexes blocks, transactions, messages, and events
  • Supports live monitoring mode (continuous indexing)
  • Handles batch extraction for historical data
  • Prometheus metrics endpoint at :2112

Data Flow:

  1. Connect to gRPC endpoint
  2. Fetch blocks via cosmos.tx.v1beta1.Service.GetBlockWithTxs
  3. Decode protobuf messages using reflection
  4. Write raw data to *_raw tables as JSONB
  5. PostgreSQL triggers parse raw data into structured *_main tables

Configuration:

  • Environment variables set in fly.toml:
    • YACI_GRPC_ENDPOINT: grpc.republic-testnet.xyz:443
    • YACI_POSTGRES_DSN: Internal flycast connection to PostgreSQL
    • YACI_CONCURRENCY: 5 (parallel block processing)
  • Entrypoint queries database for last indexed block to resume

Database Schema:

Table Description
api.blocks_raw Raw block JSONB (id = height)
api.transactions_raw Raw transaction JSONB (id = tx hash)
api.transactions_main Parsed: height, timestamp, fee, memo, error, proposal_ids
api.messages_raw Raw message JSONB per transaction
api.messages_main Parsed: type, sender, mentions array, metadata
api.events_raw Raw event JSONB
api.events_main Normalized: one row per event attribute

Triggers:

  • new_transaction_update: Parses transactions_raw -> transactions_main + messages_raw
  • new_message_update: Parses messages_raw -> messages_main
  • new_transaction_events_raw: Extracts events from transactions
  • new_event_update: Normalizes events into events_main

Views:

  • api.evm_tx_map: Maps Cosmos tx hashes to Ethereum tx hashes
  • api.evm_address_activity: EVM address statistics

Functions:

  • api.get_messages_for_address(_address): Returns all messages for an address with transaction details

2. yaci-postgrest

Repository: https://github.com/Cordtus/yaci-postgrest Deployment: yaci-postgrest.fly.dev

Purpose: Expose PostgreSQL tables as a REST API using PostgREST.

Key Features:

  • Auto-generates REST endpoints from database schema
  • Supports filtering, ordering, pagination via query params
  • Exposes PostgreSQL functions as RPC endpoints
  • Zero custom code - just configuration

Configuration:

FROM postgrest/postgrest:v12.2.3

ENV PGRST_DB_ANON_ROLE="web_anon"
ENV PGRST_DB_SCHEMAS="api"
ENV PGRST_SERVER_PORT=3000

Security (after consolidated schema):

  • web_anon role has SELECT only on _main tables
  • No access to _raw tables
  • No POST/DELETE/PATCH operations
  • Only get_messages_for_address function exposed

API Endpoints (read-only):

Endpoint Description
GET /blocks_raw Query blocks
GET /transactions_main Query transactions
GET /messages_main Query messages
GET /events_main Query events
GET /evm_tx_map EVM transaction mapping
GET /evm_address_activity EVM address stats
POST /rpc/get_messages_for_address Get all messages for address

Query Syntax:

  • Filtering: ?field=eq.value, ?field=gt.100
  • Ordering: ?order=height.desc
  • Pagination: ?limit=50&offset=100
  • Array containment: ?mentions=cs.{address}
  • Select specific columns: ?select=id,height,timestamp

3. yaci-explorer

Repository: https://github.com/Cordtus/yaci-explorer Branch: separate-postgres (pending merge to main via PR #4) Deployment: yaci-explorer.fly.dev

Purpose: Web UI for exploring blockchain data indexed by yaci.

Stack:

  • React Router 7 (client-side only, SSR disabled)
  • TanStack Query for data fetching and caching
  • Radix UI + Tailwind CSS
  • Vite 7 build tool

Configuration:

VITE_POSTGREST_URL=https://yaci-postgrest.fly.dev

Deployment:

  • Dockerfile builds React app, serves via nginx
  • nginx.conf routes /api/* to PostgREST (but currently uses direct URL)

Database Client (packages/database-client)

Location: packages/database-client/src/client.ts

Purpose: TypeScript wrapper around PostgREST API with typed methods and caching.

Current Implementation (has issues - see below):

  • Custom 10s request cache
  • Methods: getBlocks(), getTransaction(), getTransactionsByAddress(), etc.
  • Business logic for pagination, address lookups, joining related data

Known Issues:

  1. Dual caching: Both database-client (10s) and TanStack Query (10s) cache the same data
  2. N+1 queries: getTransactionsByAddress() makes 3+ round trips
  3. Client-side pagination: Fetches all data, then slices in JavaScript
  4. Client-side aggregations: Calculations that should be SQL queries

Recommended Architecture (not yet implemented):

  • Move business logic to PostgreSQL functions
  • Remove database-client caching (use TanStack Query only)
  • Single round-trip queries via PostgREST RPC endpoints

End-to-End Data Flow

Indexing Flow

1. Republic Chain emits blocks
        |
2. yaci-indexer fetches via gRPC
   - GetBlockWithTxs RPC call
   - Reflection-based protobuf decoding
        |
3. Raw data written to PostgreSQL
   - blocks_raw, transactions_raw
        |
4. PostgreSQL triggers fire
   - Parse JSONB into structured columns
   - Extract addresses, message types
   - Normalize events
        |
5. Structured data available in _main tables

Query Flow

1. User visits yaci-explorer
        |
2. React component requests data
   - useQuery({ queryFn: () => apiClient.getBlocks() })
        |
3. database-client makes HTTP request
   - fetch('https://yaci-postgrest.fly.dev/blocks_raw?order=id.desc&limit=20')
        |
4. PostgREST translates to SQL
   - SELECT * FROM api.blocks_raw ORDER BY id DESC LIMIT 20
        |
5. PostgreSQL executes query
        |
6. JSON response returned through stack
        |
7. React renders data in UI

Address Query Flow (current, inefficient)

1. User views address page
        |
2. database-client.getTransactionsByAddress(address)
        |
3. Round trip 1: GET /messages_main?or=(sender.eq.X,mentions.cs.{X})
   - Fetches ALL matching messages
        |
4. JavaScript: Extract unique tx IDs, paginate in memory
        |
5. Round trip 2: GET /transactions_main?or=(id.eq.A,id.eq.B,...)
   - Fetch transactions for paginated IDs
        |
6. Round trip 3+: Get additional related data
        |
7. Assemble and return to UI

Address Query Flow (recommended, not implemented)

1. User views address page
        |
2. POST /rpc/get_transactions_by_address
   - { p_address: "...", p_limit: 50, p_offset: 0 }
        |
3. PostgreSQL function executes
   - Single query with JOINs
   - Database does pagination
   - Returns exactly 50 rows
        |
4. Return to UI

Deployment Configuration

Fly.io Applications

App Region Machine Purpose
republic-yaci-indexer sjc shared-cpu-1x, 1GB gRPC indexer
republic-yaci-pg sjc shared-cpu-1x, 1GB PostgreSQL 17
yaci-postgrest sjc shared-cpu-1x, 256MB REST API
yaci-explorer sjc shared-cpu-1x, 256MB React frontend

Network Configuration

CI/CD

Repository Workflow Trigger
Cordtus/yaci deploy.yml Push to republic branch
Cordtus/yaci-postgrest (manual) -
Cordtus/yaci-explorer build.yml PR to main

Security Model

Database Permissions

-- web_anon role (used by PostgREST)
GRANT SELECT ON api.transactions_main TO web_anon;
GRANT SELECT ON api.messages_main TO web_anon;
GRANT SELECT ON api.events_main TO web_anon;
GRANT SELECT ON api.blocks_raw TO web_anon;
GRANT SELECT ON api.evm_tx_map TO web_anon;
GRANT SELECT ON api.evm_address_activity TO web_anon;
GRANT EXECUTE ON FUNCTION api.get_messages_for_address(TEXT) TO web_anon;

-- No access to _raw tables (except blocks_raw)
-- No INSERT/UPDATE/DELETE anywhere

API Security

  • PostgREST is public but read-only
  • No authentication required (public blockchain data)
  • CORS not configured (browser requests allowed from any origin)

Current State and Pending Work

Completed

  • Indexer deployed and indexing Republic testnet
  • PostgREST deployed with health checks
  • Explorer deployed
  • Consolidated schema with read-only permissions

Pending

  • Set FLY_API_TOKEN for yaci-explorer repo
  • Refactor database-client to eliminate N+1 queries
  • Move business logic to PostgreSQL functions
  • Remove dual caching

Known Issues

  • Business logic in browser instead of server
  • Multiple round trips per page load

Repository Links


Last updated: November 22, 2025

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment