Skip to content

Instantly share code, notes, and snippets.

@jollyjoker992

jollyjoker992/ff-indexer.md

Last active October 7, 2025 07:11
Show Gist options
  • Save jollyjoker992/e2800dd0d94c99d7f0a348b353ff0c86 to your computer and use it in GitHub Desktop.
Save jollyjoker992/e2800dd0d94c99d7f0a348b353ff0c86 to your computer and use it in GitHub Desktop.
Download ZIP
FF Indexer Acceptance
Raw
ff-indexer.md

Token Indexer — Acceptance Criteria (Orbit-2)

0) Scope & Non-Goals

In Scope (MVP):

  • Light version of ff-indexer supporting Ethereum (ERC-721/1155) and Tezos (FA2).

  • Derive token ownership and provenance exclusively from on-chain data.

  • Use 3rd-party APIs (OpenSea, Objkt, ArtBlocks, etc.) only for metadata enrichment, never for ownership or provenance.

  • Parse token metadata from on-chain URIs following:

  • Expose both REST and GraphQL interfaces.

  • Provide minimal operational endpoints for health and metrics.

Out of Scope:

  • Bitmark blockchain.
  • Historical sales or pricing data.
  • Series registry and collection indexing.
  • Any internal Bitmark-only service dependency.

1) Data Correctness

Ownership

  • Ownership must match canonical blockchain data.

  • Ethereum:

    • ERC-721: derive from Transfer events.
    • ERC-1155: derive from TransferSingle/TransferBatch events; ownership based on positive balances.

  • Tezos:

    • FA2 (TZIP-12): derive from contract ledger entries and transfer entrypoints.

  • Burned tokens must be correctly recognized and marked.

Provenance

  • Provenance must include chain-derived events: Mint, Transfer, Burn, and metadata updates.
  • Provenance entries include at minimum: transaction hash, block/level, timestamp, and participant addresses.
  • Support EIP-4906 (Ethereum) and metadata URI change detection (Tezos) for metadata update events.

Metadata Parsing

  • Metadata URI fetched from contract-defined fields (e.g., ERC-721 tokenURI, FA2/TZIP-21 metadata field).
  • Parsed fields should align with OpenSea and TZIP-21 schemas.
  • If metadata is incomplete, attempt enrichment via 3rd-party APIs, but never override on-chain truth.
  • Metadata enrichment must be flagged with a provenance source label.

2) Indexing Behavior

On-Demand Indexing

  • Must support on-demand indexing of individual tokens or all tokens owned by given addresses.
  • Must be idempotent.

Automatic Refresh

  • Token ownership and provenance refresh triggered by new blocks/levels.
  • Metadata refresh only when URI changes, a metadata update event is detected, or TTL expires.

Reorganization Handling (Optional)

  • On-chain reorgs must trigger recomputation of affected ownership and provenance data.

3) API Coverage

The service must expose the following endpoints (both REST and GraphQL equivalents):

Core APIs

  • GET /api/v1/tokens/:id Retrieve a specific token by its canonical ID. Must return accurate, normalized data reflecting current chain state.

  • GET /api/v1/tokens Retrieve tokens owned by one or more addresses. Must support sorting by lastActivityTime and pagination.

  • POST /api/v1/tokens Trigger indexing for specific tokens or owners. Must return job information and handle idempotency.

  • GET /api/v1/changes Query recent changes within a timeframe. Must support filters:

    • owners (repeatable): wallet addresses.
    • since: RFC3339 timestamp.
    • tokenIDs: optional filter for specific tokens. The endpoint should return all tokens that have changed (ownership, metadata, or provenance) since the specified timestamp. This endpoint enables incremental sync and efficient caching for clients.

Operational APIs

  • GET /healthz for liveness checks.
  • GET /readyz for readiness. (OPTIONAL)
  • GET /metrics for Prometheus-compatible metrics. (OPTIONAL)

4) Performance and Reliability

  • P95 latency for reads (GET /tokens) < 600ms for cached responses.
  • Address indexing (100–300 tokens/wallet) completes within 30s P95.
  • Minimum 99% availability over 24 hours.

Resilience Requirements:

  • Circuit breakers for all 3rd-party enrichment adapters.
  • Retry with exponential backoff on transient errors.
  • Failed enrichment must not block ownership indexing.

Observability:

  • Expose tracing (trace IDs), structured logs, and metrics for all major operations.
  • Dashboards must report indexing backlog, vendor failure rates, and request latencies.

5) Testing & Completion Criteria

Unit Tests:

  • Ownership derivation for ERC-721, ERC-1155, and FA2 contracts.
  • Provenance construction from chain events.
  • Metadata parsing for OpenSea and TZIP-21 formats.
  • Enrichment logic ensuring no override of chain truth.

Integration Tests:

  • Full wallet indexing covering mixed ERC-721/1155/FA2 assets.
  • Reorg simulation with state recovery.
  • Metadata URI changes triggering update events.

Contract Tests (OPTIONAL):

  • REST and GraphQL parity for responses.
  • Pagination and sorting consistency by lastActivityTime.

Operational Tests (OPTIONAL):

  • Health and readiness probes functional.
  • Metrics exported and monitored.

Definition of Done:

  • All tests pass in CI with >80% coverage on core modules.
  • Service meets performance SLOs and stability targets in staging.
  • Deployment verified with real wallets and testnets for both chains.
  • Documentation (OpenAPI + GraphQL schema) and setup instructions complete.
  • No dependency on internal Feral File or Bitmark-only modules.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment