Livepeer ServiceRegistry Metadata Enhancement

serviceregistry-idea.md

Livepeer ServiceRegistry Metadata Enhancement

1. Motivation

The current ServiceRegistry smart contract in Livepeer stores a single serviceURI string per orchestrator Ethereum address. This works for basic discovery but has limitations:

• Single endpoint only: Multi-node operators can’t advertise all their nodes without multiple on-chain identities.
• No structured metadata: Broadcasters can’t pre-filter by location, capabilities, or other attributes.
• On-chain churn: Any endpoint change requires a transaction.
• Opaque capabilities: Job type support, GPU specs, or other features aren’t discoverable until after connection.

Goal: Introduce a backwards-compatible way to embed richer metadata in the existing serviceURI field, enabling smarter discovery and routing without a contract migration.

2. Proposed Format

The serviceURI will be encoded as a CSV string with three parts:

<defaultURI>,<version>,<base64_encoded_json>

• defaultURI: The legacy single endpoint (used by old clients for backwards compatibility).
• version: Integer schema version for the JSON payload.
• base64_encoded_json: Base64-encoded JSON metadata.

Example:

https://orch1.example.com:8935,1,eyJ2ZXJzaW9uIjoxLCJub2RlcyI6W3siaXAiOiIyMDMuMC4xMTMuMTAiLCJwb3J0Ijo4OTM1LCJsYXQiOjQwLjcxLCJsb24iOi03NC4wMCwiY2FwYWJpbGl0aWVzVXJsIjoiaHR0cHM6Ly9vcmNoMS5leGFtcGxlLmNvbS9jYXBhYmlsaXRpZXMuanNvbiJ9XX0=

3. JSON Schema

The Base64-encoded JSON will follow this minimal schema:

{
  "version": 1,
  "nodes": [
    {
      "ip": "string",
      "port": 8935,
      "lat": 40.71,
      "lon": -74.00,
      "capabilitiesUrl": "https://orch.example.com/capabilities.json"
    }
  ]
}

• version: Schema version for future evolution.
• nodes: Array of orchestrator node descriptors.
• capabilitiesUrl: Points to an off-chain JSON document (free-form map) describing capabilities.

4. Go Struct Definitions

In discovery/types.go or a similar file:

type OrchestratorNode struct {
    IP              string  `json:"ip"`
    Port            int     `json:"port"`
    Lat             float64 `json:"lat"`
    Lon             float64 `json:"lon"`
    CapabilitiesURL string  `json:"capabilitiesUrl"`
}

type OrchestratorList struct {
    Version int                `json:"version"`
    Nodes   []OrchestratorNode `json:"nodes"`
}

5. Encoding & Decoding Logic

Encoding (Orchestrator Registration)

When an orchestrator registers its metadata:

func EncodeServiceURI(defaultURI string, version int, meta OrchestratorList) (string, error) {
    jsonBytes, err := json.Marshal(meta)
    if err != nil {
        return "", err
    }
    encoded := base64.StdEncoding.EncodeToString(jsonBytes)
    return fmt.Sprintf("%s,%d,%s", defaultURI, version, encoded), nil
}

Decoding (Broadcaster Discovery)

When a broadcaster fetches and parses the metadata:

func DecodeServiceURI(uriStr string) (string, int, *OrchestratorList, error) {
    parts := strings.SplitN(uriStr, ",", 3)
    if len(parts) < 1 {
        return "", 0, nil, fmt.Errorf("invalid serviceURI")
    }
    defaultURI := parts[0]
    if len(parts) == 3 {
        version, _ := strconv.Atoi(parts[1])
        jsonData, err := base64.StdEncoding.DecodeString(parts[2])
        if err != nil {
            return defaultURI, version, nil, err
        }
        var meta OrchestratorList
        if err := json.Unmarshal(jsonData, &meta); err != nil {
            return defaultURI, version, nil, err
        }
        return defaultURI, version, &meta, nil
    }
    return defaultURI, 0, nil, nil
}

6. Exact Call Chain in go-livepeer

SetServiceURI (Orchestrator Registration Path)

1. CLI Startup: cmd/livepeer/livepeer.go

   • Reads -serviceAddr flag (or config file).
   • Passes it into node initialization if in orchestrator mode.

2. Node Initialization: core/orchestrator.go / core/livepeernode.go

   • Calls Eth.SetServiceURI(serviceAddr) if on-chain mode is enabled.

3. Ethereum Client Wrapper: eth/client.go

   • Defines:

     func (c *client) SetServiceURI(uri string) error

   • Wraps the generated contract binding:

     tx, err := c.serviceRegistry.SetServiceURI(auth, uri)

4. Contract Binding: eth/contracts/serviceregistry.go

   • Auto-generated binding for the ServiceRegistry.sol contract.

5. On-Chain: ServiceRegistry.sol

   • Stores the string in the mapping.

Patch Point: Encode the JSON into the CSV format before calling SetServiceURI in core/orchestrator.go or eth/client.go.

GetServiceURI (Broadcaster Discovery Path)

1. Broadcaster Discovery: discovery/eth_discovery.go

   • Calls:

     func (d *ethDiscovery) GetOrchestrators(addrs []ethcommon.Address) ([]*OrchestratorInfo, error)

   • Loops over orchestrator Ethereum addresses, calling:

     uri, err := d.eth.GetServiceURI(addr)

2. Ethereum Client Wrapper: eth/client.go

   • Defines:

     func (c *client) GetServiceURI(addr common.Address) (string, error)

   • Calls the generated binding:

     return c.serviceRegistry.GetServiceURI(nil, addr)

3. Contract Binding: eth/contracts/serviceregistry.go

   • Auto-generated binding for getServiceURI(address).

Patch Point: Decode the CSV string into JSON immediately after GetServiceURI returns in discovery/eth_discovery.go. Use the defaultURI for legacy compatibility and the parsed metadata for new logic.

7. Backwards Compatibility

• Old Clients: Use defaultURI (first CSV segment) as before.
• New Clients: Detect 3-part CSV, parse Base64 JSON for richer metadata.
• Orchestrators: Can opt-in to JSON format; others remain unchanged.

8. Future Possibilities Enabled

This change lays the groundwork for advanced discovery features:

• Geo-filtering: Use lat/lon to drop nodes outside a broadcaster’s service area.
• Capability-aware selection: Fetch capabilitiesUrl to match job requirements (e.g., AI inference, GPU specs).
• Multi-node load balancing: Choose among multiple endpoints for the same on-chain identity.
• Off-chain capability updates: Update capabilities without on-chain transactions.

9. Risks & Considerations

• Schema Drift: Without governance, metadata formats may diverge. Mitigate by maintaining a minimal required schema in documentation.
• Trust: Off-chain capabilitiesUrl data can change without on-chain proof. Consider signing the JSON with the orchestrator’s Ethereum key.
• Security: Malicious URLs or large payloads could cause issues. Enforce HTTPS, set size limits, and add timeouts for fetching.
• Gas Cost: Longer strings cost slightly more gas, but the impact is minimal compared to a contract migration.

10. Rollout & Governance Recommendations

• Phase 1: Implement decoding in discovery (read-only) to support new clients.
• Phase 2: Add encoding in orchestrator registration.
• Schema Governance: Maintain a minimal required schema in documentation; allow extra fields for flexibility.
• Versioning: Use the version field to manage schema evolution.
• Community Process: Use Livepeer Improvement Proposal (LIP) for schema changes.

11. Flow Diagram (Before vs. After)

This Mermaid diagram shows the high-level discovery flow before and after the change:

flowchart TB
    subgraph Current_Flow["Current Discovery Flow"]
        A1[Broadcaster] --> B1[Get orchestrator Ethereum address from on-chain registry]
        B1 --> C1[Call ServiceRegistry.getServiceURI(address)]
        C1 --> D1[Return single serviceURI string]
        D1 --> E1[Connect to orchestrator endpoint]
    end

    subgraph Proposed_Flow["Proposed Discovery Flow (CSV + Base64 JSON)"]
        A2[Broadcaster] --> B2[Get orchestrator Ethereum address from on-chain registry]
        B2 --> C2[Call ServiceRegistry.getServiceURI(address)]
        C2 --> D2[Return CSV string: defaultURI,version,base64JSON]
        D2 --> E2[Decode Base64 JSON into OrchestratorList]
        E2 --> F2[Use defaultURI for legacy compatibility]
        E2 --> G2[Use metadata: multiple nodes, lat/lon, capabilitiesUrl]
        G2 --> H2[Optional: Geo-filtering, capability-aware selection]
        F2 --> I2[Connect to orchestrator endpoint]
        H2 --> I2
    end

Loading

Explanation:

• Left side: Current flow — one URI per orchestrator, no extra metadata.
• Right side: New flow — same on-chain call, but the returned string is parsed into richer metadata, enabling geo-filtering and capability-aware selection.

12. Sequence Diagram (Function-Level Flow)

This Mermaid sequence diagram shows the exact function calls for the current and proposed flows:

sequenceDiagram
    participant Broadcaster
    participant eth.Client
    participant ServiceRegistry

    Note over Broadcaster,ServiceRegistry: Current Flow
    Broadcaster->>eth.Client: GetServiceURI(orchestratorAddr)
    eth.Client->>ServiceRegistry: getServiceURI(address)
    ServiceRegistry-->>eth.Client: "https://orch.example.com:8935"
    eth.Client-->>Broadcaster: Return single URI string
    Broadcaster->>Broadcaster: Use URI directly for connection

    Note over Broadcaster,ServiceRegistry: Proposed Flow (CSV + Base64 JSON)
    Broadcaster->>eth.Client: GetServiceURI(orchestratorAddr)
    eth.Client->>ServiceRegistry: getServiceURI(address)
    ServiceRegistry-->>eth.Client: "<defaultURI>,<version>,<base64JSON>"
    eth.Client-->>Broadcaster: Return encoded string
    Broadcaster->>Broadcaster: DecodeServiceURI()
    Broadcaster->>Broadcaster: Parse OrchestratorList (nodes[], lat/lon, capabilitiesUrl)
    Broadcaster->>Broadcaster: Optional geo-filtering
    Broadcaster->>Broadcaster: Optional capability-aware selection
    Broadcaster->>Broadcaster: Select best node
    Broadcaster->>Orchestrator: Connect to chosen endpoint

Loading

Explanation:

• Current Flow: One getServiceURI call returns a plain URI, used directly.
• Proposed Flow: Same on-chain call, but the returned CSV string is decoded into JSON metadata, enabling smarter selection logic.

Next Steps

If needed, a combined architecture diagram showing how this change integrates with the broader Livepeer discovery ecosystem (orchestrator registration, on-chain storage, broadcaster selection) can be produced to further clarify the impact for reviewers.