request-admission-consistency-options.md

options.md

Request Admission Consistency Options

Context

Land currently performs several independent operations:

1. Allocate a new sqid.
2. Write the request receipt and read-model projections.
3. Persist the initial accepted request log.
4. Publish the request to the processing pipeline.

These operations are not atomic. If one succeeds and a later operation fails, partial state can remain. A client retry allocates a new sqid, so the partial state associated with the previous sqid does not automatically converge.

The primary user-visible concern is a request that appears as accepted in List indefinitely even though it never entered the pipeline.

We do not care about exposing requests that failed before their first accepted request log was successfully persisted.

Option A: Gate List on the accepted log

Change List visibility so a request is not included until its first accepted log has been persisted:

• request.ReceiptWriter does not create the queue summary used by List.
• The request-log materializer creates the queue summary after inserting the first accepted log.
• Receipt writes that fail before the accepted log remain invisible to List.

Advantages

• Small implementation change.
• Directly matches the desired List visibility boundary.
• Avoids arbitrary age thresholds.
• Does not complicate List pagination with response-time filtering.

Disadvantages

• If the accepted log succeeds but pipeline publication fails, the request still remains visible as accepted.
• If accepted-log insertion succeeds but projection materialization fails, the log and List projection can remain inconsistent without a retry mechanism.

Option B: Gate List and record publication failures as errors

Apply Option A and also handle pipeline publication failure explicitly:

• Persist an error request log when pipeline publication fails.
• Store the publication failure in lastError.
• Return the Land error to the client.
• Let List show the abandoned sqid as error instead of leaving it permanently accepted.

Advantages

• Handles the ordinary pipeline publication failure transparently.
• Preserves the receipt history instead of hiding failed accepted requests.
• Requires relatively little additional machinery.
• Gives operators and users an explanation through lastError.

Disadvantages

• If pipeline publication and error-log persistence both fail, the request can still remain accepted.
• Client retries still allocate a new sqid, so the failed sqid and replacement sqid remain separate requests.
• This improves the common failure path but does not provide complete convergence under compound failures.

Option C: Introduce an internal accepting state

Separate receipt creation from public acceptance:

1. Create the internal receipt with status accepting.
2. Exclude accepting requests from List, Status, and History.
3. Publish the request to the pipeline.
4. Persist accepted only after publication succeeds.
5. Allow any accepted or later pipeline event to activate and repair the public projections.

Advantages

• Gives public APIs a clear semantic boundary.
• Admission attempts that never reach the pipeline remain hidden.
• Later pipeline events can repair a missed accepted transition.

Disadvantages

• Requires additional state, filtering, materializer behavior, race handling, and tests.
• A pipeline consumer can emit a later event before Land persists accepted, so event ordering must be handled carefully.
• If publication succeeds but accepted persistence fails, the client may still retry and create another sqid.

Option D: Idempotent admission with durable publication

Design Land as a recoverable admission workflow:

• Add a client-provided idempotency key.
• Reuse the same sqid when the client retries the same Land operation.
• Persist a durable publication intent.
• Retry pipeline publication until it succeeds or reaches a terminal failure.
• Make projection materialization replayable from retained request events.

Advantages

• Client retries converge on one logical request and one sqid.
• Partial failures can be recovered rather than merely hidden.
• Provides the strongest production semantics.
• Supports reliable projection repair and operational reconciliation.

Disadvantages

• Requires API and schema changes.
• Requires a publication worker or reconciler.
• Requires lifecycle, retention, retry, and terminal-failure policies.
• Has substantially greater implementation and operational complexity.

Option E: Age-based filtering or sweeping

Hide or transition requests that remain accepted beyond a configured age.

Advantages

• Straightforward operational model.
• Can remove obviously stale requests from the common List view.

Disadvantages

• An old accepted request is not necessarily abandoned. It might be legitimately waiting in a backed-up queue.
• List filtering hides incorrect state without repairing Status, History, or storage.
• Filtering after reading a page complicates pagination because hidden rows still consume storage query limits.
• A sweeper needs a reliable signal that publication did not occur, not merely an elapsed-time threshold.

This option is not recommended unless SubmitQueue defines a guaranteed maximum duration for the accepted state.

Recommendation

Use Option B for the current pre-production stack:

1. Create the List projection only after the first accepted log is persisted.
2. If pipeline publication fails, persist an error log with lastError.
3. Return the publication error so the client can retry with a new request.

Document Option D as the intended production direction if stronger retry convergence becomes necessary.

This provides a small and understandable improvement now without introducing the state machine and recovery machinery required by the more complete designs.