Replace WebSocket Monitoring With SSE

Summary

Replace the ingest and storage monitor WebSocket streams with Server-Sent Events (SSE).

Enduro currently exposes two browser-facing monitor streams:

GET /ingest/monitor
GET /storage/monitor

Both streams deliver server-to-browser updates for dashboard state: SIPs, batches, AIPs, locations, workflows, and tasks. They are modeled in Goa as streaming results and currently transported over WebSockets.

The dashboard does not authenticate these monitor streams with an existing server-side session cookie. Normal API calls use Authorization: Bearer .... Because browser-native WebSocket cannot set arbitrary headers, Enduro uses a short-lived ticket-cookie flow:

The dashboard calls POST /ingest/monitor or POST /storage/monitor with its bearer token.
The API validates the token and stores the user claims behind a short-lived ticket.
The API sets an HTTP-only ticket cookie.
The dashboard opens the monitor stream.
The API reads the ticket cookie, restores the user claims, and filters events by user attributes.

SSE can use ordinary HTTP cookies. The ticket-cookie flow is not required because of SSE itself; it is required because Enduro's API authentication is bearer-token based and native EventSource also cannot set an Authorization header. Unless we introduce a real API session cookie or use a custom fetch-based SSE client, the ticket-cookie bridge should remain.

This proposal keeps the same high-level monitor model, event payloads, ticket handshake, and authorization filtering, but changes the streaming transport from WebSocket to SSE.

Motivation

SSE is a better fit for these monitor streams because Enduro's dashboard only needs server-to-client notifications. There is no browser-to-server messaging on these connections.

Operationally, WebSockets are more demanding than this use case requires:

They require HTTP upgrade support through every deployment layer.
Reverse proxies and load balancers need WebSocket-specific configuration.
Some managed ingress, CDN, corporate proxy, and VPN environments treat WebSockets differently from normal HTTP.
The API server needs WebSocket origin checking and upgrade plumbing.
Generated Goa code includes WebSocket-specific client and server stream implementations.
Failure modes are less visible as ordinary HTTP responses once the connection has been upgraded.

SSE keeps the same real-time behavior while staying on plain HTTP:

No protocol upgrade is required.
Cookies, status codes, headers, tracing, logging, and proxy behavior remain in the normal HTTP path.
NGINX and similar proxies need only long-lived HTTP streaming settings, such as disabled buffering and longer read timeouts.
The stream is naturally one-way, matching the monitor event model.
Goa now supports SSE for streaming results, so the API design can express the intended transport directly.

This mirrors the successful migration in the legacy Enduro project commit b023db6af7b5a877eea27dd890a4e9313df46d1e.

Legacy Enduro was simpler: its collection monitor stream did not use the newer ticket-cookie and ABAC filtering flow. This repository does, so the migration is not a blind copy. The transport change is similar; the authentication and authorization model must be preserved.

Scope

In scope:

Change the ingest monitor stream to SSE.
Change the storage monitor stream to SSE.
Keep the current event envelope shape unless Goa generation requires a small adjustment.
Keep the monitor ticket request flow.
Rename monitor ticket cookies from *-ws-ticket to *-sse-ticket.
Update dashboard connection code and tests.
Update NGINX and Vite proxy config.
Update generated Goa code, OpenAPI docs, dashboard generated client, and admin documentation.
Remove WebSocket-only server wiring and dependencies if no longer used.

Out of scope:

Changing event semantics or permission filtering.
Replacing the monitor ticket mechanism with bearer-token query parameters.
Introducing bidirectional monitor commands.
Changing upload or download streaming endpoints.

Breaking Change

The transport change is a breaking change for any external client that opens the monitor endpoints as WebSockets. Those clients must switch from ws:// or wss:// to a normal HTTP GET that accepts text/event-stream.

This proposal also renames the monitor ticket cookies from WebSocket terminology to SSE terminology.

This cookie rename is not technically required. We could keep the current cookie names and still run the monitor streams over SSE. Renaming them makes the public contract more accurate, but it is an additional break for clients that have copied the cookie names. The proposal adopts that break now rather than carrying WebSocket terminology forward.

The monitor ticket cookie names will change from:

enduro-ingest-ws-ticket
enduro-storage-ws-ticket

to:

enduro-ingest-sse-ticket
enduro-storage-sse-ticket

The monitor endpoints themselves remain:

POST /ingest/monitor
GET /ingest/monitor
POST /storage/monitor
GET /storage/monitor

Impact:

The bundled dashboard will be updated in the same change.
Any external client that has copied the old cookie names must update.
Any external client that opens ws://.../monitor or wss://.../monitor must switch to a normal HTTP GET with Accept: text/event-stream.
The generated OpenAPI and TypeScript client artifacts will reflect the new cookie names and SSE response type.

This should be called out in release notes.

Current State

Authentication and authorization

Enduro's browser-facing API uses bearer-token authentication for normal API calls. The dashboard's generated TypeScript client attaches the user's access token as an Authorization header.

The monitor streams are different because browser-native streaming APIs cannot attach that header:

WebSocket does not allow arbitrary request headers.
Native EventSource also does not allow arbitrary request headers.

The existing POST /monitor endpoints bridge that limitation. They validate the bearer token using the normal API auth path, store user claims behind a short-lived ticket, and set an HTTP-only ticket cookie. The subsequent GET /monitor reads that ticket cookie and uses the restored claims to filter events by user attributes.

SSE can use cookies already present in the browser, but Enduro does not currently have an API session cookie that represents the validated user claims. OIDC provider cookies or frontend state are not enough for the API to perform the current ABAC event filtering.

Goa design

The current monitor methods use StreamingResult(...) with regular HTTP Response(StatusOK). The descriptions explicitly say WebSocket.

internal/api/design/ingest.go
internal/api/design/storage.go

The event result types are:

IngestEvent in internal/api/design/ingest_monitor.go
StorageEvent in internal/api/design/storage_monitor.go

These are OneOf envelopes and should remain unchanged.

API server

internal/api/api.go creates a Gorilla WebSocket upgrader and passes it to the generated ingest and storage HTTP servers.

The generated WebSocket server stream hijacks the HTTP connection, so the API server write timeout no longer applies after the upgrade.

With SSE, the monitor connection remains normal HTTP. Therefore each monitor handler must explicitly opt out of the API write timeout:

ingestServer.Monitor = middleware.WriteTimeout(0)(ingestServer.Monitor)
storageServer.Monitor = middleware.WriteTimeout(0)(storageServer.Monitor)

The service-level operation timeout already skips both monitor methods in internal/api/interceptors.go, so that part does not need a design change.

Service logic

The core monitor service logic is mostly transport-neutral:

Check ticket and recover user claims.
Subscribe to event service.
Send hello and ping events.
Filter events by user attributes.
Send matching events.

This exists in:

internal/ingest/monitor.go
internal/storage/monitor.go

The implementation should switch to SendWithContext(ctx, event) where practical, but the filtering logic can stay as-is.

Dashboard

dashboard/src/monitor.ts currently:

Calls the generated monitorRequest() method to set the ticket cookie.
Converts http/https URLs to ws/wss.
Opens a native WebSocket.
Parses each message as JSON.
Reconnects using local exponential backoff.

For SSE:

Keep the monitorRequest() call before opening the stream.
Use new EventSource(url) instead of new WebSocket(url).
Keep the JSON parser against MessageEvent.data.
On error, close the EventSource, mark disconnected, and call the existing reconnect path so a fresh short-lived ticket is requested.

Native EventSource has automatic reconnects, but relying on it alone is not ideal here because the monitor ticket cookie currently has MaxAge(5). The application should own reconnects so it can re-mint a ticket before each new stream.

Proposed Design

Authentication design

Keep the existing two-step authentication flow:

POST /ingest/monitor and POST /storage/monitor remain normal authenticated API calls using Authorization: Bearer ....
Those methods issue short-lived HTTP-only ticket cookies.
GET /ingest/monitor and GET /storage/monitor remain NoSecurity() in Goa because they authenticate by checking the ticket cookie.
The monitor service restores user claims from the ticket and continues to filter events by attributes before sending them.

Keep the existing ticket cookie max age for now. It is not material to the transport migration and can be revisited later if reconnect behavior shows a real need.

This keeps the migration focused on transport. It avoids placing access tokens in query strings and avoids replacing native EventSource with a custom fetch-based SSE parser just to carry an Authorization header.

Longer-term alternatives are possible, but they are larger auth changes:

Introduce a first-class API session cookie and authenticate GET /monitor directly from it.
Use a custom SSE client built on fetch() so the dashboard can send Authorization: Bearer ....
Use a query-string token. This is not recommended because URLs are commonly logged by browsers, proxies, and servers.

Goa design changes

For both monitor methods:

StreamingResult(IngestEvent)
HTTP(func() {
    GET("/monitor")
    ServerSentEvents()
    Response("internal_error", StatusInternalServerError)
    Cookie("ticket:enduro-ingest-sse-ticket")
})

And:

StreamingResult(StorageEvent)
HTTP(func() {
    GET("/monitor")
    ServerSentEvents()
    Response("internal_error", StatusInternalServerError)
    Cookie("ticket:enduro-storage-sse-ticket")
})

For the request methods, rename the Set-Cookie outputs:

Cookie("ticket:enduro-ingest-sse-ticket")
Cookie("ticket:enduro-storage-sse-ticket")

Descriptions should say "SSE event stream" or "monitor event stream", not "WebSocket".

SSE event shape

Use default SSE message events and keep the existing JSON envelope in the event data.

Do not add custom SSE event names such as event: ingest or event: storage. The stream URL already identifies the service (/ingest/monitor or /storage/monitor), and the JSON envelope already identifies the domain event type (sip_workflow_updated_event, aip_location_updated_event, and so on).

Keeping all messages as default SSE events avoids custom generated-code changes and keeps the dashboard parser close to the current WebSocket parser.

API server changes

Remove:

github.com/gorilla/websocket import from internal/api/api.go.
WebSocket upgrader construction.
Upgrader/configurer arguments to generated server constructors.
internal/api/origin.go and internal/api/origin_test.go if no longer used.

Add:

ingestServer.Monitor = middleware.WriteTimeout(0)(ingestServer.Monitor)
storageServer.Monitor = middleware.WriteTimeout(0)(storageServer.Monitor)

Keep existing upload/download timeout exemptions.

Service changes

In internal/ingest/monitor.go and internal/storage/monitor.go:

Replace stream.Send(...) with stream.SendWithContext(ctx, ...) where the generated SSE interface supports it.
Keep defer stream.Close() if Goa keeps Close() on SSE streams. It should be a no-op.
Keep existing permission filtering.
Keep hello and ping events.

Dashboard changes

Refactor dashboard/src/monitor.ts:

Rename socket to source or eventSource.
Remove getWebSocketURL.
Build the stream URL directly from the API base URL: baseUrl + "/" + type + "/monitor".
Use EventSource.
Parse ev.data as JSON.
On open, set isConnected = true.
On error, close the source, set isConnected = false, and start the retry path unless the connection was intentionally closed.

The existing parseMonitorEvent helper should still work if the JSON envelope remains:

{
  "value": {
    "type": "sip_workflow_updated_event",
    "value": {
      "uuid": "..."
    }
  }
}

Update comments in monitor-ingest.ts, monitor-storage.ts, and WorkflowCollapse.vue from "websocket" to "monitor event" or "SSE".

Proxy changes

Replace monitor WebSocket proxy config with streaming HTTP config.

For NGINX:

location /api/ingest/monitor {
    proxy_pass http://backend/ingest/monitor;
    proxy_buffering off;
    proxy_read_timeout 24h;
    proxy_send_timeout 24h;
    proxy_set_header Host $http_host;
}

location /api/storage/monitor {
    proxy_pass http://backend/storage/monitor;
    proxy_buffering off;
    proxy_read_timeout 24h;
    proxy_send_timeout 24h;
    proxy_set_header Host $http_host;
}

Remove:

proxy_http_version 1.1;
proxy_set_header Connection "Upgrade";
proxy_set_header Upgrade $http_upgrade;

For Vite dev proxy, remove WebSocket-specific ws: true monitor proxy entries unless a dedicated route is still needed for ordering. The generic /api proxy can handle normal SSE HTTP traffic.

Dependency changes

After Goa regeneration, run:

go mod tidy

Expected result:

github.com/gorilla/websocket should be removed from direct dependencies.
It may remain indirect only if pulled by another dependency.

Dashboard:

Remove vitest-websocket-mock if no longer used.
Add no dependency if a small local EventSource test fake is enough.

Implementation Plan

Update Goa design for ingest and storage monitor methods.
Run make gen-goa.
Update internal/api/api.go for generated constructor signatures and monitor write-timeout exemptions.
Remove dead WebSocket origin-check code.
Adjust monitor service sends if needed for the generated SSE stream interface.
Run Go tests for affected packages.
Update dashboard monitor connection code from WebSocket to EventSource.
Replace WebSocket tests with an EventSource fake.
Run dashboard tests, typecheck, and build.
Run make gen-dashboard-client.
Update dashboard Dockerfile, Vite proxy config, and documentation.
Run go mod tidy.
Add a release-note or changelog entry documenting the breaking cookie/client change.

Validation Plan

Backend:

go test ./internal/api ./internal/ingest ./internal/storage

Dashboard:

cd dashboard
npm run type-check
npm run test
npm run build-only

Manual checks:

Log into the dashboard.
Confirm the dashboard calls POST /ingest/monitor and receives Set-Cookie: enduro-ingest-sse-ticket=....
Confirm GET /ingest/monitor returns Content-Type: text/event-stream.
Confirm ingest events update SIP, workflow, task, and batch views.
Confirm the dashboard calls POST /storage/monitor and receives Set-Cookie: enduro-storage-sse-ticket=....
Confirm storage events update AIP, location, workflow, and task views.
Stop and restart the API while the dashboard is open and confirm reconnect.
Confirm monitor connections stay open longer than the API server write timeout.
Confirm NGINX-proxied deployment keeps monitor streams open.

Document the breaking change clearly in release notes and API docs.

Resolved Decisions

Keep the current ticket cookie max age for this migration.
Use default SSE message events with the existing JSON envelope.
Rename monitor ticket cookies from *-ws-ticket to *-sse-ticket.
Do not support the old monitor ticket cookie names during a compatibility period.

Recommendation

Proceed with the SSE migration and keep the monitor ticket flow.

The backend service behavior is already well aligned with SSE. The main work is generation, dashboard transport replacement, proxy/docs cleanup, and tests.

Do not replace the ticket-cookie bridge with "regular cookies" unless the auth model changes to include a real API session cookie. The ticket flow exists to bridge bearer-token authentication into browser-native streaming transports that cannot set Authorization headers.

Rename the ticket cookies to enduro-ingest-sse-ticket and enduro-storage-sse-ticket in this same migration. Do not keep a compatibility fallback for the old *-ws-ticket names.

sevein/replace-websocket-monitoring-with-sse.md

Select an option

No results found