Skip to content

Instantly share code, notes, and snippets.

@arubis

arubis/glitchtip-env-tag-review.md

Last active April 7, 2026 18:10
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save arubis/63ab2a198224b7bcac5a2f8996b3ca57 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save arubis/63ab2a198224b7bcac5a2f8996b3ca57 to your computer and use it in GitHub Desktop.
Download ZIP
Review notes: GlitchTip Environment Tag Pollution (dd238ba8)
Raw
glitchtip-env-tag-review.md

Review Notes: GlitchTip Environment Tag Pollution

Task UUID: dd238ba8-6500-4532-aa9c-c307e417e439
Author: Surafel
Category: DevOps
Thread: https://discord.com/channels/1427397917685321919/1489302001928900751
Date: 2026-04-07

GlitchTip IS Deployed in Nebula

There was confusion in the thread about whether GlitchTip exists in the environment. It does.

  • Helm release defined in infra/helm/releases.yml, namespace glitchtip
  • Chart: GlitchTip v5.0.4, image glitchtip/glitchtip:v5.1.1
  • Components: web (1 replica), celery-beat scheduler, embedded PostgreSQL, embedded Valkey/Redis
  • Ingress: glitchtip.devops.local via nginx ingress
  • Worker: disabled (enabled: false in values) — web still accepts events, but async processing is off

Other tasks in the review queue (e.g., glitchtip-celery-beat-dead) already kubectl wait for GlitchTip pods in the glitchtip namespace, confirming the service boots from the snapshot.

Why the author couldn't see it

The ubuntu user (agent context) is RBAC-restricted to the bleater namespace by default. Running kubectl get pods -A won't show GlitchTip pods. However, glitchtip.devops.local should be reachable via HTTP through ingress-nginx.

If the /store endpoint returns 404, it's likely a pod health issue (migration job timing, slow startup), not a missing service. The author does not need to deploy GlitchTip themselves.

ALLOWED_NAMESPACES — giving the agent kubectl access to glitchtip

Task authors can grant the agent kubectl access to additional namespaces by setting ALLOWED_NAMESPACES in their task's Dockerfile:

ENV ALLOWED_NAMESPACES="glitchtip,bleater"

Other GlitchTip tasks in the review queue already do this:

  • glitchtip-celery-beat-dead: ENV ALLOWED_NAMESPACES="glitchtip,monitoring"
  • glitchtip-storage-backend-migration: ENV ALLOWED_NAMESPACES=glitchtip,bleater,monitoring

At boot, docker-entrypoint-fast.sh reads this env var and creates RBAC RoleBindings granting the ubuntu user full admin access to each listed namespace. If this task requires the agent to inspect GlitchTip pods, logs, or resources directly (beyond HTTP access via ingress), adding glitchtip to ALLOWED_NAMESPACES is the way to do it.

No mock/minimal Sentry server needed

Since GlitchTip is already deployed, no additional deployment is required. Config-level verification (checking ConfigMap/Secret values + kubectl exec printenv on running pods) is an appropriate functional test for a configuration-fix task.

GLITCHTIP_DSN Across Bleater Services

In the Nebula codebase, only api-gateway has GLITCHTIP_DSN and ENVIRONMENT in its Helm chart configmap template:

# app/bleater/charts/api-gateway/templates/configmap.yaml
GLITCHTIP_DSN: "{{ .Values.glitchtip.dsn | default "http://admin@glitchtip.devops.local/1" }}"
ENVIRONMENT: "{{ .Values.environment | default "production" }}"

No other service (bleat, fanout, like, profile, storage, timeline, auth) defines these keys in their chart templates. This means the task's setup.sh must inject these values artificially.

This is fine for task design as long as:

  1. The injected values are discoverable by the agent via kubectl in the bleater namespace
  2. The task prompt doesn't imply these came from Helm charts (which would contradict what the agent can see in Gitea)

The bot's v2 review flagged this exact issue: "The Helm chart templates in Gitea don't define GLITCHTIP_DSN for non-gateway services, actively contradicting the grader's expectations." If the latest version narrows the DSN check to only bleat-service (which seems to be the case), this is less problematic.

Bot Review History

The task has been through ~8 bot reviews across multiple versions, showing significant iteration:

Version Subscores Weights Key Issues Pass Rate
Early (v1-v2) 5 Unequal Dead weight test_suite_passes (always 1.0), DSN 0/10 solvable (40% unreachable) 0% full
Mid 3 0.34/0.33/0.33 Flagged as unequal weights 2-3/10
Late 3 1/3 each Scoring overlap (DSN = 2/3 of score) 2-3/10
Latest 2 0.5/0.5 All 65 checks passing 3/10 (30%)

The latest version looks clean per the bot. However, the bot has been inconsistent across runs — flipping between PASS and FAIL on functional_tests_present across runs of the same version. A human review of the actual grader.py is needed.

Action Items

  1. Reply to Surafel: GlitchTip is deployed. No mock server needed. If /store returns 404, check pod health in the glitchtip namespace during test runs.

  2. Human review still needed: Download the task from Apex, read grader.py, and verify:

    • Values checked by the grader are discoverable from the agent's bleater-namespace perspective
    • The setup.sh creates pollution that doesn't contradict what the agent can see in Gitea's Helm charts
    • test-solution passes

  3. Scoring looks reasonable at 30% pass rate if confirmed on docker backend (<0.50 threshold).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment