Paperclip Helm Chart — Guide & Production Quickstart

paperclip-helm-chart-guide.md

Paperclip Helm Chart — Guide & Production Quickstart

What is Paperclip?

Paperclip is an open-source Node.js server + React UI that orchestrates teams of AI agents to run autonomous businesses. It supports multiple AI adapters (Claude Code, Codex, Cursor, Gemini, and more), provides WebSocket-based live event streaming, and includes a built-in plugin system.

Chart Overview

The OSO DevOps Helm chart (osodevops/paperclip) deploys a production-grade Paperclip instance on Kubernetes. It is published from osodevops/helm-charts and uses the Docker image built at osodevops/docker-paperclip.

What the Chart Deploys

Resource	Description
Deployment	Paperclip server (Express.js + React UI) with health probes, rolling updates, and preStop hooks
PostgreSQL (optional)	Bitnami PostgreSQL subchart for dev/staging, or connect to an external managed database
Service	ClusterIP service on port 80 → container port 3100
Ingress	Optional, with WebSocket upgrade support
PersistentVolumeClaim	/paperclip data volume for config, backups, logs, and local storage
Secrets	Auto-generated auth secret and encryption master key (with lookup-based idempotency)
ConfigMap	30+ environment variables and a config.json for the Paperclip runtime
ServiceAccount	With IRSA / GCP Workload Identity annotation support
NetworkPolicy	Ingress on 3100/TCP, egress to DNS, PostgreSQL, and HTTPS
PodDisruptionBudget	minAvailable: 1 by default
HorizontalPodAutoscaler	Optional (disabled by default — see leader election note)
Job (Helm hook)	Pre-upgrade database migration via applyPendingMigrations()
CronJob	Optional external pg_dump backups to S3
Helm Tests	Connection and health check test pods

Architecture Diagram

                  ┌─────────────┐
                  │   Ingress   │  (optional, WebSocket-aware)
                  └──────┬──────┘
                         │
                  ┌──────▼──────┐
                  │   Service   │  ClusterIP :80
                  └──────┬──────┘
                         │
              ┌──────────▼──────────┐
              │     Deployment      │
              │  paperclip:3100     │
              │  (1 replica default)│
              └───┬─────────┬──────┘
                  │         │
         ┌────────▼───┐  ┌──▼──────────┐
         │ PostgreSQL  │  │  S3 / PVC   │
         │ (subchart   │  │  (assets)   │
         │  or external)│  └─────────────┘
         └─────────────┘

---

Prerequisites

• Kubernetes 1.28+
• Helm 3.x
• A StorageClass that supports dynamic provisioning (for PVCs)
• For production: an external PostgreSQL database and S3-compatible object storage

Add the Helm Repository

helm repo add osodevops https://osodevops.github.io/helm-charts
helm repo update

---

Quick Start — Dev / Local Testing

This deploys Paperclip with a bundled PostgreSQL and local disk storage. Not for production.

helm install paperclip osodevops/paperclip \
  --namespace paperclip --create-namespace \
  --set postgresql.enabled=true \
  --set postgresql.auth.password=changeme \
  --set postgresql.metrics.enabled=false \
  --set storage.provider=local_disk \
  --set server.publicUrl=http://localhost:3100 \
  --set server.deploymentExposure=public \
  --set secrets.betterAuthSecret=$(openssl rand -hex 32) \
  --wait --timeout 10m

Access the UI:

kubectl port-forward svc/paperclip 3100:80 -n paperclip
open http://localhost:3100

The first user to access the UI will be prompted to claim the instance as admin.

---

Production Deployment — Step by Step

1. Create the Namespace

kubectl create namespace paperclip

2. Create Secrets

Create secrets for database credentials and application keys before installing the chart:

# Database credentials
kubectl create secret generic paperclip-db-credentials \
  --namespace paperclip \
  --from-literal=password='YOUR_DB_PASSWORD'

# Application secrets
kubectl create secret generic paperclip-app-secrets \
  --namespace paperclip \
  --from-literal=better-auth-secret="$(openssl rand -hex 32)" \
  --from-literal=master-key="$(openssl rand -base64 32)"

# AI provider API keys (optional)
kubectl create secret generic paperclip-api-keys \
  --namespace paperclip \
  --from-literal=anthropic-api-key='sk-ant-...' \
  --from-literal=openai-api-key='sk-...'

CRITICAL: Back up the master-key value externally (password manager, HSM). If lost, all encrypted secrets in Paperclip are irrecoverable.

3. Create a Values File

Create values-production.yaml:

# -- Image
image:
  repository: ghcr.io/osodevops/docker-paperclip
  tag: "latest"  # Pin to a specific SHA or tag in production

# -- Server
server:
  deploymentMode: "authenticated"
  deploymentExposure: "private"          # or "public" if internet-facing
  publicUrl: "https://paperclip.example.com"
  authDisableSignUp: true                # lock down after initial setup
  companyDeletionEnabled: false

# -- External PostgreSQL (recommended)
postgresql:
  enabled: false
  external:
    host: "paperclip-db.abc123.eu-west-2.rds.amazonaws.com"
    port: 5432
    database: "paperclip"
    username: "paperclip"
    existingSecret: "paperclip-db-credentials"   # must contain key "password"
    sslMode: "require"
    connectionPooling: false                      # set true for PgBouncer

# -- S3 Object Storage
storage:
  provider: "s3"
  s3:
    bucket: "my-paperclip-assets"
    region: "eu-west-2"

# -- Secrets (reference pre-created secrets)
secrets:
  existingBetterAuthSecret: "paperclip-app-secrets"
  existingMasterKeySecret: "paperclip-app-secrets"
  existingApiKeysSecret: "paperclip-api-keys"

# -- Persistence
persistence:
  enabled: true
  size: 50Gi
  storageClass: "gp3"       # adjust for your cluster

# -- Ingress with TLS and WebSocket support
ingress:
  enabled: true
  className: "nginx"
  annotations:
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-http-version: "1.1"
    nginx.ingress.kubernetes.io/configuration-snippet: |
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection "upgrade";
  hosts:
    - host: paperclip.example.com
      paths:
        - path: /
          pathType: Prefix
  tls:
    - secretName: paperclip-tls
      hosts:
        - paperclip.example.com

# -- ServiceAccount with IRSA (for S3 access without credentials)
serviceAccount:
  create: true
  annotations:
    eks.amazonaws.com/role-arn: "arn:aws:iam::ACCOUNT:role/paperclip-s3"

# -- Backups
backup:
  builtin:
    enabled: true
    intervalMinutes: 30
    retentionDays: 30
  external:
    enabled: true
    schedule: "0 */6 * * *"
    s3Bucket: "my-paperclip-db-backups"
    s3Region: "eu-west-2"

# -- Resources (tune to your workload)
resources:
  requests:
    cpu: 500m
    memory: 1Gi
  limits:
    cpu: "4"
    memory: 4Gi

# -- Security
networkPolicy:
  enabled: true
podDisruptionBudget:
  enabled: true
  minAvailable: 1

4. Install

helm install paperclip osodevops/paperclip \
  --namespace paperclip \
  --values values-production.yaml \
  --wait --timeout 10m

5. Verify

# Check pods
kubectl get pods -n paperclip

# Check health
kubectl exec deploy/paperclip -n paperclip -- \
  curl -sf http://localhost:3100/api/health | jq .

# Expected output:
# {"status":"ok","version":"0.3.1","deploymentMode":"authenticated",...}

6. Back Up the Master Key

kubectl get secret paperclip-app-secrets -n paperclip \
  -o jsonpath='{.data.master-key}' | base64 -d

Store this value securely outside the cluster.

---

Key Configuration Reference

PostgreSQL Modes

Mode	Config	Use Case
Bundled	postgresql.enabled: true	Dev/staging — deploys Bitnami PostgreSQL as a subchart
External	postgresql.enabled: false + postgresql.external.*	Production — connects to RDS, Cloud SQL, etc.

When using a connection pooler (PgBouncer, Supavisor), set postgresql.external.connectionPooling: true to append ?prepare=false to the DATABASE_URL.

Object Storage Modes

Mode	Config	Use Case
S3	storage.provider: s3	Production — AWS S3, MinIO, R2
Local Disk	storage.provider: local_disk	Dev/staging — files stored on the PVC

For S3 with IRSA (no credentials needed), annotate the ServiceAccount with the IAM role ARN.

Deployment Modes

Mode	Description
authenticated	Requires login. First user claims admin. Use for all real deployments.
local_trusted	No auth, binds to loopback only. For local development only.

Secrets Auto-Generation

If you don't provide secrets.betterAuthSecret or secrets.masterKey, the chart auto-generates them and persists them across upgrades using Helm's lookup function. However, for production you should always pre-create and back up these secrets externally.

---

Upgrades

helm upgrade paperclip osodevops/paperclip \
  --namespace paperclip \
  --values values-production.yaml \
  --set image.tag=v2026.325.0 \
  --wait --timeout 10m

A pre-upgrade Helm hook runs database migrations before the new pods start. The server also runs migrations on startup (idempotent). Always take a database backup before upgrading.

Rollback

helm rollback paperclip <REVISION> -n paperclip --wait

Note: Database migrations are forward-only. If the new schema is backward-incompatible, you'll need to restore from a database backup.

---

Troubleshooting

Symptom	Cause	Fix
Pod CrashLoopBackOff with gosu error	Security context too restrictive	Ensure podSecurityContext.runAsNonRoot: false and containerSecurityContext.allowPrivilegeEscalation: true (defaults)
Health probe returns 403	Hostname not in allowed list	Set server.deploymentExposure: public or configure server.allowedHostnames
EACCES writing to /paperclip	fsGroup mismatch	Ensure podSecurityContext.fsGroup: 1000 (default)
auth.baseUrlMode=explicit requires publicBaseUrl	Missing public URL	Set server.publicUrl to your ingress URL
Database connection refused	PostgreSQL not ready or wrong credentials	Check postgresql.external.* values and the database-url secret

---

Important Notes

• Single replica is the safe default. The heartbeat scheduler has no distributed locking — running multiple replicas will cause duplicate heartbeats. Keep replicaCount: 1 until upstream adds leader election.
• WebSocket support is required on the Ingress. Paperclip uses WebSockets for live agent event streaming. Ensure proxy timeouts are >= 3600s.
• The master key is critical. Without it, all encrypted secrets stored in Paperclip are permanently lost. Back it up.

---

Links

• Helm chart source: osodevops/helm-charts
• Docker image: osodevops/docker-paperclip
• Upstream project: paperclipai/paperclip