gistfile1.txt

workload (SDK)
    │  HTTPS to appreg.logring.io
    ▼
Route 53 A/ALIAS ──► NLB (TCP :443, cross-zone)
                         │
                         ▼
                    Envoy pods (infra cluster)
                    ├─ TLS termination (cert-manager Secret via SDS)
                    ├─ HTTP/2 routing by :authority
                    └─ TLS origination to EKS API (with EKS CA bundle)
                         │
                         ▼
                    Target EKS API (private endpoint,
                    reachable via VPC peering/PrivateLink)
                    
                    Prerequisites
Infra EKS cluster with AWS Load Balancer Controller installed.
cert-manager installed with a working ClusterIssuer (Let's Encrypt DNS-01 via Route 53 is a good default).
Route 53 hosted zone for logring.io.
The target EKS API endpoint is reachable from the infra cluster over VPC peering/PrivateLink — verify with openssl s_client -connect <target-eks-endpoint>:443 -servername <target-eks-endpoint> from an infra-cluster pod.
Target EKS cluster CA bundle: aws eks describe-cluster --name <target> --query 'cluster.certificateAuthority.data' --output text | base64 -d.




Great problem — this is essentially referential integrity enforcement in Kubernetes. There are a few approaches, each with trade-offs. Let me walk through them from simplest to most robust.
Approach 1: Validating Admission Webhook (Recommended)
This is the cleanest approach. A validating webhook intercepts DELETE requests on ApplicationInstance and rejects them if references exist.
How it works:
When someone tries to delete an ApplicationInstance, the webhook queries the API server for all CRDs that could reference it. If any exist, it denies the delete with a clear error message.
The "we don't know how many referencing CRDs" problem is solved by making the webhook discover referencing types dynamically. You have two sub-options:
Option A — Convention-based discovery
Establish a convention: any CRD that references ApplicationInstance must use a field like spec.applicationRef. The webhook discovers all CRDs at startup (or periodically) via the discovery API, then searches each for matching refs.
go
// Pseudocode for the webhook logic
func validateDelete(appInstance *AppInstance) admission.Response {
    // 1. Discover all CRDs in the cluster
    crdList := discoverCRDs()

    // 2. For each CRD, list resources that reference this ApplicationInstance
    for _, crd := range crdList {
        // Use unstructured client to list all instances
        resources := dynamicClient.Resource(crd.GVR).List(ctx, metav1.ListOptions{})

        for _, res := range resources.Items {
            // Walk the object looking for applicationRef matching our instance
            ref, found, _ := unstructured.NestedString(res.Object,
                "spec", "applicationRef", "name")
            if found && ref == appInstance.Name {
                return admission.Denied(fmt.Sprintf(
                    "cannot delete ApplicationInstance %q: referenced by %s/%s",
                    appInstance.Name, crd.Kind, res.GetName(),
                ))
            }
        }
    }
    return admission.Allowed("")
}
Option B — Registry-based discovery (more scalable)
Create a lightweight ReferenceRegistry CRD (or a ConfigMap) that each new downstream CRD registers itself in. This avoids scanning every CRD in the cluster:
yaml
apiVersion: mycompany.io/v1
kind: ReferenceRegistry
metadata:
  name: applicationinstance-refs
spec:
  targetResource:
    group: mycompany.io
    kind: ApplicationInstance
  referencingResources:
    - group: mycompany.io
      kind: APIGateway
      fieldPath: "spec.applicationRef.name"
    - group: mycompany.io
      kind: DatabaseBinding
      fieldPath: "spec.appRef.name"
The webhook reads this registry and only searches the listed resources. When a team adds a new CRD, they just add an entry to the registry.
Approach 2: Finalizer + Controller
An alternative if you can't or don't want to run a webhook:
A mutating webhook (or the controller that creates ApplicationInstance) adds a finalizer like mycompany.io/reference-protection.
When kubectl delete is called, Kubernetes sets deletionTimestamp but doesn't remove the object because the finalizer is present.
A controller watches ApplicationInstance for deletionTimestamp != nil, checks for references, and only removes the finalizer when all references are gone.
go
func (r *Reconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
    var app myv1.ApplicationInstance
    if err := r.Get(ctx, req.NamespacedName, &app); err != nil {
        return ctrl.Result{}, client.IgnoreNotFound(err)
    }

    if app.DeletionTimestamp == nil {
        // Not being deleted — ensure finalizer is present
        if !controllerutil.ContainsFinalizer(&app, finalizerName) {
            controllerutil.AddFinalizer(&app, finalizerName)
            return ctrl.Result{}, r.Update(ctx, &app)
        }
        return ctrl.Result{}, nil
    }

    // Being deleted — check for references
    refs, err := r.findReferences(ctx, &app)
    if err != nil {
        return ctrl.Result{}, err
    }

    if len(refs) > 0 {
        // Update status to inform the user
        app.Status.Conditions = setCondition(app.Status.Conditions,
            "DeletionBlocked", "True",
            fmt.Sprintf("Still referenced by: %v", refs))
        r.Status().Update(ctx, &app)
        // Requeue — check again later in case refs are removed
        return ctrl.Result{RequeueAfter: 30 * time.Second}, nil
    }

    // No references — safe to delete
    controllerutil.RemoveFinalizer(&app, finalizerName)
    return ctrl.Result{}, r.Update(ctx, &app)
}
Trade-off Summary
Concern	Webhook (Approach 1)	Finalizer (Approach 2)
UX	Immediate rejection — user sees "denied" right away	kubectl delete appears to succeed; object lingers with deletionTimestamp
Reliability	Webhook downtime can block all deletes (use failurePolicy: Ignore carefully)	Controller can be down temporarily; reconcile catches up
Extensibility	Registry or convention makes adding new CRDs easy	Same discovery logic applies
Cleanup	N/A	You need to think about what happens if someone really wants to force-delete
My Recommendation
Use both together:
The validating webhook gives an immediate, clear rejection at delete time — great developer experience.
The finalizer acts as a safety net in case the webhook is bypassed (e.g., direct etcd access, webhook downtime with failurePolicy: Ignore).
Use the registry pattern (Option B) for discovery, so each team that introduces a new referencing CRD just adds an entry without touching the webhook code.
Want me to flesh out any of these — the full webhook implementation, the registry CRD design, or the controller?