We want the cost of applying pprof labels to be low enough that we stop conditionally labeling. Today, CockroachDB only applies labels when a profile is actively being collected, checked via an atomic. This means operations that passed the labeling point before profiling started are unlabeled, goroutine dumps are unlabeled, and when labeling IS on, it costs more than it should.
pprof.Do(ctx, Labels("req", strconv.Itoa(id)), func(ctx) { ... }) triggers 4-5 heap allocations: