Metricing: Calculate vs Hint Behavior

metricing-calculate-vs-hint-behavior.md

Metricing: Calculate vs Hint Behavior

Overview

The metricing pipeline always runs in two sequential phases for numerical metrics:

1. Calculate — compute and persist metric values, tier by tier
2. Hint — enrich existing metrics with statistical context (deltas, descriptive stats, rankings, predictions), tier by tier

Hinting never interleaves with calculation. All tiers finish calculating before hinting begins.

---

Entry Points and Their Behavior

Entry Point	Trigger	overwrite default	Calculate	Hint
Admin UI → Generate (bulk_create_all, bulk_create_for_account, create)	User clicks Generate	false	Skips metrics that already exist	Only hints current month (MIN_BACKFILL_NUMBER = 1)
Admin UI → Regenerate	User clicks Regenerate	true	Overwrites all metrics for all backfill_count months	Re-hints all backfill_count months (default 24)
GenerateAggregateMetricsJob	Background / manual	true	Always calculates	Always hints — both happen in one transaction per metric, no separate hinting phase

---

The overwrite Flag — Two Effects

Effect 1: Calculation (BaseMetric#calculate_and_persist)

def calculate_and_persist(metricing_id = nil)
  return unless data_exists?
  return unless calculatable_metric_for_facet?
  return metric unless overwrite  # ← EARLY RETURN if metric exists and overwrite=false
  # ...calculate and persist...
end

• overwrite: false → skips calculation if a metric record already exists for that account/date/facets
• overwrite: true → recalculates and overwrites regardless

Effect 2: Hinting scope (GenerateHintJob#perform)

(overwrite ? backfill_count : MIN_BACKFILL_NUMBER).times do |periods_ago|
  # queue GenerateHintCalculatorJob for each month
end

• overwrite: false → hints only MIN_BACKFILL_NUMBER months (env default: 1 — current month only)
• overwrite: true → hints for all backfill_count months (env default: 24)

---

The hintable Flag — Per-Metric Opt-Out

BaseMetric sets @hintable = true by default. Individual metric subclasses can override this:

def hint_and_persist
  return unless metric && hintable  # ← exits immediately if hintable=false
  # ...
end

If a metric subclass sets @hintable = false, hinting is skipped for that metric entirely, regardless of what the job requests. This is the per-metric-class level override.

---

Job Chain (Normal Metricing Path)

Admin UI / CLI
  └─► GenerateMetricsForAllAccountsJob          (all accounts)
  └─► GenerateMetricsForAllAccountFacetsJob      (one account, all facets)
  └─► GenerateMetricsForAccountJob               (one account, one facet) ← core orchestrator
        │
        ├─ PHASE 1: batch_calculate_tier(tier 0)
        │     └─► GenerateMetricJob              (handles backfill loop)
        │           └─► GenerateMetricCalculatorJob × backfill_count
        │                 └─► metric.calculate_and_persist(metricing_id)
        │
        ├─ [tier 0 complete] → batch_calculate_tier(tier 1) → ... → max_tier
        │
        └─ PHASE 2 (after all tiers calculated): batch_hint_tier(tier 0)
              └─► GenerateHintJob               (handles backfill loop)
                    └─► GenerateHintCalculatorJob × (overwrite ? backfill_count : MIN_BACKFILL_NUMBER)
                          └─► metric.hint_and_persist

---

Aggregate Metrics Path (Different)

GenerateAggregateMetricsJob uses a simpler, self-contained flow — no tier batching, no separate hint phase:

GenerateAggregateMetricsJob
  └─► GenerateSingleAggregateMetricJob (per metric, per date)
        └─► am.calculate_and_persist   ← always runs
        └─► am.hint_and_persist        ← always runs immediately after

Aggregates default to overwrite: true and always hint — no conditional logic.

---

Key Parameters Reference

Parameter	Controls	Default
overwrite	Whether existing metrics are recalculated; how many months get hinted	false (UI generate), true (regenerate, aggregates)
backfill_count	How many months back to calculate metrics	ENV["MAX_BACKFILL_NUMBER"] (default 24)
MIN_BACKFILL_NUMBER	How many months get hinted when overwrite=false	ENV["MIN_BACKFILL_NUMBER"] (default 1)
hintable	Per-metric-class opt-out from hinting	true in BaseMetric, overridable in subclasses
metric_factory_collection_id	Filters calculation/hinting to a specific collection	nil (all active collections)

---

Summary: Three Scenarios

1. Normal generate (overwrite: false)

• Calculates only new metrics (skips existing ones)
• Hints only the current month per metric factory

2. Regenerate (overwrite: true)

• Overwrites all metrics for all backfill_count months
• Re-hints all backfill_count months — expensive

3. Aggregate metrics (GenerateAggregateMetricsJob)

• Always calculates + hints immediately in one shot per metric
• No separate hinting phase; defaults overwrite: true

---

Rails Console Examples

Lookup helpers (useful before running jobs)

# Find a specific account
account = Account.find(123)
account = Account.find_by(slug: "acme-hvac")

# List all MetricFactoryCollections and their IDs
MetricFactoryCollection.active.pluck(:id, :name, :platform)
# => [[1, "Home Services", "home_services"], [2, "Accounting", "accounting"], ...]

# Find a specific collection by name
collection = MetricFactoryCollection.find_by(name: "Accounting")
collection.id  # use this as metric_factory_collection_id

# Reference date convention: always first of the month
reference_date = Date.current.beginning_of_month  # e.g. 2026-03-01

---

Generate for ALL accounts, ALL collections (normal — no overwrite)

GenerateMetricsForAllAccountsJob.perform_later(
  reference_date: Date.current.beginning_of_month
)

Generate for ALL accounts, ALL collections — with overwrite (full recalculation + full rehinting)

GenerateMetricsForAllAccountsJob.perform_later(
  reference_date: Date.current.beginning_of_month,
  overwrite: true
)

Generate for ALL accounts, ALL collections — backfill N months

GenerateMetricsForAllAccountsJob.perform_later(
  reference_date: Date.current.beginning_of_month,
  backfill_count: 12
)

---

Generate for ONE account, ALL collections (normal — no overwrite)

GenerateMetricsForAllAccountFacetsJob.perform_later(
  account_id: 123,
  reference_date: Date.current.beginning_of_month
)

Generate for ONE account, ALL collections — with overwrite

GenerateMetricsForAllAccountFacetsJob.perform_later(
  account_id: 123,
  reference_date: Date.current.beginning_of_month,
  overwrite: true
)

---

Generate for ONE account, ONE specific collection

First find the collection ID, then pass it to GenerateMetricsForAccountJob directly. Note: this job also requires a Metricing record and metricing_id.

account = Account.find(123)
reference_date = Date.current.beginning_of_month
collection = MetricFactoryCollection.find_by(name: "Accounting")  # or .find(2)

# Create or find the Metricing record (tracks this run)
metricing = Metricing.where(
  account_id: account.id,
  reference_date: reference_date,
  facets: {}
).first_or_create! do |m|
  m.run_date = Time.current
  m.filters  = { id: account.id }
end

GenerateMetricsForAccountJob.perform_later(
  account_id: account.id,
  reference_date: reference_date,
  metricing_id: metricing.id,
  metric_factory_collection_id: collection.id
)

Same, with overwrite

GenerateMetricsForAccountJob.perform_later(
  account_id: account.id,
  reference_date: reference_date,
  metricing_id: metricing.id,
  metric_factory_collection_id: collection.id,
  overwrite: true
)

Same, with overwrite and custom backfill (e.g. 6 months)

GenerateMetricsForAccountJob.perform_later(
  account_id: account.id,
  reference_date: reference_date,
  metricing_id: metricing.id,
  metric_factory_collection_id: collection.id,
  overwrite: true,
  backfill_count: 6
)

---

Generate for ONE account, a FEW specific collections

There is no built-in multi-collection parameter — run one job per collection:

account_id = 123
reference_date = Date.current.beginning_of_month
collection_names = ["Accounting", "Home Services"]

MetricFactoryCollection.where(name: collection_names).each do |collection|
  metricing = Metricing.where(
    account_id: account_id,
    reference_date: reference_date,
    facets: {}
  ).first_or_create! do |m|
    m.run_date = Time.current
    m.filters  = { id: account_id }
  end

  GenerateMetricsForAccountJob.perform_later(
    account_id: account_id,
    reference_date: reference_date,
    metricing_id: metricing.id,
    metric_factory_collection_id: collection.id,
    overwrite: true
  )
end

---

Synchronous execution (blocks — useful for console debugging)

Replace perform_later with perform_now to run inline without Sidekiq:

GenerateMetricsForAllAccountsJob.perform_now(
  reference_date: Date.current.beginning_of_month,
  overwrite: true
)