Plan: Migrate `provided_fields` to Static Metadata via `completion_state` Macro

Context

provided_fields on step classes returns field names a step will set on the workflow. The engine uses this to (a) block re-activation of completed steps and (b) build dependency graphs for static analysis.

Problem: Most implementations need a runtime ActiveFlowStep.find just to read the completion_state parameter — which is already on the ActiveFlowStepTrigger at definition time. This prevents static inspection of ActiveFlowDefinition without building mock instances.

Goal: A single source of truth for a step's completion state, declared on the class (overridable by trigger parameters), resolvable statically from triggers. Eliminate the 605 subclass overrides of provided_fields over time.

Key Constraints (from user)

Single value only. provided_fields → one completion state (array of 1 or 0). The existing dependency system assumes this; multi-value returns create ambiguity for static analysis.
Parameter always wins. The completion_state parameter on a trigger overrides any class-level macro. The macro is the default when the parameter isn't set.
Guard against subclass overrides. Use metaprogramming to prevent new provided_fields overrides — the macro should be the only way.
Migrate by pattern, not one-offs. Target the bulk patterns (hardcoded, standard completion_state). Leave truly custom implementations alone for now, but mark them.
Runtime behavior must not change. ActiveFlowStep records carry their own parameters copied from the trigger at launch — runtime resolution still uses that data.

Current Pattern Distribution (605 steps, re-verified)

Pattern	Count	Migration target
Hardcoded single-value (`['foo']` / `["foo"]`)	~552	→ `completion_state 'foo'` macro, delete override
Generic (uses `completion_state` param from trigger)	27 + 4 generic base classes	→ no macro, rely on parameter, delete override
`completion_state` with default	6	→ `completion_state 'default'` macro (parameter overrides)
Non-standard parameter-driven (custom field names)	~16	→ keep override, flag with `custom_provided_fields!` marker

(The 18 old get_parameters steps fold into the above categories after inspection.)

Design

1. New `completion_state` macro on `StepModule`

Follows the existing step_outcomes pattern:

# app/models/concerns/step_module.rb

# Declares this step's completion_state field name (default).
# Trigger parameters['completion_state'] overrides this at runtime.
def completion_state(value = nil)
  if value.nil?
    # reader
    return @declared_completion_state if defined?(@declared_completion_state)
    superclass.respond_to?(:completion_state) ? superclass.completion_state : nil
  else
    # writer
    @declared_completion_state = value.to_s.freeze
  end
end

Usage:

class InitializeVcccdStudentTermStep < Step
  completion_state 'initialization_complete'
end

class WaitForStudentTermCompletionStep < Step
  completion_state 'student_term_complete'  # trigger param overrides
end

class ApprovalStep < Step
  # No macro — always relies on trigger parameter (generic step)
end

2. Canonical `provided_fields` in `StepModule`

Replace the default stub with a resolver that uses the macro + runtime parameter:

# app/models/concerns/step_module.rb

def provided_fields(*args)
  fields = args[0]

  # Priority 1: runtime parameter from the step's trigger/parameters
  if fields.is_a?(Hash)
    afs = fields['active_flow_step']  # sometimes already injected as an instance
    afs ||= ActiveFlowStep.find_by(id: fields['active_flow_step_id']) if fields['active_flow_step_id']
    cs_param = afs&.get_parameter_by_name('completion_state')
    return [cs_param] if cs_param.present?
  end

  # Priority 2: class-level macro
  declared = completion_state  # reader
  return [declared] if declared.present?

  # Priority 3: nothing declared
  []
end

Note: fields['active_flow_step'] is sometimes pre-populated with an actual ActiveFlowStep instance (see active_flow_step.rb:964, failed_registration_active_flow_step.rb:42). Checking it first avoids a DB lookup.

This means once a subclass adopts the macro (or when the parameter is set on the trigger), its custom def self.provided_fields can be deleted — the base resolution handles it.

3. Metaprogramming guard — prevent overrides after migration

Conflict-based guard: if a class uses the macro AND overrides provided_fields, raise on class load:

# In StepModule
def completion_state(value = nil)
  # ... writer path ...
  if value
    @declared_completion_state = value.to_s.freeze
    if self.singleton_class.instance_methods(false).include?(:provided_fields)
      raise "#{self.name}: cannot override `provided_fields` when `completion_state` macro is declared. Remove the override."
    end
    install_override_guard!
  end
end

def install_override_guard!
  # Detect future (re-)definitions
  metaclass = self.singleton_class
  original_method_added = metaclass.method(:method_added) rescue nil
  metaclass.define_method(:method_added) do |name|
    if name == :provided_fields && @declared_completion_state
      raise "#{self.name}: do not override `provided_fields` — use the `completion_state` macro instead."
    end
    original_method_added&.call(name)
  end
end

This does not break legacy overrides — the guard only activates when the macro is used. As each class is migrated (override deleted, macro added), it becomes guarded. Newly-written step classes that try to override without using the macro are handled by a separate lint (see CLAUDE.md update below).

For a stronger long-term guard (once migration is complete), add a base-class method_added that always raises on provided_fields redefinition unless a temporary legacy_provided_fields! opt-in is declared.

4. Static resolution on `ActiveFlowStepTrigger`

# app/models/active_flow_step_trigger.rb

def static_provided_fields
  # Priority 1: trigger parameter (config always wins)
  if parameters.is_a?(Hash) && parameters['completion_state'].present?
    return [parameters['completion_state']]
  end

  # Priority 2: class-level macro
  step_klass = const_get_step_class
  declared = step_klass.respond_to?(:completion_state) ? step_klass.completion_state : nil
  return [declared] if declared.present?

  []
end

No DB round-trips. No step instance needed. Works on a raw trigger loaded from an ActiveFlowDefinition.

5. `ActiveFlowStep#get_provided_fields` — how it works today and what changes

Today:

def get_provided_fields
  all_provided_fields = []
  if self.step_class.present?
    step_class = const_get_step_class
    result = step_class.provided_fields(self.get_active_flow_fields)  # passes {..., 'active_flow_step_id' => self.id}
    all_provided_fields = all_provided_fields | result unless result.blank?
  end
  all_provided_fields
end

Each of the 605 overridden provided_fields methods does ActiveFlowStep.find(fields['active_flow_step_id']) and calls get_parameter_by_name('completion_state'). That's a DB round-trip per invocation.

ActiveFlowStep records carry their own parameters hash (copied verbatim from the trigger at launch by ActiveFlowStep#initialize). So the runtime data is already local to the step instance — the ActiveFlowStep.find in each override is technically redundant (the base resolver does it once via find_by).

After changes: get_provided_fields remains unchanged in shape. The behavior change is in StepModule.provided_fields (the base resolver) — subclasses that don't override it now get correct resolution via macro + parameter. Subclasses that still override keep working as before.

Risk surface and mitigation:

When a subclass's override is deleted in favor of the macro, the base resolver must return exactly what the override returned. → Validation rake task compares old vs new result for every active ActiveFlowStep record before each migration batch.
The base resolver does a find_by(id:) per call. This matches what the overrides already do (ActiveFlowStep.find), so no regression. Overrides that were deleted save one DB lookup compared to before (since they previously did their own find).
Steps with a completion_state_with_default helper: once given the macro as default, the trigger parameter overrides, preserving current semantics.

6. Marker for non-standard steps

For the ~16 steps that compute provided_fields from custom parameters (not completion_state), add an explicit marker so they're deliberately excluded from migration (not overlooked):

class CourseReviewApprovalStep < Step
  custom_provided_fields!  # marker — this step uses non-standard resolution
  # override remains
end

The custom_provided_fields! marker:

Has no runtime effect
Is introspectable (list all custom steps via Step.descendants.select(&:custom_provided_fields?))
Tracks tech debt explicitly

Steps to mark (initial audit): CourseReviewApprovalStep, SendMessageStep (also deprecated), UploadWorkflowDocumentStep (also unused), RegisterViaXmlStep*, and others surfaced during migration.

*RegisterViaXmlStep returns ['registration_xml_workflow_file_id'] — this is actually hardcoded, so it migrates cleanly via completion_state 'registration_xml_workflow_file_id'. Not a custom case.

7. Mark `SendMessageStep` deprecated

Uses the existing step_deprecated macro (from StepModule). Add a deprecation message pointing to the replacement (TBD at migration time). Only one fixture uses it (db/fixtures/169_csi_course_review.rb); that usage should be migrated to the replacement before the class is removed.

8. Delete unused step classes

Audit found 75 step class files with no references in fixtures (db/fixtures/), app code, lib, config, or specs, and no subclass descendants. Examples: UploadWorkflowDocumentStep, FerpaWaiverStep, StudentAgeWaiverStep, CompleteStepOnceStep, TestingRequiredStep, PushWaiversAssessmentScoresViaBannerApiStep, plus many more. Full list in section 11.

Required DB verification before deletion: A step can still be live in production via existing ActiveFlowStepTrigger records (set up historically, not in fixtures). For each candidate:

SELECT COUNT(*) FROM active_flow_step_triggers WHERE step_class = 'UploadWorkflowDocumentStep';
SELECT COUNT(*) FROM active_flow_steps WHERE step_class = 'UploadWorkflowDocumentStep';

Only delete if both counts are zero across production + QA environments.

9. `WorkflowVisualizer` + YAML handling

Update lib/workflow_visualizer.rb to call trigger.static_provided_fields as the primary source for completion_state fields.

Retain the YAML file (lib/workflow_step_provided_fields.yml) but narrow its purpose: it documents additional fields a step writes to the fields hash beyond its completion_state — e.g., a legacy step that sets fields['completion_state'] = 'complete' AND fields['some_extra_field'] = .... These secondary outputs are dependencies for downstream steps and must not be lost.

YAML becomes:

step_classes:
  SomeStep:
    additional_provides: [extra_field_a, extra_field_b]  # beyond completion_state

The visualizer composes: trigger.static_provided_fields + YAML[step_class][:additional_provides].

As steps are audited and their extra-field outputs either removed or documented via additional fields in trigger parameters, the YAML shrinks. Long-term goal: YAML contains only genuinely dynamic outputs.

10. Documentation updates

app/models/steps/CLAUDE.md — update the "Outcome Declarations" and "Step Outcomes Pattern" sections:

Replace example def self.provided_fields with completion_state 'my_status' macro
Document that trigger parameters['completion_state'] overrides the macro
Show the anti-pattern: overriding provided_fields when using the macro
Document custom_provided_fields! marker for steps that can't migrate
Update "Testing Considerations" example to use macro-style class

Wiki pages (/Users/ckozus/dev/rails/documentation.wiki/):

Workflow-Dependencies-Guide.md — update "How Steps Provide Fields" section to lead with the completion_state macro
ActiveFlow-Static-Analysis:-Explicit-Outcome-Declarations.md — add a completion_state macro section alongside step_outcomes
Update "Common Step Classes Reference" table to show macro-declared values

11. Spec file updates

When migrating a step class:

If the class has a spec file that directly calls provided_fields or asserts on its return value, update the spec to match the macro-based path (usually no change — it still returns the same array).
If the spec only tests perform/on_activate/outcomes, no changes needed.
No new specs are needed for the macro itself — the macro is declarative configuration, covered by the parity rake task and integration tests.
StepModule changes get a new spec file (spec/models/concerns/step_module_spec.rb if not present) covering: macro declaration, inheritance, parameter override precedence, conflict guard.

12. Unused Step Classes (75 candidates requiring DB verification)

AnonymousFacultyApprovalStep, AnonymousUploadTranscriptStep, ApplyStagedHsStudentNumberStep,
ApplyStagedStudentNumbersStep, ApproveCollegeStudentNumberStep, CheckStudentApprovalStep,
CheckStudentCodesViaBannerApiStep, CollegeApproveDropStep, CollegeApproveWithdrawStep,
CollegeReviewFormsRecommendationStep, CollegeReviewFormsStep,
CollegeReviewFormsTranscriptTestScoresStep, CollegeReviewInstructorQualificationsStep,
CollegeStudentApprovalStep, CompleteAcademyRequestStep, CompleteRequestToSubmitGradesStep,
CompleteStepOnceStep, ConfirmCourseSectionStep, DeclineDropWithdrawStep,
EtamuAcademicStandingStep, FerpaWaiverStep, FinalizeDropWithdrawStep,
GenerateInstructorDocumentFormStep, GenerateWorkflowFormStep,
GetStudentProgramsViaEthosApiStep, HoldsInquiryViaBannerEedmApiStep,
InitializeDropWithdrawStep, InitializeRequestToSubmitGradesStep, InstructorAgreementStep,
InstructorUploadDocumentStep, PrioritizeCoursesStep, ProvideGpaStep,
ProvideStudentNumberStep, PullWaiversAssessmentScoresViaBannerApiStep,
PushWaiversAssessmentScoresViaBannerApiStep, ReadAltCredViaEthosApiStep,
ResolveCourseSectionStep, ReviewStudentProgramsStep, SelectPaymentFeeTypeStep,
SendCollegeCourseReviewMessageStep, SetStudentProgramsViaEthosApiStep,
StudentAgeWaiverStep, SubmitApplicationViaJjcApiStep, TestingRequiredStep,
UploadDocumentsStep, UploadInstructorDocumentStep, UploadOptionalCourseReviewDocumentStep,
UploadResidencyDocumentsStep, UploadWorkflowDocumentStep, WriteAltCredViaEthosApiStep,
... (25 more)

Production verification query (run manually before deleting any class):

-- Bulk check: returns any step_class that IS referenced in prod/QA
WITH candidates(step_class) AS (
  VALUES
    ('UploadWorkflowDocumentStep'),
    ('FerpaWaiverStep'),
    ('StudentAgeWaiverStep'),
    ('CompleteStepOnceStep'),
    ('TestingRequiredStep')
    -- ... add all 75 candidates here
)
SELECT c.step_class,
       (SELECT COUNT(*) FROM active_flow_step_triggers t WHERE t.step_class = c.step_class) AS trigger_count,
       (SELECT COUNT(*) FROM active_flow_steps s WHERE s.step_class = c.step_class) AS step_count
FROM candidates c
ORDER BY trigger_count DESC, step_count DESC;

Only classes with trigger_count = 0 AND step_count = 0 in both production and QA are safe to delete. Run on each environment, cross-reference results, delete in a single cleanup PR.

Migration Tranches

Tranche 1 — Framework (no step file changes)

Add completion_state macro to StepModule
Add custom_provided_fields! marker to StepModule
Replace StepModule#provided_fields default with macro+parameter resolver
Add ActiveFlowStepTrigger#static_provided_fields
Add conflict-detection guard (macro + override = raise)
Add validation rake task: rake active_flow:verify_provided_fields_parity — iterates all ActiveFlowStep records, compares klass.provided_fields(fields) (current) vs expected-from-macro — flags mismatches

Tranche 2 — Generic steps (no macro, delete overrides)

Steps that rely on the trigger parameter always being set: ApprovalStep, UploadDocumentStep, MergeStep, BranchingStep. Delete their provided_fields overrides — base resolver handles them.

Tranche 3 — Hardcoded steps (bulk migration via script)

~552 steps. Script:

Parses def self.provided_fields method bodies returning a single literal array
Extracts the field name
Deletes the method
Adds completion_state '<field>' near top of class body
Commits in batches grouped by directory (college-specific folders first)

Run validation rake task after each batch.

Tranche 4 — completion_state with defaults (6 steps)

Add completion_state '<default>' macro, delete override. Parameter overrides at runtime.

Tranche 5 — Mark non-standard steps

Add custom_provided_fields! to the ~16 remaining steps that compute fields from non-standard parameters (approval_state, workflow_file_kind, mailer_template, etc.). Mark SendMessageStep as step_deprecated.

Tranche 6 — Delete unused classes

After DB verification, delete the 75 unused step files in one or more cleanup PRs.

Tranche 7 — Documentation

Update app/models/steps/CLAUDE.md and both wiki pages.

Tranche 8 — Workflow visualizer

Update lib/workflow_visualizer.rb to use trigger.static_provided_fields as primary source. Restructure YAML into additional_provides format. Retain YAML for extra-field documentation only.

Files to Modify (framework code)

File	Change
`app/models/concerns/step_module.rb`	Add `completion_state`, `custom_provided_fields!` macros; new `provided_fields` resolver; conflict guard
`app/models/active_flow_step_trigger.rb`	Add `static_provided_fields`
`app/models/active_flow_step.rb`	No change needed (delegates to class) — verify
`lib/workflow_visualizer.rb`	Use `static_provided_fields` + YAML `additional_provides`
`lib/workflow_step_provided_fields.yml`	Restructure to `additional_provides` only
`app/models/steps/CLAUDE.md`	Update patterns and examples
`documentation.wiki/Workflow-Dependencies-Guide.md`	Update
`documentation.wiki/ActiveFlow-Static-Analysis:-Explicit-Outcome-Declarations.md`	Update
`lib/tasks/active_flow_*.rake` (new)	Add `verify_provided_fields_parity` task

Verification

Parity rake task before/after each tranche — zero mismatches required
bundle exec rspec — no spec changes expected for untouched files
Manual: rake workflow:generate_dot for representative definitions, diff DOT output before/after framework change
Spot-check: load a production ActiveFlowStepTrigger in console and call .static_provided_fields — compare with ActiveFlowStep.find_by(active_flow_step_trigger_id: ...).get_provided_fields

QA Plan

Explicit QA steps per tranche:

Tranche 1 (framework) — highest risk

Run rake active_flow:verify_provided_fields_parity against prod DB snapshot — expect zero mismatches
Run same rake task against QA DB snapshot
Load a handful of active workflows in console, call afs.get_provided_fields on each step, verify output unchanged
Pick 3–5 representative definitions, run rake workflow:generate_dot MODE=strict before and after — diff DOT files, expect no change
Trigger a full workflow run end-to-end on QA (course registration, application submission) — verify steps activate and complete correctly
Confirm conflict guard raises: create a test step with both completion_state 'x' macro AND a def self.provided_fields override, expect exception on class load

Tranches 2–4 (step migrations) — per batch

Run parity rake task — zero mismatches
Full RSpec suite passes
For each batch, pick one migrated step, find a live workflow using it on QA, trigger activation, verify completion flows correctly and downstream steps activate
For generic steps batch (ApprovalStep, UploadDocumentStep, MergeStep, BranchingStep): trigger at least one real instance of each on QA

Tranche 5 (custom markers) — low risk

Step.descendants.select { |s| s.respond_to?(:custom_provided_fields?) && s.custom_provided_fields? } returns expected list
No behavior change — full RSpec suite passes

Tranche 6 (unused class deletion) — careful

SQL verification query run on prod + QA, results attached to PR
Fixture grep + code grep confirmed empty per class
RSpec suite passes after deletion (catches any overlooked references)

Tranche 7 (docs) — non-code

CLAUDE.md examples match actual new API
Wiki pages reviewed by a developer not on the ticket

Tranche 8 (visualizer)

Generate DOT for 10+ definitions before/after — diff and review
Lenient and strict modes both still work
YAML additional_provides correctly propagates to visualizer output

PR Description Template

Each PR in this effort should include:

Core Changes

Bullet list of the actual structural change (e.g., "Added completion_state macro to StepModule; replaced default provided_fields with parameter+macro resolver")
Files touched with one-line explanation each

Trade-offs

Why a macro + parameter-override model over alternatives (e.g., storing provided_fields on trigger DB column directly)
Why we keep legacy overrides working during migration rather than a flag-day rewrite
Why the YAML file is retained in a narrower role

Risks

What could break: subclass override removed but base resolver returns different value → caught by parity task
What's out of scope: the ~16 custom steps remain using override-style resolution
DB-referenced but code-unreferenced step classes (unused class deletion risk)
Migration sequencing: framework must deploy before any step class change

Validation Evidence

Parity rake task output (before/after, zero mismatches)
DOT diff for representative definitions (no change expected)
List of workflows exercised on QA

Rollout

Which tranche this PR is in
Dependencies on prior tranches (explicit ticket references)
Post-merge verification checklist

The PR description focuses on why these changes are safe and what reviewers should scrutinize, not on re-explaining the macro feature (which lives in CLAUDE.md / wiki).

ckozus/provided-fields-metadata-plan.md

Plan: Migrate provided_fields to Static Metadata via completion_state Macro