How agents game code quality metrics

agents-game-metrics.md

Metrics are useful smoke alarms, but as optimization targets they are terrible. Agents don't "understand refactoring". They just learn to satisfy the tool.

Example:

cyclomatic complexity goes down, but locality and intent gets destroyed.

Metric target	Agent "fix"	What got worse
Cyclomatic complexity ↓	splits one readable method into 6 tiny methods	intent fragmented, call graph bigger
Coupling ↓	hides deps in ctx[:payments]	dependency contract invisible
Duplication ↓	creates generic method with flags	abstraction now has 12 modes

1. Cyclomatic complexity ↓, call graph swamp ↑

def total_for(user, cart) # a perfectly readable, short example
  return 0 if cart.empty?

  total = cart.items.sum(&:price)
  total *= 0.9 if user.vip?
  total += 15 unless cart.free_shipping?

  total
end

Now we "fix" complexity:

def total_for(user, cart)
  # With this character's death, the thread of prophecy is severed.
  #  Restore a saved game to restore the weave of fate,
  #  or persist in the doomed world you have created.
  apply_shipping(apply_discount(base_total(cart), user), cart)
end

def base_total(cart)
  return 0 if cart.empty?
  cart.items.sum(&:price)
end

def apply_discount(total, user)
  user.vip? ? total * 0.9 : total
end

def apply_shipping(total, cart)
  cart.free_shipping? ? total : total + 15
end

2. Coupling ↓ via trash context object

class Checkout
  def initialize(ctx)
    @ctx = ctx
  end

  def call(order_id)
    order = @ctx[:orders].find(order_id)
    @ctx[:payments].charge(order)
    @ctx[:mailer].receipt(order)
  end
end

Less dependencies right? Just one ctx. But now payments is just a hash key. Anything happens to that key, and your payments are gone. Super nasty for debugging.

def initialize(orders:, payments:, mailer:)
  @orders = orders
  @payments = payments
  @mailer = mailer
end

This is much more clear, you cannot even initialize without passing a proper dependency. And this is a code design choice. You want explicit dependencies, not implicit.

And yes, your context object can be fixed with dry-validation gem. It becomes a contract. Yes, now we have to add a dependency to validate this! Arguably a good thing to do, but this is no longer metrics problem. Now it is a design problem.

And I don't want to explode this gist but here how can you actually code this as a a custom rubocop rule... which exactly proves my point. This is SPEC now. You just created code that checks your code for architectural drift. And this spec is now living document and have to be maintained.

# lib/rubocop/cop/architecture/no_dependency_bag_lookup.rb
module RuboCop
  module Cop
    module Architecture
      class NoDependencyBagLookup < Base
        MSG = "Use explicit constructor dependencies, not ctx/container/registry lookup."

        BAG_NAMES = %i[ctx context container registry service_locator].freeze
        LOOKUP_METHODS = %i[[] fetch resolve get].freeze

        def on_send(node)
          return unless LOOKUP_METHODS.include?(node.method_name)
          return unless dependency_bag?(node.receiver)

          add_offense(node)
        end

        private

        def dependency_bag?(receiver)
          return false unless receiver

          case receiver.type
          when :lvar
            BAG_NAMES.include?(receiver.children[0])
          when :ivar
            BAG_NAMES.include?(receiver.children[0].to_s.delete_prefix("@").to_sym)
          when :send
            receiver.arguments.empty? && BAG_NAMES.include?(receiver.method_name)
          else
            false
          end
        end
      end
    end
  end
end

And this should catch any combination of

ctx[:payments]
@ctx.fetch(:mailer)
container.resolve(:orders)
registry.get(:payments)

but allow a simple

Checkout.new(orders:, payments:, mailer:)

3. Duplication ↓ → abstraction with flags

def export_report(format:, include_private:, compress:)
  # 80 lines of if format == :csv / :pdf / :json
end

Enjoy a god method with hidden behaviors, and agent would still fail at abstracting this. Here's one option how to solve it:

ExportReport.new(
  visibility_policy:,
  compressor:
).call(format:)

But how would you write a rubocop for it? I tried and utterly failed. There would be so many false positivies and invariants. And the rule would most likely end up being useless or even harmful.

When you are optimizing for metrics directly, agents will reduce local complexity by increasing non-local complexity:

• more indirection
• worse names
• hidden dependencies
• fragmented intent

Metrics can flag code for review, but they cannot define the desired shape of refactoring. A custom linter / cop / code quality metric can be written, but that is exactly the point — code semantics itself might have specs and ACs.