Skip to content

Instantly share code, notes, and snippets.

@MaxGhenis

MaxGhenis/PolicyEngine-CLAUDE.md

Created July 22, 2025 20:36
Show Gist options
  • Save MaxGhenis/30eecb0e85ef1f0e5d25942805af190a to your computer and use it in GitHub Desktop.
Save MaxGhenis/30eecb0e85ef1f0e5d25942805af190a to your computer and use it in GitHub Desktop.
Download ZIP
Raw
PolicyEngine-CLAUDE.md

PolicyEngine Development Guidelines

Repository Overview

PolicyEngine is a multi-repository organization that builds microsimulation models for tax and benefit systems across multiple countries. The codebase is organized into several main components:

Core Infrastructure

  • policyengine-core: Fork of OpenFisca-Core that powers country models and apps
  • policyengine-api: Flask-based API serving policy simulations and calculations
  • policyengine-app: React-based web application (the main user interface)

Country-Specific Models

  • policyengine-us: US federal and state tax/benefit microsimulation model
  • policyengine-uk: UK tax-benefit microsimulation model
  • policyengine-canada: Canadian tax-benefit model
  • policyengine-il: Israel tax-benefit model
  • policyengine-ng: Nigeria tax-benefit model

Data & Tools

  • policyengine-us-data: Enhanced CPS microdata for US simulations
  • microdf: Python package for analyzing distributional effects
  • survey-enhance: Tools for imputation and reweighting of survey data
  • policyengine.py: Python package for programmatic PolicyEngine access

Applications & Calculators

Multiple specialized calculators and dashboards for specific policy analyses (e.g., salt-amt-calculator, child-tax-credit-expansion-impact, 2024-election-dashboard)

Development Environment Setup

Python Projects

  1. Python Version: 3.10-3.12 (check each project's pyproject.toml for specifics)
  2. Installation: make install or pip install -e ".[dev]" --config-settings editable_mode=compat
  3. Virtual Environment: Recommended to use venv or conda for isolation

React/JavaScript Projects

  1. Node Version: >=22.0.0 (use nvm for version management)
  2. Installation: npm ci (not npm install to ensure lockfile consistency)
  3. Package Manager: npm (not yarn or pnpm)

Build & Test Commands

Python Projects (Core, API, Country Models)

make install      # Install dependencies
make test         # Run all tests
make format       # Format code with Black (79 chars)
make changelog    # Update changelog before PR
pytest path/to/test.py::test_function -v  # Run specific test

React App (policyengine-app)

make install      # Install npm packages and Python tools
make debug        # Start dev server
make test         # Run Jest tests
make format       # ESLint fix + Prettier
npm run lint -- --max-warnings=0  # Check CI will pass

API Development

make debug        # Start Flask dev server
make debug-test   # Run tests with debug output
make deploy       # Deploy to Google Cloud

Code Standards

Python Code Style

  • Formatter: Black with 79-character line length
  • Imports: Grouped by stdlib, third-party, local (alphabetized within groups)
  • Naming: snake_case for functions/variables, CamelCase for classes
  • Type Hints: Use where possible, especially for public APIs
  • Docstrings: Required for all public functions/classes
  • Error Handling: Catch specific exceptions, provide helpful messages

JavaScript/React Code Style

  • Formatter: Prettier (default config) + ESLint
  • Components: Functional with hooks (no class components)
  • React Imports: Use named imports (import { useState } from "react")
  • File Naming: PascalCase for components, camelCase for utilities
  • Component Size: Keep under 150 lines after formatting
  • TypeScript: Gradually adopting - use for new files when possible

Repository Structure Patterns

Country Model Structure

policyengine-{country}/
├── policyengine_{country}/
│   ├── parameters/      # YAML files with tax rates, thresholds
│   │   └── gov/        # Organized by department/agency
│   ├── variables/      # Python formulas for calculations
│   │   └── gov/        # Matches parameter structure
│   ├── reforms/        # Pre-defined policy reforms
│   ├── tests/          # Test cases for validation
│   ├── entities.py     # Person, household definitions
│   └── model_api.py    # Main API entry point
├── Makefile
├── pyproject.toml
└── changelog.yaml

Parameter Files (YAML)

  • Lowercase, concise labels
  • Include legislative references
  • Organized by government department
  • Use correct department (e.g., education → DfE, not DWP)

Variable Files (Python)

  • One variable per file (recent refactoring standard)
  • Include regulatory citations in comments
  • Import from policyengine_core.variables import Variable
  • Use numpy functions (np.ceil, not ceil)

Testing Guidelines

Python Tests

  • Write tests for all new functionality
  • Use pytest framework
  • Place in tests/ directory matching source structure
  • Run with coverage to ensure adequate testing
  • For country models: YAML test cases for calculations

React Tests

  • Jest + React Testing Library
  • Tests in __tests__ directories with .test.js suffix
  • Test user interactions, not implementation details
  • Mock API calls appropriately
  • Coverage reports in coverage/ directory

Version Control & PR Process

Changelog Management

  1. Add changelog_entry.yaml at repo root: 
    - bump: patch|minor|major
  changes:
    added|changed|removed|fixed:
    - Description of change
  2. Run make changelog to update main changelog
  3. Commit both changelog files

Git Workflow

  1. Fork the repository
  2. Create feature branch from master/main
  3. Keep fork synced: git pull upstream master
  4. Open PR from feature branch to upstream master
  5. Include "Fixes #123" in PR description
  6. Request review when ready

Commit Messages

  • Clear, descriptive messages
  • Reference issue numbers
  • Explain "why" not just "what"

API & Data Flow

API Architecture

  • Flask-based REST API
  • Redis for caching
  • Google Cloud SQL for persistence
  • Background jobs with RQ (Redis Queue)
  • Country models loaded dynamically

Data Flow

  1. User Input → React App
  2. API Request → Flask API with country/reform/household data
  3. Simulation → Country model (OpenFisca-based)
  4. Results → Cached and returned to frontend
  5. Visualization → Plotly charts in React

Key API Endpoints

  • /calculate: Run household calculations
  • /economy/{country_id}/over/{policy_id}: Society-wide impacts
  • /parameters: Get parameter metadata
  • /variables: Get variable metadata

Common Patterns & Utilities

React App Patterns

  • State Management: No global state - lift state up
  • Routing: React Router v6 with search params
  • Country Switching: Use useCountryId hook
  • API Calls: Use fetch with proper error handling
  • Charts: Plotly.js with react-plotly.js wrapper

Python Model Patterns

  • Variables: Inherit from Variable class
  • Parameters: Defined in YAML, accessed via parameter system
  • Reforms: Modify parameters or add/replace variables
  • Entities: Person, household, tax unit hierarchies
  • Time Periods: Handle different period types (year, month, eternity)

Deployment & Infrastructure

Google Cloud Platform

  • App Engine: Hosts API and React app
  • Cloud SQL: PostgreSQL for data persistence
  • Cloud Build: CI/CD pipeline
  • Service Accounts: For deployment authentication

Environment Variables

  • Development: .env files (not committed)
  • Production: Set in GCP App Engine config
  • API keys: Stored securely, never in code

Debugging Tips

Python Debugging

  • Use pytest -vv for verbose test output
  • Add import pdb; pdb.set_trace() for breakpoints
  • Check variable values with simulation.calculate()
  • Use --durations=0 to find slow tests

React Debugging

  • React Developer Tools browser extension
  • Console.log liberally during development
  • Check Network tab for API calls
  • Use React Profiler for performance

Common Issues

  • Import Errors: Check PYTHONPATH and editable installs
  • CORS Issues: Verify API URL configuration
  • State Updates: Use spread operator for immutability
  • Enum Values: Must match exactly (case-sensitive)

Best Practices

Code Quality

  1. Run formatters before committing
  2. Write self-documenting code
  3. Add comments for complex logic
  4. Keep functions focused and small
  5. Handle edge cases explicitly

Performance

  1. Cache expensive calculations
  2. Use React.memo for pure components
  3. Lazy load large components
  4. Optimize Plotly chart rendering
  5. Profile before optimizing

Security

  1. Never commit secrets or API keys
  2. Validate all user inputs
  3. Use parameterized queries
  4. Follow OWASP guidelines
  5. Regular dependency updates

Accessibility

  1. Semantic HTML elements
  2. ARIA labels for complex widgets
  3. Keyboard navigation support
  4. Color contrast compliance
  5. Screen reader testing

Resources & Documentation

Internal Documentation

  • Each repo's README.md for specific setup
  • API documentation at /api/documentation
  • Country model docs in docs/ directories

External Resources

Getting Help

  • GitHub Issues for bugs/features
  • Internal Slack/Discord for questions
  • Code reviews for learning patterns
  • Pair programming encouraged

Quick Reference

Make Commands Cheatsheet

# All Python projects
make install    # Install dependencies
make test       # Run tests
make format     # Format code
make changelog  # Update changelog

# React app specific
make debug      # Start dev server
make build      # Production build

# API specific
make debug-test # Debug mode tests
make deploy     # Deploy to GCP

File Naming Conventions

  • Python: snake_case.py
  • React Components: PascalCase.jsx
  • Tests: test_*.py or *.test.js
  • YAML: snake_case.yaml

Common File Locations

  • Parameters: parameters/gov/{dept}/{policy}.yaml
  • Variables: variables/gov/{dept}/{concept}.py
  • React Pages: src/pages/{PageName}.jsx
  • API Routes: policyengine_api/endpoints/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment