DDEV Go-based Add-ons Implementation Proposal

ddev-go-addons-proposal.md

DDEV Go-based Add-ons Implementation Proposal

Overview

This proposal outlines an approach to enable Go-based actions as an alternative technique for DDEV add-ons, while maintaining full support for existing bash-based actions. Both techniques will be equally supported, and add-on developers can choose the approach that best fits their needs. The solution uses Docker containers with Go runtime to build and execute Go source files on-demand, requiring only Docker (which DDEV already requires) on user machines.

Current DDEV Add-on System

DDEV add-ons currently use install.yaml with bash scripts:

name: my-addon
pre_install_actions:
  - |
    #ddev-description:Check something
    if some_bash_condition; then
      echo "Success"
    else 
      exit 1
    fi
post_install_actions:
  - |
    #ddev-description:Configure something
    generate_config_file

Proposed Go Actions (Alternative Technique)

Enhanced install.yaml

name: my-addon

# Option 1: Use bash actions (existing, fully supported)
pre_install_actions:
  - |
    #ddev-description:Validate environment
    check_environment_bash_logic
post_install_actions:
  - |
    #ddev-description:Generate configs
    generate_configs_bash_logic

# Option 2: Use Go actions (new alternative technique)
go_actions:
  pre_install:
    - action_name: "validate_environment"
      go_source: "actions/validate.go"
    - action_name: "check_dependencies"
      go_source: "actions/check_deps.go"
  post_install:
    - action_name: "generate_configs"
      go_source: "actions/generate.go"

# Option 3: Mix both approaches (both techniques work together)
pre_install_actions:
  - |
    #ddev-description:Quick bash check
    simple_bash_validation
go_actions:
  post_install:
    - action_name: "complex_config_generation"
      go_source: "actions/generate_complex_configs.go"

Go Action Structure (Pseudocode)

// actions/validate.go
package main

func main() {
    context := loadDdevContext()
    
    if validateEnvironment(context) {
        printSuccess("Environment validation passed")
    } else {
        printError("Environment validation failed")
        exit(1)
    }
}

func validateEnvironment(context DdevContext) bool {
    // Type-safe validation logic
    if context.hasService("database") {
        return true
    }
    return false
}

Add-on Directory Structure

my-addon/
├── install.yaml
├── docker-compose.service.yaml
├── commands/host/my-command
└── actions/
    ├── go.mod
    ├── validate.go
    ├── check_deps.go
    └── generate.go

Implementation Details

Docker-based Execution

When DDEV encounters a go_actions section, it:

1. Mounts the add-on directory into a Go runtime container
2. Provides DDEV context as JSON file to the Go program
3. Builds and runs the Go source file using go run
4. Returns the result just like bash actions

Pseudocode for execution:

func processGoAction(action GoAction, ddevContext map[string]interface{}) error {
    contextFile := createTempContextFile(ddevContext)
    
    dockerCommand := []string{
        "docker", "run", "--rm",
        "-v", addonDirectory + ":/addon:ro",
        "-v", projectDirectory + ":/project", 
        "-v", contextFile + ":/context.json:ro",
        "-e", "DDEV_ADDON_CONTEXT=/context.json",
        "-w", "/addon",
        "ddev/go-addon-runner:latest",
        "go", "run", action.GoSource,
    }
    
    return executeDockerCommand(dockerCommand)
}

Docker Container (ddev/go-addon-runner:latest)

The container includes:

• Go 1.23+ runtime
• Common dependencies (yaml parsing, HTTP clients, etc.)
• DDEV context utilities
• Standard Unix tools (jq, curl, bash)

Go Action Interface

All Go actions follow a standard pattern:

// Pseudocode interface
type AddonAction interface {
    Execute(context DdevContext) error
    Describe() string  // For #ddev-description output
}

type DdevContext struct {
    ProjectConfig map[string]interface{}
    GlobalConfig  map[string]interface{}
    Environment   map[string]string
    ProjectRoot   string
    YamlFiles     map[string]interface{}
}

Benefits

For Simple Add-ons

• Type Safety: Compile-time error checking vs runtime bash errors
• Better Error Handling: Structured error messages with context
• Testing: Unit testable Go code vs difficult-to-test bash
• IDE Support: Full IDE support for development and debugging

For Complex Add-ons

• Configuration Parsing: Robust YAML/JSON parsing with struct validation
• Data Transformation: Complex logic becomes maintainable
• Performance: Faster execution for heavy processing
• Modularity: Clean separation of concerns

Example: Complex Configuration Transformation

// Pseudocode for Platform.sh-style complex transformation
type PlatformConfig struct {
    App      AppConfig
    Services map[string]ServiceConfig
    Routes   map[string]RouteConfig
}

func transformPlatformToDdev(platform PlatformConfig) DdevConfig {
    ddev := DdevConfig{}
    
    // Type-safe transformations
    ddev.PHPVersion = extractPHPVersion(platform.App.Type)
    ddev.Database = transformDatabase(platform.Services)
    ddev.AdditionalServices = transformServices(platform.Services)
    ddev.Hooks = transformHooks(platform.App.Hooks)
    
    return ddev
}

User Experience

For Add-on Users

• No Changes Required: Existing add-ons continue to work
• No Go Installation: Only Docker required (already needed for DDEV)
• Better Error Messages: More helpful error output from Go actions
• Faster Installation: Especially for complex add-ons

For Add-on Developers

• Choice of Technique: Can choose bash, Go, or mix both based on needs
• Standard Tooling: Use familiar Go development tools when choosing Go
• Testing: Standard Go testing framework for Go actions
• Documentation: Generated from Go code comments for Go actions

Implementation Phases

Phase 1: Core Infrastructure

• Create ddev/go-addon-runner Docker image
• Extend InstallDesc struct to support go_actions
• Modify ProcessAddonAction() to handle Go actions
• Create DDEV context interface and utilities

Phase 2: Developer Experience

• Add-on scaffolding tool (ddev addon create --go)
• Testing framework for Go actions
• Documentation and examples
• Best practices guide for choosing between bash and Go

Phase 3: Examples and Documentation

• Create Go-based version of ddev-phpmyadmin as simple example
• Create Go-based version of complex parts of ddev-platformsh as advanced example
• Community documentation and best practices for both techniques

Compatibility and Coexistence

• Full Backward Compatibility: All existing bash-based add-ons continue to work unchanged
• Equal Support: Both bash and Go actions are first-class citizens with equal support
• Mix and Match: Add-ons can use bash actions, Go actions, or both together
• No Migration Required: Existing add-ons can stay bash-based indefinitely
• Developer Choice: Add-on developers choose the best technique for their use case
• No Breaking Changes: No impact on existing DDEV functionality

Technical Requirements

User Requirements

• Docker (already required by DDEV)
• No Go installation needed
• No additional system dependencies

Add-on Developer Requirements

• Go knowledge for writing Go actions
• Basic understanding of Docker (for testing)
• Standard Go development environment (for development only)

Detailed Implementation Example

Real Go Code Example

Here's how the phpMyAdmin add-on validation would look:

// actions/validate_db.go
package main

import (
    "encoding/json"
    "fmt"
    "os"
)

type AddonContext struct {
    DdevProjectConfig map[string]interface{} `json:"DdevProjectConfig"`
    DdevGlobalConfig  map[string]interface{} `json:"DdevGlobalConfig"`
}

func main() {
    // Load context from file
    contextFile := os.Getenv("DDEV_ADDON_CONTEXT")
    if contextFile == "" {
        fmt.Fprintf(os.Stderr, "DDEV_ADDON_CONTEXT not set\n")
        os.Exit(1)
    }
    
    data, err := os.ReadFile(contextFile)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Failed to read context: %v\n", err)
        os.Exit(1)
    }
    
    var ctx AddonContext
    if err := json.Unmarshal(data, &ctx); err != nil {
        fmt.Fprintf(os.Stderr, "Failed to parse context: %v\n", err)
        os.Exit(1)
    }
    
    // Validate database service exists
    if err := validateDatabaseService(&ctx); err != nil {
        fmt.Fprintf(os.Stderr, "Database validation failed: %v\n", err)
        os.Exit(1)
    }
    
    fmt.Printf("#ddev-description:Check for db service\n")
}

func validateDatabaseService(ctx *AddonContext) error {
    // Check global config for omitted containers
    if globalConfig, ok := ctx.DdevGlobalConfig["omit_containers"]; ok {
        if containers, ok := globalConfig.([]interface{}); ok {
            for _, container := range containers {
                if container == "db" {
                    return fmt.Errorf("unable to install the add-on because no db service was found")
                }
            }
        }
    }
    
    // Check project config for omitted containers  
    if projectConfig, ok := ctx.DdevProjectConfig["omit_containers"]; ok {
        if containers, ok := projectConfig.([]interface{}); ok {
            for _, container := range containers {
                if container == "db" {
                    return fmt.Errorf("unable to install the add-on because no db service was found")
                }
            }
        }
    }
    
    return nil
}

DDEV Core Implementation

The key changes to DDEV core would be in pkg/ddevapp/addons.go:

// Enhanced InstallDesc struct
type InstallDesc struct {
    Name                  string            `yaml:"name"`
    ProjectFiles          []string          `yaml:"project_files"`
    GlobalFiles           []string          `yaml:"global_files,omitempty"`
    DdevVersionConstraint string            `yaml:"ddev_version_constraint,omitempty"`
    Dependencies          []string          `yaml:"dependencies,omitempty"`
    PreInstallActions     []string          `yaml:"pre_install_actions,omitempty"`
    PostInstallActions    []string          `yaml:"post_install_actions,omitempty"`
    RemovalActions        []string          `yaml:"removal_actions,omitempty"`
    YamlReadFiles         map[string]string `yaml:"yaml_read_files"`
    
    // New Go actions support
    GoActions             *GoActionsConfig  `yaml:"go_actions,omitempty"`
}

type GoActionsConfig struct {
    PreInstall  []GoAction `yaml:"pre_install,omitempty"`
    PostInstall []GoAction `yaml:"post_install,omitempty"`
}

type GoAction struct {
    Name     string   `yaml:"action_name"`
    GoSource string   `yaml:"go_source"`
    Args     []string `yaml:"args,omitempty"`
}

// Enhanced ProcessAddonAction function
func ProcessAddonAction(action interface{}, dict map[string]interface{}, bashPath string, verbose bool) error {
    switch a := action.(type) {
    case string:
        // Legacy bash action (existing code)
        return processBashAction(a, dict, bashPath, verbose)
    case GoAction:
        // New Go action
        return processGoAction(a, dict, verbose)
    default:
        return fmt.Errorf("unsupported action type: %T", action)
    }
}

func processGoAction(action GoAction, dict map[string]interface{}, verbose bool) error {
    // Create temporary context file
    contextFile := filepath.Join(os.TempDir(), "ddev-addon-context.json")
    contextData, _ := json.Marshal(dict)
    os.WriteFile(contextFile, contextData, 0644)
    defer os.Remove(contextFile)
    
    // Get current directories
    addonDir := getCurrentAddonDir()
    projectDir := getCurrentProjectRoot()
    
    dockerCmd := []string{
        "docker", "run", "--rm",
        "-v", fmt.Sprintf("%s:/addon:ro", addonDir),
        "-v", fmt.Sprintf("%s:/project", projectDir),
        "-v", fmt.Sprintf("%s:/context.json:ro", contextFile),
        "-e", "DDEV_ADDON_CONTEXT=/context.json",
        "-w", "/addon",
        "ddev/go-addon-runner:latest",
        "go", "run", action.GoSource,
    }
    
    dockerCmd = append(dockerCmd, action.Args...)
    
    if verbose {
        fmt.Printf("Running Go action: %s\n", action.GoSource)
    }
    
    return exec.RunCommand("docker", dockerCmd...)
}

Conclusion

This approach provides an alternative technique for DDEV add-on development, giving developers the choice between bash and Go while preserving the user-friendly, dependency-minimal nature of DDEV. Both bash and Go actions are equally supported and can coexist within the same add-on.

The key innovation is using Docker containers with Go runtime to build and execute Go source files on-demand, requiring only Docker (which DDEV already requires) on user machines. This enables more powerful, maintainable add-on logic for developers who prefer Go, while keeping bash as a fully supported option for those who prefer it.

Key Principles:

• Choice, not replacement: Go actions are an alternative, not a replacement for bash
• Equal support: Both techniques receive the same level of support and documentation
• Backward compatibility: All existing bash-based add-ons continue to work unchanged
• Flexibility: Add-ons can use bash, Go, or both techniques together