TESTING-PROCESS.md

Form Analysis Agent Flow Documentation

This document outlines the workflow and testing process for the Form Analysis Agent that maps form structures by navigating through web forms.

Agent Architecture

flowchart TD
    A[Start Agent] --> B[Initialize Browser Controller]
    B --> C[Open URL]
    C --> D[Map Form Structure]
    D --> E{Form Navigation}
    E --> F[Document Fields]
    F --> G{More Fields?}
    G -->|Yes| E
    G -->|No| H{Next Step?}
    H -->|Yes| I[Navigate to Next Step]
    I --> D
    H -->|No| J[Generate Schema]
    J --> K[Save Schema]
    K --> L[Log Results]
    L --> M[End Process]
    
    subgraph Form Navigation
    E --> E1[Identify Field]
    E1 --> E2[Wait for Field Ready]
    E2 --> E3[Fill with Fake Data]
    E3 --> E4{Field Filled?}
    E4 -->|Yes| E5[Move to Next Field]
    E4 -->|No| E6[Try Alternative Method]
    E6 --> E2
    end
    
    subgraph Special Field Handling
    E3 --> S1{Special Field?}
    S1 -->|Address| S2[Enter Address Slowly]
    S2 --> S3[Wait for Suggestions]
    S3 --> S4[Select Suggestion]
    S1 -->|Date| S5[Try Different Formats]
    S5 --> S6[Use Date Picker if Available]
    end

Loading

Schema Generation Process

flowchart LR
    A[Form Analysis] --> B[Form Steps]
    B --> C[Field Information]
    C --> D[Navigation Elements]
    D --> E[Schema JSON Generation]
    E --> F[Schema Validation]
    F --> G[Save to File]
    
    subgraph Field Information
    C1[Field Type] 
    C2[Field Name/ID]
    C3[Field Label]
    C4[Required Status]
    C5[Validation Rules]
    C6[Value Used]
    C7[Suggestions Info]
    end
    
    subgraph Schema Output
    E1[Form ID]
    E2[Total Steps]
    E3[Step Details]
    E4[Stopping Point]
    E5[Errors]
    end

Loading

Iterative Testing Process

flowchart TD
    A[Start Testing Process] --> B[Load Provider URL]
    B --> C[Run Form Analysis Agent]
    C --> D[Record Execution Time]
    D --> E[Check for Errors]
    E --> F{Errors Found?}
    F -->|Yes| G[Record as Failed]
    F -->|No| H[Check Steps Count]
    H --> I{Steps > 0?}
    I -->|Yes| J[Record as Success]
    I -->|No| G
    J --> K[Wait 1 Minute]
    G --> K
    K --> L{100 Tests Complete?}
    L -->|No| B
    L -->|Yes| M[Generate Test Report]
    M --> N[End Testing Process]

Loading

Test Result Analysis

%%{init: {"themeVariables": {"pie1": "#00ff00", "pie2": "#ff0000", "pie3": "#fff00"}} }%%
pie
    title Test Results Distribution
    "Success": 65
    "Failed (Errors)": 15
    "Failed (No Steps)": 20

Loading

Failure Modes Analysis

flowchart TD
    A[Test Failures] --> B{Failure Type}
    B -->|Browser Errors| C[Connection Issues]
    B -->|Agent Errors| D[LLM Response Issues]
    B -->|Form Errors| E[Form Structure Change]
    
    C --> C1[Timeout]
    C --> C2[Network Error]
    
    D --> D1[Invalid JSON]
    D --> D2[Incomplete Response]
    
    E --> E1[Field Not Found]
    E --> E2[Navigation Changed]
    E --> E3[New Verification Step]

Loading

Success Rate Over Time

graph TD
    subgraph Timeline
    A[Day 1] --> B[Day 2] --> C[Day 3] --> D[Day 4] --> E[Day 5]
    end
    
    subgraph Success Rate
    A1[12%] --> B1[42%] --> C1[55%] --> D1[62%] --> E1[65%]
    end
    
    A --- A1
    B --- B1
    C --- C1
    D --- D1
    E --- E1

Loading

Testing Environment

The testing process is designed to run on a provider URL with a 1-minute delay between iterations to avoid overloading the server. Each test:

1. Initializes the agent with the provider URL
2. Attempts to map the form structure
3. Records success/failure based on errors and step count
4. Waits 1 minute before the next iteration
5. Compiles results after 100 iterations

Success Criteria

• Success: The agent successfully navigates at least one form step without errors
• Failure: The agent encounters errors or fails to navigate any steps

Optimization Considerations

Based on test results, the following optimizations may be considered:

1. Adjusting wait times for form field interactions
2. Enhancing error handling for specific field types
3. Improving address field handling logic
4. Updating fake data values to better match form requirements
5. Refining stopping conditions to capture more form steps

Implementation Notes

The testing script logs detailed information about each test run, including:

• Test number and timestamp
• Execution time
• Success/failure status
• Error details (if any)
• Number of form steps discovered
• Stopping point

This data can be used to identify patterns in failures and optimize the agent's performance over time.