selemium-test-failure.md

Thank you for the additional details. With Selenium 4.18.1 and Chrome headless mode, especially when dealing with many elements, there are a few specific approaches we can take:

1. Use the new Selenium 4 Headless Mode syntax (it's more stable):

from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options

options = Options()
options.add_argument("--headless=new")  # New headless mode for Chrome

2. For handling multiple elements, implement a custom wait function:

def wait_for_all_elements(driver, locator, timeout=10):
    """Wait for all elements to be present and visible"""
    try:
        wait = WebDriverWait(driver, timeout)
        elements = wait.until(
            lambda d: all(
                element.is_displayed() and element.is_enabled()
                for element in d.find_elements(*locator)
            )
        )
        return elements
    except TimeoutException:
        print(f"Timeout waiting for elements: {locator}")
        raise

3. Add performance-oriented Chrome options:

options.add_argument('--disable-gpu')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
options.add_argument('--disable-infobars')
options.add_argument('--disable-extensions')
options.add_argument('--disable-browser-side-navigation')
options.add_argument('--enable-automation')
options.add_argument('--dns-prefetch-disable')
options.page_load_strategy = 'normal'  # Try 'eager' if this doesn't work

4. For forms with many inputs, try this approach:

def fill_form_safely(driver, input_data):
    """
    Safely fill form with multiple inputs
    input_data should be a list of tuples: [(locator, value), ...]
    """
    wait = WebDriverWait(driver, 10)
    
    for locator, value in input_data:
        try:
            # Wait for element and scroll into view
            element = wait.until(EC.presence_of_element_located(locator))
            driver.execute_script("arguments[0].scrollIntoView(true);", element)
            
            # Add a small delay for stability
            time.sleep(0.1)  # Minimal delay
            
            # Clear and fill
            wait.until(EC.element_to_be_clickable(locator))
            element.clear()
            element.send_keys(value)
            
            # Verify value was set
            wait.until(lambda d: d.find_element(*locator).get_attribute('value') == value)
            
        except Exception as e:
            print(f"Error filling element {locator}: {str(e)}")
            raise

Could you share some specific test code that's failing? That would help me provide even more targeted solutions. Also:

1. Are you seeing any specific error patterns in the Chrome logs?
2. Have you tried monitoring memory usage during test execution?
3. Are the elements that fail to load typically in any specific part of the page (top, bottom, need scrolling)?

---

There are several ways to capture Chrome logs during Selenium test execution. Here are the main approaches:

1. Enable Chrome logging through options:

from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities

# Set up logging preferences
options = Options()
options.add_argument('--enable-logging')
options.add_argument('--v=1')  # Logging level

# Enable performance and browser logging
caps = DesiredCapabilities.CHROME.copy()
caps['goog:loggingPrefs'] = {
    'browser': 'ALL',
    'performance': 'ALL'
}

# Initialize driver with these capabilities
driver = webdriver.Chrome(options=options, desired_capabilities=caps)

2. Retrieve logs during/after test execution:

# Get browser logs
browser_logs = driver.get_log('browser')
for log in browser_logs:
    print(f"Browser Log: {log}")

# Get performance logs
performance_logs = driver.get_log('performance')
for log in performance_logs:
    print(f"Performance Log: {log}")

3. Create a wrapper to capture logs when tests fail:

import logging
import sys
from datetime import datetime

def setup_logging():
    timestamp = datetime.now().strftime('%Y%m%d_%H%M%S')
    logging.basicConfig(
        filename=f'selenium_test_{timestamp}.log',
        level=logging.DEBUG,
        format='%(asctime)s - %(levelname)s - %(message)s'
    )
    # Also print to console
    console_handler = logging.StreamHandler(sys.stdout)
    logging.getLogger().addHandler(console_handler)

def test_wrapper(func):
    def wrapper(*args, **kwargs):
        try:
            return func(*args, **kwargs)
        except Exception as e:
            driver = kwargs.get('driver') or args[0]  # Assuming driver is first arg or kwarg
            
            # Get browser logs
            logging.error("=== Browser Logs ===")
            for log in driver.get_log('browser'):
                logging.error(f"Browser Log: {log}")
            
            # Get performance logs
            logging.error("=== Performance Logs ===")
            for log in driver.get_log('performance'):
                logging.error(f"Performance Log: {log}")
            
            # Get page source
            logging.error("=== Page Source ===")
            logging.error(driver.page_source)
            
            # Re-raise the exception
            raise e
    return wrapper

# Usage example:
@test_wrapper
def your_test_function(driver):
    # Your test code here
    pass

4. To capture network-related issues, enable Chrome's network logging:

options = Options()
options.set_capability('goog:loggingPrefs', {'performance': 'ALL'})

def process_network_logs(driver):
    performance_logs = driver.get_log('performance')
    for log in performance_logs:
        message = log.get('message')
        if 'Network.responseReceived' in message or 'Network.requestWillBeSent' in message:
            print(f"Network Log: {message}")

5. Create a reusable logging context manager:

from contextlib import contextmanager

@contextmanager
def log_context(driver, test_name):
    timestamp = datetime.now().strftime('%Y%m%d_%H%M%S')
    logging.info(f"Starting test: {test_name} at {timestamp}")
    try:
        yield
    except Exception as e:
        logging.error(f"Test failed: {test_name}")
        logging.error(f"Exception: {str(e)}")
        
        # Capture logs
        for log in driver.get_log('browser'):
            logging.error(f"Browser Log: {log}")
        
        # Capture screenshot
        driver.save_screenshot(f"failure_{test_name}_{timestamp}.png")
        
        raise
    finally:
        logging.info(f"Completed test: {test_name}")

# Usage:
def test_something(driver):
    with log_context(driver, "test_something"):
        # Your test code here
        pass

These logs can help you identify:

• JavaScript errors
• Network requests that fail
• Timing issues
• Resource loading problems
• DOM mutations
• Console messages

When debugging headless mode issues, pay special attention to:

1. Network timeouts
2. JavaScript errors
3. Resource loading failures
4. DOM element state changes

Would you like me to explain how to analyze specific types of logs or show how to set up more detailed logging for particular scenarios?