Skip to content

Instantly share code, notes, and snippets.

@Darkflib
Last active October 23, 2024 15:10
Show Gist options
  • Save Darkflib/6711ccbc7645d52f5a61f02cc0898646 to your computer and use it in GitHub Desktop.
Save Darkflib/6711ccbc7645d52f5a61f02cc0898646 to your computer and use it in GitHub Desktop.

Can you explain what the following script does?

#!/usr/bin/env python3

import os
import signal
import subprocess
import sys
import logging
import time

# Set up logging
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger(__name__)

# Global variable to store the daemon process
daemon_process = None

def create_config():
    """Process environment variables and create configuration."""
    try:
        config_vars = {
            'FOO': os.getenv('FOO', '/config'),
            'BAR': os.getenv('BAR', '/app')
        }
        
        # Validate required environment variables
        for var, value in config_vars.items():
            if not value:
                raise ValueError(f"Required environment variable {var} is not set")
            
            # Check if directories exist
            if not os.path.exists(value):
                logger.warning(f"Directory {value} for {var} does not exist. Creating it...")
                os.makedirs(value, exist_ok=True)
        
        logger.info("Configuration processed successfully")
        return config_vars
        
    except Exception as e:
        logger.error(f"Error processing configuration: {str(e)}")
        sys.exit(1)

def start_daemon():
    """Start the daemon process."""
    global daemon_process
    
    try:
        # Start the daemon in the foreground
        daemon_process = subprocess.Popen(
            ['flibd'],
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE,
            universal_newlines=True
        )
        
        logger.info(f"Daemon started with PID {daemon_process.pid}")
        
        # Start background thread to log daemon output
        def log_output():
            for line in daemon_process.stdout:
                logger.info(f"Daemon output: {line.strip()}")
            for line in daemon_process.stderr:
                logger.error(f"Daemon error: {line.strip()}")
                
        from threading import Thread
        Thread(target=log_output, daemon=True).start()
        
    except Exception as e:
        logger.error(f"Failed to start daemon: {str(e)}")
        sys.exit(1)

def stop_daemon():
    """Stop the daemon process."""
    global daemon_process
    
    if daemon_process:
        try:
            daemon_process.terminate()
            daemon_process.wait(timeout=10)  # Wait up to 10 seconds for graceful shutdown
            logger.info("Daemon stopped successfully")
        except subprocess.TimeoutExpired:
            logger.warning("Daemon did not stop gracefully, forcing shutdown")
            daemon_process.kill()
        except Exception as e:
            logger.error(f"Error stopping daemon: {str(e)}")

def handle_sighup(signum, frame):
    """Handle SIGHUP signal by restarting the daemon."""
    logger.info("Received SIGHUP signal")
    try:
        stop_daemon()
        start_daemon()
        logger.info("Daemon restarted successfully")
    except Exception as e:
        logger.error(f"Failed to restart daemon: {str(e)}")
        sys.exit(1)

def main():
    """Main function to run the wrapper."""
    try:
        # Process configuration
        config = create_config()
        
        # Set up signal handler
        signal.signal(signal.SIGHUP, handle_sighup)
        
        # Start daemon
        start_daemon()
        
        # Keep the script running and monitor the daemon
        while True:
            if daemon_process.poll() is not None:
                exit_code = daemon_process.returncode
                logger.error(f"Daemon process exited unexpectedly with code {exit_code}")
                sys.exit(exit_code)
            time.sleep(1)
            
    except KeyboardInterrupt:
        logger.info("Received shutdown signal")

Yes, the script does pass through the output from the supervised process (flibd), but the way it's currently implemented has a potential issue that might cause lost or buffered output. Let me explain:

The current implementation captures output using:

daemon_process = subprocess.Popen(
    ['flibd'],
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE,
    universal_newlines=True
)

This captures the output but sends it to pipes that are then read in a separate thread:

def log_output():
    for line in daemon_process.stdout:
        logger.info(f"Daemon output: {line.strip()}")
    for line in daemon_process.stderr:
        logger.error(f"Daemon error: {line.strip()}")

There are two potential problems with this approach:

  1. The output is buffered by Python's pipe implementation, which means there might be delays before output appears
  2. The sequential processing of stdout and stderr (first all of stdout, then all of stderr) isn't ideal as it doesn't preserve the actual ordering of output

Here's how you could modify it to handle output pass-through more effectively:

def start_daemon():
    """Start the daemon process."""
    global daemon_process
    
    try:
        # Start the daemon without buffering
        daemon_process = subprocess.Popen(
            ['flibd'],
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE,
            universal_newlines=True,
            bufsize=1  # Line buffering
        )
        
        logger.info(f"Daemon started with PID {daemon_process.pid}")
        
        # Improved output handling
        import select
        
        def log_output():
            # Get file descriptors for stdout and stderr
            fd_stdout = daemon_process.stdout.fileno()
            fd_stderr = daemon_process.stderr.fileno()
            
            # Use select to handle both streams simultaneously
            while True:
                reads, _, _ = select.select([fd_stdout, fd_stderr], [], [])
                
                for fd in reads:
                    if fd == fd_stdout:
                        line = daemon_process.stdout.readline()
                        if line:
                            # Pass through to stdout and log
                            sys.stdout.write(line)
                            sys.stdout.flush()
                            logger.info(f"Daemon output: {line.strip()}")
                    elif fd == fd_stderr:
                        line = daemon_process.stderr.readline()
                        if line:
                            # Pass through to stderr and log
                            sys.stderr.write(line)
                            sys.stderr.flush()
                            logger.error(f"Daemon error: {line.strip()}")
                
                # Check if process has exited
                if daemon_process.poll() is not None:
                    break
                
        from threading import Thread
        Thread(target=log_output, daemon=True).start()
        
    except Exception as e:
        logger.error(f"Failed to start daemon: {str(e)}")
        sys.exit(1)

This improved version:

  1. Uses select to handle both stdout and stderr simultaneously
  2. Preserves the original ordering of output
  3. Actually passes the output through to the parent process's stdout/stderr
  4. Uses line buffering to reduce output delays
  5. Properly terminates the output handling when the process exits

This makes the script much more suitable for scenarios where you need to see the supervised process's output in real-time, such as in container logs or interactive debugging sessions.

This script appears to be a daemon process wrapper or supervisor written in Python. Let me break down its main components and functionality:

  1. Setup & Configuration:
  • Sets up logging with timestamps and log levels
  • Processes environment variables FOO and BAR with default values /config and /app respectively
  • Creates these directories if they don't exist
  • Validates the configuration values
  1. Daemon Management:
  • Starts a daemon process called flibd using subprocess.Popen
  • Captures and logs both stdout and stderr from the daemon
  • Creates a background thread to continuously monitor and log the daemon's output
  • Provides functionality to gracefully stop the daemon with a 10-second timeout before forcing termination
  1. Signal Handling:
  • Implements SIGHUP signal handling to support daemon restart
  • When SIGHUP is received, it stops the current daemon and starts a new instance
  1. Main Loop:
  • Monitors the daemon process continuously
  • If the daemon exits unexpectedly, logs the exit code and terminates
  • Uses a sleep interval of 1 second to prevent excessive CPU usage
  • Handles keyboard interrupts (Ctrl+C) gracefully

This is a common pattern for process supervision, similar to what you might see in container environments or service managers. A few key features make this particularly robust:

  • Proper logging of all events and errors
  • Graceful shutdown handling
  • Directory validation and creation
  • Continuous monitoring of the child process
  • Signal handling for restart capability
  • Error handling throughout the codebase

A practical use case might be running this as a container entrypoint script where it manages a service that needs supervision and logging capabilities.

Would you like me to elaborate on any particular aspect of the script?

Does this pass-through the output from the supervised process?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment