Skip to content

Instantly share code, notes, and snippets.

@mierzejk
Last active March 14, 2023 14:16
Show Gist options
  • Save mierzejk/749011cfc3dbc637840e80c578d69c0d to your computer and use it in GitHub Desktop.
Save mierzejk/749011cfc3dbc637840e80c578d69c0d to your computer and use it in GitHub Desktop.
Python script to capture process real output (stdout, stderr).
import os
from contextlib import contextmanager
from typing import IO
class Captured:
__slots__ = r'_data',
def __init__(self, value=None):
self._data = value
@property
def data(self) -> str | None:
return self._data
@data.setter
def data(self, value: str) -> None:
self._data = value
@contextmanager
def capture_stream(stream: IO, *, encoding: str = r'utf-8', buffer_length: int = 1024):
"""All credit to https://devpress.csdn.net/python/63045471c67703293080b3f2.html"""
file_no = stream.fileno()
stream_saved = os.dup(file_no)
desc_r, desc_w = os.pipe()
os.dup2(desc_w, file_no)
os.close(desc_w)
captured = Captured()
try:
yield captured
finally:
os.close(file_no)
captured_stream = b''
while True:
if not (data_read := os.read(desc_r, buffer_length)):
break
captured_stream += data_read
os.close(desc_r)
os.dup2(stream_saved, file_no)
os.close(stream_saved)
captured.data = captured_stream.decode(encoding=encoding)

In Python, capturing process output with contextlib.redirect_stdout / contextlib.redirect_stderr, or manual redirection of sys.stdout and sys.stderr works only when sys file objects are actually being used by the producer to write to standard output or error. If data streams are referred directly, for instance by an underlying C/C++ library invoked by the interpreter, hence eliding sys file objects, these methods are inadequate.
Fortunately, the os module lays on a number of file descriptor operations that can be availed to capture the actual standard streams. The capture_stream.py gist defines a context manager that is capable of capturing the given IO stream (e.g. sys.stdout or sys.stderr) and populates the returned Captured's data property on exit.

Following snippet exemplifies the usage of capture_stream context manager to capture OpenCV imread errors, not exposed by the Python API:

import cv2
import sys

from pathlib import Path

root = Path(r'/home/user/Pictures/')

for img in root.rglob(r'*.jpg'):
    with capture_stream(sys.stderr) as error:
        fatal = cv2.imread(img.as_posix()) is None

    if error.data:
        print(f'{img}: {error.data}')

The context manager implementation follows the idea published in "In python, how to capture the stdout from a c++ shared library to a variable" and simply wraps the code available therein, so all credit goes to the original publisher!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment