Quick Guide to Using `line_profiler` for Python Performance Analysis

line_profiler offers a detailed, line-by-line breakdown of execution time for specific functions in your Python code. Here's how to use it:

1. Installation

pip install line_profiler

2. Marking Functions for Profiling

Mark the functions you wish to profile with the @profile decorator. This acts as a placeholder for kernprof, the utility used to execute and generate profiling data.

# NOTE: The import is optional and only for clarity. The `@profile` 
#       decorator doesn't need an actual import when using `kernprof`.
# from line_profiler import profile

@profile
def my_function():
    ...

Note: You don't need to import anything for the @profile decorator. It's a special placeholder recognized by kernprof.

3. Profiling the Code

Run your script with kernprof:

kernprof -l my_script.py

This creates an output .lprof file.

4. Viewing the Profile

Analyze the generated .lprof file:

Standard Output:

python -m line_profiler my_script.py.lprof

IPython Magic Style:

python -m line_profiler -rmt my_script.py.lprof

Example Result:

Given the example code:

read_dpx.py

...

@profile
read_dpx_image_data(file: IO[bytes], meta: Dict[str, Any]) -> np.ndarray:
    if meta['depth'] != 10 or meta['packing'] != 1 or meta['encoding'] != 0 or meta['descriptor'] != 50:
        return None

    width = meta['width']
    height = meta['height']

    file.seek(meta['offset'])
    raw = np.fromfile(file, dtype=np.int32, count=width*height, sep="")
    raw = raw.reshape(height, width)

    if meta['endianness'] == 'be':
        raw.byteswap(True)

    image = np.array([raw >> 22, raw >> 12, raw >> 2], dtype=np.uint16)
    image &= 0x3FF
    image = image.astype(np.float32)
    image /= 0x3FF

    return image.transpose(1, 2, 0)

...

image_data = read_dpx_image_data(f, meta)

...

When profiled using line_profiler, you might see an output similar to:

Timer unit: 1e-06 s

Total time: 0.0328762 s
File: .\read_dpx.py
Function: read_dpx_image_data at line 156

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
   156                                           @profile
   157                                           def read_dpx_image_data(file: IO[bytes], meta: Dict[str, Any]) -> np.ndarray:
   158         1          1.8      1.8      0.0      if meta['depth'] != 10 or meta['packing'] != 1 or meta['encoding'] != 0 or meta['descriptor'] != 50:
   159                                                   return None
   160
   161         1          0.5      0.5      0.0      width = meta['width']
   162         1          0.5      0.5      0.0      height = meta['height']
   163
   164         1          3.5      3.5      0.0      file.seek(meta['offset'])
   165         1       2937.8   2937.8      8.9      raw = np.fromfile(file, dtype=np.int32, count=width*height, sep="")
   166         1          6.3      6.3      0.0      raw = raw.reshape(height, width)
   167
   168         1          1.7      1.7      0.0      if meta['endianness'] == 'be':
   169         1        955.4    955.4      2.9          raw.byteswap(True)
   170
   171         1      15080.1  15080.1     45.9      image = np.array([raw >> 22, raw >> 12, raw >> 2], dtype=np.uint16)
   172         1       2256.7   2256.7      6.9      image &= 0x3FF
   173         1       7629.7   7629.7     23.2      image = image.astype(np.float32)
   174         1       3989.0   3989.0     12.1      image /= 0x3FF
   175
   176         1         13.2     13.2      0.0      return image.transpose(1, 2, 0)


  0.03 seconds - .\read_dpx.py:156 - read_dpx_image_data

Results Interpretation

Line #: Line number.
Hits: Number of times the line executed.
Time (μs): Total time for the line in microseconds.
Per Hit (μs): Average time per execution.
% Time: Percentage of time relative to the function's total.

Cleanup

After profiling, remember to remove or comment out the @profile decorator, especially if deploying or sharing the code.

promto-c/line_profiler_guide.md