mattip

Exception Table Migration Plan (April 13 commit b350a6b)

Architecture

Current PyPy: runtime linked list of FrameBlock objects (ExceptBlock, FinallyBlock, SysExcInfoRestorer) anchored at frame.lastblock. The compiler emits SETUP_FINALLY/SETUP_EXCEPT/POP_BLOCK to manage this list. On exception, unrollstack() walks the list to find the handler.

CPython 3.11: no block stack at all. The exception table co_exceptiontable maps bytecode offset ranges to (handler_offset, stack_depth, lasti_flag). Only consulted when an exception actually fires — zero overhead on the happy path.

---

mattip

Migration Plan: Buildbot → GitHub Actions for PyPy

Platform matrix

Platform	GitHub runner	Container image (source)
linux-x86-64	ubuntu-latest (hosted)	ghcr.io/pypy/pypy/buildslave_x86_64 (docker/Dockerfile)
linux-x86-32	ubuntu-latest (hosted)	ghcr.io/pypy/pypy/buildslave_i686 (docker/Dockerfile32)
linux-aarch64	ubuntu-24.04-arm (hosted)	ghcr.io/pypy/pypy/buildslave_aarch64 (docker/DockerfileArm64)
macOS x86-64	macos-13 (hosted, Intel)	none

mattip

Exception Table Migration Plan

Architecture

Current PyPy: runtime linked list of FrameBlock objects (ExceptBlock, FinallyBlock, SysExcInfoRestorer) anchored at frame.lastblock. The compiler emits SETUP_FINALLY/SETUP_EXCEPT/POP_BLOCK to manage this list. On exception, unrollstack() walks the list to find the handler.

CPython 3.11: no block stack at all. The exception table co_exceptiontable maps bytecode offset ranges to (handler_offset, stack_depth, lasti_flag). Only consulted when an exception actually fires — zero overhead on the happy path.

---

mattip

Summarize the differences between pypy's benchmarks and CPython's:

  PyPy benchmark suite vs pyperformance

  Framework

  All pyperformance benchmarks use pyperf.Runner which handles warmup automatically (discards early iterations) and runs multiple processes. The PyPy suite uses the old unladen_swallow util.run_benchmark
  with no warmup support.

  Methodology fixes in pyperformance

mattip

#!/usr/bin/env python3
import re
import sys
import urllib.request

BUILD_NUMBER = 11645

BASE_URL = (
    "https://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64"
    "/builds/{build}/steps/shell_6/logs/stdio/text"

mattip

# commit 6ccbd45e79cb 
# running runner.py --baseline ../pypy3.11-v7.3.19-linux64/bin/pypy --changed /tmp/venv311pypy/bin/pypy -b \
#    pickle,pickle_dict,pickle_list,cpython_doc,unpickle,unpickle_list
{
  "revision": null, 
  "results": [
    [
      "sphinx", 
      "RawResult", 
      {

mattip

From numpy/numpy#22956

Here is some code implementing a kahan summation. It is not fast

import cffi
import numpy as np
import time

ffi = cffi.FFI()
ffi.cdef("""

mattip

The problematic place in the code is in the compression, when the test does compression in many threads:
```
    def testThreading(self):
        # Issue #7205: Using a BZ2File from several threads shouldn't deadlock.
        data = b"1" * 2**20
        nthreads = 2
        with BZ2File(self.filename, 'wb') as f:
            def comp():
                for i in range(5):
                     # f.write(data)   # original test does this, change for debugging

mattip

Running `python -c "import numpy", I get a recursion error. Running a lldebug0 build under gdb, here is the backtrace.

Searching for cpyext, I see there is a chain starting around frame 75 that maybe I can do something about. 
But why are there 40 recursive calls to `pypy_g_dispatch_bytecode__AccessDirect_None` ?

#0  __pyx_pf_5numpy_6random_13bit_generator_12SeedSequence_4generate_state (__pyx_v_self=0xcfb1c0, __pyx_v_n_words=0xcfd7b0, __pyx_v_dtype=0x95de18)
    at numpy/random/bit_generator.c:6289
#1  0x00007fffec5c0cc5 in __pyx_pw_5numpy_6random_13bit_generator_12SeedSequence_5generate_state (__pyx_v_self=0xcfb1c0, __pyx_args=0xcfd960, 
    __pyx_kwds=0xcfd9b0) at numpy/random/bit_generator.c:6114
#2  0x00007ffff2b4446f in pypy_g_cpy_call_external__funcPtr__objectPtr__objectPtr_1 (

mattip

$ grep '^[A-Za-z]*Error' /tmp/stdout.txt |sort |uniq -c |sort -n -r
     65 AssertionError: DeprecationWarning not triggered
     48 AssertionError: TypeError not raised by dumps
     42 AssertionError: events did not match expectation:
     16 AssertionError: Lists differ: [1, 1] != [2, 2]
     13 ValueError: array.index(x): x not in list
     13 AssertionError: UnicodeEncodeError not raised
      8 AssertionError: TypeError not raised
      7 SyntaxError: invalid syntax
      6 AssertionError: 4294967298 != 0