mattip

For this benchmark pypy --jit off is 3x slower than cpython3.11

from docutils.core import publish_doctree                                                        
from docutils import nodes                                                                       
                                                                                                 
RST = """                                                                                        
Title One                                                                                        
=========                                                                                        
                                                                                                 
Paragraph with *emphasis* and **strong** text.                                                   

mattip

Migration Plan: Buildbot → GitHub Actions for PyPy

Platform matrix

Platform	GitHub runner	Container image (source)
linux-x86-64	ubuntu-latest (hosted)	ghcr.io/pypy/pypy/buildslave_x86_64 (docker/Dockerfile)
linux-x86-32	ubuntu-latest (hosted)	ghcr.io/pypy/pypy/buildslave_i686 (docker/Dockerfile32)
linux-aarch64	ubuntu-24.04-arm (hosted)	ghcr.io/pypy/pypy/buildslave_aarch64 (docker/DockerfileArm64)
macOS x86-64	macos-13 (hosted, Intel)	none

mattip

#!/usr/bin/env python3
import re
import sys
import urllib.request

BUILD_NUMBER = 11645

BASE_URL = (
    "https://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64"
    "/builds/{build}/steps/shell_6/logs/stdio/text"

mattip

From numpy/numpy#22956

Here is some code implementing a kahan summation. It is not fast

import cffi
import numpy as np
import time

ffi = cffi.FFI()
ffi.cdef("""

mattip

The problematic place in the code is in the compression, when the test does compression in many threads:
```
    def testThreading(self):
        # Issue #7205: Using a BZ2File from several threads shouldn't deadlock.
        data = b"1" * 2**20
        nthreads = 2
        with BZ2File(self.filename, 'wb') as f:
            def comp():
                for i in range(5):
                     # f.write(data)   # original test does this, change for debugging

mattip

Running `python -c "import numpy", I get a recursion error. Running a lldebug0 build under gdb, here is the backtrace.

Searching for cpyext, I see there is a chain starting around frame 75 that maybe I can do something about. 
But why are there 40 recursive calls to `pypy_g_dispatch_bytecode__AccessDirect_None` ?

#0  __pyx_pf_5numpy_6random_13bit_generator_12SeedSequence_4generate_state (__pyx_v_self=0xcfb1c0, __pyx_v_n_words=0xcfd7b0, __pyx_v_dtype=0x95de18)
    at numpy/random/bit_generator.c:6289
#1  0x00007fffec5c0cc5 in __pyx_pw_5numpy_6random_13bit_generator_12SeedSequence_5generate_state (__pyx_v_self=0xcfb1c0, __pyx_args=0xcfd960, 
    __pyx_kwds=0xcfd9b0) at numpy/random/bit_generator.c:6114
#2  0x00007ffff2b4446f in pypy_g_cpy_call_external__funcPtr__objectPtr__objectPtr_1 (

mattip

Build aarch64, ppc pypy feedstock

for each machine aarch64 (for instance gcc185.fsffrance.org), 140.211.169.173 (ppc64le)
    1a. Mambaforge is preferred.
    1b. screen
    2a. conda create -n build python=3.11
    2b. conda activate build
    2c. conda install -y git conda-build conda-smithy anaconda-client boa patch
    3a. git clone https://github.com/conda-forge/pypy3.6-feedstock.git
    3b. cd pypy3.6-feedstock

mattip

Cloning the hg repo as a git repo

the tools

Get this repo https://github.com/mnauw/git-remote-hg which is a fork of the original and supports modern git/hg. Copy git-hg-helper and git-remote-hg to ~/bin and make them executable with chmod +x ~/bin/git*

clone the repo as both hg and git

git clone hg::https://foss.heptapod.net/pypy/cffi cffi-git

mattip

This C function

typedef struct {
    PyObject *in;
    PyObject *out;
} ufunc_full_args;

static int
PyUFunc_GenericFunctionInternal(PyUFuncObject *ufunc, PyArrayObject **op,
        ufunc_full_args full_args, PyObject *type_tup, PyObject *extobj,

mattip

BEFORE                                                   AFTER
[4e2f663bd9fe8] {jit-summary                                 [4e2d5889878b8] {jit-summary
Tracing:      	86	0.100334                                Tracing:      	78	0.113238
Backend:      	74	0.031218                                Backend:      	66	0.032329
TOTAL:      		1.515222                                 TOTAL:      		1.509000
ops:             	124400                                  ops:             	123038
heapcached ops:  	78660                                   heapcached ops:  	79768
recorded ops:    	39196                                   recorded ops:    	40168
  calls:         	6160                                      calls:         	5844
guards:          	9656                                    guards:          	10059