Runs of sympy's risch_integrate(f, x, algebraic=True) (the experimental
exp-log-tower representation of radicals, e.g. sqrt(x) as
exp(log(x)/2)) over test corpora. Companion to the "Experimental
track" section of the RISCH_PLAN.md gist.
The experiment: represent radicals as exp-log towers, run the
transcendental Risch machinery over them, degrade every nonelementary
conclusion to a plain Integral (the proofs assume transcendence), and
accept solved results only after a tower-level verification (formal
derivative identity + denominators nonzero under the tower's algebraic
relations). Test corpus: the Rubi test suite, ~16,800 attemptable
radical cases.
Headline numbers (details in the runs below; per-case tables in the
rubi-table-*.md files):
- 970 integrals solved that sympy's non-Risch
integrate()cannot do (SOLVED-NEW), plus 2,771 solved by both. Zero wrong answers and zero false nonelementary claims across the whole corpus. - Single-radical families solve at 40–50%; the classic collections (Timofeev, Hearn, Bronstein) contribute 9 concrete new solves.
- The transcendental A/B on the radical-free exp-log chapters (master
vs the
risch-rde-cancellationbranch): zero regressions, five improvements (four crashes become decided results, two of them newly solved).
Results by corpus part (uniform capture run, per-case data in the rubi-table files):
| corpus part | attempted | SOLVED-NEW | SOLVED-both | partial | timeout | NIE |
|---|---|---|---|---|---|---|
| 0. Independent suites | 632 | 9 (1%) | 110 (17%) | 345 (55%) | 3 (0%) | 165 (26%) |
| 1.1.1 Linear binomials | 3377 | 159 (5%) | 615 (18%) | 2235 (66%) | 353 (10%) | 15 (0%) |
| 1.1.2-4 Other binomials | 5144 | 143 (3%) | 1102 (21%) | 3735 (73%) | 141 (3%) | 23 (0%) |
| 1.2 Trinomials | 6148 | 617 (10%) | 874 (14%) | 4451 (72%) | 203 (3%) | 3 (0%) |
| 1.3 Miscellaneous | 726 | 36 (5%) | 49 (7%) | 566 (78%) | 11 (2%) | 64 (9%) |
| 2-3 Exp/log radicals | 131 | 6 (5%) | 21 (16%) | 82 (63%) | 1 (1%) | 21 (16%) |
| all | 16158 | 970 (6%) | 2771 (17%) | 11414 (71%) | 712 (4%) | 291 (2%) |
Key findings:
- Input representation (
1/sqrt(x)vssqrt(x)/xetc.) and tower order have negligible effect on solvability (1 flip in 250 unsolved samples) — no retry driver needed. - ~95% of the unsolved mass is the genuine algebraic-Risch frontier:
the per-power rischDE subproblems correctly find no solution of
transcendental shape, because the true antiderivatives need terms
mixing several radical generators. Finishing the exp-log gaps buys
only the residue-side slice (~4%), the
parametric_log_derivundecidables (47 cases), and part of the timeouts; the rest needs real algebraic machinery (Trager / Bronstein Vol. II). - The corpus flushed out three genuine sympy master bugs (an spde() infinite loop, malformed Polys in is_log_deriv_k_t_radical_in_field(), and the ratint_logpart() PolynomialError of issue #26502) plus two crashes in the experimental branch itself — all fixed except the octic tail of #26502.
- sympy branch:
risch-algebraicat56520fe191(stacked onrisch-rde-cancellationat496a777876; forkasmeurer/sympy) - corpus: Upabjojr/rubi-integration-test-suite
at
0643a6b— the Rubi MathematicaSyntaxTestSuite translated to SymPy, 66k+ cases - runner:
risch_test_suite_runner.py(local, in the corpus clone; mirrors the upstream runner's role) - per-case timeout: 5 s (SIGALRM); first 120-case sample used 15 s
- case filter ("attemptable"): every exponent a concrete Rational, at
least one non-integer (an actual radical), no nan/oo; cases split
into concrete (free symbols == {x}) and parametric (symbolic
constants, e.g.
(a + b*x)**(3/2))
Classification: SOLVED = no unevaluated Integral in the result (for
algebraic towers every solved result was certified internally by the
tower-level acceptance filter: formal-derivative identity + kernel-free
denominators); partial = plain unevaluated Integral returned
(honest); CLAIMS-NE = NonElementaryIntegral present (would be a
bug: nonelementary proofs are invalid over algebraic towers);
NIE = NotImplementedError; timeout; error:*.
| chapter | total | radical-concrete | radical-parametric | symbolic-exponent | no-radical |
|---|---|---|---|---|---|
| 1.1 binomial products | 13,777 | 2,117 | 6,436 | 1,507 | 3,717 |
| 1.2 trinomial products | 10,200 | 1,163 | 5,009 | 1,003 | 3,025 |
| 1.3 miscellaneous | 1,350 | 373 | 358 | 159 | 460 |
(symbolic-exponent cases — (a + b*x)**m with symbolic m — are
unattemptable by any Risch-style method and are excluded from runs.)
Totals: 3,385 attempted, 782 SOLVED (23%), 0 CLAIMS-NE, 0 errors.
| shard (source family) | tried | concrete | parametric | notes |
|---|---|---|---|---|
1.1.1.2 (a+b x)^m (c+d x)^n (single-radical slice) |
1,247 | 126: 64 S, 48 p, 14 t | 1,121: 415 S, 626 p, 80 t | ~40–50% solve on single-radical forms |
| 1.1.1.3 (two-radical products) | 1,903 | 1,185: 131 S, 935 p, 119 t | 718: 156 S, 366 p, 196 t | new-extension frontier |
| 1.1.1.4 (improper/general) | 105 | 56: 0 S, 31 p, 13 NIE, 12 t | 49: 4 S, 26 p, 2 NIE, 17 t | |
| 1.1.1.5 (cubic numerators) | 24 | — | 24: 12 S, 12 p | |
| 1.1.1.6 (three linear factors) | 71 | — | 71: 0 S, 38 p, 33 t | profiling target |
1.1.1.7 P(x) (a+b x)^m ... |
35 | — | 35: 0 S, 35 t | profiling target |
(S = SOLVED, p = partial, t = timeout. Full raw log, including
per-shard SOLVED listings and timing progress lines, in
rubi-linear-chapter.log below.)
Reading of the results:
- The zero CLAIMS-NE / zero errors across 3,385 cases is the headline: the degrade-to-Integral policy plus the acceptance filter hold at scale.
- Symbolic constants flow through the towers fine — parametric cases solve at comparable-or-better rates than concrete ones.
- Failures partition into the two known work streams: honest partials cluster on multi-radical integrands (new-extension discovery / structure theorems), and the all-timeout families point at tower construction or machinery cost (profile one representative case).
16/30 solved, 0 wrong answers (every solved case verified by
differentiation in the harness), 0 false nonelementary claims; the
acceptance filter caused no false rejections. Newly solving vs the
original PoC branch: sqrt(x), x**(3/2), x**(5/2), x**(4/3),
sqrt(x)*log(x), d/dx[sqrt(x)*sqrt(x+1)], sqrt(1-x) + sqrt(1+x).
From test_failing_integrals.py territory: x/sqrt(a - x),
x*sqrt(x**2 + 1), x**(3/2)*log(x) solve; sqrt(1 - x**2),
sqrt(x)/(1 + sqrt(x)), sqrt(1 + sqrt(x)) honest-partial;
1/sqrt(a*x**2 + b*x + c) times out.
Runner gained integrate(f, x, risch=False) cross-checks: SOLVED-NEW
means only the algebraic towers solve it. Chapters 0 + 1.1 ran on the
tree at the spde-loop fix; 1.2/1.3/2/3 after it. Totals across the
~16,800 attempted radical cases: ~980 SOLVED-NEW, ~1,900 SOLVED-both,
0 wrong answers, 0 false nonelementary claims (the 10 apparent
CLAIMS-NE were classifier false alarms: radicals of exponentials and of
constants leave the tower transcendental, so those NonElementary
results are sound proofs; classifier fixed). Highlights: trinomial
products 617 SOLVED-NEW; linear binomials 159; chapter 0 classics 9
(Timofeev, Hearn, Bronstein). Crashes found and fixed along the way:
the spde() SymPy-Integer infinite loop (master bug), the exps-worklist
AttributeError (nested rational powers), and the ratint_logpart()
PolynomialError for radical coefficients (master bug, issue #26502,
partially fixed: quartic algebraic and symbolic-radical cases work,
octic cases still crash at later constructions).
The radical-free exp-log cases of chapters 2-3 (856 cases) through stock risch_integrate on both trees. Identical outcomes except five master->branch improvements, zero regressions: four master error:TypeError crashes in 3.2.3 (the bound_degree Poly-vs-int family fixed in Phase 0) become two sound nonelementary decisions and two SOLVED-NEW (e.g. integrands built over log((a+bx)/(c+dx)) towers that only risch solves), and one master timeout becomes a decided nonelementary result. In this mode a NonElementaryIntegral result is a legitimate proof (the towers are genuinely transcendental).
Every attempted algebraic case with SymPy expression, rendered math, and status, split by corpus part: see the rubi-table-.md files in this gist (SOLVED-NEW bolded). Raw per-case data: rubi-alg-results.jsonl / rubi-trans-.jsonl.
250 sampled unsolved (partial/NIE) cases retried under together/ expand/powsimp(force)/cancel rewritings and handle_first='exp' tower order: 1 flip (a perfect square, (a2 + 2ab*x + b2*x2)(3/2), which expand/cancel let the builder reduce), and tower order flipped nothing. Representation sensitivity exists in principle but is empirically negligible on this corpus: a retry-driver over rewritings is not worth building for the current failure mass.
200 sampled partial cases rerun with stage instrumentation:
| stage that gave up | count |
|---|---|
| integrate_hyperexponential_polynomial (in-field rischDE per t-power) | 191 |
| residue side (needs new log/atan extensions) | 8 |
| acceptance-filter reject | 1 |
So ~95% of the unsolved mass dies in the polynomial part: the per-power rischDE subproblems conclude "no solution", which over a non-transcendental tower means "no solution of the transcendental shape" -- the true antiderivatives (Rubi solves these) need terms mixing several radical generators, which the single-monomial machinery cannot produce. This is the genuine algebraic-Risch frontier (Trager/Bronstein Vol. II territory), NOT something that finishing the exp-log gaps will fix. The exp-log-completion payoff on this corpus is the smaller slice: the residue-side partials, the 47 parametric_log_deriv NIEs, and part of the timeouts.
Two follow-ups. (a) The acceptance-filter checks never fire on corpus inputs (400-case instrumented sample: 379 passes, 0 identity failures, 0 kernel failures, remainder not reaching the filter), so the kernel path is covered by a constructed regression test instead — which exposed and fixed a real hole: kernel contamination entering through the residue coefficients, invisible to both prior checks because the kernel factors cancel between a logarithm's coefficient and its argument's derivative (sympy commit 46dcb5c928).
(b) The representation question done at the representation level (Expr-level rewriting cannot express it -- sympy auto-canonicalizes same-base powers): multiply fa/fd by algebraic generators and reduce modulo the power relations, giving a different formal rational function for the same function. 2 flips in 150 unsolved cases, both by clearing the radical from the formal denominator (e.g. (x + (1 - 9x2)(3/2))/sqrt(1 - 9x**2)). Conclusion: the sensitivity is real and lives exactly where predicted, but is small; the actionable form is a canonical representative -- normalize fa/fd at tower-build time so algebraic generators do not divide the denominator -- rather than a retry driver.
Pairs of algebraically identical inputs that sympy keeps syntactically
distinct (x positive), both run through algebraic=True:
| pair | form A | form B |
|---|---|---|
sqrt(x*(x+1)) vs sqrt(x)*sqrt(x+1) |
partial | partial |
1/sqrt(x*(x+1)) vs 1/(sqrt(x)*sqrt(x+1)) |
partial | partial |
sqrt(x/(x+1)) vs sqrt(x)/sqrt(x+1) |
partial | partial |
1/(sqrt(x+1)+sqrt(x)) vs sqrt(x+1)-sqrt(x) |
partial | SOLVED |
sqrt((x+1)**2*(x+2)) vs (x+1)*sqrt(x+2) |
SOLVED | SOLVED |
sqrt(x**2+3*x+2) vs sqrt(x+1)*sqrt(x+2) |
partial | partial |
sqrt(x**2+2*x+1)/x vs (x+1)/x |
partial | SOLVED |
sqrt(exp(2*x)+2*exp(x)+1) vs exp(x)+1 |
SOLVED | SOLVED |
Discordance occurs exactly when a representation hides structure the tower builder does not normalize: rationalizable sum-of-radicals denominators, and perfect squares unexpanded inside a radicand. Structurally equivalent representations (split vs combined radicals) agree, matching the failure taxonomy. Actionable: two build-time canonical-form fixes (square-content extraction from radicands; clearing radicals from the formal denominator, cf. Run 6) would close every discordance found by any of the three methods.
The two candidate canonical-form fixes from Runs 6-7, measured over 1,500 sampled unsolved cases before deciding whether to implement:
| normalization | flips | corpus-wide estimate |
|---|---|---|
| square-content extraction (factor radicands) | 65 (4.3%) | ~507 cases |
| clearing radicals from the formal denominator | 6 (0.4%) | ~47 cases |
| either | 71 (4.7%) | ~554 cases (overall solve rate 23.2% -> ~26.6%) |
Decision (per Aaron's rule -- implement if significant, skip or
flag-gate if rare): square-content extraction implemented in
DifferentialExtension for the algebraic mode (sympy commit
409f898b96); denominator clearing skipped. With it,
sqrt(x**2 + 2*x + 1)/x integrates to
sqrt(x**2 + 2*x + 1) + log(x) - 1, and the Run 7 perfect-square
discordances close. Timing: the runner now records per-case seconds
in the JSONL for all future runs; the transcendental A/B already
showed master and the branch identical to within noise (783 s vs
785 s over the same 856 cases), so the new machinery has not slowed
the classic fast path.
Distributions (per-class study, serial, pinned worktree): solved cases are fast and failures are cheap — SOLVED-both median 0.09 s (p90 0.36 s), SOLVED-NEW median 0.26 s (p90 1.29 s), partial median 0.16 s.
Head-to-head vs integrate(risch=False) on 600 SOLVED-both:
median 0.09 s vs 0.21 s; risch >=5x faster on 116 cases (19%) vs
integrate >=5x faster on 33 (5.5%); extremes ~60x (e.g.
(1 - 2*x**2)**(-7/2) at 0.05 s vs 3.0 s). integrate() never failed
outright on the sample, so the advantage is pure speed.
Full timed rerun (parallel, 6 workers): validates the radicand
normalization at +387 SOLVED-NEW (970 -> 1,357; overall 23.2% ->
25.6%), all landing as SOLVED-NEW. Timeout counts under parallelism
are inflated by contention (953 vs 712 serial) — timing conclusions
here come from the serial runs. The rerun also flushed out a real
master bug in sympy/polys: PolyMatrix scalar multiplication with a
ground-domain fallback (EX) crashed with AttributeError, reached
through constant_system() for nested symbolic radicals like
sqrt(a + b*sqrt(d/x) + c/x) (fixed at the PolyMatrix level,
bd3dde37f8).
Deep probe of the former timeout class (120 sampled, 300 s cap, serial): 68 finish within 30 s — including 9 that fully solve (several instantly, their hidden perfect squares now collapsed by the normalization; others genuinely slow solves at 40-67 s) — 13 more by 120 s, 3 by 300 s, and 36 exceed 300 s. Stack sites at the cap: residue_reduce 17, gcdex_diophantine 8, ratint_logpart 8, spde 3 — all bounded-arithmetic shapes (subresultant PRS and extended-GCD coefficient blowup over parametric multi-radical towers), no loop-signature sites. Conclusion: the tail is slow, not hung. Roughly 2% of attempted corpus cases sit beyond 300 s; a production cap of ~30 s would recover ~60% of the former timeout class, including new solves.
The problem. The radicand factoring of Run 8 distributes fractional
powers over factors, an algebraic equivalence only where the factors
are positive: sqrt((x+1)**2*(x+2)) -> (x+1)*sqrt(x+2) has the wrong
sign for x < -1. Verified numerically: the answers for
sqrt(x**2+2*x+1)/x and Timofeev's (x**3-5*x**2+3*x+9)**(-2/3) are
wrong on whole real intervals (the latter is one of the 9 chapter-0
headline solves). The tower-level acceptance filter cannot see this:
it certifies against the tower image of the rewritten integrand.
The core machinery is NOT affected -- exp(log(u)/2) is identically
the principal sqrt(u), and core-path answers spot-checked at
negative and complex points are all faithful.
The literature answer (Jeffrey 1993, ISSAC 93; Jeffrey, Labahn,
von Mohrenschildt & Rich, "Integration of the signum, piecewise and
related functions" -- both in ~/Dropbox/papers/symbolic-computation):
carry the sign explicitly. sqrt(w**2*v) == |w|*sqrt(v) == s*w*sqrt(v) with s = sgn(w) treated as a symbolic constant
during integration; afterwards substitute s -> sgn(w) and add jump
corrections -J_k*sgn(x - x_k) at each breakpoint (root or pole of
w), where J_k = (G(x_k, s=+1) - G(x_k, s=-1))/2. That restores
continuity -- Jeffrey's "domain of maximum extent". Jeffrey 1993 §2
additionally gives the rule for combining logarithms with fractional
coefficients: a*ln f1 + b*ln f2 -> (m/n)*ln(f1**p * f2**q).
Prototype result (signum_proto2.py): the approach works end to end on our towers.
- Jeffrey's Example 1 reproduced exactly:
3*x**2*sqrt(1+1/x**2)->sgn(x)*((1+x**2)**(3/2) - 1), J = 1. sqrt((x+1)**2*(x+2)): J = -4/15; derivative correct at x = -1.9, -1.5, -0.5, 3 (the first two are where the current code is wrong), and the correction makes the answer continuous at x = -1 (jump 0.533 -> 0).sqrt(x**2+2*x+1)/x: correct at x = -3 and -1.5, the exact failing points. The signs pass through as ordinary symbolic constants, which the corpus runs already showed the towers handle well.
Consequence for the Run 8 decision: the choice is not "gate on
positivity (losing ~all corpus gains) vs keep a generically-valid
answer". The signum algorithm keeps the gains and returns answers
valid on the whole real line, at the cost of sgn factors in the
output and a jump-correction pass. Open work: complex-valued jumps
when a breakpoint sits at a singularity of the integrand (our
sqrt(x**2+2*x+1)/x case gives J = -1 + I*pi); cube roots and other
odd radicals need the sgn**(2/3)-style treatment of Jeffrey 1993 §5;
and breakpoint detection needs the roots of non-polynomial sign
arguments.