GLM Tries, Opus Triages: Behavioral Differences in Research-to-Code Agents

A controlled comparison across 19 paired runs spanning 19 repository forks — 38 individual workflow executions total — running an identical paper-implementation pipeline (remyxai/outrider — Claude Code under the hood, with glm-5.2 routed at z.ai's Coding Plan endpoint vs default Opus). The pipeline ran in two modes that probe different parts of the workflow:

Selection-pass mode (n=9): no pin; each provider freely selects its own paper from the candidate pool. Exercises the full pipeline including selection + verification gates.
Pin-method mode (n=10): same paper pinned on each fork, both providers run their full chain on identical input. Isolates implementation-side behavior on a forced pick.

The aggregate verdict comes from the n=19 union; the two mode-specific breakdowns below show where the difference comes from.

Reproducing this

smellslikeml / autoresearch-gist.md

Last active July 4, 2026 17:32

Findings from running remyxai-cli autoresearch across 5 production repos — per-repo inventory of architectural extension points missing to receive recent AI methods

Findings from running remyxai-cli explore across 6 production repos

Recent AI research lands in existing codebases through specific extension points — modules, callbacks, or data-structure fields where a new method can plug in. Which extension points a repo provides determines which methods can be tried against it without a rewrite. We ran an agentic method-search loop that dispatches recent arxiv papers as draft integrations against 6 production repos; the by-product across 36 cycles was a per-repo inventory of the specific extension points those repos are missing.

The dispatch mode

Packaged as a CLI subcommand in remyxai-cli #46:

remyxai outrider explore --repo owner/name \

smellslikeml / gist_landing_zone_standalone.md

Created July 5, 2026 03:40

Landing-zone-to-PR: shipping one focused PR per architectural cluster of gap-Issues, with cluster context preserved in the PR narrative

From surveying gaps to shipping the anchor

An earlier writeup (Findings from running remyxai-cli explore across 6 production repos) walked through what happens when you dispatch recent arxiv papers as draft integrations against a set of production repos: each paper hits the ranker, preflight identifies the extension point the paper needs, and Outrider opens an Issue naming what's missing. The by-product is a per-repo gap analysis — a catalog of the extension points those repos lack, and the papers currently blocked on them.

smellslikeml / README.md

Created August 5, 2026 16:01

DoRA factored kernel — dense vs. factored norm-computation composite (algorithmic analysis)

DoRA Factored Kernel — Algorithmic Analysis

Composite illustration for smellslikeml/peft#18 + remyxai/dora-factored-kernel.

Compares the dense DoRA norm-computation path used by HuggingFace PEFT + 5 other major frameworks (torchtune, Unsloth, SWIFT, LLaMA-Factory, Axolotl per arXiv:2603.22276 Appendix G Table 15) against the factored decomposition — 38× to 634× per-module memory reduction across d_in ∈ {1K…16K}, r=64, bf16.

smellslikeml / remyx-peft-contributions.md

Created August 6, 2026 18:30

Remyx contributions to huggingface/peft — Riemannian LoRA (merged), Super-Tuning (in review), Scaling DoRA (pending)

Remyx contributions to `huggingface/peft`

Three parameter-efficient fine-tuning methods surfaced via Outrider, drafted through the smellslikeml/peft fork, and shepherded into huggingface/peft — the fine-tuning ecosystem's hub. This gist tracks what's landed, what's in review, what's pending on external coordination, and how the shape of these contributions compares against the year's other merged tuner PRs.

Why `huggingface/peft` is where the leverage lives

Per Appendix G Table 15 of Scaling DoRA (arXiv:2603.22276), five of six DoRA-supporting fine-tuning frameworks route their implementation through PEFT:

smellslikeml smellslikeml