josherich

OP

A Galileo Moment for LLM Design

Introduction

This document discusses the significant advancements in Large Language Model (LLM) architecture design, drawing parallels to pivotal moments in the history of science, such as the Pisa Tower experiment that catalyzed modern physics. Our findings reveal the true limits of LLM architectures through a controlled synthetic pretraining environment, marking a potential turning point in LLM research that may delineate the field into “before” and “after.”

Read more about Architecture Design and the Magic of Canon Layers

---

josherich

OP

New Research Paper Announcement

Introduction

We are pleased to announce the release of our latest research paper, which explores the training of reasoning Large Language Models (LLMs) to effectively reason about their areas of uncertainty. This work is particularly relevant in high-stakes domains such as healthcare and law, where the reliability of LLMs is critical.

Key Findings

Reasoning Training and its Challenges