Ahmad Qamar atqamar

Snowball train–inference mismatch: diagnostic from existing runs

Backing evidence pack: snowball-mismatch-diagnostics-20260916.md. Source: 32 Snowball runs — 4 sync + 28 async — each with ≥20 updates carrying the ratio panel, 1,818 such updates in all. They span several runtime commits, not ba5ba171 alone. No new GPU time.

Definition of terms

ρ (rho), the ratio, and δ (delta) = log ρ. Per loss-masked token, δ = log ρ = learner_logprob − rollout_logprob: the Megatron trainer's recomputed logprob minus the vLLM engine's logprob recorded when the token was sampled

title	Final report: async non-agentic RL on Snowball
created	2026-09-14
updated	2026-09-14T19:47Z
author	planner session
status	live (updated as follow-ups land)
audience	Ahmad

Final report: asynchronous non-agentic reinforcement learning on Snowball

	WITH
	transacting_clients AS
	(
	SELECT
	client_number,
	COUNT(*) AS transaction_count,
	AVG(hammer_usd) transaction_avg
	FROM
	`sothebys-data.published.sothebys_transactions`
	WHERE

	# here we list all relevant nums (label ids)
	nums = [
	1, 3, 4, 13, 19, 20, 26, 27, 34, 35, 52, 63, 72, 73, 74, 75, 77, 79, 80, 82,
	83, 85, 89, 91, 93, 104, 106, 107, 109, 167, 168, 172, 190, 195, 196, 198, 203,
	204, 205, 206, 208, 209, 215, 218, 219, 235, 246, 266, 271, 273, 288, 289, 290,
	292, 293, 295, 296, 297, 299, 300, 301, 302, 303, 304, 306, 307, 308, 309, 310,
	311, 312, 313, 319, 320, 326, 328, 329, 331, 332, 335, 336, 337, 338, 340, 341,
	343, 344, 347, 348, 349, 350, 351, 352, 354, 355, 357, 358, 360, 361, 362, 363,
	364, 366, 367, 369, 370, 371, 372, 373, 374, 375, 378, 379, 380, 382, 383, 385,
	386, 388, 391, 393, 397, 398, 399, 420, 422, 427, 429, 434, 443, 444, 524, 530,

	import tensorflow as tf
	slim = tf.contrib.slim

	VALID_PADDINGS = set(['SAME', 'VALID'])


	# NOTES: even though pad_* functions aren't decorated by add_arg_scope, the
	# convolutions within are within the arg_scope
	#
	# layer arguments are copied from:

	from __future__ import absolute_import
	from __future__ import division
	from __future__ import print_function

	import tensorflow as tf

	# from nets import inception_utils

	from slim.nets.tg_layers import pad_conv2d, pad_max_pool2d, pad_avg_pool2d

	COLOR
	brown leather
	color acid green
	color anthracite
	color azure
	color beige
	color black
	color black leather
	color blackgold
	color blackwhite

	backpack
	bag other
	bikini
	bra
	dress
	earring
	eyewear
	footwear regular
	footwear tall
	gloves

	<# of products> <Category leaf>
	25 Boutique~~Clothing
	37 Boutique~~Clothing Accessories
	33 Boutique~~Clothing Accessories~~Belt
	19 Boutique~~Clothing Accessories~~Eyewear
	3 Boutique~~Clothing Accessories~~Gloves
	53 Boutique~~Clothing Accessories~~Keychain
	5 Boutique~~Clothing Accessories~~Lanyard
	6 Boutique~~Clothing Accessories~~Leg Warmers
	18 Boutique~~Clothing Accessories~~Scarf