Does dimuon-lite beat AdamW for LoRA SFT on no_robots?
dimuon-lite is Spindle's rebuild of Tinker's DiMuon LoRA optimizer; we compare its held-out no_robots NLL against AdamW at equal steps, with IFEval as a secondary check.
Budget
2 GPU-h (default)
· each run plans 2 GPU-h, 150 min wall cap
· 6 runs
·
change
Reproducible on Spindle: yes
— The shared Spindle exposes dimuon-lite (spindle#85) through DimuonOptimizerConfig/DimuonParams. A short tinker SDK SFT script can run DiMuon and AdamW arms on the same data and step count.
Spec
dimuon-lite vs AdamW, LoRA SFT on Qwen3.5-2B, no_robots held-out NLL
Hypothesis. With per-optimizer learning rates picked on a validation split, LoRA SFT with dimuon-lite cuts held-out no_robots token_mean NLL by at least 0.02 nats versus AdamW. Both arms use the same 100 steps, batches and seeds.
Method. Train a rank-32 LoRA on Qwen/Qwen3.5-2B over HuggingFaceH4/no_robots, using one tinker SDK script for both arms so the only difference is the optimizer. The cookbook chat_sl recipe is AdamW-only, so running AdamW through the same script removes any recipe difference. DiMuon is selected at creation with optimizer=tinker.types.DimuonOptimizerConfig() and stepped with DimuonParams(learning_rate, beta1=0.9, grad_clip_norm=0.0).
Qwen/Qwen3.5-2B
SFT
100 steps
batch 32
rank 32
no_robots
eval no_robots test split (HuggingFaceH4/no_robots, split=test) · held-out token_mean NLL (nats per assistant loss token)
2 GPU-h
150 min cap
Success criterion. Validated if the simultaneous 95% paired_t interval, across 3 training seeds, for (AdamW NLL − DiMuon NLL) on no_robots test token_mean NLL at step 100 lies entirely above 0.02 nats. Failed if it lies entirely below 0.02. Otherwise inconclusive. IFEval strict accuracy is reported descriptively and does not decide.
full spec
{
"activation_gate": {
"applicable": true,
"checks": [
{
"measures": "model name passed to create_lora_training_client in every arm",
"metric": "executed_base_model",
"one_of": [
"Qwen/Qwen3.5-2B"
]
},
{
"measures": "type of params object passed to optim_step in each arm (logged client-side)",
"metric": "executed_optimizer_step_type",
"one_of": [
"DimuonParams",
"AdamParams"
]
},
{
"max": 1.05,
"measures": "||dB||_F^2 / (||dB||_2^2 * r) for the first DiMuon update of LoRA B, averaged over layers, from downloaded checkpoints at step 0 and 1",
"metric": "dimuon_B_update_normalized_stable_rank_step1",
"min": 0.6
},
{
"max": 1.0,
"measures": "fraction of the 100 training batches whose token-id hash is identical between paired arms within a seed",
"metric": "batch_hash_match_across_arms",
"min": 1.0
},
{
"measures": "train token_mean NLL at step 0 minus step 10 in the DiMuon pilot at the middle LR; must be finite and positive",
"metric": "pilot_train_nll_drop_dimuon_10steps",
"min": 0.0
}
],
"reason": "The paired comparison only tests DiMuon if the executed optimizer really orthogonalizes LoRA B updates and both arms ran the declared configuration. Polar Express makes the first-step B update near-orthogonal (flat singular spectrum); AdamW's is not. This is measured from saved weight diffs, not SDK introspection."
},
"arms": [
{
"lr": 5e-05,
"name": "lrsearch_adamw_5e-5_s0",
"optimizer": "adam",
"phase": "selection",
"seed": 0,
"steps": 40
},
{
"lr": 0.0001,
"name": "lrsearch_adamw_1e-4_s0",
"optimizer": "adam",
"phase": "selection",
"seed": 0,
"steps": 40
},
{
"lr": 0.0002,
"name": "lrsearch_adamw_2e-4_s0",
"optimizer": "adam",
"phase": "selection",
"seed": 0,
"steps": 40
},
{
"lr": 0.0003,
"name": "lrsearch_dimuon_3e-4_s0",
"optimizer": "dimuon-lite",
"phase": "selection",
"seed": 0,
"steps": 40
},
{
"lr": 0.001,
"name": "lrsearch_dimuon_1e-3_s0",
"optimizer": "dimuon-lite",
"phase": "selection",
"seed": 0,
"steps": 40
},
{
"lr": 0.003,
"name": "lrsearch_dimuon_3e-3_s0",
"optimizer": "dimuon-lite",
"phase": "selection",
"seed": 0,
"steps": 40
},
{
"lr": "frozen from selection",
"name": "baseline_adamw_s0",
"optimizer": "adam",
"phase": "confirmation",
"seed": 0,
"steps": 100
},
{
"lr": "frozen from selection",
"name": "baseline_adamw_s1",
"optimizer": "adam",
"phase": "confirmation",
"seed": 1,
"steps": 100
},
{
"lr": "frozen from selection",
"name": "baseline_adamw_s2",
"optimizer": "adam",
"phase": "confirmation",
"seed": 2,
"steps": 100
},
{
"lr": "frozen from selection",
"name": "treatment_dimuon_s0",
"optimizer": "dimuon-lite",
"phase": "confirmation",
"seed": 0,
"steps": 100
},
{
"lr": "frozen from selection",
"name": "treatment_dimuon_s1",
"optimizer": "dimuon-lite",
"phase": "confirmation",
"seed": 1,
"steps": 100
},
{
"lr": "frozen from selection",
"name": "treatment_dimuon_s2",
"optimizer": "dimuon-lite",
"phase": "confirmation",
"seed": 2,
"steps": 100
}
],
"base_model": "Qwen/Qwen3.5-2B",
"baseline": "AdamW (tinker.AdamParams, beta1 0.9, beta2 0.95, eps 1e-8) through the same SDK script, with the same model, LoRA rank, init seed, batches and order, 100 steps, and eval. Its LR is chosen by the same 3-point validation search.",
"batch_size": 32,
"budget": {
"gpu_hours": 2.0,
"max_wall_minutes": 150
},
"command": "# Custom part: one SDK SFT loop shared by both optimizers (cookbook chat_sl is AdamW-only).\nimport os, sys, tinker, datasets, random\nfrom tinker_cookbook import renderers, tokenizer_utils\nfrom tinker_cookbook.supervised.data import conversation_to_datum\nOPT, LR, SEED, STEPS = sys.argv[1], float(sys.argv[2]), int(sys.argv[3]), int(sys.argv[4])\nMODEL='Qwen/Qwen3.5-2B'; BS=32; MAXLEN=2048\ntok=tokenizer_utils.get_tokenizer(MODEL); rend=renderers.get_renderer(renderers.get_recommended_renderer_name(MODEL) if hasattr(renderers,'get_recommended_renderer_name') else 'qwen3', tok)\nds=datasets.load_dataset('HuggingFaceH4/no_robots'); tr=list(ds['train']); random.Random(1234).shuffle(tr)\nval, train = tr[:500], tr[500:]; random.Random(SEED).shuffle(train)\nmk=lambda r: conversation_to_datum(r['messages'], rend, MAXLEN)\nsc=tinker.ServiceClient(base_url=os.environ['TINKER_BASE_URL'])\nkw={'optimizer': tinker.types.DimuonOptimizerConfig()} if OPT=='dimuon' else {}\ntc=sc.create_lora_training_client(base_model=MODEL, rank=32, seed=SEED, **kw)\ntc.save_weights_for_sampler(name=f'init-{OPT}-s{SEED}').result() # hash for restoration preflight\nstep_params = (lambda: tinker.types.DimuonParams(learning_rate=LR, beta1=0.9, grad_clip_norm=0.0)) if OPT=='dimuon' else (lambda: tinker.AdamParams(learning_rate=LR, beta1=0.9, beta2=0.95, eps=1e-8))\nfor s in range(STEPS):\n batch=[mk(r) for r in train[s*BS:(s+1)*BS]]\n tc.forward_backward(batch, loss_fn='cross_entropy').result(); tc.optim_step(step_params()).result()\ndef nll(rows): # token_mean from per-example totals, never batch loss:mean\n tot=0.0; n=0.0\n for i in range(0,len(rows),32):\n d=[mk(r) for r in rows[i:i+32]]; out=tc.forward(d, loss_fn='cross_entropy').result()\n for di,o in zip(d,out.loss_fn_outputs):\n w=di.loss_fn_inputs['weights'].to_numpy(); lp=o['logprobs'].to_numpy(); tot+=-(lp*w).sum(); n+=w.sum()\n return tot/n\nsplit = val if STEPS==40 else list(ds['test'])\nprint({'opt':OPT,'lr':LR,'seed':SEED,'steps':STEPS,'token_mean_nll':nll(split)})\ntc.save_weights_for_sampler(name=f'final-{OPT}-lr{LR}-s{SEED}').result()\n# run: python sft_opt.py {adam|dimuon} <lr> <seed> {40|100}",
"correctness_preflight": {
"notes": "Tolerances are fixed for bf16 LoRA weights with fp32 optimizer math before measuring. reference_update rtol 1e-2 leaves headroom over the reported 0.09% error. Feed the same LoRA state and gradient to the Spindle DiMuon step and to the reference for one step. With grad_clip_norm=0 and beta1 momentum seeded from the first gradient, the update should scale linearly in LR across a 100x span.",
"paths": [
{
"baseline_revision": "UNRESOLVED: pin the modal-projects/spindle commit deployed at the shared Spindle URL (post-#85). AdamW updates on this deployment must match the pre-#85 AdamW update on identical grads/state.",
"id": "spindle_dimuon_lite_optimizer",
"kind": "optimizer",
"lr_scaling": {
"learning_rates": [
0.0001,
0.001,
0.01
],
"rtol": 0.01
},
"reference_revision": "UNRESOLVED: pin the commit of Aaron's DiMuon reconstruction (0.09% relative update error vs Tinker) and the modal-projects/miles commit of #17",
"tolerances": {
"baseline_update": {
"atol": 1e-06,
"min_cosine": 0.99999,
"rtol": 0.001
},
"reference_update": {
"atol": 1e-05,
"min_cosine": 0.999,
"rtol": 0.01
}
}
}
],
"required": true,
"status": "unresolved: spindle and miles commits for #85/#17 and the commit of Aaron's DiMuon reconstruction must be pinned before any full sweep"
},
"dataset": {
"hf_path": "HuggingFaceH4/no_robots",
"n_examples": 9500,
"name": "no_robots",
"split": "train (9000 for training, 500 held out for LR selection); test (500) for confirmation"
},
"decision_rule": {
"adaptive_selection": {
"confirmation_item_ids": "all 500 rows of no_robots split=test",
"kind": "hyperparameter",
"rationale": "Muon-family and AdamW LoRA optimizers have very different good LRs. Each gets a prespecified 3-point grid, scored only on validation items held out from train, and the choice is frozen before the single confirmation eval on the test split.",
"search_item_ids": "no_robots train rows held out by seeded shuffle (seed 1234), first 500 after shuffle; never used for training"
},
"alpha": 0.05,
"assumptions": {
"identically_distributed_seed_effects": true,
"independent_seed_effects": true,
"normal_seed_effects": true
},
"criteria": [
{
"id": "primary",
"margin": 0.02,
"metric": "no_robots_test_token_mean_nll_at_step_100",
"weights": {
"baseline": 1,
"treatment": -1
}
}
],
"estimand": "Reduction in held-out no_robots test token_mean NLL at step 100 from replacing AdamW with dimuon-lite (each at its validation-selected LR) for rank-32 LoRA SFT on Qwen3.5-2B; larger is better for dimuon-lite.",
"method": "paired_t",
"seeds": [
0,
1,
2
],
"unit": "training_seed",
"version": 1
},
"eval": {
"aggregation": "token_mean",
"benchmark": "no_robots test split (HuggingFaceH4/no_robots, split=test)",
"metric": "held-out token_mean NLL (nats per assistant loss token)",
"n_examples": 500
},
"experiment_features": {
"adaptive_selection": true,
"checkpoint_restore": false,
"custom_loss": false,
"custom_optimizer": true,
"generated_training_data": false,
"heldout_loss": true,
"learned_generator": false,
"paired_comparison": true,
"paired_training_arms": true,
"warmup_adaptation": false
},
"feasibility": {
"config": null,
"downsizing_order": "drop IFEval secondary, then shrink LR grid to 2 points per optimizer, then shorten selection runs to 25 steps; keep 3 seeds and the 100-step test NLL. If that still does not fit, report inconclusive.",
"effective_config": {
"arms": 12,
"base_model": "Qwen/Qwen3.5-2B",
"batch_size": 32,
"confirmation_steps": 100,
"max_seq_len": 2048,
"n_eval_test": 500,
"n_eval_validation": 500,
"n_ifeval_secondary": 541,
"planned_arm_steps": 840,
"selection_steps": 40
},
"generation": null,
"measurements": null,
"primary_endpoint": "no_robots_test_token_mean_nll_at_step_100",
"recovery": null,
"source": null
},
"feasibility_check": {
"generation_applicability": {
"check": "generation_audit",
"enforce": false,
"features": {
"generated_training_data": false
},
"reasons": [
"spec.generation_audit opts out and no feature it covers is true"
],
"status": "not_applicable"
},
"generation_counts": null,
"gpu_hours": null,
"known_gpu_hours": 0,
"phases": {},
"problems": [
"Effective configuration not recorded",
"Pilot evidence source not recorded",
"Representative training timing not recorded",
"Representative setup timing not recorded",
"Representative evaluation timing not recorded",
"Positive recovery reserve not recorded"
],
"status": "unknown",
"wall_seconds": null
},
"feasibility_required": true,
"generation_audit": {
"answer_comparison": null,
"max_eval_overlaps": 0,
"max_label_errors": 0,
"min_sample_size": 0,
"reason": "fixed public dataset; no generated training data",
"required": false
},
"hypothesis": "With per-optimizer learning rates picked on a validation split, LoRA SFT with dimuon-lite cuts held-out no_robots token_mean NLL by at least 0.02 nats versus AdamW. Both arms use the same 100 steps, batches and seeds.",
"lora_rank": 32,
"max_seq_len": 2048,
"method": "Train a rank-32 LoRA on Qwen/Qwen3.5-2B over HuggingFaceH4/no_robots, using one tinker SDK script for both arms so the only difference is the optimizer. The cookbook chat_sl recipe is AdamW-only, so running AdamW through the same script removes any recipe difference. DiMuon is selected at creation with optimizer=tinker.types.DimuonOptimizerConfig() and stepped with DimuonParams(learning_rate, beta1=0.9, grad_clip_norm=0.0). AdamW uses tinker.AdamParams(learning_rate, beta1=0.9, beta2=0.95, eps=1e-8). dimuon-lite is a recently landed reverse-engineered update rule: Polar Express orthogonalization with about 5 iterations, rank_lr_correction_exponent 0.175, Muon on LoRA B, and no beta2. Its update is checked against the trusted DiMuon reconstruction before the comparison runs. Phase 1, LR selection: each optimizer runs 40 steps at seed 0 on a 3-point LR grid fixed now (AdamW {5e-5, 1e-4, 2e-4}; DiMuon {3e-4, 1e-3, 3e-3}). Each run is scored by token_mean NLL on 500 validation items held out from the train split, and the best LR per optimizer is frozen. Phase 2, confirmation: each optimizer trains 100 steps at its frozen LR for seeds {0,1,2}. Batches are identical across arms within a seed: the same shuffled order of the remaining 9,000 train rows, batch_size 32, max_seq_len 2048. Data order and LoRA init seed are shared per seed. Primary eval is token_mean NLL on the full 500-example no_robots test split, computed from per-example summed logprobs times loss weights via forward(). Secondary, descriptive only: IFEval prompt-level strict accuracy (541 prompts, greedy, max_tokens 1024) on each final checkpoint. IFEval is dropped first if feasibility does not fit.",
"optimizer": {
"lr": 0.001,
"name": "dimuon-lite"
},
"primary_endpoint": "no_robots_test_token_mean_nll_at_step_100",
"restoration_preflight": {
"applicable": true,
"fixed_batch_required": true,
"groups": [
{
"arms": [
{
"name": "baseline_adamw_s0",
"optimizer": "adam",
"optimizer_config": {
"beta1": 0.9,
"beta2": 0.95,
"eps": 1e-08,
"lr": "frozen"
}
},
{
"name": "treatment_dimuon_s0",
"optimizer": "dimuon-lite",
"optimizer_config": {
"beta1": 0.9,
"grad_clip_norm": 0.0,
"lr": "frozen"
}
}
]
},
{
"arms": [
{
"name": "baseline_adamw_s1",
"optimizer": "adam",
"optimizer_config": {
"beta1": 0.9,
"beta2": 0.95,
"eps": 1e-08,
"lr": "frozen"
}
},
{
"name": "treatment_dimuon_s1",
"optimizer": "dimuon-lite",
"optimizer_config": {
"beta1": 0.9,
"grad_clip_norm": 0.0,
"lr": "frozen"
}
}
]
},
{
"arms": [
{
"name": "baseline_adamw_s2",
"optimizer": "adam",
"optimizer_config": {
"beta1": 0.9,
"beta2": 0.95,
"eps": 1e-08,
"lr": "frozen"
}
},
{
"name": "treatment_dimuon_s2",
"optimizer": "dimuon-lite",
"optimizer_config": {
"beta1": 0.9,
"grad_clip_norm": 0.0,
"lr": "frozen"
}
}
]
}
],
"reason": "AdamW and DiMuon arms are trained from a shared fresh-LoRA start within each seed. Before the first update, each arm calls save_weights and downloads the adapter. The SHA-256 of the LoRA tensors must match across the paired arms; B=0 at init, and A comes from the seeded init. Base-model logprobs on a fixed tokenized batch must also match. A hash mismatch makes that seed inconclusive. Optimizer type is verified by the activation-gate step-1 spectrum check, not by invented introspection."
},
"steps": 100,
"success_criterion": "Validated if the simultaneous 95% paired_t interval, across 3 training seeds, for (AdamW NLL \u2212 DiMuon NLL) on no_robots test token_mean NLL at step 100 lies entirely above 0.02 nats. Failed if it lies entirely below 0.02. Otherwise inconclusive. IFEval strict accuracy is reported descriptively and does not decide.",
"title": "dimuon-lite vs AdamW, LoRA SFT on Qwen3.5-2B, no_robots held-out NLL",
"training_type": "sft"
}previous versions (1)
dimuon-lite vs AdamW for LoRA SFT on no_robots (Qwen3.5-0.8B, rank 32, 100 steps)
spec
{
"base_model": "Qwen/Qwen3.5-0.8B",
"baseline": "AdamW via tinker.AdamParams (default betas), LoRA rank 32 on Qwen/Qwen3.5-0.8B, best of LR {5e-5, 1e-4, 2e-4}. Same data order, batch size 64, 100 steps, constant LR, max length 2048, renderer, eval set and second-seed rerun as the dimuon-lite arm.",
"batch_size": 64,
"budget": {
"gpu_hours": 1.5,
"max_wall_minutes": 120
},
"command": "# Custom part: optimizer selection (DimuonOptimizerConfig/DimuonParams vs AdamParams). Everything else is stock tinker SDK + cookbook data utils.\nimport os, random, datasets, tinker\nfrom tinker_cookbook import renderers, model_info\nfrom tinker_cookbook.tokenizer_utils import get_tokenizer\nfrom tinker_cookbook.supervised.data import conversation_to_datum\nM = 'Qwen/Qwen3.5-0.8B'; tok = get_tokenizer(M)\nr = renderers.get_renderer(model_info.get_recommended_renderer_name(M), tok)\nds = datasets.load_dataset('HuggingFaceH4/no_robots')\ntest = [conversation_to_datum(x['messages'], r, 2048) for x in ds['test'].select(range(500))]\nsc = tinker.ServiceClient(base_url=os.environ['TINKER_BASE_URL'])\ndef run(opt, lr, seed=0, steps=100, bs=64):\n rows = list(ds['train']); random.Random(seed).shuffle(rows)\n train = [conversation_to_datum(x['messages'], r, 2048) for x in rows[:steps*bs]]\n if opt == 'dimuon':\n tc = sc.create_lora_training_client(base_model=M, rank=32, optimizer=tinker.types.DimuonOptimizerConfig())\n p = tinker.types.DimuonParams(learning_rate=lr, beta1=0.9, grad_clip_norm=0.0)\n else:\n tc = sc.create_lora_training_client(base_model=M, rank=32)\n p = tinker.AdamParams(learning_rate=lr)\n for s in range(steps):\n tc.forward_backward(train[s*bs:(s+1)*bs], 'cross_entropy').result()\n tc.optim_step(p).result()\n out = tc.forward(test, 'cross_entropy').result()\n num = sum(-(o['logprobs'].to_torch() * d.loss_fn_inputs['weights'].to_torch()).sum().item() for o, d in zip(out.loss_fn_outputs, test))\n den = sum(d.loss_fn_inputs['weights'].to_torch().sum().item() for d in test)\n path = tc.save_weights_for_sampler(name=f'{opt}-{lr}-s{seed}').result().path\n print(opt, lr, seed, 'test_nll', num/den, path)\nfor lr in [5e-5, 1e-4, 2e-4]: run('adam', lr)\nfor lr in [1e-4, 3e-4, 1e-3]: run('dimuon', lr)\n# then: rerun best LR per optimizer with seed=1; sample IFEval (google/IFEval, 541 prompts, greedy, max_tokens=1024) from the saved sampler paths via sc.create_sampling_client(model_path=path) and score prompt-level strict accuracy",
"dataset": {
"hf_path": "HuggingFaceH4/no_robots",
"n_examples": 6400,
"name": "no_robots",
"split": "train (eval on test, first 500)"
},
"eval": {
"benchmark": "no_robots test held-out NLL (primary); IFEval (secondary)",
"metric": "token-weighted NLL in nats/token on 500 test examples; IFEval prompt-level strict accuracy on 541 prompts",
"n_examples": 500
},
"hypothesis": "With each optimizer at its best learning rate from a 3-point sweep, dimuon-lite reaches a held-out no_robots test NLL at least 0.02 nats/token lower than AdamW after 100 steps of LoRA SFT, without lowering IFEval prompt-level strict accuracy by more than 2 points.",
"lora_rank": 32,
"max_seq_len": 2048,
"method": "Train a rank-32 LoRA on Qwen/Qwen3.5-0.8B with next-token cross-entropy on assistant tokens of HuggingFaceH4/no_robots train. Use one fixed shuffle (seed 0) and the cookbook's recommended renderer for the model. Max length is 2048, batch size 64, and training runs 100 optimizer steps at a constant learning rate. The single decision under test is the optimizer. Because the two optimizers have different natural LR scales, each gets a 3-point LR sweep: AdamW at {5e-5, 1e-4, 2e-4} and dimuon-lite at {1e-4, 3e-4, 1e-3}. That is 6 runs on identical batches in identical order. After step 100, compute token-weighted NLL on the first 500 examples of the no_robots test split via forward with cross_entropy. Pick the best LR per optimizer by test NLL, re-run that config once with a second data-shuffle seed (seed 1) to estimate seed noise, and report the mean. For the two best checkpoints, sample greedily (max_tokens 1024) on all 541 google/IFEval prompts and score prompt-level strict accuracy with the standard IFEval checker, client-side. Also log train-loss curves for both arms to compare convergence speed.",
"optimizer": {
"lr": 0.0003,
"name": "dimuon-lite"
},
"steps": 100,
"success_criterion": "Validated if the two-seed mean test NLL of best-LR dimuon-lite is at least 0.02 nats/token below best-LR AdamW, the seed-to-seed spread of each arm is below 0.01, and IFEval strict accuracy for dimuon-lite is no more than 2.0 points below AdamW. Failed if the NLL gap is under 0.02 in either direction (no meaningful difference) or AdamW is better. Inconclusive if seed spread exceeds the gap.",
"title": "dimuon-lite vs AdamW for LoRA SFT on no_robots (Qwen3.5-0.8B, rank 32, 100 steps)",
"training_type": "sft"
}Runs
Source
raw text
Kevin Li, Kailash, Jason and Aaron reverse-engineered Tinker's DiMuon optimizer for LoRA: Polar Express orthogonalization (~5 iterations), rank_lr_correction_exponent 0.175, Muon applied to the LoRA B matrix, no beta2. Jason has 'dimuon-lite' in miles (modal-projects/miles#17), exposed in Spindle (modal-projects/spindle#85). Aaron's reconstruction reached 0.09% relative update error against Tinker's updates. Open question: does it actually train better than AdamW on a LoRA SFT job of the same size?
triage output
{
"adjacent_experiment": "",
"reproducibility_reason": "The shared Spindle exposes dimuon-lite (spindle#85) through DimuonOptimizerConfig/DimuonParams. A short tinker SDK SFT script can run DiMuon and AdamW arms on the same data and step count.",
"reproducible_on_spindle": true,
"sources": [],
"spec": {
"activation_gate": {
"applicable": true,
"checks": [
{
"measures": "model name passed to create_lora_training_client in every arm",
"metric": "executed_base_model",
"one_of": [
"Qwen/Qwen3.5-2B"
]
},
{
"measures": "type of params object passed to optim_step in each arm (logged client-side)",
"metric": "executed_optimizer_step_type",
"one_of": [
"DimuonParams",
"AdamParams"
]
},
{
"max": 1.05,
"measures": "||dB||_F^2 / (||dB||_2^2 * r) for the first DiMuon update of LoRA B, averaged over layers, from downloaded checkpoints at step 0 and 1",
"metric": "dimuon_B_update_normalized_stable_rank_step1",
"min": 0.6
},
{
"max": 1.0,
"measures": "fraction of the 100 training batches whose token-id hash is identical between paired arms within a seed",
"metric": "batch_hash_match_across_arms",
"min": 1.0
},
{
"measures": "train token_mean NLL at step 0 minus step 10 in the DiMuon pilot at the middle LR; must be finite and positive",
"metric": "pilot_train_nll_drop_dimuon_10steps",
"min": 0.0
}
],
"reason": "The paired comparison only tests DiMuon if the executed optimizer really orthogonalizes LoRA B updates and both arms ran the declared configuration. Polar Express makes the first-step B update near-orthogonal (flat singular spectrum); AdamW's is not. This is measured from saved weight diffs, not SDK introspection."
},
"arms": [
{
"lr": 5e-05,
"name": "lrsearch_adamw_5e-5_s0",
"optimizer": "adam",
"phase": "selection",
"seed": 0,
"steps": 40
},
{
"lr": 0.0001,
"name": "lrsearch_adamw_1e-4_s0",
"optimizer": "adam",
"phase": "selection",
"seed": 0,
"steps": 40
},
{
"lr": 0.0002,
"name": "lrsearch_adamw_2e-4_s0",
"optimizer": "adam",
"phase": "selection",
"seed": 0,
"steps": 40
},
{
"lr": 0.0003,
"name": "lrsearch_dimuon_3e-4_s0",
"optimizer": "dimuon-lite",
"phase": "selection",
"seed": 0,
"steps": 40
},
{
"lr": 0.001,
"name": "lrsearch_dimuon_1e-3_s0",
"optimizer": "dimuon-lite",
"phase": "selection",
"seed": 0,
"steps": 40
},
{
"lr": 0.003,
"name": "lrsearch_dimuon_3e-3_s0",
"optimizer": "dimuon-lite",
"phase": "selection",
"seed": 0,
"steps": 40
},
{
"lr": "frozen from selection",
"name": "baseline_adamw_s0",
"optimizer": "adam",
"phase": "confirmation",
"seed": 0,
"steps": 100
},
{
"lr": "frozen from selection",
"name": "baseline_adamw_s1",
"optimizer": "adam",
"phase": "confirmation",
"seed": 1,
"steps": 100
},
{
"lr": "frozen from selection",
"name": "baseline_adamw_s2",
"optimizer": "adam",
"phase": "confirmation",
"seed": 2,
"steps": 100
},
{
"lr": "frozen from selection",
"name": "treatment_dimuon_s0",
"optimizer": "dimuon-lite",
"phase": "confirmation",
"seed": 0,
"steps": 100
},
{
"lr": "frozen from selection",
"name": "treatment_dimuon_s1",
"optimizer": "dimuon-lite",
"phase": "confirmation",
"seed": 1,
"steps": 100
},
{
"lr": "frozen from selection",
"name": "treatment_dimuon_s2",
"optimizer": "dimuon-lite",
"phase": "confirmation",
"seed": 2,
"steps": 100
}
],
"base_model": "Qwen/Qwen3.5-2B",
"baseline": "AdamW (tinker.AdamParams, beta1 0.9, beta2 0.95, eps 1e-8) through the same SDK script, with the same model, LoRA rank, init seed, batches and order, 100 steps, and eval. Its LR is chosen by the same 3-point validation search.",
"batch_size": 32,
"budget": {
"gpu_hours": 2.0,
"max_wall_minutes": 150
},
"command": "# Custom part: one SDK SFT loop shared by both optimizers (cookbook chat_sl is AdamW-only).\nimport os, sys, tinker, datasets, random\nfrom tinker_cookbook import renderers, tokenizer_utils\nfrom tinker_cookbook.supervised.data import conversation_to_datum\nOPT, LR, SEED, STEPS = sys.argv[1], float(sys.argv[2]), int(sys.argv[3]), int(sys.argv[4])\nMODEL='Qwen/Qwen3.5-2B'; BS=32; MAXLEN=2048\ntok=tokenizer_utils.get_tokenizer(MODEL); rend=renderers.get_renderer(renderers.get_recommended_renderer_name(MODEL) if hasattr(renderers,'get_recommended_renderer_name') else 'qwen3', tok)\nds=datasets.load_dataset('HuggingFaceH4/no_robots'); tr=list(ds['train']); random.Random(1234).shuffle(tr)\nval, train = tr[:500], tr[500:]; random.Random(SEED).shuffle(train)\nmk=lambda r: conversation_to_datum(r['messages'], rend, MAXLEN)\nsc=tinker.ServiceClient(base_url=os.environ['TINKER_BASE_URL'])\nkw={'optimizer': tinker.types.DimuonOptimizerConfig()} if OPT=='dimuon' else {}\ntc=sc.create_lora_training_client(base_model=MODEL, rank=32, seed=SEED, **kw)\ntc.save_weights_for_sampler(name=f'init-{OPT}-s{SEED}').result() # hash for restoration preflight\nstep_params = (lambda: tinker.types.DimuonParams(learning_rate=LR, beta1=0.9, grad_clip_norm=0.0)) if OPT=='dimuon' else (lambda: tinker.AdamParams(learning_rate=LR, beta1=0.9, beta2=0.95, eps=1e-8))\nfor s in range(STEPS):\n batch=[mk(r) for r in train[s*BS:(s+1)*BS]]\n tc.forward_backward(batch, loss_fn='cross_entropy').result(); tc.optim_step(step_params()).result()\ndef nll(rows): # token_mean from per-example totals, never batch loss:mean\n tot=0.0; n=0.0\n for i in range(0,len(rows),32):\n d=[mk(r) for r in rows[i:i+32]]; out=tc.forward(d, loss_fn='cross_entropy').result()\n for di,o in zip(d,out.loss_fn_outputs):\n w=di.loss_fn_inputs['weights'].to_numpy(); lp=o['logprobs'].to_numpy(); tot+=-(lp*w).sum(); n+=w.sum()\n return tot/n\nsplit = val if STEPS==40 else list(ds['test'])\nprint({'opt':OPT,'lr':LR,'seed':SEED,'steps':STEPS,'token_mean_nll':nll(split)})\ntc.save_weights_for_sampler(name=f'final-{OPT}-lr{LR}-s{SEED}').result()\n# run: python sft_opt.py {adam|dimuon} <lr> <seed> {40|100}",
"correctness_preflight": {
"notes": "Tolerances are fixed for bf16 LoRA weights with fp32 optimizer math before measuring. reference_update rtol 1e-2 leaves headroom over the reported 0.09% error. Feed the same LoRA state and gradient to the Spindle DiMuon step and to the reference for one step. With grad_clip_norm=0 and beta1 momentum seeded from the first gradient, the update should scale linearly in LR across a 100x span.",
"paths": [
{
"baseline_revision": "UNRESOLVED: pin the modal-projects/spindle commit deployed at the shared Spindle URL (post-#85). AdamW updates on this deployment must match the pre-#85 AdamW update on identical grads/state.",
"id": "spindle_dimuon_lite_optimizer",
"kind": "optimizer",
"lr_scaling": {
"learning_rates": [
0.0001,
0.001,
0.01
],
"rtol": 0.01
},
"reference_revision": "UNRESOLVED: pin the commit of Aaron's DiMuon reconstruction (0.09% relative update error vs Tinker) and the modal-projects/miles commit of #17",
"tolerances": {
"baseline_update": {
"atol": 1e-06,
"min_cosine": 0.99999,
"rtol": 0.001
},
"reference_update": {
"atol": 1e-05,
"min_cosine": 0.999,
"rtol": 0.01
}
}
}
],
"required": true,
"status": "unresolved: spindle and miles commits for #85/#17 and the commit of Aaron's DiMuon reconstruction must be pinned before any full sweep"
},
"dataset": {
"hf_path": "HuggingFaceH4/no_robots",
"n_examples": 9500,
"name": "no_robots",
"split": "train (9000 for training, 500 held out for LR selection); test (500) for confirmation"
},
"decision_rule": {
"adaptive_selection": {
"confirmation_item_ids": "all 500 rows of no_robots split=test",
"kind": "hyperparameter",
"rationale": "Muon-family and AdamW LoRA optimizers have very different good LRs. Each gets a prespecified 3-point grid, scored only on validation items held out from train, and the choice is frozen before the single confirmation eval on the test split.",
"search_item_ids": "no_robots train rows held out by seeded shuffle (seed 1234), first 500 after shuffle; never used for training"
},
"alpha": 0.05,
"assumptions": {
"identically_distributed_seed_effects": true,
"independent_seed_effects": true,
"normal_seed_effects": true
},
"criteria": [
{
"id": "primary",
"margin": 0.02,
"metric": "no_robots_test_token_mean_nll_at_step_100",
"weights": {
"baseline": 1,
"treatment": -1
}
}
],
"estimand": "Reduction in held-out no_robots test token_mean NLL at step 100 from replacing AdamW with dimuon-lite (each at its validation-selected LR) for rank-32 LoRA SFT on Qwen3.5-2B; larger is better for dimuon-lite.",
"method": "paired_t",
"seeds": [
0,
1,
2
],
"unit": "training_seed",
"version": 1
},
"eval": {
"aggregation": "token_mean",
"benchmark": "no_robots test split (HuggingFaceH4/no_robots, split=test)",
"metric": "held-out token_mean NLL (nats per assistant loss token)",
"n_examples": 500
},
"experiment_features": {
"adaptive_selection": true,
"checkpoint_restore": false,
"custom_loss": false,
"custom_optimizer": true,
"generated_training_data": false,
"heldout_loss": true,
"learned_generator": false,
"paired_comparison": true,
"paired_training_arms": true,
"warmup_adaptation": false
},
"feasibility": {
"config": null,
"downsizing_order": "drop IFEval secondary, then shrink LR grid to 2 points per optimizer, then shorten selection runs to 25 steps; keep 3 seeds and the 100-step test NLL. If that still does not fit, report inconclusive.",
"effective_config": {
"arms": 12,
"base_model": "Qwen/Qwen3.5-2B",
"batch_size": 32,
"confirmation_steps": 100,
"max_seq_len": 2048,
"n_eval_test": 500,
"n_eval_validation": 500,
"n_ifeval_secondary": 541,
"planned_arm_steps": 840,
"selection_steps": 40
},
"generation": null,
"measurements": null,
"primary_endpoint": "no_robots_test_token_mean_nll_at_step_100",
"recovery": null,
"source": null
},
"feasibility_check": {
"generation_applicability": {
"check": "generation_audit",
"enforce": false,
"features": {
"generated_training_data": false
},
"reasons": [
"spec.generation_audit opts out and no feature it covers is true"
],
"status": "not_applicable"
},
"generation_counts": null,
"gpu_hours": null,
"known_gpu_hours": 0,
"phases": {},
"problems": [
"Effective configuration not recorded",
"Pilot evidence source not recorded",
"Representative training timing not recorded",
"Representative setup timing not recorded",
"Representative evaluation timing not recorded",
"Positive recovery reserve not recorded"
],
"status": "unknown",
"wall_seconds": null
},
"feasibility_required": true,
"generation_audit": {
"answer_comparison": null,
"max_eval_overlaps": 0,
"max_label_errors": 0,
"min_sample_size": 0,
"reason": "fixed public dataset; no generated training data",
"required": false
},
"hypothesis": "With per-optimizer learning rates picked on a validation split, LoRA SFT with dimuon-lite cuts held-out no_robots token_mean NLL by at least 0.02 nats versus AdamW. Both arms use the same 100 steps, batches and seeds.",
"lora_rank": 32,
"max_seq_len": 2048,
"method": "Train a rank-32 LoRA on Qwen/Qwen3.5-2B over HuggingFaceH4/no_robots, using one tinker SDK script for both arms so the only difference is the optimizer. The cookbook chat_sl recipe is AdamW-only, so running AdamW through the same script removes any recipe difference. DiMuon is selected at creation with optimizer=tinker.types.DimuonOptimizerConfig() and stepped with DimuonParams(learning_rate, beta1=0.9, grad_clip_norm=0.0). AdamW uses tinker.AdamParams(learning_rate, beta1=0.9, beta2=0.95, eps=1e-8). dimuon-lite is a recently landed reverse-engineered update rule: Polar Express orthogonalization with about 5 iterations, rank_lr_correction_exponent 0.175, Muon on LoRA B, and no beta2. Its update is checked against the trusted DiMuon reconstruction before the comparison runs. Phase 1, LR selection: each optimizer runs 40 steps at seed 0 on a 3-point LR grid fixed now (AdamW {5e-5, 1e-4, 2e-4}; DiMuon {3e-4, 1e-3, 3e-3}). Each run is scored by token_mean NLL on 500 validation items held out from the train split, and the best LR per optimizer is frozen. Phase 2, confirmation: each optimizer trains 100 steps at its frozen LR for seeds {0,1,2}. Batches are identical across arms within a seed: the same shuffled order of the remaining 9,000 train rows, batch_size 32, max_seq_len 2048. Data order and LoRA init seed are shared per seed. Primary eval is token_mean NLL on the full 500-example no_robots test split, computed from per-example summed logprobs times loss weights via forward(). Secondary, descriptive only: IFEval prompt-level strict accuracy (541 prompts, greedy, max_tokens 1024) on each final checkpoint. IFEval is dropped first if feasibility does not fit.",
"optimizer": {
"lr": 0.001,
"name": "dimuon-lite"
},
"primary_endpoint": "no_robots_test_token_mean_nll_at_step_100",
"restoration_preflight": {
"applicable": true,
"fixed_batch_required": true,
"groups": [
{
"arms": [
{
"name": "baseline_adamw_s0",
"optimizer": "adam",
"optimizer_config": {
"beta1": 0.9,
"beta2": 0.95,
"eps": 1e-08,
"lr": "frozen"
}
},
{
"name": "treatment_dimuon_s0",
"optimizer": "dimuon-lite",
"optimizer_config": {
"beta1": 0.9,
"grad_clip_norm": 0.0,
"lr": "frozen"
}
}
]
},
{
"arms": [
{
"name": "baseline_adamw_s1",
"optimizer": "adam",
"optimizer_config": {
"beta1": 0.9,
"beta2": 0.95,
"eps": 1e-08,
"lr": "frozen"
}
},
{
"name": "treatment_dimuon_s1",
"optimizer": "dimuon-lite",
"optimizer_config": {
"beta1": 0.9,
"grad_clip_norm": 0.0,
"lr": "frozen"
}
}
]
},
{
"arms": [
{
"name": "baseline_adamw_s2",
"optimizer": "adam",
"optimizer_config": {
"beta1": 0.9,
"beta2": 0.95,
"eps": 1e-08,
"lr": "frozen"
}
},
{
"name": "treatment_dimuon_s2",
"optimizer": "dimuon-lite",
"optimizer_config": {
"beta1": 0.9,
"grad_clip_norm": 0.0,
"lr": "frozen"
}
}
]
}
],
"reason": "AdamW and DiMuon arms are trained from a shared fresh-LoRA start within each seed. Before the first update, each arm calls save_weights and downloads the adapter. The SHA-256 of the LoRA tensors must match across the paired arms; B=0 at init, and A comes from the seeded init. Base-model logprobs on a fixed tokenized batch must also match. A hash mismatch makes that seed inconclusive. Optimizer type is verified by the activation-gate step-1 spectrum check, not by invented introspection."
},
"steps": 100,
"success_criterion": "Validated if the simultaneous 95% paired_t interval, across 3 training seeds, for (AdamW NLL \u2212 DiMuon NLL) on no_robots test token_mean NLL at step 100 lies entirely above 0.02 nats. Failed if it lies entirely below 0.02. Otherwise inconclusive. IFEval strict accuracy is reported descriptively and does not decide.",
"title": "dimuon-lite vs AdamW, LoRA SFT on Qwen3.5-2B, no_robots held-out NLL",
"training_type": "sft"
},
"summary": "dimuon-lite is Spindle's rebuild of Tinker's DiMuon LoRA optimizer; we compare its held-out no_robots NLL against AdamW at equal steps, with IFEval as a secondary check.",
"tags": [
"optimizer",
"dimuon",
"lora",
"sft",
"eval"
],
"title": "Does dimuon-lite beat AdamW for LoRA SFT on no_robots?",
"value_estimate": 4,
"value_rationale": "Spindle users choosing an optimizer for LoRA SFT would act on a clean, LR-fair dimuon-lite vs AdamW result. It also checks that the reverse-engineered optimizer actually helps rather than just matching Tinker's updates. Triage saw only the Slack text, not the linked PRs."
}Comments
@reproducibility · 2d ago · 61/100 · Data and design are solid, but pin the Spindle/miles commit for dimuon-lite, the model and dataset r…
61/100 · Data and design are solid, but pin the Spindle/miles commit for dimuon-lite, the model and dataset revisions, and finish the IFEval step in the recorded command. Also note the 0.09% fidelity figure is only a thread claim; link the PRs or log it.
5/5 data-available: HuggingFaceH4/no_robots train for training, test first 500 for eval, plus google/IFEval 541 prompts. Exact paths and splits are named.
3/5 stock-tooling: Cookbook chat_sl is AdamW-only, so a custom SDK script is needed. Dimuon is exposed via DimuonOptimizerConfig/DimuonParams; the rest is stock, including data utils.
3/5 rerunnable: Command and shuffle seeds are recorded, but no model revision, dataset revision, or dimuon-lite commit is pinned, and the command is truncated before IFEval.
2/5 claim-evidence: Source is a Slack thread with internal PRs and one number (0.09% update error); no published results show dimuon-lite beating AdamW. The error figure is not in a comparable form.
Reproducibility Auditor rubric v1
@prior-art · 2d ago · 44/100 · Before running this, ask in #proj-train-spindle whether Kevin, Kailash, Jason or Aaron already train…
44/100 · Before running this, ask in #proj-train-spindle whether Kevin, Kailash, Jason or Aaron already trained with dimuon-lite and link their curves. Then say why orthogonalizing the LoRA B update should help at 100 steps, and cite Muon and Polar Express as the closest work.
3/5 novelty: Muon-style optimizers vs AdamW is well-studied; dimuon-lite on LoRA with Qwen3.5-0.8B is a new setting with unknown outcome, but it is mainly a validation.
3/5 positioning: Names Tinker's DiMuon, miles#17 and spindle#85, but cites no Muon/Polar Express papers or LoRA-Muon work, and doesn't state the delta beyond 'does it beat AdamW'.
3/5 duplication-risk: Four people reverse-engineered this; they likely ran sanity training. Aaron's 0.09% update-error check is the only result mentioned. Ask in #proj-train-spindle before spending GPU.
2/5 insight: No mechanism for why orthogonalized B-matrix updates should lower NLL at 100 steps, nor when it would help or not; the hypothesis is just a threshold.
Prior Art rubric v1
@scope-cutter · 2d ago · 53/100 · The single-optimizer framing is right. Cut the first pass to NLL only: AdamW at 1e-4 and 2e-4, dimuo…
53/100 · The single-optimizer framing is right. Cut the first pass to NLL only: AdamW at 1e-4 and 2e-4, dimuon at 3e-4 and 1e-3, one seed. Add IFEval and seed 1 only if a gap appears. Then spell out the ladder: a larger Qwen, more steps, then a recipe.
4/5 one-variable: Optimizer is the only difference, with identical data order, steps, and renderer. The per-optimizer LR sweep is justified, but IFEval and the seed rerun add side comparisons.
3/5 smallest-version: Qwen3.5-0.8B and 100 steps are small, but 6 runs plus reruns plus full 541-prompt IFEval exceed what a first signal needs. Start with 1-2 LRs per arm and NLL only.
2/5 incremental-path: No stated next step after a win or loss: no larger model, longer-step, or rank ladder. The outcomes are only validated, failed, or inconclusive.
Scope Cutter rubric v1
@reviewer-2 · 2d ago · 58/100 · Good pre-registered thresholds, but select LR on a held-out validation split, not the 500 test examp…
58/100 · Good pre-registered thresholds, but select LR on a held-out validation split, not the 500 test examples you report. Add a lower AdamW LR or a wider grid so neither optimum sits on a boundary, and run at least 3 seeds for both arms. Two seeds can't support a 0.01 spread claim.
5/5 falsifiable-hypothesis: Specific: dimuon-lite ≥0.02 nats/token lower test NLL than AdamW, IFEval drop ≤2 points, seed spread <0.01, with failure and inconclusive branches defined before the run.
3/5 fair-baseline: Same data, steps, model, and a 3-point LR sweep per arm. But AdamW grid is narrow (5e-5 to 2e-4, no upper bracket) and betas, weight decay, and warmup/decay are untuned.
3/5 confounds: Data order, batch size, steps, and renderer are matched. But constant LR, a 3-point grid with possible edge optima, and grad_clip 0.0 for dimuon versus Adam defaults are unmatched.
2/5 noise: Seed-1 rerun only at best LR gives n=2. The sweep is picked on seed 0 by test NLL (winner's curse). A 0.01 spread from 2 seeds is unreliable, and IFEval n=541 can't resolve 2 points.
3/5 eval-validity: Held-out no_robots test NLL is clean, but LR is selected on that same test set. IFEval is a weak downstream proxy for no_robots SFT. Greedy decoding and generation settings are fine.
Reviewer 2 rubric v1
@spindle-customer · 2d ago · 81/100 · Good pick, since customers are asking this and dimuon-lite is unproven. Please also log the Tinker-p…
81/100 · Good pick, since customers are asking this and dimuon-lite is unproven. Please also log the Tinker-parity caveat: a win for the reconstruction is not proof about DiMuon. Consider a larger model or rank, since 0.8B at 100 steps may not transfer.
4/5 customer-action: Customers ask whether the Tinker DiMuon optimizer beats AdamW. A clear answer on Qwen3.5-0.8B could change their default optimizer and LR, though one small model limits generality.
4/5 recipe-potential: A short tinker SDK script with an AdamW vs dimuon-lite LR sweep on no_robots is nearly copyable. The cookbook chat_sl recipe is AdamW-only, so this fills a gap.
5/5 product-feedback: Exercises the new, fragile DimuonOptimizerConfig/DimuonParams path at model creation and optim_step, a reconstruction that customers are about to depend on. It could surface bugs.
4/5 story: Two sentences and a number: dimuon-lite vs AdamW NLL gap and IFEval delta. The LR sweep and two-seed caveat add slight complexity.
Spindle Customer Lens rubric v1
@compute-hawk · 2d ago · 78/100 · Cheap and well scoped; run it. Pick the LR on a held-out slice rather than the reported test split, …
78/100 · Cheap and well scoped; run it. Pick the LR on a held-out slice rather than the reported test split, and extend the grid if dimuon's best LR lands at 1e-4 or 1e-3. Say plainly that 0.8B/100 steps won't settle larger-model claims.
5/5 budget-fit: About 2.9M tokens per run on a 0.8B model, 8 training runs plus a 500-example eval and 2×541 IFEval samples. Comfortably under 1.5 GPU-hours, with room for more seeds.
4/5 signal-per-gpu-hour: The 3-point LR sweep per arm is needed for a fair optimizer comparison. IFEval is a cheap secondary check. The seed rerun comes only after selecting on test NLL.
3/5 small-scale-validity: Optimizer gaps from Muon-style updates may depend on model size, rank and step count. A 0.8B, 100-step win would need confirmation at larger scale.
4/5 throughput-risk: Wall clock is mostly predictable: short sequences and a small model. Per-step Spindle latency across 8 sequential runs, and greedy IFEval rambling to 1024 tokens, deserve a quick timing check.
Compute Hawk rubric v1
@instrument-check · 1d ago · 34/100 · Tuning is the strong part. Before any training, add a one-step probe of both arms' exact configs, an…
34/100 · Tuning is the strong part. Before any training, add a one-step probe of both arms' exact configs, and pin the spindle#85 and miles#17 commits with a numeric stop threshold on the update-error check. Add an edge rule: if a best LR sits at a grid edge, extend that grid by one point.
2/5 capability-probe: Only an update-rule check against the DiMuon reconstruction is mentioned. No one-step run of both arms' exact DimuonOptimizerConfig/DimuonParams and AdamParams on Qwen3.5-2B first.
3/5 tuning-parity: Per-optimizer 3-point grids, equal 40-step budget, picked on 500 held-out validation items. But no edge rule, and AdamW's 2e-4 or DiMuon's 3e-3 could win at the edge.
2/5 riskiest-assumption: The risk (dimuon-lite faithfully matches DiMuon) is never stated as such. The preflight is marked 'unresolved', with unpinned commits and no pass threshold that stops the run.
Instrument Check rubric v1
anonymous · 11h ago
frhh
@pre-mortem · 2d ago · 38/100 · Name the assumption that dimuon-lite is faithful and correctly tuned, and test it first. Check that …