Score centering vs plain GRPO on GSM8K at an aggressive learning rate

Failed no result Devin Qwen/Qwen3.5-0.8B RL 2.0 GPU-h budget started 1h ago 20m 20s

Redo

Stopped before full training on Qwen/Qwen3.5-2B (spec's Qwen3.5-0.8B is not served): base GSM8K-500 greedy accuracy 0.684, Adam lr 1e-4 accepted, and custom-loss parity passed tolerances I proposed before measuring, but the saved spec has no accepted correctness plan (validator: unverified). Also, the specified treatment (score centering composed with the identity IS weight) is identically zero (measured |correction| ~1e-8/token vs 0.009 with weight_fn=ones), so CENTER=1 equals CENTER=0; a spec revision pairing REINFORCE with REINFORCE+SC is needed.

No result No result: comparison did not finish.

Result

No arm results yet.

base_accuracy: 0.684
base_eval_seconds: 128.8
preflight_baseline_loss_max_abs_error: 1.192e-05
preflight_baseline_gradient_max_abs_error: 5.96e-07
preflight_reference_gradient_max_abs_error: 7.451e-09
pilot_sc_abs_correction_mean_max: 1.029e-08
pilot_sc_ones_weight_abs_correction_mean_min: 0.00933
pilot_step_seconds_mean: 65.28
comparison_training_steps: 0

Changes from the spec 1

what	spec	ran with	why	recorded
Model	Qwen/Qwen3.5-0.8B	Qwen/Qwen3.5-2B	Qwen3.5-0.8B is not in shared Spindle get_server_capabilities supported_models; Qwen3.5-2B is the closest served Qwen3.5 model (per launch instructions).	before the run ended

what

spec

ran with

why

recorded

Model

Qwen/Qwen3.5-0.8B

Qwen/Qwen3.5-2B

Qwen3.5-0.8B is not in shared Spindle get_server_capabilities supported_models; Qwen3.5-2B is the closest served Qwen3.5 model (per launch instructions).

before the run ended

Original hypothesis tested: yes

Checks · activation unverified · evaluator unknown · feasibility unknown

Custom implementation correctness: unverified

Correctness preflight was not recorded.

Validates executor-reported evidence; autonomous training calls are not blocked by this server.

Executor reported inconclusive; shown as no result.

Treatment activation: unverified

The spec has no activation gate or experiment features (it predates them), so whether the treatment engaged is unknown.

Initialization / optimizer verification: unverified

Initialization and optimizer evidence was not required by this saved spec.

Research decision: unknown

No prespecified decision rule was recorded.

Adaptation batches across paired arms: unknown (not verified)

The spec does not declare whether this check applies, and the run did not verify it. The run did not report adaptation batch manifests.

Evaluator comparability: unknown

Comparison provenance is incomplete; record every declared arm's score and evaluator.

accuracy: unknown · base

Evaluator identities and per-arm scores

{
  "comparisons": [],
  "evaluators": {
    "7c9cd38084fa1c5ab0bf60eeb477cf29955e8d344a2cef7b38ad6b93cc439a36": {
      "code_sha256": "287fedf4482e8be81b02508222faea06601fda637a21b1ad0a40a21f08399a09",
      "decoding": {
        "max_tokens": 512,
        "stop": [
          248046
        ],
        "temperature": 0
      },
      "items": {
        "ids": [
          "test-0",
          "test-1",
          "test-2",
          "test-3",
          "test-4",
          "test-5",
          "test-6",
          "test-7",
          "test-8",
          "test-9",
          "test-10",
          "test-11",
          "test-12",
          "test-13",
          "test-14",
          "test-15",
          "test-16",
          "test-17",
          "test-18",
          "test-19",
          "test-20",
          "test-21",
          "test-22",
          "test-23",
          "test-24",
          "test-25",
          "test-26",
          "test-27",
          "test-28",
          "test-29",
          "test-30",
          "test-31",
          "test-32",
          "test-33",
          "test-34",
          "test-35",
          "test-36",
          "test-37",
          "test-38",
          "test-39",
          "test-40",
          "test-41",
          "test-42",
          "test-43",
          "test-44",
          "test-45",
          "test-46",
          "test-47",
          "test-48",
          "test-49",
          "test-50",
          "test-51",
          "test-52",
          "test-53",
          "test-54",
          "test-55",
          "test-56",
          "test-57",
          "test-58",
          "test-59",
          "test-60",
          "test-61",
          "test-62",
          "test-63",
          "test-64",
          "test-65",
          "test-66",
          "test-67",
          "test-68",
          "test-69",
          "test-70",
          "test-71",
          "test-72",
          "test-73",
          "test-74",
          "test-75",
          "test-76",
          "test-77",
          "test-78",
          "test-79",
          "test-80",
          "test-81",
          "test-82",
          "test-83",
          "test-84",
          "test-85",
          "test-86",
          "test-87",
          "test-88",
          "test-89",
          "test-90",
          "test-91",
          "test-92",
          "test-93",
          "test-94",
          "test-95",
          "test-96",
          "test-97",
          "test-98",
          "test-99",
          "test-100",
          "test-101",
          "test-102",
          "test-103",
          "test-104",
          "test-105",
          "test-106",
          "test-107",
          "test-108",
          "test-109",
          "test-110",
          "test-111",
          "test-112",
          "test-113",
          "test-114",
          "test-115",
          "test-116",
          "test-117",
          "test-118",
          "test-119",
          "test-120",
          "test-121",
          "test-122",
          "test-123",
          "test-124",
          "test-125",
          "test-126",
          "test-127",
          "test-128",
          "test-129",
          "test-130",
          "test-131",
          "test-132",
          "test-133",
          "test-134",
          "test-135",
          "test-136",
          "test-137",
          "test-138",
          "test-139",
          "test-140",
          "test-141",
          "test-142",
          "test-143",
          "test-144",
          "test-145",
          "test-146",
          "test-147",
          "test-148",
          "test-149",
          "test-150",
          "test-151",
          "test-152",
          "test-153",
          "test-154",
          "test-155",
          "test-156",
          "test-157",
          "test-158",
          "test-159",
          "test-160",
          "test-161",
          "test-162",
          "test-163",
          "test-164",
          "test-165",
          "test-166",
          "test-167",
          "test-168",
          "test-169",
          "test-170",
          "test-171",
          "test-172",
          "test-173",
          "test-174",
          "test-175",
          "test-176",
          "test-177",
          "test-178",
          "test-179",
          "test-180",
          "test-181",
          "test-182",
          "test-183",
          "test-184",
          "test-185",
          "test-186",
          "test-187",
          "test-188",
          "test-189",
          "test-190",
          "test-191",
          "test-192",
          "test-193",
          "test-194",
          "test-195",
          "test-196",
          "test-197",
          "test-198",
          "test-199",
          "test-200",
          "test-201",
          "test-202",
          "test-203",
          "test-204",
          "test-205",
          "test-206",
          "test-207",
          "test-208",
          "test-209",
          "test-210",
          "test-211",
          "test-212",
          "test-213",
          "test-214",
          "test-215",
          "test-216",
          "test-217",
          "test-218",
          "test-219",
          "test-220",
          "test-221",
          "test-222",
          "test-223",
          "test-224",
          "test-225",
          "test-226",
          "test-227",
          "test-228",
          "test-229",
          "test-230",
          "test-231",
          "test-232",
          "test-233",
          "test-234",
          "test-235",
          "test-236",
          "test-237",
          "test-238",
          "test-239",
          "test-240",
          "test-241",
          "test-242",
          "test-243",
          "test-244",
          "test-245",
          "test-246",
          "test-247",
          "test-248",
          "test-249",
          "test-250",
          "test-251",
          "test-252",
          "test-253",
          "test-254",
          "test-255",
          "test-256",
          "test-257",
          "test-258",
          "test-259",
          "test-260",
          "test-261",
          "test-262",
          "test-263",
          "test-264",
          "test-265",
          "test-266",
          "test-267",
          "test-268",
          "test-269",
          "test-270",
          "test-271",
          "test-272",
          "test-273",
          "test-274",
          "test-275",
          "test-276",
          "test-277",
          "test-278",
          "test-279",
          "test-280",
          "test-281",
          "test-282",
          "test-283",
          "test-284",
          "test-285",
          "test-286",
          "test-287",
          "test-288",
          "test-289",
          "test-290",
          "test-291",
          "test-292",
          "test-293",
          "test-294",
          "test-295",
          "test-296",
          "test-297",
          "test-298",
          "test-299",
          "test-300",
          "test-301",
          "test-302",
          "test-303",
          "test-304",
          "test-305",
          "test-306",
          "test-307",
          "test-308",
          "test-309",
          "test-310",
          "test-311",
          "test-312",
          "test-313",
          "test-314",
          "test-315",
          "test-316",
          "test-317",
          "test-318",
          "test-319",
          "test-320",
          "test-321",
          "test-322",
          "test-323",
          "test-324",
          "test-325",
          "test-326",
          "test-327",
          "test-328",
          "test-329",
          "test-330",
          "test-331",
          "test-332",
          "test-333",
          "test-334",
          "test-335",
          "test-336",
          "test-337",
          "test-338",
          "test-339",
          "test-340",
          "test-341",
          "test-342",
          "test-343",
          "test-344",
          "test-345",
          "test-346",
          "test-347",
          "test-348",
          "test-349",
          "test-350",
          "test-351",
          "test-352",
          "test-353",
          "test-354",
          "test-355",
          "test-356",
          "test-357",
          "test-358",
          "test-359",
          "test-360",
          "test-361",
          "test-362",
          "test-363",
          "test-364",
          "test-365",
          "test-366",
          "test-367",
          "test-368",
          "test-369",
          "test-370",
          "test-371",
          "test-372",
          "test-373",
          "test-374",
          "test-375",
          "test-376",
          "test-377",
          "test-378",
          "test-379",
          "test-380",
          "test-381",
          "test-382",
          "test-383",
          "test-384",
          "test-385",
          "test-386",
          "test-387",
          "test-388",
          "test-389",
          "test-390",
          "test-391",
          "test-392",
          "test-393",
          "test-394",
          "test-395",
          "test-396",
          "test-397",
          "test-398",
          "test-399",
          "test-400",
          "test-401",
          "test-402",
          "test-403",
          "test-404",
          "test-405",
          "test-406",
          "test-407",
          "test-408",
          "test-409",
          "test-410",
          "test-411",
          "test-412",
          "test-413",
          "test-414",
          "test-415",
          "test-416",
          "test-417",
          "test-418",
          "test-419",
          "test-420",
          "test-421",
          "test-422",
          "test-423",
          "test-424",
          "test-425",
          "test-426",
          "test-427",
          "test-428",
          "test-429",
          "test-430",
          "test-431",
          "test-432",
          "test-433",
          "test-434",
          "test-435",
          "test-436",
          "test-437",
          "test-438",
          "test-439",
          "test-440",
          "test-441",
          "test-442",
          "test-443",
          "test-444",
          "test-445",
          "test-446",
          "test-447",
          "test-448",
          "test-449",
          "test-450",
          "test-451",
          "test-452",
          "test-453",
          "test-454",
          "test-455",
          "test-456",
          "test-457",
          "test-458",
          "test-459",
          "test-460",
          "test-461",
          "test-462",
          "test-463",
          "test-464",
          "test-465",
          "test-466",
          "test-467",
          "test-468",
          "test-469",
          "test-470",
          "test-471",
          "test-472",
          "test-473",
          "test-474",
          "test-475",
          "test-476",
          "test-477",
          "test-478",
          "test-479",
          "test-480",
          "test-481",
          "test-482",
          "test-483",
          "test-484",
          "test-485",
          "test-486",
          "test-487",
          "test-488",
          "test-489",
          "test-490",
          "test-491",
          "test-492",
          "test-493",
          "test-494",
          "test-495",
          "test-496",
          "test-497",
          "test-498",
          "test-499"
        ]
      },
      "parser": "gsm8k_eval.parse_answer/is_correct (number after last '####' else last number; float equality 1e-6; reference = text after '####' with commas removed)",
      "prompt_template": {
        "chat_template": "Qwen/Qwen3.5-2B tokenizer apply_chat_template(add_generation_prompt=True, enable_thinking=False)",
        "instruction": "Solve the following math problem step by step. Put the final numeric answer on the last line after '####'.",
        "user": "{instruction}\n\n{question}"
      }
    }
  },
  "scores": [
    {
      "arm": "base",
      "evaluator_id": "7c9cd38084fa1c5ab0bf60eeb477cf29955e8d344a2cef7b38ad6b93cc439a36",
      "metric": "accuracy",
      "predictions_ref": "predictions_base.jsonl",
      "value": 0.684
    }
  ],
  "version": 1
}

Generator reward audit

Generator reward audit: Unknown whether this run needed a teacher reward audit: its spec does not declare whether a teacher or task generator is trained. Nothing was recorded, so reward contributions are unknown.

Feasibility projection: unknown

Projected costs, not measured GPU usage.

Projected wall time: unknown
Projected GPU allocation: unknown

Re-projection required: lora_rank changed or was not recorded
Re-projection required: dataset changed or was not recorded
Re-projection required: steps changed or was not recorded
Re-projection required: max_seq_len changed or was not recorded
Re-projection required: arms changed or was not recorded
Re-projection required: n_examples changed or was not recorded

Timing, sample counts and allocation evidence

{
  "generation_applicability": {
    "check": "generation_audit",
    "enforce": false,
    "features": {
      "generated_training_data": null
    },
    "reasons": [
      "spec has no experiment_features: generated_training_data"
    ],
    "status": "unknown"
  },
  "generation_counts": null,
  "gpu_hours": null,
  "known_gpu_hours": 0,
  "phases": {
    "evaluation": {
      "gpu_hours": null,
      "planned_samples": 22500.0,
      "planned_units": 22500,
      "wall_seconds": 5796.106342077255
    },
    "recovery": {
      "gpu_hours": null,
      "wall_seconds": 900
    },
    "setup": {
      "gpu_hours": null,
      "planned_samples": 0.0,
      "planned_units": 1,
      "wall_seconds": 5.449490308761597
    },
    "training": {
      "gpu_hours": null,
      "planned_samples": 102400.0,
      "planned_units": 400,
      "wall_seconds": 26113.648955027264
    }
  },
  "problems": [
    "Re-projection required: lora_rank changed or was not recorded",
    "Re-projection required: dataset changed or was not recorded",
    "Re-projection required: steps changed or was not recorded",
    "Re-projection required: max_seq_len changed or was not recorded",
    "Re-projection required: arms changed or was not recorded",
    "Re-projection required: n_examples changed or was not recorded"
  ],
  "record": {
    "config": {
      "arm": "CENTER=1",
      "base_model": "Qwen/Qwen3.5-2B",
      "batch_size": 256,
      "max_tokens": 512,
      "n_eval": 500,
      "steps": 3,
      "topk": 128
    },
    "effective_config": {
      "arms": [
        "baseline_center0_seed0",
        "treatment_center1_seed0",
        "baseline_center0_seed1",
        "treatment_center1_seed1"
      ],
      "base_model": "Qwen/Qwen3.5-2B",
      "batch_size": 256,
      "evals_per_arm": 11,
      "max_seq_len": 1024,
      "max_tokens": 512,
      "n_eval": 500,
      "steps": 100
    },
    "generation": null,
    "measurements": {
      "evaluation": {
        "allocation_source": null,
        "gpu_count": null,
        "planned_units": 22500,
        "samples": 500,
        "seconds": 128.80236315727234,
        "units": 500
      },
      "setup": {
        "allocation_source": null,
        "gpu_count": null,
        "planned_units": 1,
        "samples": 0,
        "seconds": 5.449490308761597,
        "units": 1
      },
      "training": {
        "allocation_source": null,
        "gpu_count": null,
        "planned_units": 400,
        "samples": 768,
        "seconds": 195.85236716270447,
        "units": 3
      }
    },
    "primary_endpoint": "greedy exact-match accuracy; peak-minus-final drop over the run",
    "recovery": {
      "allocation_source": null,
      "gpu_count": null,
      "seconds": 900
    },
    "source": "pilot_steps_center1.jsonl, preflight_result.json, base_eval.log"
  },
  "recorded_at": 1793761065.3422754,
  "status": "unknown",
  "wall_seconds": null
}

Generated task audit: unverified: no audit recorded

Answer agreement with recorded independent solutions does not establish semantic correctness. Normalized exact overlap does not detect paraphrases.

Spec exp_6188892aaf

Experiment version

exp_6188892aaf

Repeat the selected saved experiment version to check reproducibility. This repetition tests that version's hypothesis and methods.

Legacy context: original idea context unrecorded (unknown); launch uses the saved spec.

Hypothesis

With LoRA GRPO on Qwen3.5-0.8B at a deliberately high learning rate, score centering limits the drop from peak to final GSM8K test accuracy to at most 2 points, while the uncentered baseline drops by at least 5 points.

Success criterion

Validated if, averaged over 2 seeds, the treatment's peak-minus-final accuracy drop is ≤2 points and is ≥3 points smaller than the baseline's, and final accuracy is not more than 1.5 points below baseline. Failed if the drops differ by <3 points, which is within seed noise on 500 examples.

GSM8K 100 steps batch 256 LoRA r32 lr 0.0001 120 min wall budget

Method

Both arms use one tinker SDK GRPO script that differs only by a CENTER flag. Each step samples 32 GSM8K train problems with 8 rollouts each from the current LoRA and scores exact-match reward. Advantages are reward minus the group mean, and the importance-sampling policy-gradient loss goes through forward_backward_custom. With CENTER=1, the loss applies the score-centering correction exactly as written in Martin's paper. That correction is the only custom part, and it must be copied from the paper's equations before launch. With CENTER=0 the loss is the stock GRPO surrogate. Every 10 steps the script saves sampler weights and evaluates greedy exact match on 500 fixed GSM8K test problems. Every step it logs mean reward, gradient norm, sampler-vs-learner logprob gap and token entropy as stability signals. Each arm runs 2 seeds at learning rate 1e-4, chosen so the baseline is likely to destabilize.

Baseline

Same script, model, seeds, data order, group size 8, 32 groups per batch, max_tokens 512, LoRA rank 32, AdamW lr 1e-4, and 100 steps, with CENTER=0 (standard group-mean-centered GRPO surrogate, no score centering).

Full spec

{
  "base_model": "Qwen/Qwen3.5-0.8B",
  "baseline": "Same script, model, seeds, data order, group size 8, 32 groups per batch, max_tokens 512, LoRA rank 32, AdamW lr 1e-4, and 100 steps, with CENTER=0 (standard group-mean-centered GRPO surrogate, no score centering).",
  "batch_size": 256,
  "budget": {
    "gpu_hours": 2.0,
    "max_wall_minutes": 120
  },
  "command": "CENTER=0 SEED=0 python rl_score_centering.py && CENTER=1 SEED=0 python rl_score_centering.py (repeat SEED=1)  # rl_score_centering.py: sc=tinker.ServiceClient(base_url=os.environ['TINKER_BASE_URL']); tc=sc.create_lora_training_client(base_model='Qwen/Qwen3.5-0.8B', rank=32); loop 100 steps: samp=tc.save_weights_and_get_sampling_client(name=f'step{i}'); sample 32 GSM8K prompts x8 (max_tokens=512); reward=exact match; adv=r-mean(group); tc.forward_backward_custom(data, loss_fn) where loss_fn is the IS policy-gradient loss plus the score-centering term when CENTER=1 (custom part); tc.optim_step(tinker.AdamParams(learning_rate=1e-4)); every 10 steps eval 500 GSM8K test greedy; log grad norm, logprob gap, entropy to /tmp/rx-sc-$CENTER-$SEED",
  "dataset": {
    "hf_path": "openai/gsm8k",
    "n_examples": 3200,
    "name": "GSM8K",
    "split": "main/train (train), main/test first 500 (eval)"
  },
  "eval": {
    "benchmark": "GSM8K test (first 500)",
    "metric": "greedy exact-match accuracy; peak-minus-final drop over the run",
    "n_examples": 500
  },
  "hypothesis": "With LoRA GRPO on Qwen3.5-0.8B at a deliberately high learning rate, score centering limits the drop from peak to final GSM8K test accuracy to at most 2 points, while the uncentered baseline drops by at least 5 points.",
  "lora_rank": 32,
  "max_seq_len": 1024,
  "method": "Both arms use one tinker SDK GRPO script that differs only by a CENTER flag. Each step samples 32 GSM8K train problems with 8 rollouts each from the current LoRA and scores exact-match reward. Advantages are reward minus the group mean, and the importance-sampling policy-gradient loss goes through forward_backward_custom. With CENTER=1, the loss applies the score-centering correction exactly as written in Martin's paper. That correction is the only custom part, and it must be copied from the paper's equations before launch. With CENTER=0 the loss is the stock GRPO surrogate. Every 10 steps the script saves sampler weights and evaluates greedy exact match on 500 fixed GSM8K test problems. Every step it logs mean reward, gradient norm, sampler-vs-learner logprob gap and token entropy as stability signals. Each arm runs 2 seeds at learning rate 1e-4, chosen so the baseline is likely to destabilize.",
  "optimizer": {
    "lr": 0.0001,
    "name": "adam"
  },
  "steps": 100,
  "success_criterion": "Validated if, averaged over 2 seeds, the treatment's peak-minus-final accuracy drop is \u22642 points and is \u22653 points smaller than the baseline's, and final accuracy is not more than 1.5 points below baseline. Failed if the drops differ by <3 points, which is within seed noise on 500 examples.",
  "title": "Score centering vs plain GRPO on GSM8K at an aggressive learning rate",
  "training_type": "rl"
}

Evidence 16

Partial · Download manifest. Recorded files are retained copies; checkpoint references do not include model weights.

Not recorded: predictions, checkpoint references.

launch-instructions-a43b123dc830.md · instructions · 32010 bytes
launch-instructions.md · instructions · 32010 bytes
rl_score_centering.py · script · 9833 bytes
gsm8k_eval.py · script · 4356 bytes
preflight2.py · script · 4950 bytes
ref_parity.py · script · 4772 bytes
pilot.py · script · 565 bytes
common.py · script · 291 bytes
effective-config.json · effective config · 1333 bytes
code_revision.json · code revision · 1125 bytes
sc_nonfinite_fix.patch · code patch · 677 bytes
preflight_result.json · metrics · 4033 bytes
ref_parity_result_v2.json · metrics · 1774 bytes
correctness_preflight.json · metrics · 3390 bytes
proposed_correctness_plan.json · metrics · 837 bytes
spec_effective_with_proposed_plan.json · metrics · 3922 bytes

Per-arm evidence metrics

{
  "base": {
    "accuracy": 0.684
  }
}

Evidence import problems: file 11: attachment retrieval returned HTTP 401; file 17: invalid file name or kind; file 18: invalid file name or kind; file 19: attachment retrieval returned HTTP 401; file 20: invalid file name or kind; file 21: invalid file name or kind; file 22: attachment retrieval returned HTTP 401.

All metrics

base_accuracy: 0.684
base_eval_seconds: 128.8
preflight_baseline_loss_max_abs_error: 1.192e-05
preflight_baseline_gradient_max_abs_error: 5.96e-07
preflight_reference_gradient_max_abs_error: 7.451e-09
pilot_sc_abs_correction_mean_max: 1.029e-08
pilot_sc_ones_weight_abs_correction_mean_min: 0.00933
pilot_step_seconds_mean: 65.28
comparison_training_steps: 0

Interim updates from Devin (3, first Nov 4, 02:40 UTC)

Artifact references

instructions · launch-instructions.md sha256 a43b123dc830
script · rl_score_centering.py sha256 388f91b33c1a
script · gsm8k_eval.py sha256 1c26e3c71f3a
script · preflight2.py sha256 d924f6390c0b
script · ref_parity.py sha256 834e424488d6
script · pilot.py sha256 7f88b0bede92
script · common.py sha256 69661150722d
effective_config · effective-config.json sha256 89615187d8fe
code_revision · code_revision.json sha256 614b48ad7ab1
code_patch · sc_nonfinite_fix.patch sha256 65a4c3080f13
predictions · predictions_base.jsonl sha256 fbbd6addea6a
metrics · preflight_result.json sha256 811ead486144
metrics · ref_parity_result_v2.json sha256 2c564c3010f5
metrics · correctness_preflight.json sha256 c5257497b169
metrics · proposed_correctness_plan.json sha256 9e9bdee20fe5
metrics · spec_effective_with_proposed_plan.json sha256 4d9bfb4f605d
metrics · validator_spec.json.out sha256 6300ab307259
metrics · validator_spec_effective_with_proposed_plan.json.out sha256 f42f56fdab9e
metrics · pilot_steps_center1.jsonl sha256 172d706acfdc
metrics · preflight_fixture.json.gz sha256 c7229a7f7f20
metrics · ref_parity_fixture.npz sha256 a383045d3429
metrics · gate_diagnosis.md sha256 153b12c3e1f3

Nov 4, 02:57 UTC · failed · Stopped before full training on Qwen/Qwen3.5-2B (spec's Qwen3.5-0.8B is not served): base GSM8K-500 greedy accuracy 0.684, Adam lr 1e-4 accepted, and custom-loss parity passed tolerances I proposed b…
Nov 4, 02:48 UTC · running · Launch instructions downloaded and SHA256-verified; Qwen/Qwen3.5-0.8B is not served on shared Spindle so Qwen/Qwen3.5-2B is used. Base 2B greedy accuracy on GSM8K test first 500 is 0.684 (frozen eval…
Nov 4, 02:40 UTC · running · Launch instructions downloaded and SHA256-verified; spec read. Qwen/Qwen3.5-0.8B is not served on shared Spindle, so the closest same-family model Qwen/Qwen3.5-2B is used; preflights in progress.

Artifacts

correctness_preflight: {'required': True, 'paths': [{'id': 'score_centering_loss', 'fixed_input': {'sha256': '4e0d466b0fcac6ad708c927548447e33ef4c272077510584ee83597fd668d591', 'description': 'preflight_fixture.json: 4 GSM8K train problems (train-0..3) x 8 rollouts from the fresh LoRA r32 Qwen3.5-2B sampler after one Adam lr1e-4 probe step, 32 nonzero-advantage datums, 9390 loss tokens; same weights for stock and custom passes (no optim_step between). Reference parity fixture ref_parity_fixture.npz sha256 c1311032134b0b3925f3267a87e11ac2a6ffd8dc5080a60121049107617b6f5a (B=4,T=32,V=4096,K=128, seed 0, sampler/trainer logit mismatch 0.3 std).'}, 'baseline_revision': 'tinker-0.37.0 built-in loss_fn=importance_sampling (stock GRPO IS surrogate) on shared Spindle', 'reference_revision': 'github.com/martin-marek/score-centering@fdca22b19793babdf08e33e5ba51537e82cbed2f README score_centering_loss (JAX) + losses.py loss_fn_rl', 'implementation_revision': 'rl_score_centering.py sha256 388f91b33c1a648ebed6cd1ec0223dde89829aa5c106dc505114ee53c3c942c6', 'precision': 'server forward bf16-model logprobs returned as float32; custom loss computed in float32 torch; reference JAX float32 on CPU', 'normalization': 'token-sum over completion mask (no mean), matching stock importance_sampling loss:sum; reference parity uses sum/mask.sum on both sides', 'errors': {'baseline_loss': {'max_abs_error': 1.1920928955078123e-05, 'reference_max_abs': 264.63753843307495, 'relative_l2_error': 4.504625090477422e-08, 'cosine_similarity': 1.0}, 'baseline_gradient': {'max_abs_error': 5.960464477539062e-07, 'reference_max_abs': 1.0349624156951904, 'relative_l2_error': 1.07802118665598e-07, 'cosine_similarity': 0.999999999999994}, 'reference_loss': {'max_abs_error': 2.682209014892578e-07, 'reference_max_abs': 0.31140369176864624, 'relative_l2_error': 8.613285859453761e-07, 'cosine_similarity': 1.0}, 'reference_gradient': {'max_abs_error': 7.450580596923828e-09, 'reference_max_abs': 0.018823595717549324, 'relative_l2_error': 2.543320963813718e-07, 'cosine_similarity': 0.9999999999999708}}, 'extra_errors': {'baseline_logprobs': {'max_abs_error': 9.5367431640625e-07, 'reference_max_abs': 16.260665893554688, 'relative_l2_error': 1.0626508195941365e-07, 'cosine_similarity': 0.9999999999999942}, 'reference_gradient_vs_loss_fn_rl': {'max_abs_error': 7.450580596923828e-09, 'reference_max_abs': 0.018823595717549324, 'relative_l2_error': 2.552380770976158e-07, 'cosine_similarity': 0.9999999999999706}, 'ones_weight_reference_loss': {'max_abs_error': 7.450580596923828e-07, 'reference_max_abs': 0.20463140308856964, 'relative_l2_error': 3.640976157358912e-06, 'cosine_similarity': 1.0}, 'ones_weight_reference_gradient': {'max_abs_error': 9.313225746154783e-10, 'reference_max_abs': 0.008928446099162102, 'relative_l2_error': 5.6153296751853425e-08, 'cosine_similarity': 0.9999999999999984}}, 'baseline_gradient_scope': 'gradient w.r.t. per-token server logprobs (the only path from the loss to LoRA params); custom-path gradient on the K top-k head columns is exactly 0 for CENTER=0', 'lr_scaling': []}]}
correctness_validation: {'status': 'unverified', 'required': True, 'checks': [], 'reason': 'Correctness preflight was not recorded.', 'evidence_source': 'executor report; remote execution is not attested', 'applicability': 'unknown', 'reported_verdict': 'inconclusive'}
evaluation: {'version': 1, 'evaluators': {'7c9cd38084fa1c5ab0bf60eeb477cf29955e8d344a2cef7b38ad6b93cc439a36': {'code_sha256': '287fedf4482e8be81b02508222faea06601fda637a21b1ad0a40a21f08399a09', 'prompt_template': {'user': '{instruction}\n\n{question}', 'instruction': "Solve the following math problem step by step. Put the final numeric answer on the last line after '####'.", 'chat_template': 'Qwen/Qwen3.5-2B tokenizer apply_chat_template(add_generation_prompt=True, enable_thinking=False)'}, 'decoding': {'temperature': 0, 'max_tokens': 512, 'stop': [248046]}, 'parser': "gsm8k_eval.parse_answer/is_correct (number after last '####' else last number; float equality 1e-6; reference = text after '####' with commas removed)", 'items': {'ids': ['test-0', 'test-1', 'test-2', 'test-3', 'test-4', 'test-5', 'test-6', 'test-7', 'test-8', 'test-9', 'test-10', 'test-11', 'test-12', 'test-13', 'test-14', 'test-15', 'test-16', 'test-17', 'test-18', 'test-19', 'test-20', 'test-21', 'test-22', 'test-23', 'test-24', 'test-25', 'test-26', 'test-27', 'test-28', 'test-29', 'test-30', 'test-31', 'test-32', 'test-33', 'test-34', 'test-35', 'test-36', 'test-37', 'test-38', 'test-39', 'test-40', 'test-41', 'test-42', 'test-43', 'test-44', 'test-45', 'test-46', 'test-47', 'test-48', 'test-49', 'test-50', 'test-51', 'test-52', 'test-53', 'test-54', 'test-55', 'test-56', 'test-57', 'test-58', 'test-59', 'test-60', 'test-61', 'test-62', 'test-63', 'test-64', 'test-65', 'test-66', 'test-67', 'test-68', 'test-69', 'test-70', 'test-71', 'test-72', 'test-73', 'test-74', 'test-75', 'test-76', 'test-77', 'test-78', 'test-79', 'test-80', 'test-81', 'test-82', 'test-83', 'test-84', 'test-85', 'test-86', 'test-87', 'test-88', 'test-89', 'test-90', 'test-91', 'test-92', 'test-93', 'test-94', 'test-95', 'test-96', 'test-97', 'test-98', 'test-99', 'test-100', 'test-101', 'test-102', 'test-103', 'test-104', 'test-105', 'test-106', 'test-107', 'test-108', 'test-109', 'test-110', 'test-111', 'test-112', 'test-113', 'test-114', 'test-115', 'test-116', 'test-117', 'test-118', 'test-119', 'test-120', 'test-121', 'test-122', 'test-123', 'test-124', 'test-125', 'test-126', 'test-127', 'test-128', 'test-129', 'test-130', 'test-131', 'test-132', 'test-133', 'test-134', 'test-135', 'test-136', 'test-137', 'test-138', 'test-139', 'test-140', 'test-141', 'test-142', 'test-143', 'test-144', 'test-145', 'test-146', 'test-147', 'test-148', 'test-149', 'test-150', 'test-151', 'test-152', 'test-153', 'test-154', 'test-155', 'test-156', 'test-157', 'test-158', 'test-159', 'test-160', 'test-161', 'test-162', 'test-163', 'test-164', 'test-165', 'test-166', 'test-167', 'test-168', 'test-169', 'test-170', 'test-171', 'test-172', 'test-173', 'test-174', 'test-175', 'test-176', 'test-177', 'test-178', 'test-179', 'test-180', 'test-181', 'test-182', 'test-183', 'test-184', 'test-185', 'test-186', 'test-187', 'test-188', 'test-189', 'test-190', 'test-191', 'test-192', 'test-193', 'test-194', 'test-195', 'test-196', 'test-197', 'test-198', 'test-199', 'test-200', 'test-201', 'test-202', 'test-203', 'test-204', 'test-205', 'test-206', 'test-207', 'test-208', 'test-209', 'test-210', 'test-211', 'test-212', 'test-213', 'test-214', 'test-215', 'test-216', 'test-217', 'test-218', 'test-219', 'test-220', 'test-221', 'test-222', 'test-223', 'test-224', 'test-225', 'test-226', 'test-227', 'test-228', 'test-229', 'test-230', 'test-231', 'test-232', 'test-233', 'test-234', 'test-235', 'test-236', 'test-237', 'test-238', 'test-239', 'test-240', 'test-241', 'test-242', 'test-243', 'test-244', 'test-245', 'test-246', 'test-247', 'test-248', 'test-249', 'test-250', 'test-251', 'test-252', 'test-253', 'test-254', 'test-255', 'test-256', 'test-257', 'test-258', 'test-259', 'test-260', 'test-261', 'test-262', 'test-263', 'test-264', 'test-265', 'test-266', 'test-267', 'test-268', 'test-269', 'test-270', 'test-271', 'test-272', 'test-273', 'test-274', 'test-275', 'test-276', 'test-277', 'test-278', 'test-279', 'test-280', 'test-281', 'test-282', 'test-283', 'test-284', 'test-285', 'test-286', 'test-287', 'test-288', 'test-289', 'test-290', 'test-291', 'test-292', 'test-293', 'test-294', 'test-295', 'test-296', 'test-297', 'test-298', 'test-299', 'test-300', 'test-301', 'test-302', 'test-303', 'test-304', 'test-305', 'test-306', 'test-307', 'test-308', 'test-309', 'test-310', 'test-311', 'test-312', 'test-313', 'test-314', 'test-315', 'test-316', 'test-317', 'test-318', 'test-319', 'test-320', 'test-321', 'test-322', 'test-323', 'test-324', 'test-325', 'test-326', 'test-327', 'test-328', 'test-329', 'test-330', 'test-331', 'test-332', 'test-333', 'test-334', 'test-335', 'test-336', 'test-337', 'test-338', 'test-339', 'test-340', 'test-341', 'test-342', 'test-343', 'test-344', 'test-345', 'test-346', 'test-347', 'test-348', 'test-349', 'test-350', 'test-351', 'test-352', 'test-353', 'test-354', 'test-355', 'test-356', 'test-357', 'test-358', 'test-359', 'test-360', 'test-361', 'test-362', 'test-363', 'test-364', 'test-365', 'test-366', 'test-367', 'test-368', 'test-369', 'test-370', 'test-371', 'test-372', 'test-373', 'test-374', 'test-375', 'test-376', 'test-377', 'test-378', 'test-379', 'test-380', 'test-381', 'test-382', 'test-383', 'test-384', 'test-385', 'test-386', 'test-387', 'test-388', 'test-389', 'test-390', 'test-391', 'test-392', 'test-393', 'test-394', 'test-395', 'test-396', 'test-397', 'test-398', 'test-399', 'test-400', 'test-401', 'test-402', 'test-403', 'test-404', 'test-405', 'test-406', 'test-407', 'test-408', 'test-409', 'test-410', 'test-411', 'test-412', 'test-413', 'test-414', 'test-415', 'test-416', 'test-417', 'test-418', 'test-419', 'test-420', 'test-421', 'test-422', 'test-423', 'test-424', 'test-425', 'test-426', 'test-427', 'test-428', 'test-429', 'test-430', 'test-431', 'test-432', 'test-433', 'test-434', 'test-435', 'test-436', 'test-437', 'test-438', 'test-439', 'test-440', 'test-441', 'test-442', 'test-443', 'test-444', 'test-445', 'test-446', 'test-447', 'test-448', 'test-449', 'test-450', 'test-451', 'test-452', 'test-453', 'test-454', 'test-455', 'test-456', 'test-457', 'test-458', 'test-459', 'test-460', 'test-461', 'test-462', 'test-463', 'test-464', 'test-465', 'test-466', 'test-467', 'test-468', 'test-469', 'test-470', 'test-471', 'test-472', 'test-473', 'test-474', 'test-475', 'test-476', 'test-477', 'test-478', 'test-479', 'test-480', 'test-481', 'test-482', 'test-483', 'test-484', 'test-485', 'test-486', 'test-487', 'test-488', 'test-489', 'test-490', 'test-491', 'test-492', 'test-493', 'test-494', 'test-495', 'test-496', 'test-497', 'test-498', 'test-499']}}}, 'scores': [{'arm': 'base', 'metric': 'accuracy', 'value': 0.684, 'evaluator_id': '7c9cd38084fa1c5ab0bf60eeb477cf29955e8d344a2cef7b38ad6b93cc439a36', 'predictions_ref': 'predictions_base.jsonl'}], 'comparisons': []}
evaluator_validation: {'status': 'unknown', 'issues': ["Comparison provenance is incomplete; record every declared arm's score and evaluator."], 'comparisons': [{'metric': 'accuracy', 'status': 'unknown', 'arms': ['base'], 'evaluator_ids': ['7c9cd38084fa1c5ab0bf60eeb477cf29955e8d344a2cef7b38ad6b93cc439a36'], 'missing_arms': []}]}
feasibility: {'status': 'unknown', 'problems': ['Re-projection required: lora_rank changed or was not recorded', 'Re-projection required: dataset changed or was not recorded', 'Re-projection required: steps changed or was not recorded', 'Re-projection required: max_seq_len changed or was not recorded', 'Re-projection required: arms changed or was not recorded', 'Re-projection required: n_examples changed or was not recorded'], 'wall_seconds': None, 'gpu_hours': None, 'known_gpu_hours': 0, 'phases': {'setup': {'wall_seconds': 5.449490308761597, 'planned_units': 1, 'planned_samples': 0.0, 'gpu_hours': None}, 'training': {'wall_seconds': 26113.648955027264, 'planned_units': 400, 'planned_samples': 102400.0, 'gpu_hours': None}, 'evaluation': {'wall_seconds': 5796.106342077255, 'planned_units': 22500, 'planned_samples': 22500.0, 'gpu_hours': None}, 'recovery': {'wall_seconds': 900, 'gpu_hours': None}}, 'generation_counts': None, 'generation_applicability': {'check': 'generation_audit', 'status': 'unknown', 'enforce': False, 'reasons': ['spec has no experiment_features: generated_training_data'], 'features': {'generated_training_data': None}}, 'record': {'config': {'base_model': 'Qwen/Qwen3.5-2B', 'arm': 'CENTER=1', 'steps': 3, 'batch_size': 256, 'max_tokens': 512, 'topk': 128, 'n_eval': 500}, 'effective_config': {'base_model': 'Qwen/Qwen3.5-2B', 'arms': ['baseline_center0_seed0', 'treatment_center1_seed0', 'baseline_center0_seed1', 'treatment_center1_seed1'], 'steps': 100, 'batch_size': 256, 'max_tokens': 512, 'max_seq_len': 1024, 'n_eval': 500, 'evals_per_arm': 11}, 'primary_endpoint': 'greedy exact-match accuracy; peak-minus-final drop over the run', 'source': 'pilot_steps_center1.jsonl, preflight_result.json, base_eval.log', 'measurements': {'setup': {'seconds': 5.449490308761597, 'units': 1, 'planned_units': 1, 'samples': 0, 'gpu_count': None, 'allocation_source': None}, 'training': {'seconds': 195.85236716270447, 'units': 3, 'planned_units': 400, 'samples': 768, 'gpu_count': None, 'allocation_source': None}, 'evaluation': {'seconds': 128.80236315727234, 'units': 500, 'planned_units': 22500, 'samples': 500, 'gpu_count': None, 'allocation_source': None}}, 'generation': None, 'recovery': {'seconds': 900, 'gpu_count': None, 'allocation_source': None}}, 'recorded_at': 1793761065.3422754}
feasibility_required: True
launch: {'experiment_id': 'exp_6188892aaf', 'context_status': 'legacy_unknown', 'superseded': False, 'mode': 'redo', 'purpose': "Repeat the selected saved experiment version to check reproducibility. This repetition tests that version's hypothesis and methods."}
launch_instructions: {'name': 'launch-instructions-a43b123dc830.md', 'sha256': 'a43b123dc830f9e630619639702ede1e853ac487eafe31b58d6b79b1c4434e8f', 'url': 'https://ta-01m6fva5db548f5skm214fepcr-8000-oah9l7x00jpo7186eb6cix5v0.w.modal.host/api/runs/run_b40a4cca1b/evidence/a43b123dc830f9e630619639702ede1e853ac487eafe31b58d6b79b1c4434e8f'}
reported_outcome_reason: incomplete_execution
reported_verdict: inconclusive

Agent trace full trace with commands in Devin ↗

Details

run: run_b40a4cca1b
backend: Devin
session: devin-c836cd535d084e8bb153ab58ace85ad7
experiment: exp_6188892aaf
launch: redo
redo of: run_c01b44995b
created: Nov 4, 02:37 UTC
started: Nov 4, 02:37 UTC
finished: Nov 4, 02:57 UTC
duration: 20m 20s
GPU-h budget: 2.0
wall budget: 120 min
base model: Qwen/Qwen3.5-0.8B
training: RL
dataset: GSM8K
result: final
outcome reason: incomplete execution

Other runs of this idea

run_c01b44995b no result 5h ago
run_bf333fc33a not supported 23h ago

🔬Modal Research Exchange