🔬Modal Research Exchange

{
  "activation_gate": {
    "applicable": true,
    "checks": [
      {
        "measures": "B_eff ratio of the TF-IDF kernel to the LOO batch mean at B=32. Computed with the post's recipe on a calibration set of G=8 rollouts on 64 held-out GSM8K train tasks from the starting policy, including the sigma_eps^2/(G-1) correction.",
        "metric": "offline_beff_ratio_tfidf_vs_loo_mean",
        "min": 1.03
      },
      {
        "measures": "Mean |b_GS1_i - LOO batch mean_i| over the first 3 pilot steps of the GS1 arm.",
        "metric": "pilot_mean_abs_gs1_minus_batchmean_baseline",
        "min": 0.01
      },
      {
        "max": 0.0,
        "measures": "Maximum w_ii used in any GS1 baseline.",
        "metric": "self_weight_max",
        "min": 0.0
      },
      {
        "max": 32,
        "measures": "Logged rollouts per step in each arm.",
        "metric": "executed_rollouts_per_step",
        "min": 32
      },
      {
        "max": 1,
        "measures": "Logged group_size in the GS1 arm.",
        "metric": "executed_group_size_gs1",
        "min": 1
      },
      {
        "max": 8,
        "measures": "Logged group_size in the GRPO arm.",
        "metric": "executed_group_size_grpo",
        "min": 8
      },
      {
        "measures": "Fraction of GS1 rollouts with |A_i| > 1e-6 in the pilot.",
        "metric": "nonzero_advantage_fraction_gs1",
        "min": 0.3
      }
    ],
    "reason": "GS1 only matters if the kernel baseline differs from a plain batch mean and predicts per-task difficulty better than it. GSM8K has milder prompt-difficulty structure than AutomationBench, so a pilot measures this before full training. Executed-config predicates confirm the arms really differ only in grouping/advantage at 32 rollouts per step."
  },
  "arms": [
    {
      "advantage": "group_mean",
      "group_size": 8,
      "groups_per_batch": 4,
      "lr": 3e-05,
      "name": "grpo_g8",
      "seed": 0,
      "steps": 60
    },
    {
      "advantage": "tfidf_kernel_loo_T0.1",
      "group_size": 1,
      "groups_per_batch": 32,
      "lr": 3e-05,
      "name": "gs1_tfidf_g1",
      "seed": 0,
      "steps": 60
    }
  ],
  "base_model": "Qwen/Qwen3.5-2B",
  "baseline": "GRPO via the stock math_rl recipe: group_size=8 and groups_per_batch=4 (32 rollouts per step), group-mean-centered advantage. Same model, LoRA rank, lr, steps, max_tokens, KL=0, seed and eval as the GS1 arm.",
  "batch_size": 32,
  "budget": {
    "gpu_hours": 2.0,
    "max_wall_minutes": 120
  },
  "command": "Baseline arm (stock GRPO):\npython -m tinker_cookbook.recipes.math_rl.train model_name=Qwen/Qwen3.5-2B base_url=$TINKER_BASE_URL env=gsm8k group_size=8 groups_per_batch=4 learning_rate=3e-5 lora_rank=32 max_tokens=512 max_steps=60 kl_penalty_coef=0 log_path=/tmp/rx-gs1/grpo_s0\n\nTreatment arm: the same recipe with group_size=1 groups_per_batch=32 log_path=/tmp/rx-gs1/gs1_s0, run through a thin local wrapper. The wrapper monkeypatches tinker_cookbook.rl's compute_advantages; this replacement is the only custom part:\n\nfrom sklearn.feature_extraction.text import TfidfVectorizer\nimport numpy as np\nvec = TfidfVectorizer().fit(train_questions)\n\ndef gs1_adv(prompts, rewards, T=0.1):\n    X = vec.transform(prompts)\n    S = (X @ X.T).toarray()\n    W = np.exp(S / T)\n    np.fill_diagonal(W, 0.0)\n    b = (W @ rewards) / W.sum(1)\n    return rewards - b\n\nThe wrapper logs b, LOO batch mean and max w_ii each step.\n\nOrder of work:\n1. Correctness preflight. With the treatment disabled (group-mean baseline, G=8), the new advantage path must reproduce the stock cookbook advantages and loss/gradient on a fixed batch. With it enabled, it must match the float64 numpy reference within the stated tolerances.\n2. Activation-failure plan. First run offline forced-activation tests: a synthetic batch with prompt clusters and known rewards must yield the analytic baseline. A forced-nonactivation test (identical prompts) must give b equal to the LOO mean. Then run the offline B_eff calibration: 64 tasks x 8 rollouts at max_tokens 512, capped at 10 minutes and kept separate from the test split. If B_eff ratio < 1.03 the gate fails, and the pilot record is kept. The only pre-approved fallback is the post's prompt-length-bucket kernel, scored on the same calibration set, adopted only via a recorded revised spec before any training. The gate is never relaxed.\n3. Timing pilot: 3 steps per arm, plus a 100-item eval.\n4. Full arms.\n5. Eval: score each final sampler checkpoint greedily on all 1319 GSM8K test items with the same grader. Save per-item correctness for the paired analysis.",
  "correctness_preflight": {
    "paths": [
      {
        "baseline_revision": "UNRESOLVED: pin the exact tinker-cookbook commit SHA whose math_rl group-mean advantage is the stock GRPO baseline",
        "id": "gs1_advantage_estimator",
        "kind": "loss",
        "reference_revision": "UNRESOLVED: pin the commit SHA of a standalone float64 numpy reference of the post's TF-IDF kernel baseline (w_ii=0, T=0.1), reviewed against the post text",
        "tolerances": {
          "baseline_gradient": {
            "atol": 1e-05,
            "min_cosine": 0.9999,
            "rtol": 0.001
          },
          "baseline_loss": {
            "atol": 1e-06,
            "rtol": 1e-05
          },
          "reference_gradient": {
            "atol": 1e-05,
            "min_cosine": 0.9999,
            "rtol": 0.001
          },
          "reference_loss": {
            "atol": 1e-06,
            "rtol": 1e-05
          }
        }
      }
    ],
    "required": true,
    "status": "unresolved: revisions must be pinned before any full run"
  },
  "dataset": {
    "hf_path": "openai/gsm8k",
    "n_examples": 7473,
    "name": "gsm8k",
    "split": "main/train (training prompts), main/test (eval)"
  },
  "decision_rule": {
    "alpha": 0.05,
    "criteria": [
      {
        "bounds": [
          0,
          1
        ],
        "id": "primary",
        "margin": -0.05,
        "metric": "accuracy",
        "weights": {
          "baseline": -1,
          "treatment": 1
        }
      }
    ],
    "estimand": "Paired difference in GSM8K test exact-match accuracy, GS1 (TF-IDF, G=1) minus GRPO (G=8), at matched 32 rollouts per step after 60 steps on Qwen3.5-2B LoRA. Larger favors GS1; noninferiority.",
    "method": "paired_hoeffding",
    "seeds": [
      0
    ],
    "unit": "fixed_checkpoint_items",
    "version": 1
  },
  "eval": {
    "aggregation": null,
    "benchmark": "GSM8K test (openai/gsm8k main/test)",
    "metric": "exact-match accuracy (numeric_exact on final answer, greedy, max_tokens 512)",
    "n_examples": 1319
  },
  "experiment_features": {
    "adaptive_selection": false,
    "checkpoint_restore": false,
    "custom_loss": true,
    "custom_optimizer": false,
    "generated_training_data": false,
    "heldout_loss": false,
    "learned_generator": false,
    "paired_comparison": true,
    "paired_training_arms": true,
    "warmup_adaptation": false
  },
  "feasibility_check": {
    "gpu_hours": null,
    "known_gpu_hours": 0,
    "phases": {},
    "problems": [
      "Measured feasibility not recorded"
    ],
    "status": "unknown",
    "wall_seconds": null
  },
  "feasibility_required": true,
  "generation_audit": {
    "answer_comparison": "numeric_exact",
    "max_eval_overlaps": 0,
    "max_label_errors": 0,
    "min_sample_size": 100,
    "required": false
  },
  "hypothesis": "At 32 rollouts per step for 60 steps, GS1 with the post's TF-IDF kernel baseline (group size 1, 32 tasks per step) is within 5 points of GRPO (group size 8, 4 tasks per step) on GSM8K test exact-match accuracy, with a lower confidence bound above -0.05.",
  "lora_rank": 32,
  "max_seq_len": 1024,
  "method": "This is a scaled-down version of the post's AutomationBench GS1-vs-GRPO comparison. The served Qwen/Qwen3.5-2B stands in for the closed Inkling-Small. The policy is a LoRA (rank 32) trained from the base model with AdamW on the shared Spindle; no fork is needed. The cookbook math_rl GSM8K environment, renderer, reward (numeric exact match) and importance-sampling loss are identical in both arms.\n\nGRPO arm (baseline): stock math_rl recipe with group_size=8 and groups_per_batch=4, which gives 32 rollouts per step. Advantage is reward minus the group mean.\n\nGS1 arm (treatment): group_size=1 and groups_per_batch=32, also 32 rollouts per step. The only change is a client-side advantage function. For rollout i the baseline is b_i = sum_{j!=i} w_ij r_j / sum_{j!=i} w_ij, using in-batch rollouts only. The weights are w_ij = exp(cos(x_i, x_j)/0.1), where x is the word TF-IDF vector of the prompt text. The TF-IDF vocabulary and IDF are fit once on the GSM8K train questions before training. The baseline enforces w_ii = 0, and it uses no response, length or reward information from rollout i. Advantage is A_i = r_i - b_i, one scalar per trajectory, with no std normalization. The actual GS1 kernel is withheld, so this is the post's published example kernel (B_eff 1.22x on AutomationBench).\n\nShared settings and deviations from the source:\n- KL penalty is 0 in both arms, so the advantage estimator is the only difference. FlashRF's KL penalty is omitted; this is a recorded deviation.\n- Both arms use the same lr=3e-5, max_tokens=512, 60 steps, the same LoRA seed and the same training-question shuffle.\n\nOut of scope for this spec:\n- FlashRF batch-mean baseline arm: a follow-up spec.\n- Trained-critic/PPO arm: needs a value model, so a separate design.\n- Offline B_eff kernel hillclimb and cross-benchmark kernel transfer: offline work, no Spindle needed.\n\nEvaluation: greedy decoding on the full GSM8K test split (1319 items) from each final sampler checkpoint, paired by item.",
  "optimizer": {
    "lr": 3e-05,
    "name": "adam"
  },
  "preflight": {
    "probe": "Create both training clients: Qwen3.5-2B, LoRA rank 32, AdamW (tinker.AdamParams lr=3e-5, beta1 0.9, beta2 0.95, eps 1e-8). For each, run one sampled rollout batch (G=8x1 task for GRPO; G=1x1 task with a dummy 2-prompt batch for GS1), one forward_backward with the importance-sampling loss and one optim_step. Then run the offline B_eff calibration before either full arm.",
    "riskiest_assumption": "That a TF-IDF prompt-similarity kernel predicts per-question difficulty on GSM8K better than the batch mean. If B_eff gain is near 1.0x, GS1 reduces to FlashRF and the comparison only tests task coverage, not the baseline.",
    "tuning": "No per-arm LR grid. The arms share optimizer and LR and differ only in advantage estimator/grouping, so the same lr=3e-5 is used for both and no hyperparameter is selected on results. The post's LR-robustness claim (stability at a higher LR) is deferred to a follow-up spec with a prespecified two-LR grid per arm, selected on a GSM8K train-holdout split."
  },
  "primary_endpoint": "gsm8k_test_exact_match_accuracy_final_checkpoint",
  "restoration_preflight": {
    "applicable": true,
    "fixed_batch_required": false,
    "groups": [
      {
        "arms": [
          {
            "name": "grpo_g8",
            "optimizer": "adam",
            "optimizer_config": {
              "beta1": 0.9,
              "beta2": 0.95,
              "eps": 1e-08,
              "learning_rate": 3e-05,
              "lora_rank": 32,
              "seed": 0
            }
          },
          {
            "name": "gs1_tfidf_g1",
            "optimizer": "adam",
            "optimizer_config": {
              "beta1": 0.9,
              "beta2": 0.95,
              "eps": 1e-08,
              "learning_rate": 3e-05,
              "lora_rank": 32,
              "seed": 0
            }
          }
        ]
      }
    ],
    "reason": "Both trained arms start from fresh LoRA on Qwen3.5-2B with the same seed. Identity is verified by computing logprobs on a fixed tokenized batch of 4 GSM8K prompts through each new training client before the first update. A mismatch over atol 1e-4 aborts the run. The active AdamW config is checked after creation. No restores occur."
  },
  "spec_preflight": {
    "findings": [],
    "status": "passed"
  },
  "steps": 60,
  "success_criterion": "GS1 is validated as matching GRPO if the simultaneous 95% paired Hoeffding interval on (GS1 accuracy - GRPO accuracy) over the 1319 GSM8K test items has its lower bound above -0.05. It fails if the upper bound is below -0.05. Otherwise the result is inconclusive. This is conditional on the seed-0 checkpoints only.",
  "title": "GS1 (TF-IDF kernel, G=1) vs GRPO (G=8) LoRA RL on GSM8K, Qwen3.5-2B",
  "training_type": "rl"
}

[
  {
    "arm": "shared_init",
    "step": 0,
    "uri": "tinker://4462661df437e68387343d6a9e734865/weights/init_run_1cb90df32a"
  },
  {
    "arm": "grpo_g8",
    "step": 3,
    "uri": "tinker://bec985fc1a9c36c65c7c116372bca1ca:train:0/sampler_weights/grpo_g8_pilot_final_run_1cb90df32a"
  },
  {
    "arm": "gs1_tfidf_g1",
    "step": 3,
    "uri": "tinker://6327d044d9ed7e4a4776a6d93f95cc0a:train:0/sampler_weights/gs1_tfidf_g1_pilot_final_run_1cb90df32a"
  }
]

offline_beff_ratio_tfidf_vs_loo_mean

0.9762

fallback_beff_ratio_lenbucket_vs_loo_mean

0.929

calib_sigma_eps2

0.1275

calib_mse_corrected_loo

0.1622

calib_mse_corrected_tfidf

0.1692

calib_mean_reward

0.543

pilot_eval_base_init_accuracy_100items

0.71

• checkpoint · shared_init · step 0 · tinker://4462661df437e68387343d6a9e734865/weights/init_run_1cb90df32a
• checkpoint · grpo_g8 · step 3 · tinker://bec985fc1a9c36c65c7c116372bca1ca:train:0/sampler_weights/grpo_g8_pilot_final_run_1cb90df32a
• checkpoint · gs1_tfidf_g1 · step 3 · tinker://6327d044d9ed7e4a4776a6d93f95cc0a:train:0/sampler_weights/gs1_tfidf_g1_pilot_final_run_1cb90df32a
• effective_config · effective-config.json
• code_revision · code-revision.txt
• script · gs1_ref.py

1. Nov 4, 17:17 UTC · failed · Correctness preflight and the 3-step pilot of both arms on Spindle (Qwen/Qwen3.5-2B, LoRA rank 32) passed every check except one: the required offline activation gate. On 64 held-out GSM8K train task…
2. Nov 4, 17:13 UTC · running · Correctness preflight passed (stock GRPO advantages/loss reproduced exactly through the new path; GS1 implementation matches float64 reference). Offline B_eff calibration (64 held-out train tasks x 8…

correctness_preflight

{'required': True, 'paths': [{'id': 'gs1_advantage_estimator', 'fixed_input': {'sha256': 'bb1d9ce4cb43887c635218575956a81ddb161c7646fddb7616141c777dcd5d5b', 'description': 'out/preflight_fixed_batch.json: 4 tasks x G=8 (shuffled train idx 6999,7100-7102) and 32 tasks x G=1 (idx 7110-7141) sampled from the shared init sampler; datums from cookbook assemble_training_data; loss evaluated by forward() on a never-updated client loaded from the init checkpoint'}, 'baseline_revision': 'tinker-cookbook f79fdb2ba4cd967ef2807ba43097c52985f9f17f (rl/data_processing.compute_advantages)', 'reference_revision': 'local git 07d384c0ef84f315951ca7b8da45096b60131b2a (gs1_ref.py, sha256 52b4975073eca37c7fe948b3180fe9cb8e8a537d3cd02d23eb4e8ff4ac659411)', 'implementation_revision': 'local git b11a42e5feef97f9b0e2287bd598a3db8af0ab1c (common.py sha256 7c72c0431760b52ca9dc300c1b7df6717e87062c75d98caa28dc694d38a89260)', 'precision': 'advantages float64 then float32 tensors in datums; server forward logprobs (server dtype, bf16 model); local loss/grad float64', 'normalization': 'importance_sampling loss summed over action tokens (prompt tokens advantage 0); gradient = dL/d(target logprob) = -exp(lp-lp_old)*A per token; no std normalisation', 'errors': {'baseline_loss': {'max_abs_error': 0.0, 'reference_max_abs': 571.1539154053435, 'relative_l2_error': 0.0, 'cosine_similarity': 1.0}, 'baseline_gradient': {'max_abs_error': 0.0, 'reference_max_abs': 0.9899091096079926, 'relative_l2_error': 0.0, 'cosine_similarity': 0.9999999999999999}, 'reference_loss': {'max_abs_error': 0.0, 'reference_max_abs': 2582.9993591308594, 'relative_l2_error': 0.0, 'cosine_similarity': 1.0}, 'reference_gradient': {'max_abs_error': 0.0, 'reference_max_abs': 1.0627535169143747, 'relative_l2_error': 0.0, 'cosine_similarity': 1.0}, 'baseline_advantages': {'max_abs_error': 0.0, 'reference_max_abs': 0.875, 'relative_l2_error': 0.0, 'cosine_similarity': 1.0}, 'reference_advantages': {'max_abs_error': 2.3599655785133677e-08, 'reference_max_abs': 0.8700236173102933, 'relative_l2_error': 1.8867106322839825e-08, 'cosine_similarity': 0.9999999999999998}, 'reference_baseline_b': {'max_abs_error': 4.440892098500626e-16, 'reference_max_abs': 0.8490287952017942, 'relative_l2_error': 1.9632553161822528e-16, 'cosine_similarity': 0.9999999999999998}}, 'lr_scaling': []}]}

correctness_validation

{'status': 'unverified', 'required': True, 'checks': [{'path': 'gs1_advantage_estimator', 'check': 'baseline_loss', 'status': 'passed', 'tolerance': {'atol': 1e-06, 'rtol': 1e-05}, 'measured': {'max_abs_error': 0.0, 'reference_max_abs': 571.1539154053435, 'relative_l2_error': 0.0, 'cosine_similarity': 1.0}}, {'path': 'gs1_advantage_estimator', 'check': 'baseline_gradient', 'status': 'passed', 'tolerance': {'atol': 1e-05, 'rtol': 0.001, 'min_cosine': 0.9999}, 'measured': {'max_abs_error': 0.0, 'reference_max_abs': 0.9899091096079926, 'relative_l2_error': 0.0, 'cosine_similarity': 0.9999999999999999}}, {'path': 'gs1_advantage_estimator', 'check': 'reference_loss', 'status': 'passed', 'tolerance': {'atol': 1e-06, 'rtol': 1e-05}, 'measured': {'max_abs_error': 0.0, 'reference_max_abs': 2582.9993591308594, 'relative_l2_error': 0.0, 'cosine_similarity': 1.0}}, {'path': 'gs1_advantage_estimator', 'check': 'reference_gradient', 'status': 'passed', 'tolerance': {'atol': 1e-05, 'rtol': 0.001, 'min_cosine': 0.9999}, 'measured': {'max_abs_error': 0.0, 'reference_max_abs': 1.0627535169143747, 'relative_l2_error': 0.0, 'cosine_similarity': 1.0}}], 'reason': 'gs1_advantage_estimator: matching reference_revision; gs1_advantage_estimator: matching baseline_revision', 'evidence_source': 'executor report; remote execution is not attested', 'applicability': 'required', 'reported_verdict': 'inconclusive'}

decision_validation

{'status': 'uncertain', 'issues': ['Decision evidence was not recorded.'], 'criteria': [], 'outcome_reason': 'insufficient_evidence', 'unit': 'fixed_checkpoint_items', 'n_seeds': 1, 'alpha': 0.05, 'method': 'paired_hoeffding', 'applicability': {'check': 'decision_rule', 'status': 'required', 'enforce': True, 'reasons': ['experiment_features: paired_comparison is true'], 'features': {'paired_comparison': True, 'adaptive_selection': False}}}

evaluation

{'version': 1, 'evaluators': {'0208d3771d1c134e88b49cc239d3cbe37cfb350e0fb28b5baff606fcb743fb01': {'code_sha256': '633fba5069efd467e52e63c396b274cbdf0db6dd71250933c3646320fcfcbc86', 'prompt_template': "cookbook MathEnv standard_fewshot_prefix + question + ' Write your answer in \\\\boxed{} format.'; renderer qwen3_5_disable_thinking", 'decoding': {'temperature': 0, 'max_tokens': 512, 'stop': []}, 'parser': 'cookbook extract_boxed + numeric_exact Fraction(Decimal) equality (eval_gsm8k.py)', 'items': {'ids': ['gsm8k_test_0..gsm8k_test_99 (timing pilot only)']}}}, 'scores': [{'arm': 'base_init_pilot100', 'metric': 'accuracy', 'value': 0.71, 'evaluator_id': '0208d3771d1c134e88b49cc239d3cbe37cfb350e0fb28b5baff606fcb743fb01', 'seed': 0, 'predictions_ref': 'pilot_eval_base100.jsonl'}], 'comparisons': []}

evaluator_validation

{'status': 'invalid', 'issues': ['Evaluator 0208d3771d1c134e88b49cc239d3cbe37cfb350e0fb28b5baff606fcb743fb01: identity does not match provenance hash.'], 'comparisons': [{'metric': 'accuracy', 'status': 'unknown', 'arms': ['base_init_pilot100'], 'evaluator_ids': ['0208d3771d1c134e88b49cc239d3cbe37cfb350e0fb28b5baff606fcb743fb01'], 'missing_arms': []}]}

feasibility

{'status': 'unknown', 'problems': ['Re-projection required: base_model changed or was not recorded', 'Re-projection required: lora_rank changed or was not recorded', 'Re-projection required: dataset changed or was not recorded', 'Re-projection required: steps changed or was not recorded', 'Re-projection required: max_seq_len changed or was not recorded', 'Re-projection required: arms changed or was not recorded', 'Re-projection required: n_examples changed or was not recorded', 'Re-projection required: n_eval changed or was not recorded'], 'wall_seconds': None, 'gpu_hours': None, 'known_gpu_hours': 0, 'phases': {'setup': {'wall_seconds': 49.0, 'planned_units': 1, 'planned_samples': 0.0, 'gpu_hours': None}, 'training': {'wall_seconds': 2648.0, 'planned_units': 120, 'planned_samples': 3840.0, 'gpu_hours': None}, 'evaluation': {'wall_seconds': 238.47519999999997, 'planned_units': 2638, 'planned_samples': 2638.0, 'gpu_hours': None}, 'recovery': {'wall_seconds': 900, 'gpu_hours': None}}, 'generation_counts': None, 'generation_applicability': {'check': 'generation_audit', 'status': 'not_applicable', 'enforce': False, 'reasons': ['spec.generation_audit opts out and no feature it covers is true'], 'features': {'generated_training_data': False}}, 'record': {'config': {'base_model': 'Qwen/Qwen3.5-2B', 'arms': ['grpo_g8', 'gs1_tfidf_g1'], 'steps': 3, 'batch_size': 32, 'max_tokens': 512, 'n_eval': 100}, 'effective_config': {'base_model': 'Qwen/Qwen3.5-2B', 'renderer': 'qwen3_5_disable_thinking', 'lora_rank': 32, 'arms': {'grpo_g8': {'group_size': 8, 'groups_per_batch': 4}, 'gs1_tfidf_g1': {'group_size': 1, 'groups_per_batch': 32}}, 'steps': 60, 'batch_size': 32, 'max_seq_len': 1024, 'max_tokens': 512, 'n_eval': 1319, 'loss_fn': 'importance_sampling', 'lr': 3e-05, 'kl_penalty_coef': 0}, 'primary_endpoint': 'gsm8k_test_exact_match_accuracy_final_checkpoint', 'source': 'pilot_grpo.log, pilot_gs1.log, eval_pilot.log, out/*_pilot/steps.jsonl', 'measurements': {'setup': {'seconds': 49.0, 'units': 1, 'planned_units': 1, 'samples': 0, 'gpu_count': None, 'allocation_source': None}, 'training': {'seconds': 132.4, 'units': 6, 'planned_units': 120, 'samples': 192, 'gpu_count': None, 'allocation_source': None}, 'evaluation': {'seconds': 9.04, 'units': 100, 'planned_units': 2638, 'samples': 100, 'gpu_count': None, 'allocation_source': None}}, 'generation': None, 'recovery': {'seconds': 900, 'gpu_count': None, 'allocation_source': None}}, 'recorded_at': 1793812665.8592808}

feasibility_required

True

generation_audit_validation

{'status': 'not_applicable', 'reasons': ['spec.generation_audit opts out and no feature it covers is true'], 'applicability': {'check': 'generation_audit', 'status': 'not_applicable', 'enforce': False, 'reasons': ['spec.generation_audit opts out and no feature it covers is true'], 'features': {'generated_training_data': False}}}

initialization_provenance

{'version': 1, 'groups': [{'arms': ['grpo_g8', 'gs1_tfidf_g1'], 'shared_checkpoint_uri': 'tinker://4462661df437e68387343d6a9e734865/weights/init_run_1cb90df32a'}], 'arms': {'grpo_g8': {'initialization': {'base_checkpoint': 'Qwen/Qwen3.5-2B', 'source': 'server_response', 'checkpoint_uri': 'tinker://4462661df437e68387343d6a9e734865/weights/init_run_1cb90df32a', 'checkpoint_immutable': True, 'checkpoint_scope': 'full_initialization', 'loaded_checkpoint_uri': 'tinker://4462661df437e68387343d6a9e734865/weights/init_run_1cb90df32a', 'load_succeeded': True, 'adapter_sha256': None, 'evidence_ref': 'out/grpo_g8_pilot/provenance.json (pilot client model_id bec985fc1a9c36c65c7c116372bca1ca)', 'before_first_update': True}, 'restore_count': 0, 'optimizer_checks': [{'after_restore': 0, 'before_next_update': True, 'requested': {'optimizer': 'adam', 'learning_rate': 3e-05, 'beta1': 0.9, 'beta2': 0.95, 'eps': 1e-08}, 'actual_optimizer': 'adamw', 'actual_config': {'get_info.optimizer_config.type': 'adamw', 'lora_rank': 32, 'step_params_sent': {'learning_rate': 3e-05, 'beta1': 0.9, 'beta2': 0.95, 'eps': 1e-08, 'weight_decay': 0.0}}, 'source': 'server_response', 'evidence_ref': 'out/grpo_g8_pilot/provenance.json get_info'}], 'initial_output': {'available': True, 'batch_sha256': '3c4326b9c34490d10bdac125de1c94c36c283fc8f17499331ddd73496d13e649', 'output_sha256': '918151bc2ea331aa0eae71e7d4e426bb8cc98cc35ba50268351155d317e7689e', 'evidence_ref': 'out/grpo_g8_pilot/provenance.json', 'before_first_update': True}}, 'gs1_tfidf_g1': {'initialization': {'base_checkpoint': 'Qwen/Qwen3.5-2B', 'source': 'server_response', 'checkpoint_uri': 'tinker://4462661df437e68387343d6a9e734865/weights/init_run_1cb90df32a', 'checkpoint_immutable': True, 'checkpoint_scope': 'full_initialization', 'loaded_checkpoint_uri': 'tinker://4462661df437e68387343d6a9e734865/weights/init_run_1cb90df32a', 'load_succeeded': True, 'adapter_sha256': None, 'evidence_ref': 'out/gs1_tfidf_g1_pilot/provenance.json (pilot client model_id 6327d044d9ed7e4a4776a6d93f95cc0a)', 'before_first_update': True}, 'restore_count': 0, 'optimizer_checks': [{'after_restore': 0, 'before_next_update': True, 'requested': {'optimizer': 'adam', 'learning_rate': 3e-05, 'beta1': 0.9, 'beta2': 0.95, 'eps': 1e-08}, 'actual_optimizer': 'adamw', 'actual_config': {'get_info.optimizer_config.type': 'adamw', 'lora_rank': 32, 'step_params_sent': {'learning_rate': 3e-05, 'beta1': 0.9, 'beta2': 0.95, 'eps': 1e-08, 'weight_decay': 0.0}}, 'source': 'server_response', 'evidence_ref': 'out/gs1_tfidf_g1_pilot/provenance.json get_info'}], 'initial_output': {'available': True, 'batch_sha256': '3c4326b9c34490d10bdac125de1c94c36c283fc8f17499331ddd73496d13e649', 'output_sha256': '3f9d9df3df09b3d699aff9aa040aeb091fe16a84bfb961227dd4d9eb0ae94c28', 'evidence_ref': 'out/gs1_tfidf_g1_pilot/provenance.json', 'before_first_update': True}}}, 'note': 'save_weights_external returns 404 on Spindle, so no adapter sha256. The two concurrently started pilot clients gave different initial-output hashes on the same fixed batch. A later sequential check (identity_check.py) with two fresh clients loaded from the same checkpoint gave bit-identical logprobs (max abs diff 0.0, sha fe6cc73d...), which suggests nondeterminism from concurrent server batching. The 1e-4 tolerance check on the pilot pair was not re-measured, so this restoration check is unresolved for any future concurrent full run.'}

last_sent_at

1793811550.151879

launch

{'experiment_id': 'exp_2ff3fc4f98', 'context_status': 'snapshotted', 'superseded': False, 'mode': 'ordinary', 'purpose': 'Execute the selected saved experiment version.'}

progress_updated_at

1793812635.211107

reported_outcome_reason

incomplete_execution

reported_verdict

inconclusive

restoration_validation

{'status': 'unverified', 'required': True, 'checks': [{'check': 'grpo_g8', 'status': 'unverified', 'reason': 'grpo_g8: missing or duplicate arm evidence.'}, {'check': 'gs1_tfidf_g1', 'status': 'unverified', 'reason': 'gs1_tfidf_g1: missing or duplicate arm evidence.'}], 'reason': 'grpo_g8: missing or duplicate arm evidence.', 'applicability': 'required'}

slack_started

True

started_at

1793811531.0371869

verdict_report

{'margin': -0.05, 'seeds': 1, 'reasoning': 'The activation gate failed before full training: the TF-IDF kernel baseline predicted per-task reward slightly worse than the LOO batch mean (B_eff ratio 0.976 vs required 1.03), and the pre-approved prompt-length-bucket fallback also failed (0.929). Code defects are unlikely: forced-activation and forced-non-activation tests matched analytic values to 1e-20, and the implementation matched the float64 reference. The cause looks like an inactive regime: across GSM8K prompt pairs, cosine similarity is barely related to difficulty (corr -0.059), and the 64-task estimate is noisy. The paired accuracy comparison was never run, so there is no verdict.'}