Curation
Every Slack message the scraper sees is scored 0–1 by Sonnet against this rubric. Ideas scoring above the threshold are listed and triaged; the rest are stored but hidden. With no votes, the list is ranked by this score. Manual and agent submissions are scored for ranking but never hidden.
Rubric
Full prompt sent to Sonnet
You curate the Modal Research Exchange. You read one Slack message (with its thread replies) from Modal's
training channels and score how well it fits the team's rubric below. Be calibrated: most Slack messages are
not research ideas and should score low.
<rubric>
{rubric}
</rubric>
Return exactly this JSON object:
{
"score": float, // 0.0 to 1.0, following the rubric
"rationale": str, // one sentence, at most 30 words: why this score
"title": str, // at most 12 words, how a person would say it, often a question
"summary": str, // one plain sentence, at most 30 words: what to try and what it is compared against
"tags": [str] // 1-4 tags from exactly this list: sft, rl, dpo, optimizer, eval, data, infra, paper, model-release, not-training
}
Threshold
0
1
00.250.50.751
| score | draft | post | votes |
|---|
Versions
v2
v1
After triage, Sonnet tags every post with the model it prefers, how flexible it is about the model, and the context length one example needs. These tags decide which Spindle servers with spare LoRA slots a post can use and which it fits best. Edit the instructions below; the output format is fixed.
Instructions
Full prompt sent to Sonnet
You tag research ideas for the Modal Research Exchange with the base model and context length they need, so
ideas can be matched to Spindle training servers with spare capacity. Follow the team's instructions below.
<instructions>
{guide}
</instructions>
Served base models (preferred_model must be exactly one of these):
- Qwen/Qwen3.5-2B
- Qwen/Qwen3.5-4B
- Qwen/Qwen3.5-9B
- Qwen/Qwen3.5-9B-Base
- Qwen/Qwen3-4B
- Qwen/Qwen3-8B
- Qwen/Qwen3.8-27B
- openai/gpt-oss-20b
Return exactly this JSON object:
{
"preferred_model": str, // exactly one served model
"source_model": str | null, // the model the idea or its source names, verbatim, if any
"model_flexibility": "any" | "size_range" | "pinned",
"size_min_b": float | null, // size_range only: smallest acceptable total parameters, billions
"size_max_b": float | null, // size_range only: largest acceptable total parameters, billions
"requirements": [str], // hard needs a model name doesn't show; usually []
"min_context": int, // tokens one example needs, a power of two >= 1024
"rationale": str // one sentence, at most 30 words: why this model and flexibility
}
| tag | draft | post |
|---|