# Record
**Author:** @cameron.stream (`did:plc:gfrmhdmjvxn2sjedzboeudef`)

## `knowledge-tinker-curriculum-task-set-01`
**Collection:** `site.standard.document`
**AT URI:** `at://did:plc:gfrmhdmjvxn2sjedzboeudef/site.standard.document/knowledge-tinker-curriculum-task-set-01`

**Title:** Flat JSON transformation (Tinker curriculum task set 1)
**Published:** Sun, 26 Jul 2026 03:56:24 GMT
**Updated:** Sun, 26 Jul 2026 04:17:00 GMT
**Description:** The first Tinker curriculum task set: generating Python functions that transform flat records, evaluated by a narrow AST interpreter against hidden cases.
**Publication:** `at://did:plc:gfrmhdmjvxn2sjedzboeudef/site.standard.publication/3mr4py6clps2f`
**Path:** /tinker-curriculum/task-set-01
**Tags:** knowledge, project, ai, machine-learning, tinker, curriculum-learning, code-generation

**Content:**
````json
{
  "text": "Flat JSON transformation is the first task set in the [Tinker curriculum](https://cameron.stream/knowledge/tinker-curriculum). The model receives a short natural-language instruction and must emit exactly one Python function that transforms a flat input record into a new flat dictionary. The generated code is never executed as arbitrary Python; a narrow AST interpreter accepts only the task-set grammar and evaluates it against hidden test cases.\n\n## Task structure\n\nEach task presents a mapping instruction such as \"copy required field `id` unchanged\" or \"set `region` from optional input field `area`; if it is missing, use literal `unknown`.\" The model must produce a function with this signature:\n\n```python\ndef transform(record):\n    return {\n        \"customer_id\": record[\"id\"],\n        \"region\": record.get(\"region\", \"unknown\"),\n        \"schema_version\": 1,\n    }\n```\n\nThe task set stays inside one flat-record transformation family while combining eight operations:\n\n- **Copy** a required field unchanged.\n- **Rename** a required field to a new destination key.\n- **Copy an optional field** with a literal default, using `record.get(key, default)`.\n- **Add a literal field** with a constant value.\n- **Normalize** a string by trimming whitespace and applying `lower`, `upper`, or `title` case.\n- **Concatenate** two trimmed fields with a requested separator.\n- **Apply a rounded affine formula** of the form `round(value * multiplier + offset, 2)`.\n- **Choose a literal** from a numeric threshold rule, using a conditional expression `value if record[key] >= threshold else alternative`.\n\nThe eight operations are combined into eight patterns of increasing compositionality, from two-field tasks to four-field tasks that mix string, numeric, and conditional logic.\n\n## Evaluation\n\nThe evaluator parses the generated code with Python's `ast` module and enforces a strict grammar contract:\n\n- exactly one function named `transform` with one parameter named `record`;\n- the function body is a single `return` of a dictionary literal;\n- dictionary keys are string literals and must be unique;\n- values may be: string or numeric constants, `record[key]` subscripts, `record.get(key, default)` calls, `.strip().lower()` / `.upper()` / `.title()` method chains, `+` concatenation, `*` multiplication, `round(expr, 2)` calls, and `value if record[key] >= threshold else alternative` conditionals;\n- no imports, decorators, annotations, comprehensions, loops, or arbitrary function calls are permitted.\n\nIf the code passes the grammar check, the AST interpreter evaluates it against five hidden cases per task. A task passes only if all five cases produce the exact expected output. This means success is behavioral: the generated transformation must produce correct results, not match the reference implementation's text.\n\nThe evaluator boundary is intentionally narrower than Python itself:\n\n```python\ntree = ast.parse(generated_code)\ntransform = validate_restricted_transform(tree)\n\nfor case in hidden_cases:\n    assert interpret(transform, case.input) == case.expected\n```\n\n`validate_restricted_transform` rejects every syntax form outside the task grammar before the interpreter sees a case. The evaluator never calls `exec` or imports generated code.\n\n## Corpus\n\nThe deterministic corpus contains 96 training tasks, 32 development-validation tasks, and 32 sealed promotion tasks. Field vocabularies are disjoint across splits—training, validation, and test use entirely different source and destination field names—so memorizing training field names cannot help on held-out splits. The dataset is generated from fixed seeds and hash-checked for reproducibility.\n\n## Training and results\n\nThe base model is [Qwen3.5-35B-A3B-Base](https://huggingface.co/Qwen/Qwen3.5-35B-A3B-Base), a raw 35B-parameter mixture-of-experts model with approximately 3B active parameters per token. The choice of a raw base model—rather than an instruction-tuned model that might already solve the task—is deliberate: the point is to teach a compact active model a task through supervised fine-tuning.\n\nAn initial projection-only draft was rejected before training. Both a smaller 4B model and the raw 35B base scored 24 out of 24 on the simpler initial task set, indicating the task was too easy to measure improvement. The task set was then expanded within the same flat-transformation family to require compositional string, numeric, and conditional behavior. The rejected draft remains only as an ignored baseline receipt, not as a second task set.\n\nA rank-16 [LoRA](https://arxiv.org/abs/2106.09685) adapter was trained on task set 1 only. The promotion contract required at least 85% task accuracy and at least five percentage points improvement over the base model on development validation before the sealed test could be opened.\n\n**Task\\-set\\-1 accuracy.** Task accuracy for the untouched base model and rank\\-16 adapter on development and the sealed promotion set\\.\n\n| Measure | Base model | Trained adapter | Change |\n| --- | --- | --- | --- |\n| Development | 18\\.75% | 100% | 81\\.25% |\n| Sealed test | 12\\.5% | 100% | 87\\.5% |\n\n*Source: Task\\-set\\-1 machine\\-readable evaluation and lineage receipts; 32 tasks and 160 hidden cases per split\\. Note: The sealed set was opened only after the trained adapter passed the 85% development threshold and five\\-point improvement gate\\.*\n\n| Split | Model | Tasks passed | Hidden cases passed | Valid code |\n|---|---|---:|---:|---:|\n| Development | Base | 6/32 (18.75%) | 30/160 (18.75%) | 6/32 |\n| Development | Trained | 32/32 (100%) | 160/160 (100%) | 32/32 |\n| Sealed test | Base | 4/32 (12.5%) | 20/160 (12.5%) | 4/32 |\n| Sealed test | Trained | 32/32 (100%) | 160/160 (100%) | 32/32 |\n\nThe trained adapter passed the promotion gate with an 81.25 percentage-point improvement on development validation. After the gate passed, the sealed test was opened, and the trained adapter achieved 100% on both tasks and hidden cases.\n\n## Promotion gate\n\nThe promotion contract for task set 1 is:\n\n1. Evaluate the untouched base model on development validation.\n2. Train one LoRA candidate on task set 1 only.\n3. Compare the candidate with the base on the same development set.\n4. Open the sealed promotion set only if the candidate reaches at least 85% task accuracy and improves by at least five percentage points.\n5. Preserve task set 1's frozen promotion set when task set 2 is eventually added.\n\nThe gate passed. The frozen test split is preserved for all future task-set additions.\n\n## Receipts\n\nThe training and evaluation results are recorded as machine-readable JSON receipts. The lineage receipt records the base model, dataset hash, hyperparameters, checkpoint references, validation scores, and test scores. The evaluation receipt records every prompt, model output, semantic score, and usage estimate. Dataset hashes are preserved for reproducibility.\n\nThe task-set-1 dataset SHA-256 is `60fb4f205777f7414db175d76b05538cba9fd784b771d836b7060130dd89fc65`. The Tinker Cookbook commit is `3e04119ce293a2b6ba5284e35267c9ba6d27c5da`.\n\n## Evidence limits\n\nTask set 1 is the cleaner of the two task sets. The task is deterministic, the evaluator is mechanical, and the dataset is reproducible from fixed seeds. The main limitation is that the task family is narrow: it tests whether a model can learn to emit a restricted Python grammar that produces correct flat-record transformations, not whether it can write general-purpose code. The 100% test score should not be generalized beyond this grammar.\n\n## Sources\n\n- [Tinker](<https://tinker-docs.thinkingmachines.ai/>)\n- [Qwen3\\.5\\-35B\\-A3B\\-Base on Hugging Face](<https://huggingface.co/Qwen/Qwen3.5-35B-A3B-Base>)\n- [LoRA — Low\\-Rank Adaptation of Large Language Models](<https://arxiv.org/abs/2106.09685>)",
  "$type": "site.standard.content.markdown",
  "version": "1.0"
}
````

---
*Fetched from https://enoki.us-east.host.bsky.network via `com.atproto.repo.getRecord`*