Loop Engineering Lab Notes
Small, reproducible tests of the site's own generators and review rules. Each note publishes its setup, method, actual artifact, observed result, and limits.
What these notes can and cannot show
Every artifact below is produced by code that ships with this site or by a short, visible checker rule. The notes can demonstrate what the site generates and how a bounded review responds. They do not claim that an external model, provider, or production repository will behave the same way. Those environments need their own evidence.
From a Vague CI Request to a Bounded Goal
Can the site's real goal generator turn a broad request to fix CI into a reviewable artifact with evidence, boundaries, and exits?
We generate one Codex goal from the site's CI preset, then inspect whether the output preserves the original test, limits retries, names validation evidence, and stops before merge.
Open lab noteHow Retry Limits Change an Example Token Budget
How much does the maximum iteration setting change the same loop's example monthly cost when every other input stays fixed?
We run the site's deterministic budget formula four times with identical token, price, frequency, and retry assumptions, changing only maximum iterations from one to ten.
Open lab noteA Maker-Checker Review That Rejects Green CI
Will an explicit checker stop a polished CI proposal when the maker removes the evidence needed to judge success?
We evaluate a deliberately flawed maker artifact that proposes skipping the failing test and merging after aggregate CI turns green.
Open lab noteRepeat the same question before changing the rule.
- 01
Fix the input
Record the preset, values, example prices, or flawed artifact before evaluating the result.
- 02
Name the check
State the observable condition that would pass, fail, or stop the exercise without rewriting it afterward.
- 03
Save the artifact
Keep the generated output, calculation, or rule result that a second reader needs to inspect.
- 04
Publish the limit
Separate what the small exercise observed from claims about external agents, providers, and production systems.
Read a lab note as a bounded receipt, not a benchmark.
Strong enough to inspect
A reader can see the input, method, actual result, and limitation, then repeat the same site-side exercise.
Too weak to generalize
One deterministic example cannot prove that a live model will follow instructions, that a price is current, or that a production change is safe.
Reason to update
Revise a note when its generator, formula, source, or checker changes. Keep the old conclusion only if the recorded reproduction still supports it.
Use a note to answer a narrow question such as “does the exported goal retain the human merge gate?” or “how does the example formula react to a larger iteration cap?” Do not use it to claim that one agent is reliable in every repository. For that, repeat the method in the real environment, preserve the baseline and final evidence, and let the responsible person review permissions, quality, and side effects.