Skip to content
Lab Notes/2026-07-21

From a Vague CI Request to a Bounded Goal

Can the site's real goal generator turn a broad request to fix CI into a reviewable artifact with evidence, boundaries, and exits?

Maintained by TianMingAI · reviewed 2026-07-21

We generate one Codex goal from the site's CI preset, then inspect whether the output preserves the original test, limits retries, names validation evidence, and stops before merge.

Setup

The input is the built-in CI Fix preset with Codex selected, three maximum iterations, a repeated-failure stop, an example token cap, and human approval enabled.

Method

  1. 1Generate the artifact with the same generateCodexGoal function used by the public goal generator.
  2. 2Search the output for the original-test boundary, maximum iterations, validation evidence, fallback, and human approval.
  3. 3Treat any missing boundary or merge permission as a failed review rather than repairing the output by hand.

Actual artifact

codex-goal.txt
/goal Complete the following objective with a clear validation loop:
 
Task type: CI Fix
Target tool: Codex
 
Objective:
Fix the failing CI checks for this pull request with the smallest safe change.
 
Design hypothesis:
A small, isolated CI fix loop can repair this pull request faster than a broad manual investigation, as long as validation and review stay separate.
 
Smallest useful run:
Run one pass against the latest failing CI job only. Do not expand to unrelated lint warnings, refactors, or cleanup work.
 
Loop cycle:
- Discovery: Read the latest CI failure, related pull request comments, and recent commits before choosing the next action.
- Handoff: Assign the work to one coding agent in an isolated branch or worktree. Keep the final merge decision with a human reviewer.
- Verification: A separate reviewer checks the diff, confirms validation results, and rejects shortcuts such as deleting tests or weakening checks.
- Persistence: Write a short run note with the error seen, files changed, checks run, and the next recommended action.
- Scheduling: Run manually for each failing pull request. Move to a scheduled check only after the loop is reliable.
 
Context:
Prefer small focused changes. Reproduce the current failure and read existing project patterns before editing.
 
Done when:
Run the exact failing command first, then run the project's complete required checks.
 
Validation evidence:
Capture the failing command before the fix, the passing command after the fix, and a short note explaining what changed.
 
Independent checker:
A separate reviewer checks the diff, confirms validation results, and rejects shortcuts such as deleting tests or weakening checks.
 
Constraints:
Do not delete tests.
Do not bypass lint or type checks.
Do not modify unrelated files.
Do not merge without human approval.
 
Stop conditions:
- Stop when all validation checks pass.
- Stop when all validation commands pass. Stop earlier when the same failure repeats without new evidence.
- Stop after 3 failed iterations.
- Stop before exceeding this budget: Example only: three iterations maximum and a fixed per-run token cap agreed before execution.
 
Human approval:
Required before merge, deploy, delete, purchase, or external communication.
 
Fallback:
If blocked, summarize the current errors, attempted fixes, and recommended human decision.
 
Loop Validation Log:
- Hypothesis: A small, isolated CI fix loop can repair this pull request faster than a broad manual investigation, as long as validation and review stay separate.
- Smallest useful run: Run one pass against the latest failing CI job only. Do not expand to unrelated lint warnings, refactors, or cleanup work.
- Expected evidence: Capture the failing command before the fix, the passing command after the fix, and a short note explaining what changed.
- Actual evidence: [fill in after the run]
- Passed? [yes / no / partial]
- Feedback: After the run, note whether the hypothesis held, what slowed the loop down, and what should change before the next pass.
- Next step: [stop / adjust the loop / run the next pass]

Observed result

  • The generated artifact explicitly forbids deleting tests and modifying unrelated files.
  • It contains a three-iteration stop, a separate repeated-failure stop, a fallback report, and human approval before merge.
  • The artifact is a specification only; it does not execute code or prove that an external agent will follow every instruction.

What this does not prove

  • This checks the site's deterministic output, not the behavior of a live Codex run.
  • A real repository still needs its own commands, permissions, baseline failure, and independent diff review.

Reproduce it

Open the same site tool