Skip to content
SEOLow risk

Sitemap Change Loop

Check competitor or reference sitemaps and report new URLs, changed patterns, and possible keyword opportunities.

What this Loop Engineering template does

Detect new and removed URLs across reference sitemaps and summarize patterns and opportunities.

Only read public sitemaps. Respect robots and rate limits. Do not scrape private pages.

When to use it

Competitor monitoring
Content gap discovery
URL pattern tracking

When not to use it

Scraping private content
High-frequency hammering of sites
Guaranteed opportunity claims

Validation checks

validation
New URLs are identified
Removed URLs are identified
URL patterns are grouped
Recommended actions are summarized

Boundaries & stop rule

!Do not scrape private or restricted pages
!Do not overload target websites
!Respect robots and rate limits
Stop rule — Stop when the diff and grouped patterns are reported. If a sitemap is unavailable or restricted, skip it and note why.

Copy the loop prompt

claude-goal.txt
/goal Detect new and removed URLs across reference sitemaps and summarize patterns and opportunities.
 
Task type: Sitemap Monitoring
Target tool: Generic Agent
 
Work toward this goal until all validation checks pass or the stop rule is reached.
 
Design hypothesis:
This sitemap change loop can produce a safer result if the scope stays narrow, validation is explicit, and a checker can reject shortcut work.
 
Smallest useful run:
Run one bounded pass on the newest relevant signal before expanding scope or adding a schedule.
 
Loop cycle:
1. Discovery — Read the latest signal for this template before acting: CI output, issue detail, review comment, dataset report, or content brief.
2. Handoff — Hand the work to one agent in an isolated branch, worktree, or clearly scoped session. Keep final approval with a human.
3. Verification — Use an independent review pass to confirm the result, inspect the diff or artifact, and reject shortcut work.
4. Persistence — Save a short run note with the signal reviewed, actions taken, validation result, and next recommended step.
5. Scheduling — Run manually until the loop is reliable; only then consider a scheduled or event-triggered run.
 
Context:
Only read public sitemaps. Respect robots and rate limits. Do not scrape private pages.
 
Validation:
New URLs are identified
Removed URLs are identified
URL patterns are grouped
Recommended actions are summarized
 
Validation evidence:
Record the original signal, checks run, final result, changed files or artifacts, and any checker rejection.
 
Independent checker:
Use an independent review pass to confirm the result, inspect the diff or artifact, and reject shortcut work.
 
Boundaries:
Do not scrape private or restricted pages
Do not overload target websites
Respect robots and rate limits
 
Stop rule:
Stop when the diff and grouped patterns are reported.
Maximum iterations: 3
 
Budget:
Example only: stop before exceeding the agreed per-run token budget.
 
Human approval:
Required before merge, deploy, delete, purchase, or external communication.
 
Fallback:
If a sitemap is unavailable or restricted, skip it and note why.
 
Loop Validation Log:
- Hypothesis: This sitemap change loop can produce a safer result if the scope stays narrow, validation is explicit, and a checker can reject shortcut work.
- Smallest useful run: Run one bounded pass on the newest relevant signal before expanding scope or adding a schedule.
- Expected evidence: Record the original signal, checks run, final result, changed files or artifacts, and any checker rejection.
- Actual evidence: [fill in after the run]
- Passed? [yes / no / partial]
- Feedback: After the run, note what the loop learned, what failed, and what should change before the next pass.
- Next step: [stop / adjust the loop / run the next pass]
 
Do not delete tests, bypass checks, or modify unrelated files just to satisfy the validation condition. If blocked, stop and summarize the blocker, attempted fixes, and recommended next action.

Failure modes to watch

Scraping restricted pages
Ignoring rate limits
Mislabeled URL patterns
Noisy, unactionable output
Worked scenario review · maintained by TianMingAI · 2026-07-21

Review receipt

This is a bounded review exercise for the template, not a claim about a production deployment.

Scenario

Twenty-three thin routes are consolidated into stronger parent pages and must disappear from discovery without becoming dead ends.

Baseline evidence

The review inventories every old URL, destination, internal link, sitemap entry, generated page, and both trailing-slash forms before the consolidation is released.

Validation result

All retired URLs have direct permanent redirects, none remain in the generated sitemap or internal navigation, retained destinations render, and the page count falls as planned.

Shortcut rejected

Removing URLs from the sitemap alone is rejected because crawlers and saved user links would still reach thin or missing pages.

Human gate

A site owner reviews redirect relevance and approves publication; Search Console removal or resubmission waits until the live redirects and new sitemap are verified.

Final decisionPass

Loop Engineering FAQ

Public sitemaps are intended to be read, but you must still respect robots rules and rate limits, and never fetch private or restricted pages.