Skip to content
SEOLow risk

Sitemap Change Loop

Check competitor or reference sitemaps and report new URLs, changed patterns, and possible keyword opportunities.

What this Loop Engineering template does

Detect new and removed URLs across reference sitemaps and summarize patterns and opportunities.

Only read public sitemaps. Respect robots and rate limits. Do not scrape private pages.

When to use it

Competitor monitoring
Content gap discovery
URL pattern tracking

When not to use it

Scraping private content
High-frequency hammering of sites
Guaranteed opportunity claims

Validation checks

validation
New URLs are identified
Removed URLs are identified
URL patterns are grouped
Recommended actions are summarized

Boundaries & stop rule

!Do not scrape private or restricted pages
!Do not overload target websites
!Respect robots and rate limits
Stop rule — Stop when the diff and grouped patterns are reported. If a sitemap is unavailable or restricted, skip it and note why.

Copy the loop prompt

claude-goal.txt
/goal Detect new and removed URLs across reference sitemaps and summarize patterns and opportunities.
 
Work toward this goal until all validation checks pass or the stop rule is reached.
 
Loop cycle:
1. Discovery — Read the latest signal for this template before acting: CI output, issue detail, review comment, dataset report, or content brief.
2. Handoff — Hand the work to one agent in an isolated branch, worktree, or clearly scoped session. Keep final approval with a human.
3. Verification — Use an independent review pass to confirm the result, inspect the diff or artifact, and reject shortcut work.
4. Persistence — Save a short run note with the signal reviewed, actions taken, validation result, and next recommended step.
5. Scheduling — Run manually until the loop is reliable; only then consider a scheduled or event-triggered run.
 
Context:
Only read public sitemaps. Respect robots and rate limits. Do not scrape private pages.
 
Validation:
New URLs are identified
Removed URLs are identified
URL patterns are grouped
Recommended actions are summarized
 
Independent checker:
Use an independent review pass to confirm the result, inspect the diff or artifact, and reject shortcut work.
 
Boundaries:
Do not scrape private or restricted pages
Do not overload target websites
Respect robots and rate limits
 
Stop rule:
Stop when the diff and grouped patterns are reported.
Maximum iterations: 3
 
Budget:
Stop before exceeding the agreed per-run token budget.
 
Human approval:
Required before merge, deploy, delete, purchase, or external communication.
 
Fallback:
If a sitemap is unavailable or restricted, skip it and note why.
 
Do not delete tests, bypass checks, or modify unrelated files just to satisfy the validation condition. If blocked, stop and summarize the blocker, attempted fixes, and recommended next action.

Failure modes to watch

Scraping restricted pages
Ignoring rate limits
Mislabeled URL patterns
Noisy, unactionable output

Loop Engineering FAQ

Public sitemaps are intended to be read, but you must still respect robots rules and rate limits, and never fetch private or restricted pages.