SEOLow risk
Sitemap Change Loop
Check competitor or reference sitemaps and report new URLs, changed patterns, and possible keyword opportunities.
What this Loop Engineering template does
Detect new and removed URLs across reference sitemaps and summarize patterns and opportunities.
Only read public sitemaps. Respect robots and rate limits. Do not scrape private pages.
When to use it
Competitor monitoring
Content gap discovery
URL pattern tracking
When not to use it
Scraping private content
High-frequency hammering of sites
Guaranteed opportunity claims
Validation checks
validation
✓New URLs are identified
✓Removed URLs are identified
✓URL patterns are grouped
✓Recommended actions are summarized
Boundaries & stop rule
!Do not scrape private or restricted pages
!Do not overload target websites
!Respect robots and rate limits
Stop rule — Stop when the diff and grouped patterns are reported. If a sitemap is unavailable or restricted, skip it and note why.
Copy the loop prompt
claude-goal.txt
/goal Detect new and removed URLs across reference sitemaps and summarize patterns and opportunities.Work toward this goal until all validation checks pass or the stop rule is reached.Loop cycle:1. Discovery — Read the latest signal for this template before acting: CI output, issue detail, review comment, dataset report, or content brief.2. Handoff — Hand the work to one agent in an isolated branch, worktree, or clearly scoped session. Keep final approval with a human.3. Verification — Use an independent review pass to confirm the result, inspect the diff or artifact, and reject shortcut work.4. Persistence — Save a short run note with the signal reviewed, actions taken, validation result, and next recommended step.5. Scheduling — Run manually until the loop is reliable; only then consider a scheduled or event-triggered run.Context:Only read public sitemaps. Respect robots and rate limits. Do not scrape private pages.Validation:New URLs are identifiedRemoved URLs are identifiedURL patterns are groupedRecommended actions are summarizedIndependent checker:Use an independent review pass to confirm the result, inspect the diff or artifact, and reject shortcut work.Boundaries:Do not scrape private or restricted pagesDo not overload target websitesRespect robots and rate limitsStop rule:Stop when the diff and grouped patterns are reported.Maximum iterations: 3Budget:Stop before exceeding the agreed per-run token budget.Human approval:Required before merge, deploy, delete, purchase, or external communication.Fallback:If a sitemap is unavailable or restricted, skip it and note why.Do not delete tests, bypass checks, or modify unrelated files just to satisfy the validation condition. If blocked, stop and summarize the blocker, attempted fixes, and recommended next action.
Failure modes to watch
Scraping restricted pages
Ignoring rate limits
Mislabeled URL patterns
Noisy, unactionable output
Loop Engineering FAQ
Public sitemaps are intended to be read, but you must still respect robots rules and rate limits, and never fetch private or restricted pages.