Skip to content

chore(site): allow AI content signals#1758

Merged
chaliy merged 1 commit into
mainfrom
2026-05-25-allow-robots-ai
May 25, 2026
Merged

chore(site): allow AI content signals#1758
chaliy merged 1 commit into
mainfrom
2026-05-25-allow-robots-ai

Conversation

@chaliy
Copy link
Copy Markdown
Contributor

@chaliy chaliy commented May 25, 2026

What

Allow bashkit.sh Content Signals for search indexing, AI training, and AI-input/RAG use.

Why

The current robots policy allows search crawlers but opts out of AI training and AI-input use. The site policy should be permissive.

How

  • Set robots.txt Content-Signal values to ai-train=yes, search=yes, ai-input=yes.
  • Update the postbuild verifier to enforce the permissive values.

Risk

  • Low
  • Public site robots metadata changes only. Search and AI consumers that honor Content Signals may now treat the site as allowed for those uses.

Checklist

  • Tests added or updated
  • Backward compatibility considered

Verification:

  • npm run build
  • just test
  • just pre-pr

@cloudflare-workers-and-pages
Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
bashkit eda0302 Commit Preview URL May 25 2026, 03:04 AM

@chaliy chaliy merged commit 90175c8 into main May 25, 2026
19 checks passed
@chaliy chaliy deleted the 2026-05-25-allow-robots-ai branch May 25, 2026 03:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant