ITADN
ai-robots-txt/ai.robots.txt

版本发布 8

v1.41: adds Brightbot info, meta-webindexer, TerraCotta and DeepSeekBotv1.41
? · 2025-09-28

## What's Changed * Update Brightbot operator and details; add meta-webindexer entry by @karolyi in https://github.com/ai-robots-txt/ai.robots.txt/pull/174 * Add TerraCotta by @emersion in https://github.com/ai-robots-txt/ai.robots.txt/pull/176 * chore(robots.json): add DeepSeekBot by @cdransf in https://github.com/ai-robots-txt/ai.robots.txt/pull/179 ## New Contributors * @karolyi made their first contribution in https://github.com/ai-robots-txt/ai.robots.txt/pull/174 * @emersion made their first contribution in https://github.com/ai-robots-txt/ai.robots.txt/pull/176 **Full Changelog**: https://github.com/ai-robots-txt/ai.robots.txt/compare/v1.40...v1.41

v1.37: Adds additional SemrushBot agents, MyCentralAIScraperBot and Poseidon Research Crawlerv1.37
? · 2025-06-14

## What's Changed * chore(robots.json): adds additional SemrushBot user agents by @cdransf in https://github.com/ai-robots-txt/ai.robots.txt/pull/150 * chore(robots.json): adds MyCentralAIScraperBot by @cdransf in https://github.com/ai-robots-txt/ai.robots.txt/pull/152 * Update robots.json with new crawler by @sergiospagnuolo in https://github.com/ai-robots-txt/ai.robots.txt/pull/153 ## New Contributors * @sergiospagnuolo made their first contribution in https://github.com/ai-robots-txt/ai.robots.txt/pull/153 **Full Changelog**: https://github.com/ai-robots-txt/ai.robots.txt/compare/v1.36...v1.37

v1.36: adds EchoboxBotv1.36
? · 2025-06-10

## What's Changed * chore(robots.json): adds EchoboxBot by @cdransf in https://github.com/ai-robots-txt/ai.robots.txt/pull/148 **Full Changelog**: https://github.com/ai-robots-txt/ai.robots.txt/compare/v1.35...v1.36

v1.34: adds Quillbot and Panscient crawlersv1.34
? · 2025-06-04

## What's Changed * chore(robots.json): adds Quillbot by @cdransf in https://github.com/ai-robots-txt/ai.robots.txt/pull/142 * chore(robots.json): adds Panscient by @cdransf in https://github.com/ai-robots-txt/ai.robots.txt/pull/143 **Full Changelog**: https://github.com/ai-robots-txt/ai.robots.txt/compare/v1.33...v1.34

v1.33: adds SBIntuitionsBotv1.33
? · 2025-06-03

## What's Changed * Add SBIntuitionsBot by @Ivan-Chupin in https://github.com/ai-robots-txt/ai.robots.txt/pull/141 ## New Contributors * @Ivan-Chupin made their first contribution in https://github.com/ai-robots-txt/ai.robots.txt/pull/141 **Full Changelog**: https://github.com/ai-robots-txt/ai.robots.txt/compare/v1.32...v1.33

v1.32: Adds AndiBot, PhindBot, YandexAdditional + YandexAdditionalBotv1.32
? · 2025-06-03

## What's Changed * Add AndiBot and PhindBot by @not-not-the-imp in https://github.com/ai-robots-txt/ai.robots.txt/pull/134 * chore(robots.json): revert "adds imgproxy crawler" by @cdransf in https://github.com/ai-robots-txt/ai.robots.txt/pull/136 * chore(ai_robots_update.yml): correct workflow by revising git flags + adding guard by @cdransf in https://github.com/ai-robots-txt/ai.robots.txt/pull/139 * chore(robots.json): adds YandexAdditional crawlers by @cdransf in https://github.com/ai-robots-txt/ai.robots.txt/pull/140 ## New Contributors * @not-not-the-imp made their first contribution in https://github.com/ai-robots-txt/ai.robots.txt/pull/134 **Full Changelog**: https://github.com/ai-robots-txt/ai.robots.txt/compare/v1.31...v1.32

v1.31: adds new bots, clarifies blocking methods for Bing, updates Claude botsv1.31
? · 2025-05-22

## What's Changed * chore(robots.json): adds Google-CloudVertexBot by @cdransf in https://github.com/ai-robots-txt/ai.robots.txt/pull/129 * Mention X-Robots-Tag header as alternative for bing by @Mihitoko in https://github.com/ai-robots-txt/ai.robots.txt/pull/131 * chore(README): updates the opening line of our README to clarify the types of agents we block by @cdransf in https://github.com/ai-robots-txt/ai.robots.txt/pull/132 * chore(robots.json): adds MistralAI-User/1.0 crawler by @cdransf in https://github.com/ai-robots-txt/ai.robots.txt/pull/126 * Updating Claude Bots by @kylebuckingham in https://github.com/ai-robots-txt/ai.robots.txt/pull/98 * chore(robots.json): adds wpbot by @cdransf in https://github.com/ai-robots-txt/ai.robots.txt/pull/133 ## New Contributors * @Mihitoko made their first contribution in https://github.com/ai-robots-txt/ai.robots.txt/pull/131 * @kylebuckingham made their first contribution in https://github.com/ai-robots-txt/ai.robots.txt/pull/98 **Full Changelog**: https://github.com/ai-robots-txt/ai.robots.txt/compare/v1.30...v1.31

v1.30: Adds HAProxy converter, Caddyfile, Bing AI opt-out instructions, Traefik plugin and QualifiedBot crawlerv1.30
? · 2025-05-15

## What's Changed * Fixed meta-external* being titlecase, and removed period for consistency by @maiavixen in https://github.com/ai-robots-txt/ai.robots.txt/pull/112 * HAProxy converter added. by @rwijnen-um in https://github.com/ai-robots-txt/ai.robots.txt/pull/113 * Added a Caddyfile converter by @Crazyroostereye1 in https://github.com/ai-robots-txt/ai.robots.txt/pull/110 * Fix Python syntax error by @glyn in https://github.com/ai-robots-txt/ai.robots.txt/pull/115 * Bing AI opt-out instructions by @cdransf in https://github.com/ai-robots-txt/ai.robots.txt/pull/119 * Add Traefik plugin to the README.md file by @FlorentPoinsaut in https://github.com/ai-robots-txt/ai.robots.txt/pull/121 * chore(robots.json): adds QualifiedBot crawler by @cdransf in https://github.com/ai-robots-txt/ai.robots.txt/pull/122 * Fix JSON syntax error by @joehoyle in https://github.com/ai-robots-txt/ai.robots.txt/pull/123 * lint robots.json during pull requests by @holysoles in https://github.com/ai-robots-txt/ai.robots.txt/pull/125 ## New Contributors * @maiavixen made their first contribution in https://github.com/ai-robots-txt/ai.robots.txt/pull/112 * @rwijnen-um made their first contribution in https://github.com/ai-robots-txt/ai.robots.txt/pull/113 * @Crazyroostereye1 made their first contribution in https://github.com/ai-robots-txt/ai.robots.txt/pull/110 * @FlorentPoinsaut made their first contribution in https://github.com/ai-robots-txt/ai.robots.txt/pull/121 * @joehoyle made their first contribution in https://github.com/ai-robots-txt/ai.robots.txt/pull/123 * @holysoles made their first contribution in https://github.com/ai-robots-txt/ai.robots.txt/pull/125 **Full Changelog**: https://github.com/ai-robots-txt/ai.robots.txt/compare/v1.29...v1.30