ITADN

Matching a string of "spider" catches too much

#212Openpetdance 创建于 2026-01-27
bughelp wanted
P
petdancecommented
There's an entry in robots.json ```json "Spider": { "operator": "Unclear at this time.", "respect": "Unclear at this time.", "function": "AI Data Scrapers", "frequency": "Unclear at this time.", "description": "Description unavailable from darkvisitors.com More info can be found at https://darkvisitors.com/agents/agents/spider" }, ``` which turns out to turn into an overly restrictive regex match in nginx-block-ai-bots.conf, such that any agent with "spider" will get 403ed. I discovered this when an update from your repo started making some of our automated tests using an agent of "OurCompanyName Test Spider" to fail.
4 条评论