ITADN

Possible spiders list

#599OpenVahidN 创建于 2024-08-21
V
VahidNcommented
These are the UA's of crawlers which are not detected by this library as spiders. I will continue report them here in this thread. ``` python-requests/2.18.4 Go-http-client/2.0 FeedViewer/1.0 (+http://www.feedviewer.app/license) FeedBurner/1.0 (http://www.FeedBurner.com) Iframely/1.3.1 (+https://iframely.com/docs/about) Atlassian Mozilla/5.0 (compatible; Feedspot/1.0 (+https://www.feedspot.com/fs/fetcher; like FeedFetcher-Google) Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.6533.119 Mobile Safari/537.36 (compatible; GoogleOther) Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot) meta-externalagent/1.1 (+https://developers.facebook.com/docs/sharing/webmasters/crawler) Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:49.0) Gecko/20100101 Firefox/49.0 (FlipboardProxy/1.2; +http://flipboard.com/browserproxy) ```
5 条评论