Releases: ai-robots-txt/ai.robots.txt
Releases · ai-robots-txt/ai.robots.txt
v1.20: adds PanguBot
What's Changed
- Clarify our rationale by @glyn in #54
- Create workflow to upload
robots.txt
file as release artefact by @norwd in #55 - Allow facebookexternalhit by @glyn in #56
- Restore attribution by @fabianegli in #58
New Contributors
Full Changelog: v1.19...v1.20
v1.19: adds DuckAssistBot, improves automation
What's Changed
- Update robots.json with Claude respect link by @lxjv in #43
- Sort the content of robots.json by keys, case-insensitively by @isagalaev in #46
- Update the FAQ by @glyn in #45
- log the diff in the update actions by @fabianegli in #47
- PHP diagnostics by @glyn in #49
- Fix typo and trigger rerun of main job by @glyn in #50
- PHP to Python plus tests and stuff by @fabianegli in #51
New Contributors
- @lxjv made their first contribution in #43
- @isagalaev made their first contribution in #46
- @fabianegli made their first contribution in #47
Full Changelog: v1.18...v1.19
v1.18: adds ISSCyberRiskCrawler
Adds ISSCyberRiskCrawler. Thanks to Brian at Skewray Research for calling this one out!
v1.17: add Sidetrade indexer bot + other updates
What's Changed
- Add Sidetrade indexer bot (Thank you to Brian at Skewray Research for calling this one out.)
- Add instructions for AI bot blocking on Vercel by @cramforce in #36
- Update README.md by @cityrolr in #41
- feat: make CCBot entry more accurate by @wumpus in #42
- Add an additional resource - README.md by @urvish-p80 in #38
New Contributors
- @cramforce made their first contribution in #36
- @cityrolr made their first contribution in #41
- @wumpus made their first contribution in #42
- @urvish-p80 made their first contribution in #38
Full Changelog: v1.16...v1.17
v1.16: adds iaskspider/2.0
Thank you to Brian at Skewray Research for calling this one out.
What's Changed
- Improve main workflow by @nisbet-hubbard in #35
Full Changelog: v1.15...v1.16
v1.15: adds AI2Bot
Thank you to Brian at Skewray Research for calling this one out.
Full Changelog: v1.14...v1.15
v1.14: adds Webzio-Extended
What's Changed
- Adds the
Webzio-Extended
bot - only build on changes to robots.json by @jbowdre in #29
- Add FAQ by @glyn in #31
- Tracking Dark Visitors Automatically by @ChenghaoMou in #32
- Add links for reporting and FAQ to README.md by @nisbet-hubbard in #33
New Contributors
- @jbowdre made their first contribution in #29
- @ChenghaoMou made their first contribution in #32
Full Changelog: v1.13...v1.14
v1.13: corrects agent name
v1.12: adds ICC-Crawler
What's Changed
- Add last line of defence to FAQ by @nisbet-hubbard in #24
- Add ICC-Crawler by @mirium999 in #25
New Contributors
- @mirium999 made their first contribution in #25
Full Changelog: v1.11...v1.12
v1.11: adds PetalBot and facebookexternalhit
What's Changed
- Add
PetalBot
(andfacebookexternalhit
?) by @nisbet-hubbard in #22
Full Changelog: v1.10...v1.11