Description
Web robots can check and follow robots.txt rules from Ruby applications. It is dependency support for crawlers, where target permissions, rate limits, and user agents need review.
FICHA · AUR
ruby-webrobots
A Ruby library to help write robots.txt compliant web robots
- Ruby robots.txt library
- LIBRARY
- RUBY
- Web
- Dependency only
official+codex · reviewed · Jun 4, 2026 description in
en Permissions
Permissions not analysed for this source yet.