- Configure Allow2Ban in Rack::Attack to ban IPs requesting >500 pages per day for 1 week (7 days).
- Add honeypot route /dont-crawl-me disallowed in robots.txt and configure Fail2Ban in Rack::Attack to ban IPs visiting it for 7 days upon 1 hit.
- Update PhotosController#index to raise ActiveRecord::RecordNotFound when page parameter is out of bounds, returning 404 Not Found to crawlers instead of 200 OK.
- Add request and controller specs for Rack::Attack rules and pagination 404 responses.
Co-authored-by: CloCkWeRX <365751+CloCkWeRX@users.noreply.github.com>
New route to /robots.txt and some specific logic to switch
on if the subdomain is staging or not.
Staging will not allow any crawlers at all, production does not
have any current limitations
