The generated data (owasp_rules.json, waf_patterns/**) is derived from
third-party projects but was implicitly covered by the blanket MIT LICENSE
with no NOTICE. Add the missing attribution — additively and reversibly:
- THIRD_PARTY_NOTICES.md (new): per-source attribution
- OWASP CoreRuleSet -> Apache-2.0 (owasp_rules.json + waf_patterns/**)
- JayBizzle/Crawler-Detect -> MIT (bad-bot list)
- mitchellkrogza/...bad-bot -> MIT (bad-bot list)
- matomo/referrer-spam-... -> Public Domain (referrer spam)
- LICENSES/Apache-2.0.txt (new): canonical license copy (Apache-2.0 §4a).
- README: License section now scopes MIT to the original project code; the
generated data is redistributed under its upstream licenses. Resources now
credit the real bad-bot sources (removed the unused ai.robots.txt mention).
- Converters (owasp2json, json2nginx, json2apache, json2traefik, json2haproxy)
emit a provenance header on every generated file (Apache-2.0 §4b "state
changes"); owasp_rules.json gains a top-level _provenance key. Loaders accept
both the {_provenance, rules} object and the legacy bare-array form.
- owasp_rules.json wrapped as {_provenance, rules} (rule content unchanged).
- update_patterns.yml jq updated for the object shape.
LICENSE stays MIT (GitHub detection unaffected). Propagation of the headers to
waf_patterns/** is left to the daily update_patterns workflow.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
feat: Implement OWASP CRS to HAProxy WAF conversion with enhanced features
This commit introduces significant improvements to the script for converting OWASP Core Rule Set (CRS) rules into HAProxy Web Application Firewall (WAF) configurations.
Key changes include:
- **Expanded Operator Mapping:** Added more comprehensive mappings between ModSecurity operators and HAProxy equivalents, improving the translation of OWASP rules.
- **Location-Based ACLs:** Implemented support for inspecting different request parameters (User-Agent, Request-URI, Host, etc.) based on the `location` field in the JSON rules, increasing the WAF's coverage.
- **Rule Prioritization:** Introduced rule prioritization based on severity (high, medium, low), allowing for different actions (deny, log, tarpit) to be triggered based on the assessed risk.
- **Improved Regex Handling:** Enhanced regex validation to identify and skip overly complex or invalid patterns, preventing performance issues and potential errors.
- **Clearer ACL Logic:** Restructured the generated `waf.acl` file for better organization, separating ACL definitions from deny logic and grouping rules by request parameter location.
- **Detailed Logging:** Improved logging to provide more specific information about skipped rules, invalid patterns, and other issues, aiding in debugging and configuration.
- **Integer Comparison:** Added capability to use http-request to perform integer comparison instead of strings in the rules.
These enhancements result in a more effective, maintainable, and configurable HAProxy WAF implementation based on the OWASP CRS.
Please note that thorough testing and tuning are still crucial to ensure the WAF is working correctly and not causing false positives.
This commit addresses the following issues:
- Addresses overly aggressive rules causing false positives.
- Implements missing support for ModSecurity operators.
- Enables inspection of request parameters beyond the User-Agent header.
- Provides a more organized and maintainable HAProxy WAF configuration.
Changes made for optimization.
1. Added `functools.lru_cache` decorator to cache results of `validate_regex` function calls. This ensures that repeated validations of the same pattern are resolved quickly and avoid redundant regex compilations.
