* Fix NAT64 feed fetching on IPv6-only hosts
Closes#9371
Regression from https://github.com/FreshRSS/FreshRSS/pull/9195
Changes proposed in this pull request:
* Fix feed fetching on IPv6-only hosts using NAT64.
* Preserve SSRF protection by validating the IPv4 address embedded in RFC 6052 NAT64 addresses before allowing it in `CURLOPT_RESOLVE`.
* Add regression tests for both public and private/reserved NAT64 addresses.
* Minor formating
* Add test tearDown
* RFC8215 64:ff9b:1::/48
Credits: @onurcangnc
---------
Co-authored-by: Viktor Platov <TapuGitHub@users.noreply.github.com>
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
* Fix domain-wide Retry-After
- Fix the domain-wide HTTP `Retry-After` for public servers, which has not applied since #8195 (FreshRSS 1.28.0).
- `getRetryAfterFile()` passed the bare host name to `Minz_Request::serverIsPublic()`. That function expects a URL, so it always returned `false`.
- So a 429 or 503 paused only the URL that got it, despite the log saying "For that domain, will first retry after …", and FreshRSS went on requesting the other feeds of that server. With several Reddit feeds, for instance, each one got its own 429 in turn.
- `serverIsPublic()` now gets the URL. Its answer is kept per host for the rest of the request, because it resolves the host name (`dns_get_record()`), and a refresh may load many feeds of the same host.
- Fix the per-URL `Retry-After` of servers on a local network (#8195), which never applied to feeds whose URL, as stored, differs from the URL as requested.
- Stored URLs are HTML-encoded, e.g. `&` in an RSS-Bridge URL, and may end with `#force_feed`.
- `FreshRSS_Feed::load()` checked the stored URL, while the pause is stored for the URL as requested. It now checks the requested URL, as `loadJson()` and `loadHtmlXpath()` already do.
* Minor final
---------
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
TRUSTED_PROXY=192.168.1.1 is ignored and only 192.168.1.1/32 is honoured. checkCIDR() splits the range on the slash and rejects the value when the mask part is missing, so an address written without a subnet never matches.
An address without a subnet is now read as a single host: /32 for IPv4 and /128 for IPv6. That is the format docs/en/admins/09_AccessControl.md points at.
Host names such as gateway.localdomain are still not resolved.
tests/app/Utils/httpUtilTest.php gains 16 cases for checkCIDR, two of which fail without this change.
Refs #9210
* Dev updates: composer, hadolint, typos...
Composer compatibility PHP 8.6.
Manual updates not supported by Dependabot
* Fix hadolint
* Update lib/Minz/Url.php
Co-authored-by: Frans de Jonge <fransdejonge@gmail.com>
* Add sha256sum for composer
---------
Co-authored-by: Frans de Jonge <fransdejonge@gmail.com>
* Block CGNAT (RFC6598) and NAT64 (RFC6052) ranges in SSRF guard
getCurlResolveInfo() relies on PHP FILTER_FLAG_NO_PRIV_RANGE plus an explicit
PRIVATE_SUBNETS list. PHP's filter does not cover CGNAT 100.64.0.0/10
(RFC6598, used by Tailscale and some corporate/ISP networks) nor NAT64
64:ff9b::/96 (RFC6052, which maps IPv4 addresses incl. 169.254.169.254),
and neither range was in PRIVATE_SUBNETS, so an authenticated user could
still SSRF to those ranges (e.g. Tailscale peers, or NAT64-mapped cloud
metadata). Add both ranges to PRIVATE_SUBNETS.
Repro: filter_var('100.64.0.1', FILTER_VALIDATE_IP, FILTER_FLAG_NO_PRIV_RANGE
| FILTER_FLAG_NO_RES_RANGE) returns the IP (allowed); same for
'64:ff9b::a9fe:a9fe'.
Ref: GHSA-hcv2-vrhw-mjq8
* Minor change comments
---------
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
* WebSub: ignore http/https scheme difference in Self URL comparison
The PubSubHubbub endpoint logged 'Self URL does not match registered
canonical URL' whenever a feed's self URL differed from the registered
canonical URL only by http vs https (common with http->https redirects),
spamming the log. Compare the URLs ignoring the http/https scheme while
still requiring host/path/query to match exactly.
Closes#3087
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Stylistic preferences
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
Two "uses cache" LOG_DEBUG syslog() calls were emitted on every feed
refresh regardless of the simplepie_syslog_enabled setting, so users who
disabled it still saw "FreshRSS SimplePie uses cache for ..." spam in
syslog (issue #8540). Guard both with simplepie_syslog_enabled, like the
neighbouring SimplePie GET logs already are. Real error and fatal logs
are left untouched.
For https://github.com/FreshRSS/FreshRSS/issues/8540
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* Add SSRF mitigations using `filter_var` and `CURLOPT_RESOLVE`
The idea is to prevent FreshRSS from sending any HTTP requests to internal services, except for the ones that are explicitly allowed in the config.
Based on https://github.com/moodle/moodle/blob/6e82b46a480826d1a85394d9e5087f7d82d1dd52/lib/filelib.php#L3818 and https://github.com/symfony/symfony/blob/8.1/src/Symfony/Component/HttpClient/NoPrivateNetworkHttpClient.phphttps://github.com/FreshRSS/simplepie/pull/76https://github.com/FreshRSS/simplepie/pull/78
* Add allowlist setting in Web UI
* make readme
* Update app/i18n/fr/admin.php
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
* make readme again
* make readme
* Further work
Still WIP and needs testing etc.
* Readd previous if check for domain combination allowlist
* Turn POST to GET after redirect
* Improve
* Update config.default.php
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
* make readme
* Skip SSRF check if `CURLOPT_PROXY` is set
* make readme
* Fix `!empty()` mistake
* Respect max redirects feed option when fetching with `httpGet()`
* Respect max redirects during SimplePie fetching + fix bypass
bypass fix: `CURLOPT_FOLLOWLOCATION` was moved below so that emulated redirects are enforced.
* Avoid FreshRSS and Minz code in SimplePie
https://github.com/FreshRSS/FreshRSS/pull/8400#discussion_r2935375980
* Corrected hook code
* phpdoc wrong return type
* Add CIDR support in allowlist
* Implement simple DNS caching
* Suppress `dns_get_record()` warnings
* A bit of proof-reading
* Minor typo
* Fix proxy logic
* Fix HTTP POST redirect logic
* Proofread checkCIDR
Add fixes for several situations
* Remove credentials from URL in logs
* Ensure `CURLOPT_FOLLOWLOCATION` is `false` by setting it at the end
* Fix codesniffer long line
* Fix potential bypass due to wrong return value
If there were no records returned by `dns_get_record()`, no overrides to `CURLOPT_RESOLVE` would get passed,
and a potential bypass could occur, when cURL would try to resolve the domain by itself.
* Put the URL at the end in logs
* Add documentation and environment variable support
* make readme
* Fix wrong behavior in case of IP
* Fix duplicate selector in CSS
* Minor type check change
* i18n fr, en
* Minor type check change
* Fix whitespace i18n fr
* make fix-all
* Fix `$ips_ok` not being returned after domain records were cached
* make readme
* PHPStan fix
* make readme
* Minor syntax in SimplePie
* Only return `null` if no allowed IPs were found
* Add wildcard *, help message
* Consistent docs with help message
* i18n: pl
* SimplePie compatibility PHP 7.2
* make fix-all
* Sync SimplePie
* https://github.com/FreshRSS/simplepie/pull/76
* 💥 Breaking change in the Changelog
* Document `INTERNAL_HOST_ALLOWLIST` in Docker docs
* Remove `Cookie` and `Authorization` headers in `httpGet()` during cross-origin redirect
* Minor whitespace
And same comment convention than below
* Remove authentication headers and change POST to GET on redirect in SimplePie
* Remove .local in Docker example
* Fill in default ports when comparing URL origins
* Remove .local from other places than the Docker example
* Rewrite WebSub subscribe to use `httpGet()`
* make fix-all
* Also unset `CURLOPT_USERPWD` during redirects
* phpcs fix
* Always unset `CURLOPT_FOLLOWLOCATION`
* Bump SimplePie
https://github.com/FreshRSS/simplepie/pull/78
* Update logic for CURLOPT_FOLLOWLOCATION
* Fix PHPStan
* Changelog fix security section
* Update most common RSS Bridge case
https://hub.docker.com/r/rssbridge/rss-bridge
* Replace misleading 127.0.0.1:8080 example for Docker
This does not make sense for a Docker container
---------
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
* Exclude local networks for domain-wide Retry-After
Retry-After will be applied by URL and not by domain for local networks.
fix https://github.com/FreshRSS/FreshRSS/issues/7880
* Improved logic for detection of local domains
* Support ip6-localhost and a couple more variants
* On more: .lan
* Resolve IP address
* Add .intranet
* Housekeeping lib_rss.php
`lib_rss.php` had become much too large, especially after https://github.com/FreshRSS/FreshRSS/pull/7924
Moved most functions to other places.
Mostly no change of code otherwise (see comments).
* Extension: composer run-script phpstan-third-party
* Create separate `Retry-After` files for proxies
Bad proxies are able to send a false `Retry-After` header and affect the availability of feeds (domain-wide) for other users.
This PR starts including the address of the proxy if present in filenames for `Retry-After` to mitigate the issue.
* Reduce code changes
* Sync SimplePie fork
https://github.com/FreshRSS/simplepie/pull/62
---------
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
* Implement support for HTTP 429 Too Many Requests
Will obey the corresponding HTTP `Retry-After` header at domain level.
* Implement 503 Service Unavailable
* Sanitize Retry-After
* Reduce default value when Retry-After is absent
And make configuration parameter
* Retry-After also for favicons
Alexandre Alapetite