Commit Graph

267 Commits

Author SHA1 Message Date
Alexandre Alapetite
7aaed6092f SimplePie replace iframe allow attribute (#6274)
* SimplePie strip iframe allow attribute
https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe#allow

Besides security, the `allow autoplay` atttribute is especially problematic on mobile (Firefox on Android) as it asks to open the YouTube app as soon as the article is opened.

Example of code before:

```html
<iframe data-original="https://www.youtube.com/embed/??????feature=oembed" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen="" sandbox="allow-scripts allow-same-origin"></iframe>
```

* Replace allow attribute

* Allow more
2024-04-11 08:48:50 +02:00
Alexandre Alapetite
e3c86a164d HTTP Get allow UTF-8 even when charset is far from top (#6271)
* HTTP Get allow UTF-8 even when charset is far from top
fix https://github.com/FreshRSS/FreshRSS/issues/5586

The case was an HTML document with 15k whitespace then 1.2k of scripts before the `<meta charset="utf-8">` (far from the 1024 bytes suggested by the spec..., and too far for DOMDocument)

* Rewording

* Trim also vertical tab + comment
2024-04-06 23:02:50 +02:00
Alexandre Alapetite
d0072b9fb7 Refactor some cURL options and use CURLOPT_USERPWD (#6177)
* Refactor some cURL options and use CURLOPT_USERPWD
fix https://github.com/FreshRSS/FreshRSS/issues/6176

* Fixes
2024-03-10 23:04:17 +01:00
Alexandre Alapetite
5e54d5bc58 Reduce API memory consumption (#6137)
`echo json_encode(...)` is very memory demanding for large responses, so optimised.
Contributes to https://github.com/FreshRSS/FreshRSS/issues/6136
https://github.com/FreshRSS/FreshRSS/pull/6013#discussion_r1506779881
2024-03-01 10:08:25 +01:00
Alexandre Alapetite
7d6a64a522 Web scraping support encodings such as EUC-JP (#6112)
* Web scraping support encodings such as EUC-JP
fix https://github.com/FreshRSS/FreshRSS/issues/6106

* Typo
2024-02-18 10:53:44 +01:00
Alexandre Alapetite
06570b30f0 composer update (#6075)
Update PHPStan, fixing some bugs needed for https://github.com/FreshRSS/FreshRSS/pull/6052
(One syntax fix caught by new version)
Update also PHPUnit
2024-01-30 12:57:14 +01:00
Alexandre Alapetite
314077a457 PHPStan prepare exceptions (#6037)
Take advantage of
https://phpstan.org/blog/bring-your-exceptions-under-control

Minimum changes to pass `tooWideThrowType` and `implicitThrows`.

Revert some mistakes from:
https://github.com/FreshRSS/FreshRSS/pull/5504
Preparation needed before new PRs of the same type:
https://github.com/FreshRSS/FreshRSS/pull/5962

Fix several wrong PHPDocs and catches:

> Method ... has ...Exception in PHPDoc @throws tag but it's not thrown.

> Dead catch - ...Exception is never thrown in the try block.
2024-01-15 10:36:30 +01:00
eta-orionis
9c97d8ca72 JSONFeeds, JSON scraping, and POST requests for feeds (#5662)
* allow POST requests for feeds

* added json dotpath and jsonfeed subscriptions. No translation strings yet

* debug and fix jsonfeed parser

* bugfix params saved when editing feed

* added translations for JSON features

* Update docs for web scraping

* make fix-all
and revert unrelated changes, plus a few manual fixes, but there are still several type errors

* Fix some i18n

* refactor json parsing for both feed types

* cleanup unnecessary comment

* refactored generation of SimplePie for XPath and JSON feeds

* Fix merge error

* Update to newer FreshRSS code

* A bit of refactoring

* doc, whitespace

* JSON Feed is in two words

* Add support for array syntax

* Whitespace

* Add OPML export/import

* Work on i18n

* Accept application/feed+json

* Rework POST

* Fix update

* OPML for cURL options

* Fix types

* Fix Typos

---------

Co-authored-by: Erion Elmasllari <elmasllari@factorsixty.com>
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
2024-01-10 08:23:45 +01:00
Alexandre Alapetite
a80a5f48a1 Pass PHPStan level 8 (#5946)
* Pass PHPStan level 8
And prepare for PHPStan level 9 https://phpstan.org/user-guide/rule-levels

* Revert wrong replace in comment

* Fix PHPStan level 8

* Update PHPStan and other dev dependencies

* Remove obsolete comment

* noVariableVariables and towards bleedingEdge
https://github.com/phpstan/phpstan-strict-rules
https://phpstan.org/blog/what-is-bleeding-edge

* More bleedingEdge

* A bit more PHPStan level 9

* More PHPStan level 9

* Prepare for booleansInConditions
Ignore int and null

* Revert wrong line

* More fixes

* Fix keep_max_n_unread

* Stricter attribute functions

* Stricter callHooks and more PHPStan level 9

* More typing

* A tiny more
2023-12-18 17:59:16 +01:00
Alexandre Alapetite
b65ea97901 Fix PHP 7 compatibility strict_types (#5893)
* Fix PHP 7 compatibility
https://github.com/FreshRSS/FreshRSS/discussions/5892

* Multiple PHP 7 fixes

* PHPStan
2023-11-18 23:21:20 +01:00
Luc SANCHEZ
30c7a61a9b Use strict_types (#5830)
* Little's optimisations and booleans in conditions

* Apply strict type

* Apply strict type

* Apply strict type

* Fix multiple bugs with PHP 8.2 and 8.3

* Many declares missing, more errors fixed

* Apply strict type

* Another approach

* Stronger typing for Minz_Session

* Fix case of SQLite

---------

Co-authored-by: Luc <sanchezluc+freshrss@gmail.com>
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
2023-11-16 22:43:00 +01:00
Joe Stump
641b891972 Fix trusted cidrs check (#5853)
* Fix ignored TRUSTED_PROXY issue

* Add a sub-section to the docs no property mappings for Authentik

* Typo

* Fix typing

* A few changes to the doc

---------

Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
2023-11-11 08:40:51 +01:00
Alexandre Alapetite
1c7c1016f4 Fix JSON export/import (#5626)
* Fix import with empty content
fix https://github.com/FreshRSS/FreshRSS/issues/5622
Cherry picks on https://github.com/FreshRSS/FreshRSS/pull/5584

* Fix export of tags / labels
Article-defined tags were wrongly exported as user-defined labels.

* Fix export of tags / labels
Article-defined tags were wrongly exported as user-defined labels.

* Fix bug with many labels

* Better typing

* Comments
2023-09-04 10:09:37 +02:00
Alexandre Alapetite
e7689459f2 Rework trusted proxies (#5549)
* Rework trusted proxies
Fix https://github.com/FreshRSS/FreshRSS/issues/5502
Follow-up of https://github.com/FreshRSS/FreshRSS/pull/3226

New environment variable `TRUSTED_PROXY`: set to 0 to disable, or to a list of trusted IP ranges compatible with https://httpd.apache.org/docs/current/mod/mod_remoteip.html#remoteiptrustedproxy

New internal environment variable `CONN_REMOTE_ADDR` to remember the true IP address of the connection (e.g. last proxy), even when using mod_remoteip.

Current working setups should not observe any significant change.

* Minor whitespace

* Safer trusted sources during install
Rework of https://github.com/FreshRSS/FreshRSS/pull/5358
https://github.com/FreshRSS/FreshRSS/issues/5357

* Minor readme
2023-07-30 12:59:18 +02:00
Luc SANCHEZ
7f9594b8c7 fix many "Only booleans are allowed in an if condition" (#5501)
* fix many "Only booleans are allowed in an if condition"

* Update cli/create-user.php

Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>

* Update cli/i18n/I18nUsageValidator.php

Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>

* Fix several regressions and other minor things

* Fix another regression

* Update lib/http-conditional.php

Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>

---------

Co-authored-by: Luc <sanchezluc+freshrss@gmail.com>
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
2023-07-07 21:53:17 +02:00
Alexandre Alapetite
0292b2f1f3 Improve Dev Container (#5423)
* Improve Dev Container
PHPStan was failing in Dev Container

* Update Docker to Alpine Linux 3.18
* New DATA_PATH environment variable

* README
2023-05-28 18:03:34 +02:00
Alexandre Alapetite
6e2f2f1c1e A few additional PHPStan rules (#5388)
A subset of
https://github.com/phpstan/phpstan-strict-rules
2023-05-11 13:02:04 +02:00
maTh
54c8de86c7 docs: language table added (#5375)
* docs: language table added

* Update 05_Configuration.md

* Update 05_Configuration.md

* french docs

* Unicode quote and a few fixes
(Same search&replace aslo applied to a few other files)

---------

Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
2023-05-07 21:35:08 +02:00
Alexandre Alapetite
30c69ef147 Use PHPStan dynamicConstantNames (#5370)
https://phpstan.org/config-reference#constants
Avoid a few phpstan-ignore-next-line
2023-05-04 19:47:54 +02:00
Alexandre Alapetite
0a38aa7456 Automatic trusted_sources during install (#5358)
* Automatic trusted_sources during install
Fix https://github.com/FreshRSS/FreshRSS/issues/5357

* Fix install for http_auth

* Update lib/lib_rss.php

Co-authored-by: Luc SANCHEZ <4697568+ColonelMoutarde@users.noreply.github.com>

* Fill null exception e-mail

* Revert some syntax

* Minor parentheses
2023-05-03 00:37:04 +02:00
Alexandre Alapetite
5579dc88ab Better XPath failure (#5317)
Report feeds as errored for more cases
2023-04-20 08:21:37 +02:00
Alexandre Alapetite
ecd956c736 PHPStan 7 configureController (#5318)
And fix bugs for saving user queries
2023-04-20 00:55:10 +02:00
Alexandre Alapetite
687d0b40a8 Fix safe_ascii (#5311)
Fix https://github.com/FreshRSS/FreshRSS/issues/5310
2023-04-17 19:53:54 +02:00
Alexandre Alapetite
f3760f138d Complete PHPStan Level 6 (#5305)
* Complete PHPStan Level 6
Fix https://github.com/FreshRSS/FreshRSS/issues/4112
And initiate PHPStan Level 7

* PHPStan Level 6 for tests
* Use phpstan/phpstan-phpunit
* Update to PHPStan version 1.10

* Fix mixed bug

* Fix mixed return bug

* Fix paginator bug

* Fix FreshRSS_UserConfiguration

* A couple more Minz_Configuration bug fixes

* A few trivial PHPStan Level 7 fixes

* A few more simple PHPStan Level 7

* More files passing PHPStan Level 7
Add interface to replace removed class from https://github.com/FreshRSS/FreshRSS/pull/5251

* A few more PHPStan Level 7 preparations

* A few last details
2023-04-17 08:30:21 +02:00
Luc SANCHEZ
5f898dcc5e Modernize Constants and use new constant for string 'currentUser' (#5089)
* Modernize Constants and use new constant 'currentUser'

* Add FreshRSS_Context::currentUser() function and use

* Add FreshRSS_Context::currentUser() function and use

* Add FreshRSS_Context::currentUser() function and use

* Add FreshRSS_Context::currentUser() function and use

* Add FreshRSS_Context::currentUser() function and use

* Update app/Controllers/userController.php

* Update app/Controllers/userController.php

* Update app/Controllers/userController.php

* Update app/Models/Auth.php

* Update p/api/greader.php

* Update p/api/greader.php

* Update p/api/greader.php

* Update app/Models/Context.php

* Update app/Models/LogDAO.php

* Update lib/Minz/Log.php

* Update p/api/greader.php

* Update app/layout/header.phtml

* Update app/views/helpers/export/articles.phtml

* Update cli/do-install.php

* Remarque's from Alkarex

* Remarque's from Alkarex

* Refactor using new Minz_User class

* Consistent naming of public constants

---------

Co-authored-by: Luc <sanchezluc+freshrss@gmail.com>
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
2023-03-26 14:17:22 +02:00
maTh
a7e1428485 defined('CLEANCACHE_HOURS') (#5144)
https://github.com/FreshRSS/FreshRSS/issues/4627
2023-03-04 13:46:46 +01:00
Alexandre Alapetite
05ae1b0d26 XML+XPath (#5076)
* XML+XPath
#fix https://github.com/FreshRSS/FreshRSS/issues/5075
Implementation allowing to take an XML document as input using an XML parser (instead of an HTML parser for HTML+XPath)

* Remove noise from another PR

* Better MIME for XML

* And add glob *.xml for cache cleaning

* Minor syntax

* Add glob json for clean cache
2023-02-09 13:57:20 +01:00
Alexandre Alapetite
4ad66c24bf Workaround disabled openlog syslog (#5054)
* Workaround disabled openlog syslog
#fix https://github.com/FreshRSS/FreshRSS/issues/5053
#fix https://github.com/FreshRSS/FreshRSS/issues/5027

* COPY_SYSLOG_TO_STDERR

* Better return

* Simplify openlog
2023-01-30 20:31:46 +01:00
Alexandre Alapetite
4f316b2ed3 PHPStan level 9 for ./p/ and lib_rss.php (#5049)
And app/FreshRSS.php
Contributes to https://github.com/FreshRSS/FreshRSS/issues/4112
2023-01-29 18:53:51 +01:00
Alexandre Alapetite
07efaf71ea Fix error handling when updating URL (#5039)
Fix 3 related error handling when updating the feed URL with an invalid URL. Previously leading to unclear 500 page with additional PHP errors.
2023-01-26 08:59:34 +01:00
berumuron
daaa391e33 tec: Update the lib_opml (#4403)
* fix: Fix undefined GLOB_BRACE on Alpine

The manual states that:

> Note: The GLOB_BRACE flag is not available on some non GNU systems,
> like Solaris or Alpine Linux.

This generated an error on Alpine.

Reference: https://www.php.net/manual/function.glob.php

* fix: List details of feeds for OPML exportation

The details are necessary to export the XPath information, the CSS full
content path and read actions filters.

* Update LibOpml to 0.4.0

* Refactor OPML importation to be more robust

First, it fixes two regressions introduced by the update of lib_opml:

- title attribute is used when text attribute is missing;
- the OPML category attribute is used as a fallback for feeds categories.

In a related way, if also fixes a problem when a feed had both a parent
category outline and a category attribute. Before, it only considered the
attribute as its category, but now it considers the parent outline.

Then, it counts category limit correctly by not increasing
`$nb_categories` if the category already exists.

* Exclude lib_opml from the CodeSniffer

* Fix variable names when logging some errors

* Fix catch of LibOpml Exception

* Make sure to declare the category

* Exclude lib_opml from PHPStan analyze

* Disable markdownlint for lib_opml

* Fix typos

* Use auto-loading and allow updates via Composer

* Fix broken links to lib_opml

* Bring back the ability to import the OPML frss:opmlUrl attribute

* Refactor the logs of OPML errors

* Update lib_opml to the version 0.5.0

Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
2023-01-18 10:12:21 +01:00
Alexandre Alapetite
075cf4c800 API avoid logging passwords (#5001)
* API avoid logging passwords
* Strip passwords and tokens from API logs
* Only log failed requests information when in debug mode

* Remove debug SHA

* Clean also Apache logs

* Better comments

* Redact also token parameters

* shfmt

* Simplify whitespace

* redacted
2023-01-11 23:27:14 +01:00
Alexandre Alapetite
8864d514c8 NFS-friendly is_writable() checks (#4780)
#fix https://github.com/FreshRSS/FreshRSS/issues/4779
2022-11-14 14:57:45 +01:00
Alexandre Alapetite
97fc0bc95e GLOB_BRACE is not available on all platforms (#4628)
https://php.net/glob

#fix https://github.com/FreshRSS/FreshRSS/issues/4627

Improvement of https://github.com/FreshRSS/FreshRSS/pull/4220
2022-09-20 12:03:46 +02:00
berumuron
6813e16e95 fix: Fix preg_match in enforceHttpEncoding (#4623)
`preg_match` can return `1`, `0` or `false`. In this last case,
`enforceHttpEncoding` was trying to access `$matches[1]`, even if the regex wasn't matching.
2022-09-19 11:50:46 +02:00
maTh
4214954ea1 Improved: error page (#4465)
* error page: true HTML page

* error page: http500 erorr

* error page: add CSP header

* 'log.txt' replaced by LOG_FILENAME

* use ADMIN_LOG

* log.txt => LOG_FILENAME

* error message: add <title>

* Docs created

* delete: documentation on error message page

* line break added

* added: new line at the end

* typo fixed

* Update lib/lib_rss.php

Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>

* Update lib/lib_rss.php

Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>

* Minz HTTP 500

Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
2022-08-21 22:44:03 +02:00
Alexandre Alapetite
509c8cae63 Dynamic OPML (#4407)
* Dynamic OPML draft
#fix https://github.com/FreshRSS/FreshRSS/issues/4191

* Export dynamic OPML
http://opml.org/spec2.opml#1629043127000

* Restart with simpler approach

* Minor revert

* Export dynamic OPML also for single feeds

* Special category type for importing dynamic OPML

* Parameter for excludeMutedFeeds

* Details

* More draft

* i18n

* Fix update

* Draft manual import working

* Working manual refresh

* Draft automatic update

* Working Web refresh + fixes

* Import/export dynamic OPML settings

* Annoying numerous lines in SQL logs

* Fix minor JavaScript error

* Fix auto adding new columns

* Add require

* Add missing 🗲

* Missing space

* Disable adding new feeds to dynamic categories

* Link from import

* i18n typo

* Improve theme icon function

* Fix pink-dark
2022-07-04 09:53:26 +02:00
Alexandre Alapetite
f365a9aeb4 Update all test dependencies (#4419)
* Update all test dependencies

* Remove old false-positive

* Minor update lock files

* Increase PHPStan memory for Fedora
https://github.com/FreshRSS/FreshRSS/pull/4400#issuecomment-1159514197

* Require PHP8+ for tests
Due to small changes of signature in `ob_implicit_flush` and `simplexml_load_string`, cf. https://github.com/FreshRSS/FreshRSS/pull/4123

* Missing lint in CSS files
2022-06-19 20:08:42 +02:00
Alexandre Alapetite
cbf2bc085a Remove HTTP Referer for HTML download (#4372)
Having this header leads to errors from sites such as LinkedIn. The usage of this header is also decreasing. Finaly, we were not sending the actual Referer but just the same URL as the one being fetched.
I suggest to remove for now, and see whether anyone needs it, in which case an option would be possible.
2022-05-19 21:31:04 +02:00
Artur Weigandt
d6fd78b968 Allow CssXPath updates with composer (#4368)
* Install CssXPath with composer

* Fix code style in tests

* fix pathnames for tests and linting

* add irrelevant files to .gitignore

* Alphabetic order

* let composer sort the packages alphabetically

Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
2022-05-16 12:04:43 +02:00
Artur Weigandt
5110d1db3e Update to PHPMailer 6.6.0 with composer (#4329)
* Install PHPMailer 6.5.1 with composer

* Fix folder name for phpmailer

* Update PHPMailer to v6.6.0, add LICENSE and README.md

* remove composer and autoload.php after install/update

* Fix path to PHPMailer for PHPStan

* Remove lib/composer.lock, lock PHPMailer at version 6.6.0

* Improve PSR-4 autoloading for PHPMailer

* Markdownlint ignore

* Delete some obvious comments

* ignores

* Support composer update --no-autoloader

* Automatic cleaning
With rules based on https://github.com/PHPMailer/PHPMailer/blob/master/README.md#minimal-installation

* Readme gitignore hint

Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
2022-05-14 23:38:16 +02:00
Alexandre Alapetite
0b86e347ef Fix lazyimg single quotes (#4330)
Little bug for cases with single quote in URL such as

```html
<img src="123?format='jpg'" />
```

Could probably be replaced by `loading="lazy"` in the future, and with a better DOM-aware method, for instance during SimplePie `add_attributes()`
2022-04-26 08:58:08 +02:00
drosoCode
2aba861bc9 Add HTTP_REMOTE_USER header for auth (#4063)
* add HTTP_REMOTE_USER header for auth

* add ip whitelist for HTTP_REMOTE_USER header

* add IPv6 support for header auth

* fix formatting

* A few fixes

* Add some default trusted sources

* Fix IPv6 doc

* More standard header names

Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
2022-04-02 21:40:30 +02:00
Alexandre Alapetite
efb57f965a Rename conflicting function (#4289)
#fix https://github.com/FreshRSS/FreshRSS/issues/4284
2022-03-24 21:32:37 +01:00
Alexandre Alapetite
92b0ffe05c Fix handling of authors with ampersand & (#4287)
Especially needed for HTML+XPath Web scraping as it makes the rest of the pipeline fail to handle special characters.
2022-03-23 11:29:33 +01:00
Alexandre Alapetite
e8af54a476 WebSub: Use hash instead of base64 to handle long URLs (#4282)
* WebSub: Use hash instead of base64 to handle long URLs

* Use 410 Gone
(Part of the WebSub specification https://www.w3.org/TR/websub/ )
2022-03-22 23:13:13 +01:00
Alexandre Alapetite
4363e13c34 Set feed error state when XPath does not match (#4275)
Gives the ability to filter HTML+XPath feeds that are not working
2022-03-21 12:51:41 +01:00
Alexandre Alapetite
ae54a590b9 Replace lib_phpQuery by PhpGt/CssXPath (#4261)
https://github.com/PhpGt/CssXPath
2022-03-16 15:10:41 +01:00
Alexandre Alapetite
1fe66ad020 Implement Web scraping "HTML + XPath" (#4220)
* More PHP type hints for Fever
Follow-up of https://github.com/FreshRSS/FreshRSS/pull/4201
Related to https://github.com/FreshRSS/FreshRSS/issues/4200

* Detail

* Draft

* Progress

* More draft

* Fix thumbnail PHP type hint
https://github.com/FreshRSS/FreshRSS/issues/4215

* More types

* A bit more

* Refactor FreshRSS_Entry::fromArray

* Progress

* Starts to work

* Categories

* Fonctional

* Layout update

* Fix relative URLs

* Cache system

* Forgotten files

* Remove a debug line

* Automatic form validation of XPath expressions

* data-leave-validation

* Fix reload action

* Simpler examples

* Fix column type for PostgreSQL

* Enforce HTTP encoding

* Readme

* Fix get full content

* target="_blank"

* gitignore

* htmlspecialchars_utf8

* Implement HTML <base>
And fix/revert `xml:base` support in SimplePie e49c578817

* SimplePie upstream PR merged
https://github.com/simplepie/simplepie/pull/723
2022-02-28 20:22:43 +01:00
Alexandre Alapetite
946b0a0876 Hot fix: fix type hint null regression (#4207) 2022-02-07 01:35:29 +01:00