* Bump to API level 4 for Fever; add with_ids so we can mass-change read/unread/saved/unsaved on lists of articles.
As discussed in https://github.com/FreshRSS/FreshRSS/issues/8305
A small enhancement to the Fever API to vastly increase efficiency when bulk marking items.
-
How to test the feature manually:
The client at:
https://sources.vsta.org:7100/FeedMonkey/file?name=js/Fever.js&ci=tip
has code to talk to this API enhancement (when the API level says it's supported). In particular walkArticles().
* Update indentation per PR check on Github
* make fix-all
* Line length
* Fix many typing issues
* is_numeric
* Update comment
---------
Co-authored-by: Andy Valencia <ajv-899-334-8894@vsta.org>
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
* Add new visibility priority *Show in its feed*
fix https://github.com/FreshRSS/FreshRSS/pull/7970#issuecomment-3293917428 (you can't directly filter a hidden feed, it just shows a 404 page)
And add a new visibility *Show in its feed* to show the feed in the list but not its articles.
Ensure that visibility *hidden* is not shown to API.
* TODO for later
* Update app/i18n/pl/sub.php
Co-authored-by: Inverle <inverle@proton.me>
1. `include`, `include_once`, `require` and `require_once` are expressions not functions, parentheses are not necessary.
2. to move up the directory tree, it's better to use the `dirname` function instead of relying on `/..`.
* Puts CSP everywhere in `p/api`
* including the HTML query page ❗
* Also in `p/ext.php`
* Puts `X-Content-Type-Options: nosniff` everywhere
* Fixes custom icon configuration not showing `blob:` icon in statsController (idle feeds)
* Also removes `style-src 'unsafe-inline'` since it doesn't seem to be needed
* Improves CSP of `p/f.php`
* Add `sandbox` directive
* Fix favicon hashing in GReader API (#7570)
This allows the correct iconUrl to be returned from the GReader API for
a given feed.
* Fix method signature
* Fix Fever API
---------
Co-authored-by: CarelessCaution <189675655+CarelessCaution@users.noreply.github.com>
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
Add option to sort results by received date (existing, default), publication date, title, URL (link), random.
fix https://github.com/FreshRSS/FreshRSS/issues/1771
fix https://github.com/FreshRSS/FreshRSS/issues/2083
fix https://github.com/FreshRSS/FreshRSS/issues/2119
fix https://github.com/FreshRSS/FreshRSS/issues/2596
fix https://github.com/FreshRSS/FreshRSS/issues/3204
fix https://github.com/FreshRSS/FreshRSS/issues/4405
fix https://github.com/FreshRSS/FreshRSS/issues/5529
fix https://github.com/FreshRSS/FreshRSS/issues/5864
fix https://github.com/FreshRSS/Extensions/issues/161
URL parameters:
* `&sort=id` (current behaviour, sorting according to newest received articles)
* `&sort=date` (publication date, which is not indicative of how new an article is)
* `&sort=title`
* `&sort=link`
* `&sort=rand` (random order - which disables infinite scrolling, at least for now)
combined with `&order=ASC` or `&order=DESC`
## Implementation notes
The sorting criteria by *received date* (id), which is the default, and which was the only one before this PR, is the one that has the best sorting characteristics:
* *uniqueness*: no entries have the exact same received date
* *monotonicity*: new entries always have a higher received date
* *performance*: this field is efficiently indexed in database for fast usage, including for paging (indexing could also be done to other fields, but with lower effective performance)
In contrary, sorting criteria such as by *publication date*, by *title*, or by *link* are neither unique nor monotonic. In particular, multiple articles may share the same *publication date*, and we may receive articles with a *publication date* far in the future, and then later some new articles with a *publication date* far in the past.
To understand why sorting by *publication date* is problematic, it helps to think about sorting by *title* or by *link*, as sorting by *title* and by *publication date* share more or less the same characteristics.
### Problem 1: new articles
New articles may be received in the background after what is shown on screen, and before the next user action such as *mark all as read*. Due to the lack of *monotonicity* when sorting by e.g. *publication date* or *title*, users risk marking as read a batch of articles containing some fresh articles without seeing them.
Mitigation: A parameter `idMax` tracks the maximum ID related to a batch of actions such as *mark all as read* to exclude articles received after those that are displayed.
### Problem 2: paging / pagination
When navigating articles, only a few articles are displayed, and a new "page" of articles needs to be received from the database when scrolling down or when clicking the button to show more articles. When sorting by e.g. *publication date* or *title*, it is not trivial to show the next page without re-showing some of the same articles, and without skipping any. Indeed, views are often with additional criteria such as showing only unread articles, and users may mark some articles as read while viewing them, hereby removing some articles from the previous pages. And like for *Problem 1*, new articles may have been received in the background. Consequently, it is not possible to use `OFFSET` to implement pagination (so the patches suggested by a few users were wrong due to that, in particular).
Mitigation: `idMax` is also used (just like for *Problem 1*) and a *Keyset Pagination* approach is used, combining an unstable sorting criterion such as *publication date* or *title*, together with *id* to ensure stable sorting. (So, 2 sorting criteria + 1 filter criteria)
See e.g. https://www.alwaysdeveloping.net/dailydrop/2022/07/01-keyset-pagination/
### Problem 3: performance
Sorting by anything else than *received date* (id) is doomed to be slow(er) due to the combination of 3 criteria (see *Problem 2*). An `OFFSET` approach (which is not possible anyway as explained) would be even slower. Furthermore, we have no SQL index at the moment, but they would not necessarily help much due to the multiple sorting criteria needed and involving some `OR` logic which is difficult to optimise for databases.
The nicest syntax would be using tuples and corresponding indexes, but that is poorly supported by MySQL https://bugs.mysql.com/bug.php?id=104128
Mitigation: a compatibility SQL syntax is used to implement *Keyset Pagination*
### Problem 4: user confusion
Several users have shown that they do not fully understand the difference between *received date* and *publication date*, and particularly not the pitfalls of *publication date*.
Mitigation: the menus to mark-as-read *before 1 day* and *before 1 week* are disabled when sorting by anything else than *received date*. Likewise, the separation headers *Today* and *Yesterday* and *Before yesterday* are only shown when sorting by *received date*.
Again here, to better understand why, it helps to think about sorting by *title* or by *link*, as sorting by *title* and by *publication date* share more or less the same characteristics.
* [ ] We should write a Q&A and/or documentation about the problems associated to *sorting by publication date*: risks of not noticing new publication, of inadvertently marking them as read, of having some articles with a date in the future hanging at the top of the views (vice versa when sorting in ascending order), performance, etc.
### Problem 5: APIs
Sorting by anything else than *received date* breaks the guarantees needed for a successful synchronisation via API.
Mitigation: sorting by *received date* is ensured for all API calls.
* PHPStan 2.0
fix https://github.com/FreshRSS/FreshRSS/issues/6989https://github.com/phpstan/phpstan/releases/tag/2.0.0https://github.com/phpstan/phpstan/blob/2.0.x/UPGRADING.md
* More
* More
* Done
* fix i18n CLI
* Restore a PHPStan Next test
For work towards PHPStan Level 10
* 4 more on Level 10
* fix getTagsForEntry
* API at Level 10
* More Level 10
* Finish Minz at Level 10
* Finish CLI at Level 10
* Finish Controllers at Level 10
* More Level 10
* More
* Pass bleedingEdge
* Clean PHPStan options and add TODOs
* Level 10 for main config
* More
* Consitency array vs. list
* Sanitize themes get_infos
* Simplify TagDAO->getTagsForEntries()
* Finish reportAnyTypeWideningInVarTag
* Prepare checkBenevolentUnionTypes and checkImplicitMixed
* Fixes
* Refix
* Another fix
* Casing of __METHOD__ constant
before
count(...)' is used in a loop and is a low performing construction.
after
Foreach instead (easier to read and support)
Co-authored-by: LucS <l.sanchez-prestataire@alptis.fr>
* Minor update whitespace PHPCS rules
To simplify our configuration, apply more rules, and be clearer about what is added or removed compared with PSR12.
Does not change our current conventions, but just a bit more consistent.
* Forgotten *.phtml
* Sort exclusion patterns + add a few for Extensions repo
* Relaxed some rules
* Pass PHPStan level 8
And prepare for PHPStan level 9 https://phpstan.org/user-guide/rule-levels
* Revert wrong replace in comment
* Fix PHPStan level 8
* Update PHPStan and other dev dependencies
* Remove obsolete comment
* noVariableVariables and towards bleedingEdge
https://github.com/phpstan/phpstan-strict-ruleshttps://phpstan.org/blog/what-is-bleeding-edge
* More bleedingEdge
* A bit more PHPStan level 9
* More PHPStan level 9
* Prepare for booleansInConditions
Ignore int and null
* Revert wrong line
* More fixes
* Fix keep_max_n_unread
* Stricter attribute functions
* Stricter callHooks and more PHPStan level 9
* More typing
* A tiny more
* Little's optimisations and booleans in conditions
* Apply strict type
* Apply strict type
* Apply strict type
* Fix multiple bugs with PHP 8.2 and 8.3
* Many declares missing, more errors fixed
* Apply strict type
* Another approach
* Stronger typing for Minz_Session
* Fix case of SQLite
---------
Co-authored-by: Luc <sanchezluc+freshrss@gmail.com>
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
* Complete PHPStan Level 6
Fix https://github.com/FreshRSS/FreshRSS/issues/4112
And initiate PHPStan Level 7
* PHPStan Level 6 for tests
* Use phpstan/phpstan-phpunit
* Update to PHPStan version 1.10
* Fix mixed bug
* Fix mixed return bug
* Fix paginator bug
* Fix FreshRSS_UserConfiguration
* A couple more Minz_Configuration bug fixes
* A few trivial PHPStan Level 7 fixes
* A few more simple PHPStan Level 7
* More files passing PHPStan Level 7
Add interface to replace removed class from https://github.com/FreshRSS/FreshRSS/pull/5251
* A few more PHPStan Level 7 preparations
* A few last details
* Modernize Constants and use new constant 'currentUser'
* Add FreshRSS_Context::currentUser() function and use
* Add FreshRSS_Context::currentUser() function and use
* Add FreshRSS_Context::currentUser() function and use
* Add FreshRSS_Context::currentUser() function and use
* Add FreshRSS_Context::currentUser() function and use
* Update app/Controllers/userController.php
* Update app/Controllers/userController.php
* Update app/Controllers/userController.php
* Update app/Models/Auth.php
* Update p/api/greader.php
* Update p/api/greader.php
* Update p/api/greader.php
* Update app/Models/Context.php
* Update app/Models/LogDAO.php
* Update lib/Minz/Log.php
* Update p/api/greader.php
* Update app/layout/header.phtml
* Update app/views/helpers/export/articles.phtml
* Update cli/do-install.php
* Remarque's from Alkarex
* Remarque's from Alkarex
* Refactor using new Minz_User class
* Consistent naming of public constants
---------
Co-authored-by: Luc <sanchezluc+freshrss@gmail.com>
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
* Remove FreshRSS_Searchable for better types
The interface was not used, and it was preventing more precise types for the different `searchById()` methods, as they each have different input and output types.
* Consistent entry ID
Entry IDs (which are 64-bit integers) must be processed as string to be compatible with 32-bit platforms
* Fix type
* A few more related types
* PHPStan level 6
* Some more casts needed
* String cast for htmlspecialchars
* Remove FreshRSS_Searchable for better types
The interface was not used, and it was preventing more precise types for the different `searchById()` methods, as they each have different input and output types.
* Fix type
#fix https://github.com/FreshRSS/FreshRSS/issues/5183
And set protected methods as private since there is no subclass, and in order to get warnings for unused methods, which would have spotted the bug.
* API avoid logging passwords
* Strip passwords and tokens from API logs
* Only log failed requests information when in debug mode
* Remove debug SHA
* Clean also Apache logs
* Better comments
* Redact also token parameters
* shfmt
* Simplify whitespace
* redacted
* Refactor entry to GReader API format
Some code was copied in two locations and not completely uniform.
Cleaning of related variables and functions (e.g. better types for entries and categories as objects vs. as IDs).
Usecase: I need to call the same GReader-compatible serialization from an extension
* Fixed some edge cases
* Keep summary instead of content
`summary` and `content` seems to be used interchangeably in the Google Reader API. We have been using `summary` for our client API and `content` in our export/import, so stick to that.
* New possibility to invoke user queries from a search expression
From the search field: `S:"My query"`.
Can be combined with other filters such as `S:"My query" date:P3d` as long as the user queries do not contain `OR`.
A use-case is to have an RSS filter with a stable address or an external API call with the ability to update the user query.
* Draft of parenthesis logic
* More draft
* Working parenthesis (a OR b) (c OR d)
* Working (A) OR (B)
* Support nested parentheses + unit tests + documentation
* search:MySearch and S:3
* More PHP type hints for Fever
Follow-up of https://github.com/FreshRSS/FreshRSS/pull/4201
Related to https://github.com/FreshRSS/FreshRSS/issues/4200
* Detail
* Draft
* Progress
* More draft
* Fix thumbnail PHP type hint
https://github.com/FreshRSS/FreshRSS/issues/4215
* More types
* A bit more
* Refactor FreshRSS_Entry::fromArray
* Progress
* Starts to work
* Categories
* Fonctional
* Layout update
* Fix relative URLs
* Cache system
* Forgotten files
* Remove a debug line
* Automatic form validation of XPath expressions
* data-leave-validation
* Fix reload action
* Simpler examples
* Fix column type for PostgreSQL
* Enforce HTTP encoding
* Readme
* Fix get full content
* target="_blank"
* gitignore
* htmlspecialchars_utf8
* Implement HTML <base>
And fix/revert `xml:base` support in SimplePie e49c578817
* SimplePie upstream PR merged
https://github.com/simplepie/simplepie/pull/723
* Update Travis line length
* Also check whitespace in CSS files
* Fix line length ext.php
* More syntax, string templates
* Fix exclude-pattern
* Test JS files as well
* Avoid manual intialisations of system or user configuration
More consistent use of Context
* Simplify FreshRSS_Context::initUser
* Remove a few manual get_user_configuration
* A bit of debugging
* Fix context user init
* Fix install
* Fix concurrency
Concurrent requests could lead to bad race condition
* Fix actualize cron
Fix case when system i initialised several times
* Minz allow parallel sessions
#fix https://github.com/FreshRSS/FreshRSS/issues/3093
* Array optimisation
* Array optimisation missing
* Reduce direct access to $_SESSION except in install process
* Fix session start headers warning
* Use cookie only the first time the session is started:
`PHP Warning: session_start(): Cannot start session when headers
already sent in /var/www/FreshRSS/lib/Minz/Session.php on line 39`
* New concept of volatile session for API calls
Optimisation: do not use cookies or local storage at all for API calls
without a Web session
Fix warning:
```
PHP Warning: session_destroy(): Trying to destroy uninitialized session
in Unknown on line 0
```
* Only call Minz_Session::init once in our index
It was called twice (once indirectly via FreshRSS->init())
* Whitespace
* Mutex for notifications
Implement mutex for notifications
https://github.com/FreshRSS/FreshRSS/pull/3208#discussion_r499509809
* Typo
* Install script is not ready for using Minz_Session
* Fever integer type for feed ID
#Fix
https://github.com/FreshRSS/FreshRSS/issues/2940#issuecomment-623022435
* Fix feed_id
* Ensure string for entry ID
To be compatible with 32-bit platforms.
Before, the type was inconsistent depending on architecture + database
* Integer for entry count functions
* PDO refactor
* Automatic prefix when using the syntax `_tableName`
* Uniformity: MySQL is now PDO::ATTR_EMULATE_PREPARES = false just like SQLite and PostgreSQL, with consequences such as only one statement per query
* Use PDO methods exec(), query(), prepare() + execute() in a more efficient way
* Remove auto-update SQL code for versions older than FreshRSS 1.5 (3 years old)
* The name of the default category is set in PHP instead of in the DB (simplies SQL and allows changing the name according to the FreshRSS language)
* Rename `->bd` to `->pdo` (less of a frenshism, and more informative)
* Fix some requests, which were not compatible with MySQL prepared statements
* Whitespace
* Fix syntax for PostgreSQL sequences
+ MySQL install
* Minor formatting
* Fix lastInsertId for PostgreSQL
* Use PHP 5.6+ const
Take advantage of https://github.com/FreshRSS/FreshRSS/pull/2527https://www.php.net/manual/en/migration56.new-features.php
* A bit of forgotten PHP 5.6 simplification for cURL
* Forgotten $s
* Mini fix custom user config
https://github.com/FreshRSS/FreshRSS/pull/2490/files#r326290346
* More work on install.php but not finished
* install.php working
* More cleaning of PDO in install
* Even more simplification
Take advantage of PDO->exec() to run multiple statements
* Disallow changing the name of the default category
https://github.com/FreshRSS/FreshRSS/pull/2522#discussion_r326967724
