fix https://github.com/FreshRSS/FreshRSS/issues/7887
We have two concepts: how much a feed is shown or not (controlled by priority), and how often a feed is refreshed (or not, in which case it is archived).
This PR removes the wording *Archived* from the *visibility* parameter, since this is not what it does.
Add option to sort results by received date (existing, default), publication date, title, URL (link), random.
fix https://github.com/FreshRSS/FreshRSS/issues/1771
fix https://github.com/FreshRSS/FreshRSS/issues/2083
fix https://github.com/FreshRSS/FreshRSS/issues/2119
fix https://github.com/FreshRSS/FreshRSS/issues/2596
fix https://github.com/FreshRSS/FreshRSS/issues/3204
fix https://github.com/FreshRSS/FreshRSS/issues/4405
fix https://github.com/FreshRSS/FreshRSS/issues/5529
fix https://github.com/FreshRSS/FreshRSS/issues/5864
fix https://github.com/FreshRSS/Extensions/issues/161
URL parameters:
* `&sort=id` (current behaviour, sorting according to newest received articles)
* `&sort=date` (publication date, which is not indicative of how new an article is)
* `&sort=title`
* `&sort=link`
* `&sort=rand` (random order - which disables infinite scrolling, at least for now)
combined with `&order=ASC` or `&order=DESC`
## Implementation notes
The sorting criteria by *received date* (id), which is the default, and which was the only one before this PR, is the one that has the best sorting characteristics:
* *uniqueness*: no entries have the exact same received date
* *monotonicity*: new entries always have a higher received date
* *performance*: this field is efficiently indexed in database for fast usage, including for paging (indexing could also be done to other fields, but with lower effective performance)
In contrary, sorting criteria such as by *publication date*, by *title*, or by *link* are neither unique nor monotonic. In particular, multiple articles may share the same *publication date*, and we may receive articles with a *publication date* far in the future, and then later some new articles with a *publication date* far in the past.
To understand why sorting by *publication date* is problematic, it helps to think about sorting by *title* or by *link*, as sorting by *title* and by *publication date* share more or less the same characteristics.
### Problem 1: new articles
New articles may be received in the background after what is shown on screen, and before the next user action such as *mark all as read*. Due to the lack of *monotonicity* when sorting by e.g. *publication date* or *title*, users risk marking as read a batch of articles containing some fresh articles without seeing them.
Mitigation: A parameter `idMax` tracks the maximum ID related to a batch of actions such as *mark all as read* to exclude articles received after those that are displayed.
### Problem 2: paging / pagination
When navigating articles, only a few articles are displayed, and a new "page" of articles needs to be received from the database when scrolling down or when clicking the button to show more articles. When sorting by e.g. *publication date* or *title*, it is not trivial to show the next page without re-showing some of the same articles, and without skipping any. Indeed, views are often with additional criteria such as showing only unread articles, and users may mark some articles as read while viewing them, hereby removing some articles from the previous pages. And like for *Problem 1*, new articles may have been received in the background. Consequently, it is not possible to use `OFFSET` to implement pagination (so the patches suggested by a few users were wrong due to that, in particular).
Mitigation: `idMax` is also used (just like for *Problem 1*) and a *Keyset Pagination* approach is used, combining an unstable sorting criterion such as *publication date* or *title*, together with *id* to ensure stable sorting. (So, 2 sorting criteria + 1 filter criteria)
See e.g. https://www.alwaysdeveloping.net/dailydrop/2022/07/01-keyset-pagination/
### Problem 3: performance
Sorting by anything else than *received date* (id) is doomed to be slow(er) due to the combination of 3 criteria (see *Problem 2*). An `OFFSET` approach (which is not possible anyway as explained) would be even slower. Furthermore, we have no SQL index at the moment, but they would not necessarily help much due to the multiple sorting criteria needed and involving some `OR` logic which is difficult to optimise for databases.
The nicest syntax would be using tuples and corresponding indexes, but that is poorly supported by MySQL https://bugs.mysql.com/bug.php?id=104128
Mitigation: a compatibility SQL syntax is used to implement *Keyset Pagination*
### Problem 4: user confusion
Several users have shown that they do not fully understand the difference between *received date* and *publication date*, and particularly not the pitfalls of *publication date*.
Mitigation: the menus to mark-as-read *before 1 day* and *before 1 week* are disabled when sorting by anything else than *received date*. Likewise, the separation headers *Today* and *Yesterday* and *Before yesterday* are only shown when sorting by *received date*.
Again here, to better understand why, it helps to think about sorting by *title* or by *link*, as sorting by *title* and by *publication date* share more or less the same characteristics.
* [ ] We should write a Q&A and/or documentation about the problems associated to *sorting by publication date*: risks of not noticing new publication, of inadvertently marking them as read, of having some articles with a date in the future hanging at the top of the views (vice versa when sorting in ascending order), performance, etc.
### Problem 5: APIs
Sorting by anything else than *received date* breaks the guarantees needed for a successful synchronisation via API.
Mitigation: sorting by *received date* is ensured for all API calls.
* PHPStan 2.0
fix https://github.com/FreshRSS/FreshRSS/issues/6989https://github.com/phpstan/phpstan/releases/tag/2.0.0https://github.com/phpstan/phpstan/blob/2.0.x/UPGRADING.md
* More
* More
* Done
* fix i18n CLI
* Restore a PHPStan Next test
For work towards PHPStan Level 10
* 4 more on Level 10
* fix getTagsForEntry
* API at Level 10
* More Level 10
* Finish Minz at Level 10
* Finish CLI at Level 10
* Finish Controllers at Level 10
* More Level 10
* More
* Pass bleedingEdge
* Clean PHPStan options and add TODOs
* Level 10 for main config
* More
* Consitency array vs. list
* Sanitize themes get_infos
* Simplify TagDAO->getTagsForEntries()
* Finish reportAnyTypeWideningInVarTag
* Prepare checkBenevolentUnionTypes and checkImplicitMixed
* Fixes
* Refix
* Another fix
* Casing of __METHOD__ constant
* Add move to next unread Label on mark as read.
The Labels, unlike the Feeds and Categories, don't move to the next
unread when "move to next unread on mark all as read" user feature is
enabled.
Labels are more complex than Feeds and Categories because Entries can be
in more than Label at a time. So when marking all Entries in the Label
as read, it can cause other Labels to end up with all their Entries
marked as read as well. The calculation of what the next
Label/Feed/Category is to jump to normally happens when generating the
link for the "Mark as Read" buttons, but it can't for Labels.
To address the problem for Labels, use a placeholder value during the
pre-calculation of the "Mark as Read" button link. When that placeholder
value is encountered during the "Mark as Read" action, the next Label
with unread Entries will be calculated immediately after the mark as
read action has been processed.
Fix all the translations of the 'jump_next' text to remove the '(feed or
categories' part that no longer applies.
Attempt to fix the inconsistent Russian, Italian, and Polish
translations of 'jump_next' text, which phrased the '(feed or
categories)' part differently.
* Minor code formattting
* Fixes
* Optimize next label lookup.
Only get the tag list once, and actually error check that it returned successfully.
Fix a typo in a comment as well.
* Fix fallback when all Labels are read.
Fix the missing check for whether we're in the fallback case or not.
* Update app/i18n/ru/conf.php
* Update app/Controllers/entryController.php
* Minor changes
* One more minor
---------
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
* Minor update whitespace PHPCS rules
To simplify our configuration, apply more rules, and be clearer about what is added or removed compared with PSR12.
Does not change our current conventions, but just a bit more consistent.
* Forgotten *.phtml
* Sort exclusion patterns + add a few for Extensions repo
* Relaxed some rules
* Pass PHPStan level 8
And prepare for PHPStan level 9 https://phpstan.org/user-guide/rule-levels
* Revert wrong replace in comment
* Fix PHPStan level 8
* Update PHPStan and other dev dependencies
* Remove obsolete comment
* noVariableVariables and towards bleedingEdge
https://github.com/phpstan/phpstan-strict-ruleshttps://phpstan.org/blog/what-is-bleeding-edge
* More bleedingEdge
* A bit more PHPStan level 9
* More PHPStan level 9
* Prepare for booleansInConditions
Ignore int and null
* Revert wrong line
* More fixes
* Fix keep_max_n_unread
* Stricter attribute functions
* Stricter callHooks and more PHPStan level 9
* More typing
* A tiny more
* Little's optimisations and booleans in conditions
* Apply strict type
* Apply strict type
* Apply strict type
* Fix multiple bugs with PHP 8.2 and 8.3
* Many declares missing, more errors fixed
* Apply strict type
* Another approach
* Stronger typing for Minz_Session
* Fix case of SQLite
---------
Co-authored-by: Luc <sanchezluc+freshrss@gmail.com>
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
* New feature important feeds
* Fix PHPStan
* Initial style for important feeds + keep unread
* Change UI order
* Count important unread
* Never mark as read important feeds during scroll
* Fix i18n conf.iew.normal regression
* Fix reader view
* More fix reader view
* Create important.svg
* Fix title
* Fix counter
* Account for important during mark-all-as-read
* Fix underline colour
* 📌
* Changelog
---------
Co-authored-by: math-gh <>
Co-authored-by: maTh <1645099+math-GH@users.noreply.github.com>
* processing of depreciations and updating of code to php7.2 minimum
* Autoformat many strange array indenting
And revert a few unwanted changes
---------
Co-authored-by: Luc <sanchezluc+freshrss@gmail.com>
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
* Complete PHPStan Level 6
Fix https://github.com/FreshRSS/FreshRSS/issues/4112
And initiate PHPStan Level 7
* PHPStan Level 6 for tests
* Use phpstan/phpstan-phpunit
* Update to PHPStan version 1.10
* Fix mixed bug
* Fix mixed return bug
* Fix paginator bug
* Fix FreshRSS_UserConfiguration
* A couple more Minz_Configuration bug fixes
* A few trivial PHPStan Level 7 fixes
* A few more simple PHPStan Level 7
* More files passing PHPStan Level 7
Add interface to replace removed class from https://github.com/FreshRSS/FreshRSS/pull/5251
* A few more PHPStan Level 7 preparations
* A few last details
* Typehint to Controllers
* Remarque's from Alkarex
* Remarque's from Alkarex
* Remarque's from Alkarex
* Remarque's from Alkarex
* Remarque's from Alkarex
* Remarque's from Alkarex
---------
Co-authored-by: Luc <sanchezluc+freshrss@gmail.com>
Translations had quite a lot of cleaning in late:
* Removed some unused keys
* Added some ignores
* Applied a `cli/manipulate.translation.php -a format`
* Change archiving config page layout
I've changed some wording and moved actions into a
maintenance section.
* Update purge action
Now we have more control on the purge action. The configuration allows
us to choose what to keep and what to discard in a more precise way.
At the moment, the configuration applies for all feeds.
* Add purge configuration on feed level
Now the extend purge configuration is available on feed level.
It is stored as attributes and will be used in the purge action.
* Update purge action
Now the purge action uses the feed configuration if it exists and
defaults on user configuration if not.
* Add empty option in period list
* Fix configuration warnings
* Add archiving configuration on categories
See #2369
* Add user info back
* Add explanations in UI
* Fixes for SQLite + error + misc.
* Fix invalid feed reference
* Short array syntax
Only for new code, so far
* Fix prefix error
* Query performance, default values
Work in progress
* Fix default values and confirm before leaving
Form cancel and confirm changes before leaving were broken.
And start taking advantage of the short echo syntax `<?= ?>` as we have
moved to PHP 5.4+
* More work
* Tuning SQL
* Fix MariaDB + performance issue
* SQL performance
* Fix SQLite bug
* Fix some attributes JSON encoding bugs
Especially for SQLite export/import
* More uniform, fix bugs
More uniform between global, category, feed settings
* Drop special cases for old articles during refresh
Instead will use lastSeen date with the new archiving logic.
This was generating problems anyway
https://github.com/FreshRSS/FreshRSS/issues/2154
* Draft drop index keep_history
Not needed anymore
* MySQL typo
Now properly tested with MySQL, PostgreSQL, SQLite
* More work for legacy values
Important to avoid overriding user's preference and risking deleting
data erroneously
* Fix PHP 7.3 / 7.4 warnings
@aledeg "Trying to use values of type null, bool, int, float or resource
as an array (such as $null["key"]) will now generate a notice. "
https://php.net/migration74.incompatible
* Reintroduce min articles and take care of legacy parameters
* A few changes forgotten
* Draft of migration + DROP of feed.keep_history
* Fix several errors
And give up using const for SQL to allow multiple database types (and we
cannot redefine a const)
* Add keep_min to categories + factorise archiving logic
* Legacy fix
* Fix bug yield from
* Minor: Use JSON_UNESCAPED_SLASHE for attributes
And make more uniform
* Fix sign and missing variable
* Fine tune the logic
* MySQL GUID case sensitive
latin1_bin
https://github.com/FreshRSS/FreshRSS/issues/2077
* Prepare update for existing bases
* Perform DB update during actualize
* Reduce frequency slightly
* No optimize at the same time
* Take advantage of the SQL modifications in 1.12
* Move higher up
* Move to purge, which all users can manually call
* First draft of custom tags
https://github.com/FreshRSS/FreshRSS/issues/928https://github.com/FreshRSS/FreshRSS/issues/1367
* SMALLINT to BIGINT for id_entry
And uppercase SQL types
* Fix layout for unreads
* Start UI menu
* Change menu order
* Clean database helpers
https://github.com/FreshRSS/FreshRSS/pull/2027#discussion_r217971535
* Travis rules do not understand PostgreSQL constants
Grrr
* Tag controller + UI
* Add column attributes to tags
* Use only favicon for now, for label
* Fix styling for different themes
* Constant for maximum InnoDB index length in Unicode
https://github.com/FreshRSS/FreshRSS/pull/2027#discussion_r219052200
(I would have personnally prefered keeping the readability of a real
value instead of a constant, in this case of many SQL fields)
* Use FreshRSS_Factory::createCategoryDao
* Add view of all articles containing any tag
* Fix search in tags
* Mark as read tags
* Partial auto-update unread tags
* More auto update tag unreads
* Add tag deletion
* Do not purge tagged articles
* Minor comment
* Fix SQLite and UI bug
* Google Reader API support for user tags
Add SQL check that tag names must be distinct from category names
* whitespace
* Add missing API for EasyRSS
* Compatibility SQLite
Problematic parentheses
* Add SQL DISTINCT for cases with multiple tags
* Fix for PostgreSQL
PostgreSQL needs some additional type hint to avoid "could not determine
data type of parameter $1"
http://www.postgresql-archive.org/Could-not-determine-data-type-of-parameter-1-tp2171092p2171094.html
I think the use of a magic value repeated many times in the code is prone to have some errors made by people not knowing its meaning. Using a constant is a bit more safe. Judging by some comments in the code, I am not the only one.
- FreshRSS_Context::$conf is replaced by FreshRSS_Context::$user_conf
- Introduce FreshRSS_Context::$system_conf
- Remove FreshRSS_Configuration object
See https://github.com/FreshRSS/FreshRSS/issues/730
