Add option to sort results by received date (existing, default), publication date, title, URL (link), random.
fix https://github.com/FreshRSS/FreshRSS/issues/1771
fix https://github.com/FreshRSS/FreshRSS/issues/2083
fix https://github.com/FreshRSS/FreshRSS/issues/2119
fix https://github.com/FreshRSS/FreshRSS/issues/2596
fix https://github.com/FreshRSS/FreshRSS/issues/3204
fix https://github.com/FreshRSS/FreshRSS/issues/4405
fix https://github.com/FreshRSS/FreshRSS/issues/5529
fix https://github.com/FreshRSS/FreshRSS/issues/5864
fix https://github.com/FreshRSS/Extensions/issues/161
URL parameters:
* `&sort=id` (current behaviour, sorting according to newest received articles)
* `&sort=date` (publication date, which is not indicative of how new an article is)
* `&sort=title`
* `&sort=link`
* `&sort=rand` (random order - which disables infinite scrolling, at least for now)
combined with `&order=ASC` or `&order=DESC`
## Implementation notes
The sorting criteria by *received date* (id), which is the default, and which was the only one before this PR, is the one that has the best sorting characteristics:
* *uniqueness*: no entries have the exact same received date
* *monotonicity*: new entries always have a higher received date
* *performance*: this field is efficiently indexed in database for fast usage, including for paging (indexing could also be done to other fields, but with lower effective performance)
In contrary, sorting criteria such as by *publication date*, by *title*, or by *link* are neither unique nor monotonic. In particular, multiple articles may share the same *publication date*, and we may receive articles with a *publication date* far in the future, and then later some new articles with a *publication date* far in the past.
To understand why sorting by *publication date* is problematic, it helps to think about sorting by *title* or by *link*, as sorting by *title* and by *publication date* share more or less the same characteristics.
### Problem 1: new articles
New articles may be received in the background after what is shown on screen, and before the next user action such as *mark all as read*. Due to the lack of *monotonicity* when sorting by e.g. *publication date* or *title*, users risk marking as read a batch of articles containing some fresh articles without seeing them.
Mitigation: A parameter `idMax` tracks the maximum ID related to a batch of actions such as *mark all as read* to exclude articles received after those that are displayed.
### Problem 2: paging / pagination
When navigating articles, only a few articles are displayed, and a new "page" of articles needs to be received from the database when scrolling down or when clicking the button to show more articles. When sorting by e.g. *publication date* or *title*, it is not trivial to show the next page without re-showing some of the same articles, and without skipping any. Indeed, views are often with additional criteria such as showing only unread articles, and users may mark some articles as read while viewing them, hereby removing some articles from the previous pages. And like for *Problem 1*, new articles may have been received in the background. Consequently, it is not possible to use `OFFSET` to implement pagination (so the patches suggested by a few users were wrong due to that, in particular).
Mitigation: `idMax` is also used (just like for *Problem 1*) and a *Keyset Pagination* approach is used, combining an unstable sorting criterion such as *publication date* or *title*, together with *id* to ensure stable sorting. (So, 2 sorting criteria + 1 filter criteria)
See e.g. https://www.alwaysdeveloping.net/dailydrop/2022/07/01-keyset-pagination/
### Problem 3: performance
Sorting by anything else than *received date* (id) is doomed to be slow(er) due to the combination of 3 criteria (see *Problem 2*). An `OFFSET` approach (which is not possible anyway as explained) would be even slower. Furthermore, we have no SQL index at the moment, but they would not necessarily help much due to the multiple sorting criteria needed and involving some `OR` logic which is difficult to optimise for databases.
The nicest syntax would be using tuples and corresponding indexes, but that is poorly supported by MySQL https://bugs.mysql.com/bug.php?id=104128
Mitigation: a compatibility SQL syntax is used to implement *Keyset Pagination*
### Problem 4: user confusion
Several users have shown that they do not fully understand the difference between *received date* and *publication date*, and particularly not the pitfalls of *publication date*.
Mitigation: the menus to mark-as-read *before 1 day* and *before 1 week* are disabled when sorting by anything else than *received date*. Likewise, the separation headers *Today* and *Yesterday* and *Before yesterday* are only shown when sorting by *received date*.
Again here, to better understand why, it helps to think about sorting by *title* or by *link*, as sorting by *title* and by *publication date* share more or less the same characteristics.
* [ ] We should write a Q&A and/or documentation about the problems associated to *sorting by publication date*: risks of not noticing new publication, of inadvertently marking them as read, of having some articles with a date in the future hanging at the top of the views (vice versa when sorting in ascending order), performance, etc.
### Problem 5: APIs
Sorting by anything else than *received date* breaks the guarantees needed for a successful synchronisation via API.
Mitigation: sorting by *received date* is ensured for all API calls.
* PHPStan 2.0
fix https://github.com/FreshRSS/FreshRSS/issues/6989https://github.com/phpstan/phpstan/releases/tag/2.0.0https://github.com/phpstan/phpstan/blob/2.0.x/UPGRADING.md
* More
* More
* Done
* fix i18n CLI
* Restore a PHPStan Next test
For work towards PHPStan Level 10
* 4 more on Level 10
* fix getTagsForEntry
* API at Level 10
* More Level 10
* Finish Minz at Level 10
* Finish CLI at Level 10
* Finish Controllers at Level 10
* More Level 10
* More
* Pass bleedingEdge
* Clean PHPStan options and add TODOs
* Level 10 for main config
* More
* Consitency array vs. list
* Sanitize themes get_infos
* Simplify TagDAO->getTagsForEntries()
* Finish reportAnyTypeWideningInVarTag
* Prepare checkBenevolentUnionTypes and checkImplicitMixed
* Fixes
* Refix
* Another fix
* Casing of __METHOD__ constant
* Minor update whitespace PHPCS rules
To simplify our configuration, apply more rules, and be clearer about what is added or removed compared with PSR12.
Does not change our current conventions, but just a bit more consistent.
* Forgotten *.phtml
* Sort exclusion patterns + add a few for Extensions repo
* Relaxed some rules
* New feature: shareable user query
Share the output of a user query by RSS / HTML / OPML with other people through unique URLs.
Replaces the global admin token, which was the only option (but unsafe) to share RSS outputs with other people.
Also add a new HTML output for people without an RSS reader.
fix https://github.com/FreshRSS/FreshRSS/issues/3066#issuecomment-648977890
fix https://github.com/FreshRSS/FreshRSS/issues/3178#issuecomment-769435504
* Remove unused method
* Fix token saving
* Implement HTML view
* Update i18n for master token
* Revert i18n get_favorite
* Fix missing i18n for user queries from before this PR
* Remove irrelevant tests
* Add link to RSS version
* Fix getGet
* Fix getState
* Fix getSearch
* Alternative getSearch
* Default getOrder
* Explicit default state
* Fix test
* Add OPML sharing
* Remove many redundant SQL queries from original implementation of user queries
* Fix article tags
* Use default user settings
* Prepare public search
* Fixes
* Allow user search on article tags
* Implement user search
* Revert filter bug
* Revert wrong SQL left outer join change
* Implement checkboxes
* Safe check of OPML
* Fix label
* Remove RSS button to favour new sharing method
That sharing button was using a global admin token
* First version of HTTP 304
* Disallow some recusrivity
fix https://github.com/FreshRSS/FreshRSS/issues/6086
* Draft of nav
* Minor httpConditional
* Add support for offset for pagination
* Fix offset pagination
* Fix explicit order ASC
* Add documentation
* Help links i18n
* Note about deprecated master token
* Typo
* Doc about format
* Pass PHPStan level 8
And prepare for PHPStan level 9 https://phpstan.org/user-guide/rule-levels
* Revert wrong replace in comment
* Fix PHPStan level 8
* Update PHPStan and other dev dependencies
* Remove obsolete comment
* noVariableVariables and towards bleedingEdge
https://github.com/phpstan/phpstan-strict-ruleshttps://phpstan.org/blog/what-is-bleeding-edge
* More bleedingEdge
* A bit more PHPStan level 9
* More PHPStan level 9
* Prepare for booleansInConditions
Ignore int and null
* Revert wrong line
* More fixes
* Fix keep_max_n_unread
* Stricter attribute functions
* Stricter callHooks and more PHPStan level 9
* More typing
* A tiny more
* Little's optimisations and booleans in conditions
* Apply strict type
* Apply strict type
* Apply strict type
* Fix multiple bugs with PHP 8.2 and 8.3
* Many declares missing, more errors fixed
* Apply strict type
* Another approach
* Stronger typing for Minz_Session
* Fix case of SQLite
---------
Co-authored-by: Luc <sanchezluc+freshrss@gmail.com>
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
* processing of depreciations and updating of code to php7.2 minimum
* Autoformat many strange array indenting
And revert a few unwanted changes
---------
Co-authored-by: Luc <sanchezluc+freshrss@gmail.com>
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
* Complete PHPStan Level 6
Fix https://github.com/FreshRSS/FreshRSS/issues/4112
And initiate PHPStan Level 7
* PHPStan Level 6 for tests
* Use phpstan/phpstan-phpunit
* Update to PHPStan version 1.10
* Fix mixed bug
* Fix mixed return bug
* Fix paginator bug
* Fix FreshRSS_UserConfiguration
* A couple more Minz_Configuration bug fixes
* A few trivial PHPStan Level 7 fixes
* A few more simple PHPStan Level 7
* More files passing PHPStan Level 7
Add interface to replace removed class from https://github.com/FreshRSS/FreshRSS/pull/5251
* A few more PHPStan Level 7 preparations
* A few last details
* Remove FreshRSS_Searchable for better types
The interface was not used, and it was preventing more precise types for the different `searchById()` methods, as they each have different input and output types.
* Consistent entry ID
Entry IDs (which are 64-bit integers) must be processed as string to be compatible with 32-bit platforms
* Fix type
* A few more related types
* PHPStan level 6
* Some more casts needed
* String cast for htmlspecialchars
* Use constants for path to TOS
* improve comments
* TOS title moved to template
* TOS available via config menu
* CSS: improve handling of content of TOS/about
* give info about set/unset TOS in system config
* fix target
* i18n FR
* i18n DE
* fix: Fix undefined GLOB_BRACE on Alpine
The manual states that:
> Note: The GLOB_BRACE flag is not available on some non GNU systems,
> like Solaris or Alpine Linux.
This generated an error on Alpine.
Reference: https://www.php.net/manual/function.glob.php
* fix: List details of feeds for OPML exportation
The details are necessary to export the XPath information, the CSS full
content path and read actions filters.
* Update LibOpml to 0.4.0
* Refactor OPML importation to be more robust
First, it fixes two regressions introduced by the update of lib_opml:
- title attribute is used when text attribute is missing;
- the OPML category attribute is used as a fallback for feeds categories.
In a related way, if also fixes a problem when a feed had both a parent
category outline and a category attribute. Before, it only considered the
attribute as its category, but now it considers the parent outline.
Then, it counts category limit correctly by not increasing
`$nb_categories` if the category already exists.
* Exclude lib_opml from the CodeSniffer
* Fix variable names when logging some errors
* Fix catch of LibOpml Exception
* Make sure to declare the category
* Exclude lib_opml from PHPStan analyze
* Disable markdownlint for lib_opml
* Fix typos
* Use auto-loading and allow updates via Composer
* Fix broken links to lib_opml
* Bring back the ability to import the OPML frss:opmlUrl attribute
* Refactor the logs of OPML errors
* Update lib_opml to the version 0.5.0
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
* add docs of function param()
* page < 0 is impossible now
* show prev/next only, when there is a prev/next page
* set always page params
* Revert "page < 0 is impossible now"
This reverts commit fdb920ee50.
* Update logs_pagination.phtml
* it works
* small improvements
* Update slider.js
* fixed JS syntax
* slider.js included in main.js
* fix syntax
* delete including of slider.js
* Update extra.js
* More PHP type hints for Fever
Follow-up of https://github.com/FreshRSS/FreshRSS/pull/4201
Related to https://github.com/FreshRSS/FreshRSS/issues/4200
* Detail
* Draft
* Progress
* More draft
* Fix thumbnail PHP type hint
https://github.com/FreshRSS/FreshRSS/issues/4215
* More types
* A bit more
* Refactor FreshRSS_Entry::fromArray
* Progress
* Starts to work
* Categories
* Fonctional
* Layout update
* Fix relative URLs
* Cache system
* Forgotten files
* Remove a debug line
* Automatic form validation of XPath expressions
* data-leave-validation
* Fix reload action
* Simpler examples
* Fix column type for PostgreSQL
* Enforce HTTP encoding
* Readme
* Fix get full content
* target="_blank"
* gitignore
* htmlspecialchars_utf8
* Implement HTML <base>
And fix/revert `xml:base` support in SimplePie e49c578817
* SimplePie upstream PR merged
https://github.com/simplepie/simplepie/pull/723
* Frontend changes done
* Load more should work now also without JS
* Update template.rtl.css
* improved the themes' CSS
* Update template.rtl.css
* fix CI
* CSS prop. order fixed
* Rename pagination.phtml to stream-footer.phtml
* use the new template name
* rename key of i18n
* fixed CI, that does not like the white space
* rename pagination variable
* Update indexController.php
* Fix nav_menu mark-as-read
#Fix https://github.com/FreshRSS/FreshRSS/issues/2905
Fix regression from https://github.com/FreshRSS/FreshRSS/pull/2588
We need info about the first item (id_max) before being able to output
nav_menu. Before https://github.com/FreshRSS/FreshRSS/pull/2588 we used
to output everything in memory before starting to produce an output. Now
that we stream the output, we need a temporary buffer until we have
received the first item/article.
* Repair loading page
* Simplify CSS
Make it work in Chrome as well
* Lint
* Partial revert
* Base max_id solely on current time
* Better performance with yield
Largely decrease the time to first byte, and reduced memory consumtion.
Before, we used to make several copies in memory of the whole list of
articles before sending them to the client. Now streamed as they are
processed.
* Travis
For an extension, I needed to call a script from an external domain.
Unfortunately, the CSP headers didn't allow this domain and I had to
patch manually the FreshRSS FrontController for my extension. It's
obviously not a long-term solution since it has nothing to do in the
core of FRSS, and I don't want to apply this patch manually at each
update.
With this patch, I allow changing the CSP header from inside the
controller actions. It allows extensions to modify headers. It's also an
opportunity to remove a bit of code from the FrontController. I wasn't
happy with the previous implementation anyhow.
Reference: https://github.com/flusio/xExtension-Flus/commit/ed12d56#diff-ff12e33ed31b23bda327499fa6e84eccR143
* Add category order
Each category has a new 'priority' attribute. It is used to sort categories in
views. Categories with the same priority are sorted alphabetically. Categories
with no priority are displayed after those with one.
For example, if we have the following categories:
- A (priority: 2)
- B (no priority)
- C (priority: 1)
- D (priority: 2)
- E (no priority)
- F (priority: 1)
They will be displayed in the following order:
- C
- F
- A
- D
- B
- E
See #190
* Shorten help text
It took too much room and will not be so necessary once we have drag &
drop
This feature is optional. It is based on the presence of a
`data/tos.html` file that an administrator can create. If this file
exists, FreshRSS will automatically add a "ToS" checkbox on the
registration page that users must check to be able to create their
account.
* Less jQuery
Follow-up of https://github.com/FreshRSS/FreshRSS/pull/2199
* Even less jQuery + global view unread title fix
* Even less jQuery
* Yet even less jQuery
* Even less jQuery
* Reduce some events
* Even less jQuery
* jQuery gone from main view
+Fixed English i18n
* Fix feed folded view
* Remove Firefox 64 workaround
Remove workaround for Gecko bug 1514498 in Firefox 64, fixed in Firefox
65
* Split to extra.js
Avoid loading unneeded JavaScript code for the main view.
+ several adjustements
* Improve CSS transition fold category
* Rewrite shortcuts
Remove library. Much faster, shorter, one listener instead of many.
Control of the shortcut context.
Fix https://github.com/FreshRSS/FreshRSS/issues/2215
* Remove debug
* Minor syntax
* Filter out unwanted shortcut modifiers
* Menu overflow fix
* Typo
* Fix unfolding in mobile view
* Remove jQuery from category.js
* Remove jQuery from Global view
* Ensure fast flush of HTTP headers and HTML head
The fast flush optimisation done in
https://github.com/FreshRSS/FreshRSS/pull/1133 does not seem to work
anymore (need to check if it is related to a PHP version).
Work-around when PHP flush() is not working
Can be tested by adding a `sleep(5);` after:
ee902ee7c4/app/layout/layout.phtml (L27)
Follow-up of the performance checks of
https://github.com/FreshRSS/FreshRSS/pull/2040
* output_buffering in .user.ini for PHP CGI / FPM
* Reuse .user.ini for Docker PHP config
* Longer flush
Flush a bit later, to be compatible with the default value of 4096 for
PHP output_buffering, and thus avoid the need of tuning the value.
* First draft of custom tags
https://github.com/FreshRSS/FreshRSS/issues/928https://github.com/FreshRSS/FreshRSS/issues/1367
* SMALLINT to BIGINT for id_entry
And uppercase SQL types
* Fix layout for unreads
* Start UI menu
* Change menu order
* Clean database helpers
https://github.com/FreshRSS/FreshRSS/pull/2027#discussion_r217971535
* Travis rules do not understand PostgreSQL constants
Grrr
* Tag controller + UI
* Add column attributes to tags
* Use only favicon for now, for label
* Fix styling for different themes
* Constant for maximum InnoDB index length in Unicode
https://github.com/FreshRSS/FreshRSS/pull/2027#discussion_r219052200
(I would have personnally prefered keeping the readability of a real
value instead of a constant, in this case of many SQL fields)
* Use FreshRSS_Factory::createCategoryDao
* Add view of all articles containing any tag
* Fix search in tags
* Mark as read tags
* Partial auto-update unread tags
* More auto update tag unreads
* Add tag deletion
* Do not purge tagged articles
* Minor comment
* Fix SQLite and UI bug
* Google Reader API support for user tags
Add SQL check that tag names must be distinct from category names
* whitespace
* Add missing API for EasyRSS
* Compatibility SQLite
Problematic parentheses
* Add SQL DISTINCT for cases with multiple tags
* Fix for PostgreSQL
PostgreSQL needs some additional type hint to avoid "could not determine
data type of parameter $1"
http://www.postgresql-archive.org/Could-not-determine-data-type-of-parameter-1-tp2171092p2171094.html
When retrieving the list of entries when the context was 'all' or 'starred', there was the following warning:
> Warning: count(): Parameter must be an array or an object that implements Countable in /home/alexis/FreshRSS/app/Controllers/indexController.php on line 206
I fixed that by changing how the array is tested.
* [ci] Add Travis
* Exclude some libs
* Semi-auto whitespace fixes
* line length in SQLite
* Exclude tests from line length
* Feed.php line length
* Feed.php: get rid of unnecessary concat
* Feed.php: line length
* bootstrap.php: no newline at end of file
* Allow concatenating across multiple lines
* Add Travis badge
* do-install line length
* update-or-create-user line length
* cli/create-user line length
* tests/app/Models/SearchTest.php fix indentation
* tests/app/Models/UserQueryTest.php fix indentation
* tests/app/Models/CategoryTest.php fix indentation
* [fix] PHP 5.3 on precise
* cli/do-install no spaces
* cli/list-users line length
* cli/reconfigure line length
* empty catch statements
* api/index line length nonsense
* spaces before semicolon
* app/Models/EntryDAO bunch of indentation
* extra blank lines
* spaces before comma in function call
* testing tabwidth
* increase to 10
* comment out tabwidth line
* try older phpcs version 3.0.0RC4
* line length exception for app/install.php
* proper spaces
* stray spaces in i18n
* Minz/ModelPdo line length
* Minz whitespace
* greader line length
* greader elseif placement
* app/Models/Feed.php spacing in function argument
* ignore php 5.3
* app/Models/ConfigurationSetter.php stray whitespace
* EntryDAOSQLite line length
* I vote for higher max line length =P
* ignore SQL
* remove classname complaint
* line length/more legible SQL
* ignore line length nonsense
* greader line length
* feedController issues
* uppercase TRUE, FALSE, NULL
* revert
* importExportController lowercase null
* Share.php default value not necessary because ! is_array () a few lines down
* CategoryDAO constants should be UPPERCASE
* EntryDAO reduce line length
* contentious autofix
* Allow failures on all versions of PHP except 7.1 because reasons
