Contributes to #6139 (export part; the import will follow in another PR)
Exporting favourites was collecting the whole file in memory. With many articles, it fails with the default PHP memory limit.
## Changes
- New `Minz_View::helperToFile()`, similar to `helperToString()`, writing the output of a helper into a file by chunks of 64 KiB instead of keeping it in memory.
- `FreshRSS_Export_Service`: the `generate*()` methods now return the path of a temporary file instead of the content, and `zip()` builds the archive from those files (`ZipArchive::addFile()` instead of `addFromString()`). Temporary files are created in `TMP_PATH` (already required to be writable) and deleted at the end of the script.
- The export is sent with `readfile()`: Web UI, `cli/export-zip-for-user.php`, `cli/export-opml-for-user.php` and the OPML export of the Google Reader API.
- Note: this changes what `FreshRSS_Export_Service` returns (a path instead of the content). All callers in the repository are updated, and I could not find any public extension using this service.
## Measurements
Peak PHP memory (`memory_get_peak_usage(true)`), SQLite database with synthetic starred articles:
| Scenario | Before | After |
|---|---|---|
| Web, export your favourites, 10,000 articles (69 MB JSON) | 136 MB (fails with the default `memory_limit` of 128 MB) | 6 MB |
| Web, ZIP (OPML + favourites + labelled articles + 4 feeds), 10,000 articles | 152 MB | 6 MB |
| CLI `export-zip-for-user.php`, 10,000 articles | 189 MB (fails with 128 MB) | 8 MB |
| Web, export your favourites, 30,000 articles (208 MB JSON) | 404 MB | 10 MB |
Co-authored-by: Alejandro Borrego <210245044+abp002@users.noreply.github.com>
1. `include`, `include_once`, `require` and `require_once` are expressions not functions, parentheses are not necessary.
2. to move up the directory tree, it's better to use the `dirname` function instead of relying on `/..`.
* Adds logic for validation
* Adds validation to do-install
* Adds help to do-install
* Adds validation & help to reconfigure
* Adds validation to check.translation
* Adds validation to manipulate.translation
* Small fixes to help texts
* Refactors language option validation
* Adds default options to validation
* Fixes validation with regex
* Refactors readAs functions
* Updates to new regex validation format
* Fixes typing around default values
* Adds file extension validation
* Restandardises validation & parsing typing around array of strings
* Adds NotOneOf validation
* Adds ArrayOfString read as
* Refactors existing validation
* Adds validation throughout cli
* Removes unused file
* Adds new CL parser with goal of wrapping CLI behaviour
* Hides parsing and validation
* Rewites CL parser to make better use of classes
* Rolls out new parser across CL
* Fixes error during unknown option check
* Fixes misnamed property calls
* Seperates validations into more appropriate locations
* Adds common boolean forms to validation
* Moves CommandLineParser and Option classes into their own files
* Fixes error when validating Int type
* Rewrites appendTypedValues -> appendTypedValidValues now filters invalid values from output
* Renames -> for clarity
* Adds some docs clarifying option defaults and value taking behaviour
* Refactors getUsageMessage for readability
* Minor formatting changes
* Adds tests for CommandLineParser
* Adds more tests
* Adds minor fixs
* Reconfigure now correctly updates config
* More fixes to reconfigure
* Fixes required files for CommandLineParserTest
* Use .php extension for PHP file
* PHPStan ignore instead of wrong typing
* Refactors to support php 7.4
* Moves away from dynamic properties by adding 'Definintions' to all commands
* Renames target to definition for clarity
* Stops null from being returned as a valid value in a certain edge case
* Adds PHPStan ignore instead of incorrect typing
* Refactors tests to take account of new typing solution
* Marks file as executable
* Draft CLI rework
* Finish rewrite as object-oriented
* Fix PHPStan ignore and make more strongly typed
* Rename class Option to CliOption
* Light renaming + anonymous classes
---------
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
* Pass PHPStan level 8
And prepare for PHPStan level 9 https://phpstan.org/user-guide/rule-levels
* Revert wrong replace in comment
* Fix PHPStan level 8
* Update PHPStan and other dev dependencies
* Remove obsolete comment
* noVariableVariables and towards bleedingEdge
https://github.com/phpstan/phpstan-strict-ruleshttps://phpstan.org/blog/what-is-bleeding-edge
* More bleedingEdge
* A bit more PHPStan level 9
* More PHPStan level 9
* Prepare for booleansInConditions
Ignore int and null
* Revert wrong line
* More fixes
* Fix keep_max_n_unread
* Stricter attribute functions
* Stricter callHooks and more PHPStan level 9
* More typing
* A tiny more
* Little's optimisations and booleans in conditions
* Apply strict type
* Apply strict type
* Apply strict type
* Fix multiple bugs with PHP 8.2 and 8.3
* Many declares missing, more errors fixed
* Apply strict type
* Another approach
* Stronger typing for Minz_Session
* Fix case of SQLite
---------
Co-authored-by: Luc <sanchezluc+freshrss@gmail.com>
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
* Complete PHPStan Level 6
Fix https://github.com/FreshRSS/FreshRSS/issues/4112
And initiate PHPStan Level 7
* PHPStan Level 6 for tests
* Use phpstan/phpstan-phpunit
* Update to PHPStan version 1.10
* Fix mixed bug
* Fix mixed return bug
* Fix paginator bug
* Fix FreshRSS_UserConfiguration
* A couple more Minz_Configuration bug fixes
* A few trivial PHPStan Level 7 fixes
* A few more simple PHPStan Level 7
* More files passing PHPStan Level 7
Add interface to replace removed class from https://github.com/FreshRSS/FreshRSS/pull/5251
* A few more PHPStan Level 7 preparations
* A few last details
Even if the issue #3035 seemed pretty simple at a first glance, it was
more complicated than I expected. Because we send CSP headers AFTER
running the controller actions, it means we can't "echo" any content
from the controller. It's in fact a good practice, but it was easier at
the time we developed the feature.
To fix that, the only thing I had to do was to move the `print()` and
`readfile()` function into the view. The problem was that we needed to
output the content from the CLI too. Then, things became more
complicated. I decided to extract the export-related methods in a
`FreshRSS_Export_Service` class, in order to use it from both the
controller and the CLI. It was an opportunity to refactor the whole
feature in order to make it a bit more linear and easy to read.
Reference: https://github.com/FreshRSS/FreshRSS/issues/3035
Filenames were created with the username of the current user. However,
when we export the files with the CLI, the current user is "_".
This commit makes the username always required in the `exportFile`
method so we make sure to always manipulate a real value. Consequently,
the filenames can be formatted correctly.
Obviously, this has absolutely no impacts since the CLI doesn't consider
the HTTP headers. It just makes things a bit more clear. It's a first
step to remove the concept of "default user".
If an option used on cli is not recognized, the command
aborts and displays an error message.
If the typed option is similar to one of the recognized
options, a hint is displayed.
At the moment, there is a limitation on long options.
Short options are not validated at the moment.
See #2046
