* feat(import): accept .txt URL lists alongside OPML/JSON/ZIP
Detects .txt by extension and wraps the URL list into a minimal OPML
document so the existing import pipeline handles dedup, categories and
feed limits unchanged. Blank lines, `#` comments and a UTF-8 BOM are
skipped; lines that don't parse as URLs are logged and dropped without
aborting the batch.
Works through both `cli/import-for-user.php` and the web import form.
* utf8BOM
* ENT_COMPAT
---------
Co-authored-by: Bjørn A. Andersen <polybjorn@users.noreply.github.com>
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
Add an opt-in CLI that exports each user's database to
`data/users/<user>/sqlite-backups/<YYYYMMDDTHHMMSSZ>.sqlite` (UTC) and
prunes older files to a configured count. Gated by two new settings,
`auto_sqlite_export.enabled` and `auto_sqlite_export.retention`.
Kept separate from `cli/db-backup.php` / `cli/db-restore.php`, which
stay the fixed-filename migration tool. First step of #8183.
Co-authored-by: Bjørn A. Andersen <polybjorn@users.noreply.github.com>
Detect when an `app/i18n/<lang>/` directory has no matching `gen.lang.<lang>`
key in the reference language (or vice versa), and refuse to regenerate the
README from that invalid state.
This catches a class of silent corruption where the README translation
table renders literal i18n keys instead of localised language names. The
trigger is most often a case-folded directory on macOS APFS - git tracks
`zh-TW`, the local FS reads back `zh-tw`, the script's `_t('gen.lang.zh-tw')`
lookup misses, and the README ends up with `gen.lang.zh-tw (zh-tw)` instead
of `正體中文 (zh-TW)`. The same check also flags orphan directories (no
display-name key) and orphan keys (no directory).
The new validateLanguageNames() method on I18nData performs a bidirectional
set comparison and returns human-readable issues. cli/check.translation.php
prints them to STDERR and gates --generate-readme on the result, leaving
routine completeness validation behaviour unchanged. Adds four PHPUnit
tests covering: clean state, case mismatch, orphan directory, orphan key.
Co-authored-by: Bjørn A. Andersen <polybjorn@users.noreply.github.com>
- Adds `cli/purge.php` so the purge policy can be applied from cron, mirroring `cli/db-optimize.php`.
- Documents it in `cli/README.md`.
Aims to be a small, contained answer to #3636: reuses the same `Feed::cleanOldEntries()` logic the GUI's "Purge now" button calls, packaged as a standard `cli/` script.
Mirror the existing db-optimize.php pattern so administrators can run the
purge policy on a schedule instead of waiting for the 1/30 random trigger
in feedController, or clicking 'Purge now' in the GUI.
The script applies the same archiving policy as the GUI button: per-feed
attribute, per-category attribute, then user config; with the existing
keep_favourites / keep_unreads / keep_labels / keep_min / keep_period /
keep_max guards. With no policy configured, it is a no-op.
Refs: https://github.com/FreshRSS/FreshRSS/issues/3636
Co-authored-by: Bjørn A. Andersen <polybjorn@users.noreply.github.com>
Closes https://github.com/FreshRSS/FreshRSS/issues/8508
Changes proposed in this pull request:
- Use an integer for `Feed::error` everywhere (follow up to #8646)
- Extract `Entry::machineReadableDate()` into function for use in HTML templates
- Add `timeago()` function that converts a unix timestamp into a "4 weeks ago" string
- Show the last successful feed update, and the last erroneous update
How to test the feature manually:
1. Update a feed
2. Modify the feed URL in the database and set it to a non-existing URL
3. Update the feed again
4. Open the "Manage feed" and see the expanded error message:
> Blast! This feed has encountered a problem. If this situation persists, please verify that it is still reachable.
> Last successful update 3 hours ago, last erroneous update 1 hour ago.
You can hover the relative dates to see the timestamp.
* Make Feed::error an int everywhere
Related: https://github.com/FreshRSS/FreshRSS/pull/8646
* Extract timestamptomachinedate()
.. for later usage in the feed error time display.
* Show time since when a feed has problems
We add our own "timeago" function that converts a unix timestamp
into a "4 weeks ago" string.
Resolves: https://github.com/FreshRSS/FreshRSS/issues/8508
* Add new translation keys
* i18n fr, en-US
* Minor XHTML preference
* Slightly shorter rewrite, also hopefully easier to read
* Rewrite to allow (simple) plural
I also moved some functions around for hopefully a more generic and better structure.
I made some changes for the sake of speed (e.g. second-based logic instead of datetime intervals).
Note: I used automatic translation as I was worried it would be too complicated to explain to translators... I proofread the few languages I have some familiarity with.
* Add reference to CLDR
* Slightly more compact syntax
* Always show last update, fix case of unknown error date
* Remove forgotten span
* No need for multi-lines anymore
* Fix error date thresshold
* plurals forms
* Extract gettext formula conversion script to cli
* Simplify a bit
* Escort excess parentheses to the door
* Simplify
* Avoid being too clever in localization
* Fix German
* Fix plural TODO parsing
* Ignore en-US translation
* make fix-all
* git update-index --chmod=+x cli/compile.plurals.php
* Heredoc indent PHP 7.3+
* compileAll: Continue on error
* PHP strict comparisons
* Light logical simplification
* Cache plural_message_families
* Avoid case of empty value
* A bit of documentation
---------
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
Co-authored-by: Frans de Jonge <frans@clevercast.com>
Co-authored-by: Frans de Jonge <fransdejonge@gmail.com>
* Better comments in our Docker images
* Make `cli/access-permissions.sh` compatible with other Apache groups such as `http` for Linux Arch
* Better `/Docker/entrypoint.sh` supporting various Apache configuration paths (and slightly faster).
* Add test image for Linux Arch (not sure we will keep it)
See
* https://github.com/FreshRSS/FreshRSS/pull/8279#issuecomment-3620674818
Fix https://github.com/FreshRSS/FreshRSS/issues/8268
To better support user management on FreshRSS instance with many users.
SQL speed improved. On a reduced test with 5 users, including some large accounts (PostgreSQL on a very tiny and slow server), improving from ~2.3s to ~1.8s, which gives ~20% speed improvement.
Then tested with 1000 users, with only the default feed (on my old desktop computer):
```sh
for i in {1..1000}; do ./cli/create-user.php --user=freshrss$i --password=freshrss; done
app/actualize_script.php
cli/access-permissions.sh
```
SQLite:
```console
$ time cli/user-info.php | wc -l
1001
real 0m1.366s
user 0m0.908s
sys 0m0.475s
```
PostgreSQL:
```console
$ time cli/user-info.php | wc -l
1001
real 0m28.498s
user 0m12.137s
sys 0m2.217s
```
MariaDB:
```console
# time ./cli/user-info.php | wc -l
1001
real 0m49.485s
user 0m1.276s
sys 0m2.258s
```
Yes, SQLite is much faster - not a surprise for such use-cases, where the TCP connection is not re-used.
I have added some CLI options to disable some statistics:
```sh
cli/user-info.php --no-db-size --no-db-counts
```
For the Web UI, I have disabled detailed user statistics if it takes too long, and retrieve missing user statistics asynchronously via JavaScript. Lazy loading of the user details based on IntersectionObserver, with maximum 10 requests in parallel.
Web UI tested on 1000 users as well. Checked with SeaMonkey.
So that renaming something like `conf.shortcut.toggle_sidebar` to `conf.shortcut.toggle_aside` can be done easily even after already having added `conf.shortcut.toggle_sidebar` and translated it in multiple languages.
Example of usage:
```console
./cli/manipulate.translation.php -a move -k conf.shortcut.toggle_sidebar -n conf.shortcut.toggle_aside
```
```console
make i18n-move-key key="conf.shortcut.toggle_sidebar" new-key="conf.shortcut.toggle_aside"
```
The key will be moved and all values/states will be kept.
* Housekeeping lib_rss.php
`lib_rss.php` had become much too large, especially after https://github.com/FreshRSS/FreshRSS/pull/7924
Moved most functions to other places.
Mostly no change of code otherwise (see comments).
* Extension: composer run-script phpstan-third-party
* fix: repair cli/health.php for OIDC installations
Connected with #8039
This changes will restore healthcheck process both for classic and OIDC logins
* Finalise fixes
---------
Co-authored-by: alexx_b <alexx_b@mir-lin1.mir.int>
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
* Change regional language codes
According to `ISO-639-1` and `ISO-3166-1 alpha-2` recommendations, regional
languages must be written with the language code in lowercase and the region
in uppercase. For instance, we should have `en-US` instead of `en-us`.
Folders have been updated to reflect those recommendations and regional
language codes have been updated in the translation files.
* Update README files
* Fix configuration for typos
* Revert language order in documentation
* Remove unnecessary code
* Change language configuration getter
* Fix phpstan error
* Fix typo
* Add types
* escape regex
* Move language logic to avoid magic or deprecated methods
* Minor fix on regex
* API optimisation: more streaming of outputs
I spotted a memory issue when testing https://github.com/FreshRSS/FreshRSS/pull/7714
Attempt to stream results more, instead of keeping too much in memory.
Could be further improved.
* Apply suggestions from code review
Co-authored-by: Alexis Degrugillier <aledeg@users.noreply.github.com>
* Minor whitespace JSON formatting
---------
Co-authored-by: Alexis Degrugillier <aledeg@users.noreply.github.com>
The installer was merging the initial configuration into the custom
configuration, which meant that any keys set in the system configuration
would take precedence over custom configuration. Practically, this meant it
was not possible to preconfigure the database connection via
`config.custom.php`.
This commit reverses the order of the arguments to the `array_merge`
function so that keys in the custom configuration will override keys in the
initial configuration.
See also: https://github.com/FreshRSS/FreshRSS/discussions/8030
When manipulating I18N files, it is now possible to add a new file to all
languages. This action is available both in the manipulation script and
the makefile.
1. `include`, `include_once`, `require` and `require_once` are expressions not functions, parentheses are not necessary.
2. to move up the directory tree, it's better to use the `dirname` function instead of relying on `/..`.
* Optimize how much data needs to be `chown`/`chmod`ed on container startup
This works around an issue where `chmod`/`chown` operations inside a
container can be extremely slow when using the `overlay2` storage
driver, resulting in 10min+ container startup times.
It modifies the owner of the webapp when building the container so that
only the `data` and `extensions` directories (which are commonly mapped
as volumes into the container) have to be modified by the
`access-permissions.sh` script at container startup.
When not running via docker the behaviour of the `access-permissions.sh`
script is unchanged.
* Take DATA_PATH environment variable into account when fixing permissions
* Revert change to using bash for arrays
(the alpine image doesn't include `bash`)
* A few more improvements
* Slightly tweak reapply permissions variable
- lowercase to indicate it's not an env variable
- use 0/1 to address potentially-irrational paranoia about unset variables
* Remove conditional logic to skip reapplying permissions
Also documents why in a comment so it's not missed in the future.
---------
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
* Implement support for HTTP 429 Too Many Requests
Will obey the corresponding HTTP `Retry-After` header at domain level.
* Implement 503 Service Unavailable
* Sanitize Retry-After
* Reduce default value when Retry-After is absent
And make configuration parameter
* Retry-After also for favicons
* Show translation status in README.md
* Fix colon
* markdownlint: Allow tag `<translations>`
* Use mostly Unicode flags instead
* Only `oc.svg` remains in an image format
* `check.translation.php` still supports `.png` even though there aren't any PNGs as of right now
* Fix CodeSniffer
* Attempt approach with generating local SVGs
* Fixes for local SVG approach
* Cleanup old code
* PHPStan fix
* Remove decimal precision from percentages
* Suggestions + better error messages
* codesniffer fix v2
* Revert `ghSearchUrl` change
* Generate readme
* Fix syntax highlight, maybe
* Regenerate
* Update help message
* Use existing translation files instead of .txt
* Add test against wrong Unicode flag
---------
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
* PHPStan 2.0
fix https://github.com/FreshRSS/FreshRSS/issues/6989https://github.com/phpstan/phpstan/releases/tag/2.0.0https://github.com/phpstan/phpstan/blob/2.0.x/UPGRADING.md
* More
* More
* Done
* fix i18n CLI
* Restore a PHPStan Next test
For work towards PHPStan Level 10
* 4 more on Level 10
* fix getTagsForEntry
* API at Level 10
* More Level 10
* Finish Minz at Level 10
* Finish CLI at Level 10
* Finish Controllers at Level 10
* More Level 10
* More
* Pass bleedingEdge
* Clean PHPStan options and add TODOs
* Level 10 for main config
* More
* Consitency array vs. list
* Sanitize themes get_infos
* Simplify TagDAO->getTagsForEntries()
* Finish reportAnyTypeWideningInVarTag
* Prepare checkBenevolentUnionTypes and checkImplicitMixed
* Fixes
* Refix
* Another fix
* Casing of __METHOD__ constant
* Apache protect more non-public folders
* Also protect root
* Do the same for /p/
* Simplify Require all denied
In case of Apache 2.2, it will just make an error 500 instead of 403
* .htaccess.dist
* Simplify
* Better comment
* Minor update whitespace PHPCS rules
To simplify our configuration, apply more rules, and be clearer about what is added or removed compared with PSR12.
Does not change our current conventions, but just a bit more consistent.
* Forgotten *.phtml
* Sort exclusion patterns + add a few for Extensions repo
* Relaxed some rules
* CLI database backup and restore
Can also be used to migrate from one database to another (e.g. MySQL to PostgreSQL) or to ease upgrade to a major PostgreSQL version (e.g. 15 to 16).
* +x
* Fix some cases
* Update to docker-compose-v2
* More documentation
* Fix CLI install with prefix
It was not possible to pass a blank prefix
* Fix regression EXIT_CODE_ALREADY_EXISTS
The dedicated exit code was not sent anymore when a user already exists
