Commit Graph

141 Commits

Author SHA1 Message Date
Alexandre Alapetite
8f9c4143fc Better enclosures (#4944)
* Better enclosures
#fix https://github.com/FreshRSS/FreshRSS/issues/4702
Improvement of https://github.com/FreshRSS/FreshRSS/pull/2898

* A few fixes

* Better enclosure titles

* Improve thumbnails

* Implement thumbnail for HTML+XPath

* Avoid duplicate enclosures
#fix https://github.com/FreshRSS/FreshRSS/issues/1668

* Fix regex

* Add basic support for media:credit
And use <figure> for enclosures

* Fix link encoding + simplify code

* Fix some SimplePie bugs
Encoding errors in enclosure links

* Remove debugging syslog

* Remove debugging syslog

* SimplePie fix multiple RSS2 enclosures
#fix https://github.com/FreshRSS/FreshRSS/issues/4974

* Improve thumbnails

* Performance with yield
Avoid generating all enclosures if not used

* API keep providing enclosures inside content
Clients are typically not showing the enclosures to the users (tested with News+, FeedMe, Readrops, Fluent Reader Lite)

* Lint

* Fix API output enclosure

* Fix API content strcut

* API tolerate enclosures without a type
2023-01-06 19:53:43 +01:00
Alexandre Alapetite
4bf678f8e4 HTML+XPath allow content as HTML (#4878)
* HTML+XPath allow content as HTML
#fix https://github.com/FreshRSS/FreshRSS/issues/4869

* Wrong variable reuse

* Allow both HTML and expressions
2022-11-22 08:15:22 +01:00
Alexandre Alapetite
5897487f2f Fix path_entries encoding (#4823)
* Fix path_entries encoding
#fix https://github.com/FreshRSS/FreshRSS/issues/4815

* Fix preview
2022-11-07 08:34:12 +01:00
Alexandre Alapetite
e96b6266b5 Fix curlopt options for HTML+XPath (#4759)
Improvement of https://github.com/FreshRSS/FreshRSS/pull/4220
2022-10-21 15:11:52 +02:00
Alexandre Alapetite
648a876d77 Add support for custom XPath date/time format (#4703)
* Add support for custom XPath date/time format
#fix https://github.com/FreshRSS/FreshRSS/issues/4701
Improvement of https://github.com/FreshRSS/FreshRSS/pull/4220

* Format is not XPath

* Remove TODOs in en-GB
2022-10-09 15:43:30 +02:00
Alexandre Alapetite
e626fd249e Fix regression 4688 (#4700)
Minor regression from https://github.com/FreshRSS/FreshRSS/pull/4688
2022-10-07 12:59:31 +02:00
Alexandre Alapetite
82c4a54fea Compat PHP 8.1 strip_tags (#4688)
#fix https://github.com/FreshRSS/FreshRSS/issues/4687
2022-10-04 23:07:17 +02:00
Alexandre Alapetite
1603c10bba XPath ability to define the UID manually (#4507)
* XPath ability to define the UID manually

* Fix error in i18n
2022-08-18 12:10:55 +02:00
Alexandre Alapetite
82ac1d1e67 Refactor entry-to-GReader API format (#4490)
* Refactor entry to GReader API format
Some code was copied in two locations and not completely uniform.
Cleaning of related variables and functions (e.g. better types for entries and categories as objects vs. as IDs).
Usecase: I need to call the same GReader-compatible serialization from an extension

* Fixed some edge cases

* Keep summary instead of content
`summary` and `content` seems to be used interchangeably in the Google Reader API. We have been using `summary` for our client API and  `content` in our export/import, so stick to that.
2022-08-08 12:04:02 +02:00
Alexandre Alapetite
509c8cae63 Dynamic OPML (#4407)
* Dynamic OPML draft
#fix https://github.com/FreshRSS/FreshRSS/issues/4191

* Export dynamic OPML
http://opml.org/spec2.opml#1629043127000

* Restart with simpler approach

* Minor revert

* Export dynamic OPML also for single feeds

* Special category type for importing dynamic OPML

* Parameter for excludeMutedFeeds

* Details

* More draft

* i18n

* Fix update

* Draft manual import working

* Working manual refresh

* Draft automatic update

* Working Web refresh + fixes

* Import/export dynamic OPML settings

* Annoying numerous lines in SQL logs

* Fix minor JavaScript error

* Fix auto adding new columns

* Add require

* Add missing 🗲

* Missing space

* Disable adding new feeds to dynamic categories

* Link from import

* i18n typo

* Improve theme icon function

* Fix pink-dark
2022-07-04 09:53:26 +02:00
Alexandre Alapetite
d785ddde2a New option to automatically mark as read gone articles (#4426)
* New option to automatically mark as read gone articles
Option to automatically and immediately mark as read entries / articles that are no longer provided in their upstream RSS / ATOM / XPath feed

* Reduce SQL queries
Optimisation: Perform cache update only once
2022-06-25 11:15:51 +02:00
Alexandre Alapetite
2a0b47a8a4 No XPath error on empty list (#4425)
Set feed error state to true if the *list* of items cannot be find by XPath, but do not set the error state to true if that list happens to be empty (the resulting feed will be with an *empty* state instead of *error* state)
Follow-up of https://github.com/FreshRSS/FreshRSS/pull/4220
2022-06-23 13:26:22 +02:00
Alexandre Alapetite
f365a9aeb4 Update all test dependencies (#4419)
* Update all test dependencies

* Remove old false-positive

* Minor update lock files

* Increase PHPStan memory for Fedora
https://github.com/FreshRSS/FreshRSS/pull/4400#issuecomment-1159514197

* Require PHP8+ for tests
Due to small changes of signature in `ob_implicit_flush` and `simplexml_load_string`, cf. https://github.com/FreshRSS/FreshRSS/pull/4123

* Missing lint in CSS files
2022-06-19 20:08:42 +02:00
Alexandre Alapetite
7f0c378482 Fix favicons update (#4358)
Better update the favicons when the URL changes
2022-05-13 23:14:09 +02:00
Alexandre Alapetite
4a87206f28 OPML export/import of some proprietary FreshRSS attributes (#4342)
* OPML export/import of some proprietary FreshRSS attributes
#fix https://github.com/FreshRSS/FreshRSS/issues/4077
And one of the TODOs of https://github.com/FreshRSS/FreshRSS/pull/4220
XPath options, CSS Selector, and action filters

* Bump library patch version

* OPML namespace + documentation

* Add example
2022-05-12 22:15:10 +02:00
Alexandre Alapetite
954cc0510d HTML+XPath fallback to feed title (#4288)
* HTML+XPath fallback to feed title
Except when adding the feed, the following refreshes should fallback to feed title when generating the internal RSS representation.
Improvement of https://github.com/FreshRSS/FreshRSS/pull/4220
2022-03-23 11:39:31 +01:00
Alexandre Alapetite
e8af54a476 WebSub: Use hash instead of base64 to handle long URLs (#4282)
* WebSub: Use hash instead of base64 to handle long URLs

* Use 410 Gone
(Part of the WebSub specification https://www.w3.org/TR/websub/ )
2022-03-22 23:13:13 +01:00
Alexandre Alapetite
b0a63355b6 SimplePie fix parsing of HTTP Links (#4283)
* SimplePie fix parsing of HTTP Links
* https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Link
* https://datatracker.ietf.org/doc/html/rfc8288

Before, SimplePie was not able to parse something like

```
Link: <https://pubsubhubbub.appspot.com>; rel="hub", <https://pubsubhubbub.superfeedr.com>; rel=hub, <https://websubhub.com/hub>; rel="hub"
```
2022-03-22 23:12:22 +01:00
Alexandre Alapetite
cc6deadf69 Faster $simplePie->get_items() (#4263)
Implement one of the TODOs from https://github.com/FreshRSS/FreshRSS/pull/4220 :
use `$simplePie->get_items()` instead of very slow `$simplePie->get_item($i)`
2022-03-06 23:36:35 +01:00
Alexandre Alapetite
fd945ffb93 Fix XPath context for tags (#4246)
#fix https://github.com/FreshRSS/FreshRSS/issues/4245
Follow-up of https://github.com/FreshRSS/FreshRSS/pull/4220
2022-03-02 17:44:52 +01:00
Alexandre Alapetite
1fe66ad020 Implement Web scraping "HTML + XPath" (#4220)
* More PHP type hints for Fever
Follow-up of https://github.com/FreshRSS/FreshRSS/pull/4201
Related to https://github.com/FreshRSS/FreshRSS/issues/4200

* Detail

* Draft

* Progress

* More draft

* Fix thumbnail PHP type hint
https://github.com/FreshRSS/FreshRSS/issues/4215

* More types

* A bit more

* Refactor FreshRSS_Entry::fromArray

* Progress

* Starts to work

* Categories

* Fonctional

* Layout update

* Fix relative URLs

* Cache system

* Forgotten files

* Remove a debug line

* Automatic form validation of XPath expressions

* data-leave-validation

* Fix reload action

* Simpler examples

* Fix column type for PostgreSQL

* Enforce HTTP encoding

* Readme

* Fix get full content

* target="_blank"

* gitignore

* htmlspecialchars_utf8

* Implement HTML <base>
And fix/revert `xml:base` support in SimplePie e49c578817

* SimplePie upstream PR merged
https://github.com/simplepie/simplepie/pull/723
2022-02-28 20:22:43 +01:00
Alexandre Alapetite
946b0a0876 Hot fix: fix type hint null regression (#4207) 2022-02-07 01:35:29 +01:00
Alexandre Alapetite
1c5cf71859 Fix Fever 32 bit ID issue + more PHP type hints (#4201)
* Fix Fever 32 bit ID issue + more PHP type hints
#fix https://github.com/FreshRSS/FreshRSS/issues/4200
Follow up and fix regression from https://github.com/FreshRSS/FreshRSS/pull/4110

* More PHP type hints with PHPStan

* Fix pull problem

* Avoid more nulls
2022-02-06 14:31:36 +01:00
Alexandre Alapetite
1acd3ab09b Use typographic quotes (#4133)
* Use typographic quotes

* A few fixes

* Fix

* Fix not saved

* Implement feedback

* Detail

* Revert spoken English fixes
Left for a future dedicated discussion

* More reverts

* Final reverts

* Final minor
2022-01-08 21:58:55 +01:00
Frans de Jonge
d339b6dd45 [CI] PHPCS: check for opening brace on same line (#4122)
* [CI] PHPCS: check for opening brace on same line

* make fix-all

* Minor comments

Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
2022-01-05 00:52:24 +01:00
Alexandre Alapetite
1335a0e3cf PHPStan level 5 (#4110)
* Fix most PHPDocs errors
Contributes to https://github.com/FreshRSS/FreshRSS/issues/4103
https://phpstan.org/writing-php-code/phpdoc-types

* Avoid func_get_args
Use variadic syntax instead https://php.net/manual/functions.arguments#functions.variable-arg-list
And avoid dynamic functions names when possible to more easily identify calls and unused functions.
Contributes to https://github.com/FreshRSS/FreshRSS/issues/4103

* PHPStan level 3

* PHPStand level 4

* Update default to PHPStan level 4

* Towards level 5

* Fix level 4 regression

* Towards level 5

* Pass PHPStan level 5

* Towards level 6

* Remove erronenous regression from changelog
https://github.com/FreshRSS/FreshRSS/pull/4116
2022-01-04 13:59:09 +01:00
Alexandre Alapetite
4e00c2d337 Fix more PHP 8.1 null warnings (#4018)
Follow-up of https://github.com/FreshRSS/FreshRSS/pull/4012
2021-12-02 21:05:28 +01:00
Alexandre Alapetite
a473b90ff9 Detect enclosure as images using file extension (#3861)
* Detect enclosure as images using file extension
Support e.g. https://www.sciencesetavenir.fr/atom.xml , which does not include any type hint.

```html
<link rel="enclosure" href="https://example.net/assets/img/ZZZ.jpg" title="YYY"/>
```

Useful for https://github.com/FreshRSS/FreshRSS/pull/3805

* Add AVIF and WebP
2021-09-27 22:24:26 +02:00
Alexandre Alapetite
a7aca6c0ab Improved feed action filters (#3303)
* Re-order some feed options
* Option to auto mark as read existing titles
* Option to keep at max n unread articles per feed
2021-09-19 10:56:38 +02:00
Alexandre Alapetite
ee2d2db780 Fallback for feeds with empty title (#3787)
* Fallback for feeds with empty title
Address a part of https://github.com/FreshRSS/FreshRSS/issues/3776 for existing feeds

* Also strip www prefix

* Reuse fallback logic
2021-08-22 13:33:58 +02:00
Alexandre Alapetite
87d1036647 Fix regression media attachments (#3565)
#fix https://github.com/FreshRSS/FreshRSS/issues/3564
Bug introduced by https://github.com/FreshRSS/FreshRSS/pull/3361
2021-03-27 22:16:44 +01:00
Alexandre Alapetite
eeff1a17b0 Suport standard HTTP 410 Gone (#3561)
When a feed returns an HTTP 410 Gone, mute the corresponding feed, i.e. stop refreshing it.
Example of such feed, Les Décodeurs (Libération) https://rss.liberation.fr/rss/100893/
2021-03-26 19:13:23 +01:00
Alexandre Alapetite
b7fdfbb894 SimplePie prevent cache pollution (#3502)
* SimplePie prevent cache polution
#fix https://github.com/FreshRSS/FreshRSS/pull/3367#issuecomment-766250249
#fix https://github.com/FreshRSS/FreshRSS/pull/3494#issuecomment-790113663

* Fix bug

* Minor improvement

* Update cache filename in FreshRSS (1/2)

* cacheFilename temp

* New SimplePie get_cache_filename()

* Fix typos

* Update lib/SimplePie/SimplePie.php

Typo

* Include user-agent and timeout

* fix array_merge

* Declaration

* force_feed was lost in a commit
2021-03-09 08:41:47 +01:00
Alexandre Alapetite
ef4a826e34 Reload full content when changed (#3506)
* Reload full content when changed
If an article is changed, reload also its full content when applicable.

* Compute hash before getting full content

* Revert mix two PRs

* Update app/Controllers/feedController.php
2021-03-09 08:41:01 +01:00
Alexandre Alapetite
6d8b1ba834 Fix images in enclosures without MIME type (#3361)
#fix https://github.com/FreshRSS/FreshRSS/issues/3358
Assume enclosures without a mime type, without a length, but with either
a width or a height are images
2021-01-12 09:34:13 +01:00
Alexandre Alapetite
c523f5a4e7 Relaxed to allow underscore _ in URL hostnames (#3133)
* Relaxed to allow underscore _ in URL hostnames

#fix https://github.com/FreshRSS/FreshRSS/issues/3132
https://bugs.php.net/bug.php?id=64948

* Minor fix in favicons redirect

* Minor fix in relative HREF for favicons
2020-07-31 16:33:45 +02:00
Alexandre Alapetite
3f9518b377 Hide feed credentials when adding a new feed (#3099)
Avoid showing HTTP Basic Auth credentials in feed error log and in feed
title (when there is no title provided)
2020-07-07 10:54:28 +02:00
Alexandre Alapetite
83b5944dcb Stream instead of memory copy of SimplePie entries (#2972)
* Stream instead of memory copy of SimplePie entries

https://github.com/FreshRSS/FreshRSS/issues/2952

* Undo lines delete

* Typo

* Remove unaccessible code

https://github.com/FreshRSS/FreshRSS/pull/2972/files#r425624163

* Back-compatibility for Feed->entries

https://github.com/FreshRSS/FreshRSS/pull/2972/files#r425631913
2020-05-17 00:04:51 +02:00
Alexandre Alapetite
aea3806590 Attempt to reduce max memory usage during actualize (#2955)
* Attempt to reduce max memory usage during actualize

#Fix https://github.com/FreshRSS/FreshRSS/issues/2952

* Use memory_get_peak_usage
2020-05-10 21:34:12 +02:00
Alexandre Alapetite
b1aac20839 Fever integer type for numbers (#2946)
* Fever integer type for feed ID

#Fix
https://github.com/FreshRSS/FreshRSS/issues/2940#issuecomment-623022435

* Fix feed_id

* Ensure string for entry ID

To be compatible with 32-bit platforms.
Before, the type was inconsistent depending on architecture + database

* Integer for entry count functions
2020-05-03 13:39:16 +02:00
Alexandre Alapetite
83ba09c2a5 Fever integer type for last_refreshed_on_time 2 (#2945)
#Fix fever_last_refreshed_on_time
Try again https://github.com/FreshRSS/FreshRSS/pull/2944
2020-05-02 23:55:10 +02:00
Alexandre Alapetite
dba40e5870 WebSub move fixes (#2922)
Small fixes related to when WebSub changes address:
* When unsubscribing, one must use the current/old address, not the
newly found selfUrl;
* This change
https://github.com/FreshRSS/FreshRSS/pull/2659#discussion_r347263068 was
wrong, so reverted to the first version. We must obey the rules also for
feeds for which the initial WebSub enabling is not finished, otherwise
we never have a chance to redirect the feed to the proper selfUrl.
2020-04-24 16:29:19 +02:00
Alexandre Alapetite
bdc4da6ad0 Expose podcast in API (#2898)
* Expose podcast in API

Expose RSS enclosures in our API, e.g. for clients supporting podcasts

* PHP CS?

* Annoying linter

* Light optimisation
2020-04-16 18:11:26 +02:00
Alexandre Alapetite
925319653d No pre for enclosure-description (#2808)
* No pre for enclosure-description

#fix https://github.com/FreshRSS/FreshRSS/issues/2807
In browsers, the styling `white-space:pre-line` will be applied by CSS,
but no styling will be done for clients via API.

* Corresponding CSS change
2020-02-29 15:58:31 +01:00
Alexandre Alapetite
2aff347b2e Fix wrong getHeader refactoring (#2749)
* Fix wrong getHeader refactoring

Fix regression introduced by
https://github.com/FreshRSS/FreshRSS/pull/2373
The refactoring required a call to init() even for static functions,
which is most of the time not done.
Removed premature abstraction of `$_SERVER`, which was the root cause of
the bug.
https://github.com/FreshRSS/FreshRSS/issues/2748#issuecomment-569898931

* Refactoring: Move serverIsPublic to Minz_Request

* Add mitigations for wrong configurations

Due to the regression, we have some existing configurations with a bad
base_url

* Forgot one instance
2020-01-04 23:35:42 +01:00
Alexis Degrugillier
cc0db9af4f Feature/new archiving (#2335)
* Change archiving config page layout

I've changed some wording and moved actions into a
maintenance section.

* Update purge action

Now we have more control on the purge action. The configuration allows
us to choose what to keep and what to discard in a more precise way.
At the moment, the configuration applies for all feeds.

* Add purge configuration on feed level

Now the extend purge configuration is available on feed level.
It is stored as attributes and will be used in the purge action.

* Update purge action

Now the purge action uses the feed configuration if it exists and
defaults on user configuration if not.

* Add empty option in period list

* Fix configuration warnings

* Add archiving configuration on categories

See #2369

* Add user info back

* Add explanations in UI

* Fixes for SQLite + error + misc.

* Fix invalid feed reference

* Short array syntax

Only for new code, so far

* Fix prefix error

* Query performance, default values

Work in progress

* Fix default values and confirm before leaving

Form cancel and confirm changes before leaving were broken.
And start taking advantage of the short echo syntax `<?= ?>` as we have
moved to PHP 5.4+

* More work

* Tuning SQL

* Fix MariaDB + performance issue

* SQL performance

* Fix SQLite bug

* Fix some attributes JSON encoding bugs

Especially for SQLite export/import

* More uniform, fix bugs

More uniform between global, category, feed settings

* Drop special cases for old articles during refresh

Instead will use lastSeen date with the new archiving logic.
This was generating problems anyway
https://github.com/FreshRSS/FreshRSS/issues/2154

* Draft drop index keep_history

Not needed anymore

* MySQL typo

Now properly tested with MySQL, PostgreSQL, SQLite

* More work for legacy values

Important to avoid overriding user's preference and risking deleting
data erroneously

* Fix PHP 7.3 / 7.4 warnings

@aledeg "Trying to use values of type null, bool, int, float or resource
as an array (such as $null["key"]) will now generate a notice. "
https://php.net/migration74.incompatible

* Reintroduce min articles and take care of legacy parameters

* A few changes forgotten

* Draft of migration + DROP of feed.keep_history

* Fix several errors

And give up using const for SQL to allow multiple database types (and we
cannot redefine a const)

* Add keep_min to categories + factorise archiving logic

* Legacy fix

* Fix bug yield from

* Minor: Use JSON_UNESCAPED_SLASHE for attributes

And make more uniform

* Fix sign and missing variable

* Fine tune the logic
2019-10-23 00:52:15 +02:00
Alexandre Alapetite
e3e5954394 PDO refactoring for code simplification (#2522)
* PDO refactor

* Automatic prefix when using the syntax `_tableName`
* Uniformity: MySQL is now PDO::ATTR_EMULATE_PREPARES = false just like SQLite and PostgreSQL, with consequences such as only one statement per query
* Use PDO methods exec(), query(), prepare() + execute() in a more efficient way
* Remove auto-update SQL code for versions older than FreshRSS 1.5 (3 years old)
* The name of the default category is set in PHP instead of in the DB (simplies SQL and allows changing the name according to the FreshRSS language)
* Rename `->bd` to `->pdo` (less of a frenshism, and more informative)
* Fix some requests, which were not compatible with MySQL prepared statements

* Whitespace

* Fix syntax for PostgreSQL sequences

+ MySQL install

* Minor formatting

* Fix lastInsertId for PostgreSQL

* Use PHP 5.6+ const

Take advantage of https://github.com/FreshRSS/FreshRSS/pull/2527
https://www.php.net/manual/en/migration56.new-features.php

* A bit of forgotten PHP 5.6 simplification for cURL

* Forgotten $s

* Mini fix custom user config

https://github.com/FreshRSS/FreshRSS/pull/2490/files#r326290346

* More work on install.php but not finished

* install.php working

* More cleaning of PDO in install

* Even more simplification

Take advantage of PDO->exec() to run multiple statements

* Disallow changing the name of the default category

https://github.com/FreshRSS/FreshRSS/pull/2522#discussion_r326967724
2019-09-29 16:22:50 +02:00
Alexandre Alapetite
1804c0e0bc Filter actions (#2275)
* Draft of filter actions

* Travis

* Implement UI + finish logic

* Travis
2019-03-23 22:52:47 +01:00
Alexandre Alapetite
512d047f02 Update naming to WebSub (#2184)
Instead of PubSubHubbub / PuSH
2018-12-16 17:02:03 +01:00
Alexandre Alapetite
6122a45d9a Minor additional WebSub debug 2018-11-15 17:07:05 +01:00