Commit Graph

76 Commits

Author SHA1 Message Date
Alexandre Alapetite
4f316b2ed3 PHPStan level 9 for ./p/ and lib_rss.php (#5049)
And app/FreshRSS.php
Contributes to https://github.com/FreshRSS/FreshRSS/issues/4112
2023-01-29 18:53:51 +01:00
Alexandre Alapetite
1d9d4e3e3c Update dev dependencies (#4993)
Related to https://github.com/FreshRSS/FreshRSS/pull/4991
Required a few changes in code to pass the tests
2023-01-09 12:59:30 +01:00
Alexandre Alapetite
8f9c4143fc Better enclosures (#4944)
* Better enclosures
#fix https://github.com/FreshRSS/FreshRSS/issues/4702
Improvement of https://github.com/FreshRSS/FreshRSS/pull/2898

* A few fixes

* Better enclosure titles

* Improve thumbnails

* Implement thumbnail for HTML+XPath

* Avoid duplicate enclosures
#fix https://github.com/FreshRSS/FreshRSS/issues/1668

* Fix regex

* Add basic support for media:credit
And use <figure> for enclosures

* Fix link encoding + simplify code

* Fix some SimplePie bugs
Encoding errors in enclosure links

* Remove debugging syslog

* Remove debugging syslog

* SimplePie fix multiple RSS2 enclosures
#fix https://github.com/FreshRSS/FreshRSS/issues/4974

* Improve thumbnails

* Performance with yield
Avoid generating all enclosures if not used

* API keep providing enclosures inside content
Clients are typically not showing the enclosures to the users (tested with News+, FeedMe, Readrops, Fluent Reader Lite)

* Lint

* Fix API output enclosure

* Fix API content strcut

* API tolerate enclosures without a type
2023-01-06 19:53:43 +01:00
Alexandre Alapetite
5897487f2f Fix path_entries encoding (#4823)
* Fix path_entries encoding
#fix https://github.com/FreshRSS/FreshRSS/issues/4815

* Fix preview
2022-11-07 08:34:12 +01:00
Alexandre Alapetite
3b81708c1a API protection limit content size (#4583)
* API protection limit content size
We had no limit to the length of an article sent through our GReader API to clients, which was a problem:

The article https://devblogs.microsoft.com/dotnet/performance_improvements_in_net_7/
from the feed https://devblogs.microsoft.com/dotnet/feed/
is about 700KB long, which makes some clients (tested with News+) fail.

We should try other clients to verify that 500KB is a safe enough limit.

* Light optimisation

* Use class constant
2022-09-05 13:13:07 +02:00
Alexandre Alapetite
a8353e4b4c Compat Readrops (#4568)
While waiting for https://github.com/readrops/Readrops/pull/163
#fix https://github.com/FreshRSS/FreshRSS/issues/4567
2022-08-31 13:00:46 +02:00
Alexandre Alapetite
e27eb1ca91 Basic support for negative searches with parentheses (#4503)
* Basic support for negative searches with parentheses
* `!((author:Alice intitle:hello) OR (author:Bob intitle:world))`
* `(author:Alice intitle:hello) !(author:Bob intitle:world)`
* `!(S:1 OR S:2)`

* Minor documentation / comment

* Remove syslog debug line
2022-08-16 10:56:07 +02:00
papaschloss
8587efa621 Article css filtering (#4501)
* Update feedController.php

* Update subscriptionController.php

* Update DatabaseDAO.php

* Update Entry.php

* Update Feed.php

* Update FeedDAO.php

* Update install.sql.mysql.php

* Update install.sql.pgsql.php

* Update install.sql.sqlite.php

* Update sub.php

* Update opml.phtml

* Update ImportService.php

* Update update.phtml

* Update feed.js

* Update install.sql.mysql.php

* Update install.sql.pgsql.php

* Update install.sql.sqlite.php

* Update sub.php

* Update sub.php

* Update sub.php

* Update sub.php

* Update sub.php

* Update sub.php

* Update sub.php

* Update sub.php

* Update sub.php

* Update sub.php

* Update sub.php

* Update sub.php

* Update sub.php

* Update sub.php

* Update sub.php

* Update sub.php

* Update sub.php

* Update FeedDAO.php

* Update feedController.php

* Update subscriptionController.php

* Update Entry.php

* Update Feed.php

* Update feedController.php

* Update subscriptionController.php

* Update ImportService.php

* Update opml.phtml

* Update update.phtml

* Update update.phtml

* Update update.phtml

* Update DatabaseDAO.php

* Update app/Models/Entry.php

Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>

* Update app/i18n/fr/sub.php

Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>

* Update p/scripts/feed.js

Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>

* Update app/Controllers/feedController.php

* make fix-all

* Update documentation

* css_path_filter help message

* i18n en-us ignore

Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
2022-08-15 21:01:18 +02:00
Alexandre Alapetite
96e0efa6f0 Get content by parsing: fix base for protocol-relative URLs (#4500)
* Get content by parsing: fix base for protocol-relative URLs

* Guess missing URL scheme for base

* Light refactoring
2022-08-12 13:06:04 +02:00
Alexandre Alapetite
82ac1d1e67 Refactor entry-to-GReader API format (#4490)
* Refactor entry to GReader API format
Some code was copied in two locations and not completely uniform.
Cleaning of related variables and functions (e.g. better types for entries and categories as objects vs. as IDs).
Usecase: I need to call the same GReader-compatible serialization from an extension

* Fixed some edge cases

* Keep summary instead of content
`summary` and `content` seems to be used interchangeably in the Google Reader API. We have been using `summary` for our client API and  `content` in our export/import, so stick to that.
2022-08-08 12:04:02 +02:00
Alexandre Alapetite
0866fdaee8 Add database field attributes for entries (#4444)
* Add database field attributes for entries
Just like we already have for categories, feeds, etc.
No core use yet, but allows in particular extensions to save per-entry data

* Fix PHPStand

* Fix wrong variable
2022-07-19 15:17:08 +02:00
Alexandre Alapetite
509c8cae63 Dynamic OPML (#4407)
* Dynamic OPML draft
#fix https://github.com/FreshRSS/FreshRSS/issues/4191

* Export dynamic OPML
http://opml.org/spec2.opml#1629043127000

* Restart with simpler approach

* Minor revert

* Export dynamic OPML also for single feeds

* Special category type for importing dynamic OPML

* Parameter for excludeMutedFeeds

* Details

* More draft

* i18n

* Fix update

* Draft manual import working

* Working manual refresh

* Draft automatic update

* Working Web refresh + fixes

* Import/export dynamic OPML settings

* Annoying numerous lines in SQL logs

* Fix minor JavaScript error

* Fix auto adding new columns

* Add require

* Add missing 🗲

* Missing space

* Disable adding new feeds to dynamic categories

* Link from import

* i18n typo

* Improve theme icon function

* Fix pink-dark
2022-07-04 09:53:26 +02:00
Alexandre Alapetite
f85c510ed4 New search engine (#4378)
* New possibility to invoke user queries from a search expression
From the search field: `S:"My query"`.
Can be combined with other filters such as `S:"My query" date:P3d` as long as the user queries do not contain `OR`.
A use-case is to have an RSS filter with a stable address or an external API call with the ability to update the user query.

* Draft of parenthesis logic

* More draft

* Working parenthesis (a OR b) (c OR d)

* Working (A) OR (B)

* Support nested parentheses + unit tests + documentation

* search:MySearch and S:3
2022-06-02 08:41:08 +02:00
Alexandre Alapetite
92b0ffe05c Fix handling of authors with ampersand & (#4287)
Especially needed for HTML+XPath Web scraping as it makes the rest of the pipeline fail to handle special characters.
2022-03-23 11:29:33 +01:00
Alexandre Alapetite
ae54a590b9 Replace lib_phpQuery by PhpGt/CssXPath (#4261)
https://github.com/PhpGt/CssXPath
2022-03-16 15:10:41 +01:00
Alexandre Alapetite
dc849c3d87 Fix HTML base for get full content (#4260)
#fix https://github.com/FreshRSS/FreshRSS/issues/4252
2022-03-05 16:00:01 +01:00
Alexandre Alapetite
12000df805 Fix XPath thumbnail bug (#4250)
#fix https://github.com/FreshRSS/FreshRSS/issues/4248
2022-03-03 14:27:55 +01:00
Alexandre Alapetite
467ca9d0de Specific HTML load for lib_phpQuery.php (#4243)
And reverts a less appropriate workaround for the same thing in https://github.com/FreshRSS/FreshRSS/pull/4220
Avoids XML errors when believing that a document might be XML/XHTML.
2022-03-01 16:03:44 +01:00
Alexandre Alapetite
1fe66ad020 Implement Web scraping "HTML + XPath" (#4220)
* More PHP type hints for Fever
Follow-up of https://github.com/FreshRSS/FreshRSS/pull/4201
Related to https://github.com/FreshRSS/FreshRSS/issues/4200

* Detail

* Draft

* Progress

* More draft

* Fix thumbnail PHP type hint
https://github.com/FreshRSS/FreshRSS/issues/4215

* More types

* A bit more

* Refactor FreshRSS_Entry::fromArray

* Progress

* Starts to work

* Categories

* Fonctional

* Layout update

* Fix relative URLs

* Cache system

* Forgotten files

* Remove a debug line

* Automatic form validation of XPath expressions

* data-leave-validation

* Fix reload action

* Simpler examples

* Fix column type for PostgreSQL

* Enforce HTTP encoding

* Readme

* Fix get full content

* target="_blank"

* gitignore

* htmlspecialchars_utf8

* Implement HTML <base>
And fix/revert `xml:base` support in SimplePie e49c578817

* SimplePie upstream PR merged
https://github.com/simplepie/simplepie/pull/723
2022-02-28 20:22:43 +01:00
Alexandre Alapetite
4b9d66faca Fix thumbnail PHP type hint (#4216)
* Fix thumbnail PHP type hint
 https://github.com/FreshRSS/FreshRSS/issues/4215

* Blank
2022-02-09 23:37:13 +01:00
Alexandre Alapetite
1c5cf71859 Fix Fever 32 bit ID issue + more PHP type hints (#4201)
* Fix Fever 32 bit ID issue + more PHP type hints
#fix https://github.com/FreshRSS/FreshRSS/issues/4200
Follow up and fix regression from https://github.com/FreshRSS/FreshRSS/pull/4110

* More PHP type hints with PHPStan

* Fix pull problem

* Avoid more nulls
2022-02-06 14:31:36 +01:00
Alexandre Alapetite
1acd3ab09b Use typographic quotes (#4133)
* Use typographic quotes

* A few fixes

* Fix

* Fix not saved

* Implement feedback

* Detail

* Revert spoken English fixes
Left for a future dedicated discussion

* More reverts

* Final reverts

* Final minor
2022-01-08 21:58:55 +01:00
Alexandre Alapetite
1335a0e3cf PHPStan level 5 (#4110)
* Fix most PHPDocs errors
Contributes to https://github.com/FreshRSS/FreshRSS/issues/4103
https://phpstan.org/writing-php-code/phpdoc-types

* Avoid func_get_args
Use variadic syntax instead https://php.net/manual/functions.arguments#functions.variable-arg-list
And avoid dynamic functions names when possible to more easily identify calls and unused functions.
Contributes to https://github.com/FreshRSS/FreshRSS/issues/4103

* PHPStan level 3

* PHPStand level 4

* Update default to PHPStan level 4

* Towards level 5

* Fix level 4 regression

* Towards level 5

* Pass PHPStan level 5

* Towards level 6

* Remove erronenous regression from changelog
https://github.com/FreshRSS/FreshRSS/pull/4116
2022-01-04 13:59:09 +01:00
Alexandre Alapetite
f13ef3b102 Fix article filter (#4092)
#fix https://github.com/FreshRSS/FreshRSS/issues/4057
Several search filters where missing for the automatic article actions
2021-12-31 13:13:03 +01:00
Alexandre Alapetite
a7aca6c0ab Improved feed action filters (#3303)
* Re-order some feed options
* Option to auto mark as read existing titles
* Option to keep at max n unread articles per feed
2021-09-19 10:56:38 +02:00
ORelio
50ba6bbe07 UI: Add optional thumbnail and summary on feed items (#3805)
* UI: Add optional thumbnail and summary on feed items

Implements #561

* UI: Thumbnail: Own column, Custom size, Lazy load

* UI: Thumbnail: Remove unnecessary CSS rule

Remove rule already defined in base theme, no override needed

* CSS lint + RTL

* Improve thumbail and summary generation

* Support img alt

* Missing htmlspecialchars

Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
2021-08-30 10:58:06 +02:00
Alexandre Alapetite
812eda1fa0 Fix wrong method name (#3808)
Fix error from https://github.com/FreshRSS/FreshRSS/pull/2275 but that line was not used just yet anyway
2021-08-30 08:02:39 +02:00
Alexandre Alapetite
ef4a826e34 Reload full content when changed (#3506)
* Reload full content when changed
If an article is changed, reload also its full content when applicable.

* Compute hash before getting full content

* Revert mix two PRs

* Update app/Controllers/feedController.php
2021-03-09 08:41:01 +01:00
Alexandre Alapetite
947e918f05 Travis: Enforce phpcs line length + whitespace (#3488)
* Update Travis line length

* Also check whitespace in CSS files

* Fix line length ext.php

* More syntax, string templates

* Fix exclude-pattern

* Test JS files as well
2021-02-28 12:26:24 +01:00
Alexis Degrugillier
9682fcfebb Add full content delimiter and action (#3463)
Before, when appending or prepending the content of the CSS selector
content, it was added to the content. It was working fine for the
first call but every subsequent calls were pilling the retrieved
content on top of the already retrieved content. Thus we had an ever
growing content with a lot of duplication.
Now, the CSS selector content is identified by an HTML comment which
is used to remove the content for every subsequent calls.

The bug was introduced in #3453
2021-02-20 00:30:18 +01:00
Alexis Degrugillier
fe1fba9828 Add a content action parameter to work with CSS selector (#3453)
Before, when retrieving article contents with CSS selector, the content
of the RSS feed was replaced by the content of the CSS selector. It works
well in most of the cases but if there is a different content in the feed
and in the selector, the former is lost.
Now, there is a parameter to decide which action is performed after retrieving
the content. By default, the previous behavior is kept. But now, it is
possible to append or prepend the CSS selector content to the feed content.

See #3451
2021-02-16 18:00:49 +01:00
Eris
ee175dd616 CURLOPT parameters per feed (#3367)
* Working curlopt_params

* Examples

* curl_params for fetching the article

* cleanup

* clarification

* Remove debugging

* Options corrected

* Removed Debugging

* i18n not needed (right now)

* Translations and UI rework

* Checks in update.phtml

* Unset Proxy&Cookies

* remove clutter

* minor fuckup

* i18n added properly

* resolve Errors

* linting errors

* linting errors, again

* Review

* Minor revert

* Minor i18n: de

Co-authored-by: maru <maru@nyx.im>
Co-authored-by: Aeris <a3x@eris.cc>
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
2021-01-16 22:32:18 +01:00
Alexandre Alapetite
ee7938ed5f Do not import feed causing DB error (#3347)
* Do not import feed causing DB error

The DB error might be that the new feed tries to redirect to an already
existing feed, in which case
#fix https://github.com/FreshRSS/FreshRSS/issues/3339

* Add feed bug
2021-01-09 20:51:33 +01:00
Alexandre Alapetite
9c6682e7ed Avoid manual initialisations of system or user configuration (#3070)
* Avoid manual intialisations of system or user configuration

More consistent use of Context

* Simplify FreshRSS_Context::initUser

* Remove a few manual get_user_configuration

* A bit of debugging

* Fix context user init

* Fix install

* Fix concurrency

Concurrent requests could lead to bad race condition

* Fix actualize cron

Fix case when system i initialised several times
2021-01-02 21:20:19 +01:00
Alexandre Alapetite
be5f9f2dce When ssl_verify is false, decrease SSL security (#3171)
When ssl_verify option is set to false for a feed, allow lower security such as SHA-1 signatures.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=900984
https://stackoverflow.com/questions/58342699/php-curl-curl-error-35-error1414d172ssl-routinestls12-check-peer-sigalgwr
Fix error of type `cURL error 35: error:1414D172:SSL routines:tls12_check_peer_sigalg:wrong signature`
Example of feeds:
https://www.version2.dk/it-nyheder/rss
https://ing.dk/rss/nyheder
2020-09-03 10:35:23 +02:00
Alexis Degrugillier
8d8211b2a5 Add machine-readable dates in views (#3106)
This will allow to make some transformations on dates when needed.
See #3103 and #853
2020-07-14 10:51:45 +02:00
Alexandre Alapetite
e1ef88d815 Fix memory leak when using lib_phpQuery (#3004)
* Fix memory leak when using lib_phpQuery

This library keeps documents in static class, so it requires a special
call to clean them.

* Another unload
2020-05-25 16:02:52 +02:00
Alexandre Alapetite
b906d79d61 getContentByParsing follow HTML redirections (#2985)
* getContentByParsing follow HTML redirections

Add the ability to follow HTML redirections in getContentByParsing:

```html
<meta http-equiv="Refresh" content="1; url=https://example.net/article123.html" />
```

* Better regex

* Trim http-equiv
2020-05-21 11:56:55 +02:00
Alexandre Alapetite
83b5944dcb Stream instead of memory copy of SimplePie entries (#2972)
* Stream instead of memory copy of SimplePie entries

https://github.com/FreshRSS/FreshRSS/issues/2952

* Undo lines delete

* Typo

* Remove unaccessible code

https://github.com/FreshRSS/FreshRSS/pull/2972/files#r425624163

* Back-compatibility for Feed->entries

https://github.com/FreshRSS/FreshRSS/pull/2972/files#r425631913
2020-05-17 00:04:51 +02:00
Alexandre Alapetite
b1aac20839 Fever integer type for numbers (#2946)
* Fever integer type for feed ID

#Fix
https://github.com/FreshRSS/FreshRSS/issues/2940#issuecomment-623022435

* Fix feed_id

* Ensure string for entry ID

To be compatible with 32-bit platforms.
Before, the type was inconsistent depending on architecture + database

* Integer for entry count functions
2020-05-03 13:39:16 +02:00
Alexandre Alapetite
bdc4da6ad0 Expose podcast in API (#2898)
* Expose podcast in API

Expose RSS enclosures in our API, e.g. for clients supporting podcasts

* PHP CS?

* Annoying linter

* Light optimisation
2020-04-16 18:11:26 +02:00
Alexandre Alapetite
be582713fe Fix action filter when filtering on author (#2863)
#fix https://github.com/FreshRSS/FreshRSS/issues/2806
Bug: array instead of string
2020-03-31 17:57:05 +02:00
Alexandre Alapetite
e9f879b411 Fallback to GUID when entry title is empty (#2813)
When entries have an empty title, our UI was a bit broken. In this
(rare) case, we revert to GUID, which is always non-empty.
2020-02-29 18:14:29 +01:00
shn7798
0b6d39a795 Fix wrong foreach in applyFilterActions (#2809) 2020-02-29 16:12:04 +01:00
Julien-Pierre Avérous
d30ac40772 Enhance content path feature (#2778)
- Add a maintenance section to be able to clear cache and force reload a feed.
- Add an icon next to path field to show a pop-up with the result of the content path.

Co-authored-by: Frans de Jonge <fransdejonge@gmail.com>
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
Co-authored-by: Marien Fressinaud <dev@marienfressinaud.fr>
2020-02-13 18:22:35 +01:00
Julien-Pierre Avérous
f3ff4a59e6 Fix 'dateAdded()' getter (#2774)
Be homogeneous in field used, to prevent weird behavior in the future.
2020-01-20 11:23:55 +01:00
Julien-Pierre Avérous
01576e6317 [API, extension] Use 'dateAdded()' to forge 'crawlTimeMsec' & 'timestampUsec' field in greader API. (#2773)
- Add the ability to customize 'dateAdded()', so an extension can change it if needed.
2020-01-18 11:58:46 +01:00
Alexandre Alapetite
acc50df0ef Fix logging environment constants (#2745)
* Fix logging environment constants

* COPY_LOG_TO_SYSLOG was only working when Syslog was used
* FRESHRSS_ENV was not properly used to set logging level

* Simplify code

Always call `openlog()` even when it is not used

* Always specify syslog properties

https://github.com/FreshRSS/FreshRSS/pull/2745#discussion_r362207445

* $username is also needed further down

* No new line in syslog
2020-01-03 18:50:33 +01:00
Alexandre Alapetite
e3e5954394 PDO refactoring for code simplification (#2522)
* PDO refactor

* Automatic prefix when using the syntax `_tableName`
* Uniformity: MySQL is now PDO::ATTR_EMULATE_PREPARES = false just like SQLite and PostgreSQL, with consequences such as only one statement per query
* Use PDO methods exec(), query(), prepare() + execute() in a more efficient way
* Remove auto-update SQL code for versions older than FreshRSS 1.5 (3 years old)
* The name of the default category is set in PHP instead of in the DB (simplies SQL and allows changing the name according to the FreshRSS language)
* Rename `->bd` to `->pdo` (less of a frenshism, and more informative)
* Fix some requests, which were not compatible with MySQL prepared statements

* Whitespace

* Fix syntax for PostgreSQL sequences

+ MySQL install

* Minor formatting

* Fix lastInsertId for PostgreSQL

* Use PHP 5.6+ const

Take advantage of https://github.com/FreshRSS/FreshRSS/pull/2527
https://www.php.net/manual/en/migration56.new-features.php

* A bit of forgotten PHP 5.6 simplification for cURL

* Forgotten $s

* Mini fix custom user config

https://github.com/FreshRSS/FreshRSS/pull/2490/files#r326290346

* More work on install.php but not finished

* install.php working

* More cleaning of PDO in install

* Even more simplification

Take advantage of PDO->exec() to run multiple statements

* Disallow changing the name of the default category

https://github.com/FreshRSS/FreshRSS/pull/2522#discussion_r326967724
2019-09-29 16:22:50 +02:00
Alexandre Alapetite
1804c0e0bc Filter actions (#2275)
* Draft of filter actions

* Travis

* Implement UI + finish logic

* Travis
2019-03-23 22:52:47 +01:00