Commit Graph

132 Commits

Author SHA1 Message Date
Alexandre Alapetite
08a5cf5946 Ensure maximum date int (#3259)
#fix https://github.com/FreshRSS/FreshRSS/issues/3258
2020-11-23 23:04:06 +01:00
Alexandre Alapetite
6edbeaaf6a Add error log (#3055)
For https://github.com/FreshRSS/FreshRSS/issues/3054
2020-06-13 21:19:19 +02:00
Alexandre Alapetite
b1aac20839 Fever integer type for numbers (#2946)
* Fever integer type for feed ID

#Fix
https://github.com/FreshRSS/FreshRSS/issues/2940#issuecomment-623022435

* Fix feed_id

* Ensure string for entry ID

To be compatible with 32-bit platforms.
Before, the type was inconsistent depending on architecture + database

* Integer for entry count functions
2020-05-03 13:39:16 +02:00
Alexandre Alapetite
ae70374b03 Filter by multiple feed IDs (#2892)
Add the possibility to filter by feed ID like `f:123 more-search`
or multiple feed IDs, like `f:123,234,345 more-search` or an exclusion
like `!f:456,789 more-search`
2020-04-17 10:57:35 +02:00
Alexandre Alapetite
61c8026ac9 Implement negation for searching by date intervals (#2869)
* Implement negation for searching by date intervals

#fix https://github.com/FreshRSS/FreshRSS/issues/2866
Allow searching for e.g. `!date:P1W` to exlude all articles newer than
one week.
More generally, allows exclusion on some date intervals.

* Fix OR
2020-04-01 17:26:19 +02:00
Alexandre Alapetite
0f94402b7e Better performance with yield (#2588)
* Better performance with yield

Largely decrease the time to first byte, and reduced memory consumtion.
Before, we used to make several copies in memory of the whole list of
articles before sending them to the client. Now streamed as they are
processed.

* Travis
2020-02-29 18:19:09 +01:00
Alexandre Alapetite
cd5ebcee05 Fix SQLite CONCAT used for search (#2777)
Used e.g. for searches `inurl:xxx`

Co-authored-by: Marien Fressinaud <dev@marienfressinaud.fr>
2020-01-20 11:42:44 +01:00
Alexandre Alapetite
bec7566429 Sanitize UTF-8 before commiting entries (#2649)
Fix https://github.com/FreshRSS/FreshRSS/issues/2645
2019-11-11 19:48:45 +01:00
Alexandre Alapetite
fdfd8ce9be Workaround for MySQL limitation 1093 (#2586)
* Workaround for MySQL limitation 1093
https://github.com/FreshRSS/FreshRSS/pull/2335#issuecomment-545194606
including minor Travis fixes
2019-10-23 02:12:49 +02:00
Alexis Degrugillier
cc0db9af4f Feature/new archiving (#2335)
* Change archiving config page layout

I've changed some wording and moved actions into a
maintenance section.

* Update purge action

Now we have more control on the purge action. The configuration allows
us to choose what to keep and what to discard in a more precise way.
At the moment, the configuration applies for all feeds.

* Add purge configuration on feed level

Now the extend purge configuration is available on feed level.
It is stored as attributes and will be used in the purge action.

* Update purge action

Now the purge action uses the feed configuration if it exists and
defaults on user configuration if not.

* Add empty option in period list

* Fix configuration warnings

* Add archiving configuration on categories

See #2369

* Add user info back

* Add explanations in UI

* Fixes for SQLite + error + misc.

* Fix invalid feed reference

* Short array syntax

Only for new code, so far

* Fix prefix error

* Query performance, default values

Work in progress

* Fix default values and confirm before leaving

Form cancel and confirm changes before leaving were broken.
And start taking advantage of the short echo syntax `<?= ?>` as we have
moved to PHP 5.4+

* More work

* Tuning SQL

* Fix MariaDB + performance issue

* SQL performance

* Fix SQLite bug

* Fix some attributes JSON encoding bugs

Especially for SQLite export/import

* More uniform, fix bugs

More uniform between global, category, feed settings

* Drop special cases for old articles during refresh

Instead will use lastSeen date with the new archiving logic.
This was generating problems anyway
https://github.com/FreshRSS/FreshRSS/issues/2154

* Draft drop index keep_history

Not needed anymore

* MySQL typo

Now properly tested with MySQL, PostgreSQL, SQLite

* More work for legacy values

Important to avoid overriding user's preference and risking deleting
data erroneously

* Fix PHP 7.3 / 7.4 warnings

@aledeg "Trying to use values of type null, bool, int, float or resource
as an array (such as $null["key"]) will now generate a notice. "
https://php.net/migration74.incompatible

* Reintroduce min articles and take care of legacy parameters

* A few changes forgotten

* Draft of migration + DROP of feed.keep_history

* Fix several errors

And give up using const for SQL to allow multiple database types (and we
cannot redefine a const)

* Add keep_min to categories + factorise archiving logic

* Legacy fix

* Fix bug yield from

* Minor: Use JSON_UNESCAPED_SLASHE for attributes

And make more uniform

* Fix sign and missing variable

* Fine tune the logic
2019-10-23 00:52:15 +02:00
Alexandre Alapetite
3b65f2e586 Fix PostgreSQL and SQLite DB size estimation (#2562) 2019-10-06 22:17:28 +02:00
Alexandre Alapetite
be4c942cb3 SQL allow recreating existing user (#2555)
* SQL allow recreating existing user

Taking advantage of https://github.com/FreshRSS/FreshRSS/pull/2554
In a case when FreshRSS data is lost, but database data still intact (in
particular MySQL or PostgreSQL), this patch allows recreating previous
users without error

* Better error retrieval

Especially when error occur during the prepare statement
2019-10-06 00:14:19 +02:00
Alexandre Alapetite
38932ee3bb MySQL entry content MEDIUMBLOB (#2551)
Fix https://github.com/FreshRSS/FreshRSS/issues/2448
2019-10-05 10:37:15 +02:00
Alexandre Alapetite
7ae4e9f159 Quick fix: forgotten PDO values (#2546)
From https://github.com/FreshRSS/FreshRSS/pull/2522
2019-09-29 23:11:35 +02:00
Alexandre Alapetite
e3e5954394 PDO refactoring for code simplification (#2522)
* PDO refactor

* Automatic prefix when using the syntax `_tableName`
* Uniformity: MySQL is now PDO::ATTR_EMULATE_PREPARES = false just like SQLite and PostgreSQL, with consequences such as only one statement per query
* Use PDO methods exec(), query(), prepare() + execute() in a more efficient way
* Remove auto-update SQL code for versions older than FreshRSS 1.5 (3 years old)
* The name of the default category is set in PHP instead of in the DB (simplies SQL and allows changing the name according to the FreshRSS language)
* Rename `->bd` to `->pdo` (less of a frenshism, and more informative)
* Fix some requests, which were not compatible with MySQL prepared statements

* Whitespace

* Fix syntax for PostgreSQL sequences

+ MySQL install

* Minor formatting

* Fix lastInsertId for PostgreSQL

* Use PHP 5.6+ const

Take advantage of https://github.com/FreshRSS/FreshRSS/pull/2527
https://www.php.net/manual/en/migration56.new-features.php

* A bit of forgotten PHP 5.6 simplification for cURL

* Forgotten $s

* Mini fix custom user config

https://github.com/FreshRSS/FreshRSS/pull/2490/files#r326290346

* More work on install.php but not finished

* install.php working

* More cleaning of PDO in install

* Even more simplification

Take advantage of PDO->exec() to run multiple statements

* Disallow changing the name of the default category

https://github.com/FreshRSS/FreshRSS/pull/2522#discussion_r326967724
2019-09-29 16:22:50 +02:00
Alexandre Alapetite
c76a318193 CLI to export/import any database to/from SQLite (#2496)
* CLI to export/import any database to/from SQLite

Require PHP 5.5+ https://github.com/FreshRSS/FreshRSS/pull/2495

* Travis

* Execution rights

* Fix wrong static fields

* Fix MySQL bad default buffering

https://stackoverflow.com/questions/6895098/pdo-mysql-memory-consumption-with-large-result-set/6935271#6935271
https://php.net/manual/ref.pdo-mysql

* Fix count on progression

* Avoid static DB information

To ease working with two DBs at the same time

* Less static, simplify

Needs some testing

* Small corrections

* Special case for SQLite to SQLite

* Modify special case for SQLite

* Remove special case for SQLite

More uniform logic for the 3 databases.
Fix wrong DROP TABLE for SQLite.

* Drop indexes

* Revert "Drop indexes"

This reverts commit f28d2bae09.

* Fix deletion

* Fix classic export

* Update cli/README.md

Co-Authored-By: Marien Fressinaud <dev@marienfressinaud.fr>

* Addressing part of review

* Remove goto 😢

* Travis

* Comment for SQLite case

* Fix missing fields when inserting
2019-09-15 21:36:53 +02:00
Alexandre Alapetite
fd33d92d41 Require PHP 5.5+ (#2495)
* Require PHP 5.5+

https://github.com/FreshRSS/FreshRSS/issues/2469#issuecomment-522255093
I think it would be reasonable to require PHP 5.5+ for the core of
FreshRSS after all.

As Frenzie said, WordPress currently requires PHP 5.6.20+, and it is the
most popular PHP application.

We would loose about 20% of the PHP servers according to
https://w3techs.com/technologies/details/pl-php/5/all but I expect this
number to drop fast after the release of CentOS 8 (CentOS accounts for
17% of Linux servers
https://w3techs.com/technologies/details/os-linux/all/all ).

Distributions:
* no impact on Ubuntu, Fedora, Alpine, OpenWRT, FreeBSD, OpenSuze,
Mageia, as all active versions have PHP > 7
* no impact on OpenSuze, Synology, as all active versions have PHP > 5.5
* we drop Debian 8 Jessie (-2020) - we keep supporting Debian 9 Stretch
(2017-06) - current is Debian 10 Buster
* we drop Red Hat 7 (-2024) - we keep supporting RHEL 8 (2019-05)
* we drop CentOS 7 (-2024) - we will support CentOS 8 (to be released
soonish)

When dropping older versions, I can better like when it is for a good
reason, and there is actually one with PHP 5.5, namely generators
(yield) https://php.net/language.generators.overview which I consider
using.

* Version note for JSON.php

* hex2bin

* Update .travis.yml

Co-Authored-By: Frans de Jonge <fransdejonge@gmail.com>
2019-08-20 14:55:43 +02:00
Alexandre Alapetite
49013bf716 Cherry pick from 2335 (#2356)
Cherry pick bug fix from https://github.com/FreshRSS/FreshRSS/pull/2335
This code is currently not used.
2019-04-07 14:01:05 +02:00
Alexandre Alapetite
9cc72c0c48 Fix EntryDAO tags warning for Fever API
https://github.com/FreshRSS/FreshRSS/issues/2239
2019-02-03 21:24:06 +01:00
Alexandre Alapetite
34fe41ac78 Merge branch 'FreshRSS/dev' into FreshRSS/dev-1.14.0 2019-01-17 21:36:29 +01:00
Alexandre Alapetite
743c1b740b Export labels (#2217)
* Export labels

https://github.com/FreshRSS/FreshRSS/issues/2196

* Small fixes

* Backport code from 1.14.0

4888f919f1

* More fixes
2019-01-16 22:26:43 +01:00
Alexandre Alapetite
4888f919f1 Prepare for batch mark as read 2018-12-25 01:30:28 +01:00
Alexandre Alapetite
b672fc190d Tweaks for Vienna RSS (#2093)
* Tweaks for Vienna RSS

https://github.com/FreshRSS/FreshRSS/issues/2091
https://github.com/ViennaRSS/vienna-rss/issues/1197

* Fix get feed by URL

* Fix get item ids returning starred elements

* API add item ids by feed URL

* Add API filter `it`

https://feedhq.readthedocs.io/en/latest/api/reference.html#stream-items-ids

* API add `nt=` filter + refactoring

* No ; prefix for author

https://github.com/FreshRSS/FreshRSS/issues/2091#issuecomment-435562495

* Add id long form prefix and accept short id form

https://github.com/FreshRSS/FreshRSS/issues/2091#issuecomment-435631259

* Fix quote problem

https://github.com/FreshRSS/FreshRSS/issues/2091#issuecomment-435683930

* Isolate bug fix for News+

https://github.com/FreshRSS/FreshRSS/issues/2091#issuecomment-435687041

* Rework encoding conventions

https://github.com/FreshRSS/FreshRSS/issues/2091#issuecomment-437441834

* Unicode escaping alternative

Alternative approach to encode XML special characters and other
problematic characters into their Unicode fullwidth version when we
cannot use HTML-encoding because clients disagree wether they should
HTML-decode or not.
https://github.com/FreshRSS/FreshRSS/issues/2091#issuecomment-436059559
2018-11-11 17:31:50 +01:00
Alexandre Alapetite
cf899d8d25 TT-RSS import (#2099)
* TT-RSS import

Import of Tiny Tiny RSS favourites
https://github.com/FreshRSS/FreshRSS/issues/2018#issuecomment-432710462

* Fallback feed_url

* Simpler JSON

* TT-RSS import custom labels

* Fix syntax
2018-11-05 18:10:38 +01:00
Alexandre Alapetite
c8b54ae807 Fix MySQL create table feeds (#2047)
https://github.com/FreshRSS/FreshRSS/issues/2042
2018-10-14 13:48:59 +02:00
Alexandre Alapetite
8ee8a573f1 Custom labels (#2027)
* First draft of custom tags

https://github.com/FreshRSS/FreshRSS/issues/928
https://github.com/FreshRSS/FreshRSS/issues/1367

* SMALLINT to BIGINT for id_entry

And uppercase SQL types

* Fix layout for unreads

* Start UI menu

* Change menu order

* Clean database helpers

https://github.com/FreshRSS/FreshRSS/pull/2027#discussion_r217971535

* Travis rules do not understand PostgreSQL constants

Grrr

* Tag controller + UI

* Add column attributes to tags

* Use only favicon for now, for label

* Fix styling for different themes

* Constant for maximum InnoDB index length in Unicode

https://github.com/FreshRSS/FreshRSS/pull/2027#discussion_r219052200
(I would have personnally prefered keeping the readability of a real
value instead of a constant, in this case of many SQL fields)

* Use FreshRSS_Factory::createCategoryDao

* Add view of all articles containing any tag

* Fix search in tags

* Mark as read tags

* Partial auto-update unread tags

* More auto update tag unreads

* Add tag deletion

* Do not purge tagged articles

* Minor comment

* Fix SQLite and UI bug

* Google Reader API support for user tags

Add SQL check that tag names must be distinct from category names

* whitespace

* Add missing API for EasyRSS

* Compatibility SQLite

Problematic parentheses

* Add SQL DISTINCT for cases with multiple tags

* Fix for PostgreSQL

PostgreSQL needs some additional type hint to avoid "could not determine
data type of parameter $1"

http://www.postgresql-archive.org/Could-not-determine-data-type-of-parameter-1-tp2171092p2171094.html
2018-09-29 20:47:17 +02:00
Alexandre Alapetite
c65f535890 Fix count unreads in SQLite (#2009)
* Fix count unreads in SQLite

Was wrong in e.g. `./cli/user-info.php` because UNION output order in
MySQL and SQLite is differents...

* Changelog 2009
2018-09-08 23:05:56 +02:00
Alexandre Alapetite
d3f5bd840d Add mark-as-unread (#1995)
* Add mark-as-unread

https://github.com/FreshRSS/FreshRSS/issues/1966

* Change sentence

https://github.com/FreshRSS/FreshRSS/pull/1995#discussion_r214515954

* Enable mark-as-unread only when unread articles are shown

In order to prevent erroneous marking-as-unread.
We might find a better logic later.

* Disable instead of hide mark-as-unread option

To make it easier to discover
2018-09-03 20:48:04 +02:00
Alexandre Alapetite
32d9c3b790 Use mb_strcut (#1996)
* Use mb_strcut

Avoid cutting in the middle of a multi-byte UTF-8 character

* Forgotten php5-*

* Typo

* Whitespace

* More mb_strcut
2018-09-02 13:19:58 +02:00
Alexandre Alapetite
c0f2df3ef0 Fever fixes (#1931)
* Fever fixes

Was hardcoded for MySQL. Bug in "before" parameter. Bug in mark all as
read.

* Fix construct

* Changelog 1930

https://github.com/FreshRSS/FreshRSS/issues/193
https://github.com/FreshRSS/FreshRSS/pull/1931
2018-06-12 21:11:43 +02:00
Kevin Papst
8f1bad60d0 Add Fever API and user documentation (#1836)
* added fever api and documentation

* spaces to tabs

* fixed code format

* added links

* added utf8 to header

* removed XML support

* removed before check, as we have to convert it afterwards

* added sandboxed setting (currently disabled)
added support for extensions using entry_before_display

* listFeedsOrderUpdate LIMIT

https://github.com/FreshRSS/FreshRSS/pull/1836/files#r175287881

* removed custom sql by using FreshRSS_FeedDAO::listFeedsOrderUpdate()

* fixed mark all as read

* replaced custom sql for getUnread() and getStarred() with dao functions

* removed sanitization functions

* Rework fever login

* Fix config bug

Plus documentation

* Fix array syntax

For compatibility with PHP 5.3

* Disable cookies and session for API

* Fix currentUser

* added response header and error log

* adjusted phpdoc to match new authentication

* Mechanism to delete old keys

* replace PHP_INT_MAX with zero to disable limit

* replace method_exists with check for explicit methods

* removed Press support and smaller refactoring + updated docu

* Rewrite bindParamArray

Avoid one of the SQL injection risks

* Docs and readme

* Fix API link

* Simplify reverse key check

Using userConfig
2018-05-24 21:53:47 +02:00
Alexandre Alapetite
84d891f8cf Light Boolean search implementation (#1828)
* Light Boolean search implementation

"Hello intitle:World OR date:P1D example"
https://github.com/FreshRSS/FreshRSS/issues/879

* Doc Boolean search

* Doc typos
2018-03-14 17:20:41 +01:00
Alexandre Alapetite
5872a11eb7 cleanOldEntries call autoUpdateDb (#1804)
* cleanOldEntries call autoUpdateDb

https://github.com/FreshRSS/FreshRSS/issues/1803

* Fix feedDAO autoUpdateDb

* Move cleanOldEntries to EntryDAO

Only the entry table is concerned
2018-03-01 19:25:40 +01:00
Alexandre Alapetite
79f8b440d1 API /reader/api/0/stream/items/contents (#1774)
* API /reader/api/0/stream/items/contents

For FeedMe

* Fix continuation

* Continuation in stream/items/ids

* Fix multiple continuations

* Allow empty POST tokens

For FeedMe.
This token is not used by e.g. The Old Reader API.
There is the Authorization header anyway.
TODO: Check security consequences

* API compatibility FeedMe: add/remove feed

FeedMe uses GET for some parameters typically given by POST

* A bit of sanitization

* Links to FeedMe

* API favicons more robust when base_url is not set

* Changelog FeedMe
2018-02-08 20:11:05 +01:00
Alexis Degrugillier
8c2113f9e6 Add mute strategy configuration (#1750) 2018-01-01 20:34:06 +01:00
Alexandre Alapetite
f632a34626 CLI optimize database (#1663)
CLI optimize database https://github.com/FreshRSS/FreshRSS/issues/1583
And VACUUM in SQLite https://github.com/FreshRSS/FreshRSS/issues/918
Add VACUUM for PostgreSQL (Not tested yet)
2017-10-12 20:11:06 +02:00
Frans de Jonge
4e174ed9dd [ci] Add Travis (#1619)
* [ci] Add Travis

* Exclude some libs

* Semi-auto whitespace fixes

* line length in SQLite

* Exclude tests from line length

* Feed.php line length

* Feed.php: get rid of unnecessary concat

* Feed.php: line length

* bootstrap.php: no newline at end of file

* Allow concatenating across multiple lines

* Add Travis badge

* do-install line length

* update-or-create-user line length

* cli/create-user line length

* tests/app/Models/SearchTest.php fix indentation

* tests/app/Models/UserQueryTest.php fix indentation

* tests/app/Models/CategoryTest.php fix indentation

* [fix] PHP 5.3 on precise

* cli/do-install no spaces

* cli/list-users line length

* cli/reconfigure line length

* empty catch statements

* api/index line length nonsense

* spaces before semicolon

* app/Models/EntryDAO bunch of indentation

* extra blank lines

* spaces before comma in function call

* testing tabwidth

* increase to 10

* comment out tabwidth line

* try older phpcs version 3.0.0RC4

* line length exception for app/install.php

* proper spaces

* stray spaces in i18n

* Minz/ModelPdo line length

* Minz whitespace

* greader line length

* greader elseif placement

* app/Models/Feed.php spacing in function argument

* ignore php 5.3

* app/Models/ConfigurationSetter.php stray whitespace

* EntryDAOSQLite line length

* I vote for higher max line length =P

* ignore SQL

* remove classname complaint

* line length/more legible SQL

* ignore line length nonsense

* greader line length

* feedController issues

* uppercase TRUE, FALSE, NULL

* revert

* importExportController lowercase null

* Share.php default value not necessary because ! is_array () a few lines down

* CategoryDAO constants should be UPPERCASE

* EntryDAO reduce line length

* contentious autofix

* Allow failures on all versions of PHP except 7.1 because reasons
2017-09-22 12:13:46 +02:00
Alexandre Alapetite
aadba9fb70 Negative searches
https://github.com/FreshRSS/FreshRSS/issues/1381
Possibility to exclude authors, titles, tags, urls, words by prefixing
them by ! or - (like Google Search):

* !intitle:unwanted
* -intitle:unwanted
* -author:unwanted
* -#unwanted
* -unwanted

And one can use many of each and combine them with positive searches
2017-04-01 23:41:19 +02:00
Alexandre Alapetite
d9c0d25b85 Improve search: intitle, author, inurl
Allow multiple values of intitle: , author:, inurl:

Note: Tests for UserQueryTest are broken due to
https://github.com/sebastianbergmann/phpunit/wiki/Release-Announcement-for-PHPUnit-4.0.0#backwards-compatibility-issues
2017-04-01 22:31:12 +02:00
Alexandre Alapetite
fbdc22f75a Merge branch 'FreshRSS/dev' into defered-insertion 2017-03-26 19:36:44 +02:00
Alexandre Alapetite
7a6751b50d PDO fix PHP 7.1
http://php.net/manual/migration71.changed-functions.php#migration71.changed-functions.pdo
2017-03-26 18:48:34 +02:00
Alexandre Alapetite
cc235c6af3 SimplePie light manual update 2017-03-26 15:02:35 +02:00
Alexandre Alapetite
e956aee53d More defered insertion. New SQL index
New index entry_feed_read_index
TODO: Auto add this index to existing version
2017-03-26 14:07:06 +02:00
Alexandre Alapetite
a20fd9db9f Defered insertion MySQL bug
The update of cached values remains to be optimized
2017-03-26 01:41:08 +01:00
Alexandre Alapetite
22b41f3bfc Candidate implementation of defered insertion
https://github.com/FreshRSS/FreshRSS/issues/530
2017-03-26 00:01:11 +01:00
Thomas Citharel
ae5a8572d5 postgres fixes 2017-01-27 10:26:31 +01:00
Alexandre Alapetite
f59de4e2b6 Stream JSON export
Avoid large in-memory copies
https://github.com/FreshRSS/FreshRSS/issues/1372
2016-11-15 20:43:06 +01:00
Alexandre Alapetite
aeda49a7d2 Add CLI for user information + Fix last user activity
https://github.com/FreshRSS/FreshRSS/issues/1345
2016-11-05 20:48:46 +01:00
Alexandre Alapetite
8a410969dd Use bindValue instead of bindParam
https://github.com/FreshRSS/FreshRSS/issues/1349
We should later replace all bindParam by bindValue
2016-11-02 21:29:21 +01:00
Alexandre Alapetite
7f2b0439ec Extract function safe_ascii() 2016-10-20 01:38:23 +02:00