Feature/new archiving (#2335)

* Change archiving config page layout

I've changed some wording and moved actions into a
maintenance section.

* Update purge action

Now we have more control on the purge action. The configuration allows
us to choose what to keep and what to discard in a more precise way.
At the moment, the configuration applies for all feeds.

* Add purge configuration on feed level

Now the extend purge configuration is available on feed level.
It is stored as attributes and will be used in the purge action.

* Update purge action

Now the purge action uses the feed configuration if it exists and
defaults on user configuration if not.

* Add empty option in period list

* Fix configuration warnings

* Add archiving configuration on categories

See #2369

* Add user info back

* Add explanations in UI

* Fixes for SQLite + error + misc.

* Fix invalid feed reference

* Short array syntax

Only for new code, so far

* Fix prefix error

* Query performance, default values

Work in progress

* Fix default values and confirm before leaving

Form cancel and confirm changes before leaving were broken.
And start taking advantage of the short echo syntax `<?= ?>` as we have
moved to PHP 5.4+

* More work

* Tuning SQL

* Fix MariaDB + performance issue

* SQL performance

* Fix SQLite bug

* Fix some attributes JSON encoding bugs

Especially for SQLite export/import

* More uniform, fix bugs

More uniform between global, category, feed settings

* Drop special cases for old articles during refresh

Instead will use lastSeen date with the new archiving logic.
This was generating problems anyway
https://github.com/FreshRSS/FreshRSS/issues/2154

* Draft drop index keep_history

Not needed anymore

* MySQL typo

Now properly tested with MySQL, PostgreSQL, SQLite

* More work for legacy values

Important to avoid overriding user's preference and risking deleting
data erroneously

* Fix PHP 7.3 / 7.4 warnings

@aledeg "Trying to use values of type null, bool, int, float or resource
as an array (such as $null["key"]) will now generate a notice. "
https://php.net/migration74.incompatible

* Reintroduce min articles and take care of legacy parameters

* A few changes forgotten

* Draft of migration + DROP of feed.keep_history

* Fix several errors

And give up using const for SQL to allow multiple database types (and we
cannot redefine a const)

* Add keep_min to categories + factorise archiving logic

* Legacy fix

* Fix bug yield from

* Minor: Use JSON_UNESCAPED_SLASHE for attributes

And make more uniform

* Fix sign and missing variable

* Fine tune the logic
This commit is contained in:
Alexis Degrugillier
2019-10-23 00:52:15 +02:00
committed by Alexandre Alapetite
parent d7f8883926
commit cc0db9af4f
78 changed files with 1063 additions and 278 deletions

View File

@@ -1,12 +1,13 @@
<?php
const SQL_CREATE_DB = <<<'SQL'
$SQL_CREATE_DB = <<<'SQL'
SELECT 1; -- Do nothing for SQLite
SQL;
const SQL_CREATE_TABLES = <<<'SQL'
$SQL_CREATE_TABLES = <<<'SQL'
CREATE TABLE IF NOT EXISTS `category` (
`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
`name` VARCHAR(255) NOT NULL,
`attributes` TEXT, -- v1.15.0
UNIQUE (`name`)
);
@@ -22,7 +23,6 @@ CREATE TABLE IF NOT EXISTS `feed` (
`pathEntries` VARCHAR(511) DEFAULT NULL,
`httpAuth` VARCHAR(511) DEFAULT NULL,
`error` BOOLEAN DEFAULT 0,
`keep_history` MEDIUMINT NOT NULL DEFAULT -2,
`ttl` INT NOT NULL DEFAULT 0,
`attributes` TEXT, -- v1.11.0
`cache_nbEntries` INT DEFAULT 0,
@@ -32,7 +32,6 @@ CREATE TABLE IF NOT EXISTS `feed` (
);
CREATE INDEX IF NOT EXISTS feed_name_index ON `feed`(`name`);
CREATE INDEX IF NOT EXISTS feed_priority_index ON `feed`(`priority`);
CREATE INDEX IF NOT EXISTS feed_keep_history_index ON `feed`(`keep_history`);
CREATE TABLE IF NOT EXISTS `entry` (
`id` BIGINT NOT NULL,
@@ -60,11 +59,11 @@ CREATE INDEX IF NOT EXISTS entry_feed_read_index ON `entry`(`id_feed`,`is_read`)
INSERT OR IGNORE INTO `category` (id, name) VALUES(1, "Uncategorized");
SQL;
const SQL_CREATE_INDEX_ENTRY_1 = <<<'SQL'
$SQL_CREATE_INDEX_ENTRY_1 = <<<'SQL'
CREATE INDEX IF NOT EXISTS entry_feed_read_index ON `entry`(`id_feed`,`is_read`); -- v1.7
SQL;
const SQL_CREATE_TABLE_ENTRYTMP = <<<'SQL'
$SQL_CREATE_TABLE_ENTRYTMP = <<<'SQL'
CREATE TABLE IF NOT EXISTS `entrytmp` ( -- v1.7
`id` BIGINT NOT NULL,
`guid` VARCHAR(760) NOT NULL,
@@ -86,7 +85,7 @@ CREATE TABLE IF NOT EXISTS `entrytmp` ( -- v1.7
CREATE INDEX IF NOT EXISTS entrytmp_date_index ON `entrytmp`(`date`);
SQL;
const SQL_CREATE_TABLE_TAGS = <<<'SQL'
$SQL_CREATE_TABLE_TAGS = <<<'SQL'
CREATE TABLE IF NOT EXISTS `tag` ( -- v1.12
`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
`name` VARCHAR(63) NOT NULL,
@@ -103,12 +102,12 @@ CREATE TABLE IF NOT EXISTS `entrytag` (
CREATE INDEX IF NOT EXISTS entrytag_id_entry_index ON `entrytag` (`id_entry`);
SQL;
const SQL_INSERT_FEED = <<<'SQL'
$SQL_INSERT_FEED = <<<'SQL'
INSERT OR IGNORE INTO `feed` (url, category, name, website, description, ttl)
VALUES(:url, 1, :name, :website, :description, 86400);
SQL;
const SQL_DROP_TABLES = <<<'SQL'
$SQL_DROP_TABLES = <<<'SQL'
DROP TABLE IF EXISTS `entrytag`;
DROP TABLE IF EXISTS `tag`;
DROP TABLE IF EXISTS `entrytmp`;