Feature/new archiving (#2335)

* Change archiving config page layout

I've changed some wording and moved actions into a
maintenance section.

* Update purge action

Now we have more control on the purge action. The configuration allows
us to choose what to keep and what to discard in a more precise way.
At the moment, the configuration applies for all feeds.

* Add purge configuration on feed level

Now the extend purge configuration is available on feed level.
It is stored as attributes and will be used in the purge action.

* Update purge action

Now the purge action uses the feed configuration if it exists and
defaults on user configuration if not.

* Add empty option in period list

* Fix configuration warnings

* Add archiving configuration on categories

See #2369

* Add user info back

* Add explanations in UI

* Fixes for SQLite + error + misc.

* Fix invalid feed reference

* Short array syntax

Only for new code, so far

* Fix prefix error

* Query performance, default values

Work in progress

* Fix default values and confirm before leaving

Form cancel and confirm changes before leaving were broken.
And start taking advantage of the short echo syntax `<?= ?>` as we have
moved to PHP 5.4+

* More work

* Tuning SQL

* Fix MariaDB + performance issue

* SQL performance

* Fix SQLite bug

* Fix some attributes JSON encoding bugs

Especially for SQLite export/import

* More uniform, fix bugs

More uniform between global, category, feed settings

* Drop special cases for old articles during refresh

Instead will use lastSeen date with the new archiving logic.
This was generating problems anyway
https://github.com/FreshRSS/FreshRSS/issues/2154

* Draft drop index keep_history

Not needed anymore

* MySQL typo

Now properly tested with MySQL, PostgreSQL, SQLite

* More work for legacy values

Important to avoid overriding user's preference and risking deleting
data erroneously

* Fix PHP 7.3 / 7.4 warnings

@aledeg "Trying to use values of type null, bool, int, float or resource
as an array (such as $null["key"]) will now generate a notice. "
https://php.net/migration74.incompatible

* Reintroduce min articles and take care of legacy parameters

* A few changes forgotten

* Draft of migration + DROP of feed.keep_history

* Fix several errors

And give up using const for SQL to allow multiple database types (and we
cannot redefine a const)

* Add keep_min to categories + factorise archiving logic

* Legacy fix

* Fix bug yield from

* Minor: Use JSON_UNESCAPED_SLASHE for attributes

And make more uniform

* Fix sign and missing variable

* Fine tune the logic
This commit is contained in:
Alexis Degrugillier
2019-10-23 00:52:15 +02:00
committed by Alexandre Alapetite
parent d7f8883926
commit cc0db9af4f
78 changed files with 1063 additions and 278 deletions

View File

@@ -1,12 +1,13 @@
<?php
const SQL_CREATE_DB = <<<'SQL'
$SQL_CREATE_DB = <<<'SQL'
CREATE DATABASE "%1$s" ENCODING 'UTF8';
SQL;
const SQL_CREATE_TABLES = <<<'SQL'
$SQL_CREATE_TABLES = <<<'SQL'
CREATE TABLE IF NOT EXISTS `_category` (
"id" SERIAL PRIMARY KEY,
"name" VARCHAR(255) UNIQUE NOT NULL
"name" VARCHAR(255) UNIQUE NOT NULL,
"attributes" TEXT -- v1.15.0
);
CREATE TABLE IF NOT EXISTS `_feed` (
@@ -21,7 +22,6 @@ CREATE TABLE IF NOT EXISTS `_feed` (
"pathEntries" VARCHAR(511) DEFAULT NULL,
"httpAuth" VARCHAR(511) DEFAULT NULL,
"error" SMALLINT DEFAULT 0,
"keep_history" INT NOT NULL DEFAULT -2,
"ttl" INT NOT NULL DEFAULT 0,
"attributes" TEXT, -- v1.11.0
"cache_nbEntries" INT DEFAULT 0,
@@ -30,7 +30,6 @@ CREATE TABLE IF NOT EXISTS `_feed` (
);
CREATE INDEX IF NOT EXISTS `_name_index` ON `_feed` ("name");
CREATE INDEX IF NOT EXISTS `_priority_index` ON `_feed` ("priority");
CREATE INDEX IF NOT EXISTS `_keep_history_index` ON `_feed` ("keep_history");
CREATE TABLE IF NOT EXISTS `_entry` (
"id" BIGINT NOT NULL PRIMARY KEY,
@@ -60,11 +59,11 @@ INSERT INTO `_category` (id, name)
RETURNING nextval('`_category_id_seq`');
SQL;
const SQL_CREATE_INDEX_ENTRY_1 = <<<'SQL'
$SQL_CREATE_INDEX_ENTRY_1 = <<<'SQL'
CREATE INDEX IF NOT EXISTS `_entry_feed_read_index` ON `_entry` ("id_feed","is_read"); -- v1.7
SQL;
const SQL_CREATE_TABLE_ENTRYTMP = <<<'SQL'
$SQL_CREATE_TABLE_ENTRYTMP = <<<'SQL'
CREATE TABLE IF NOT EXISTS `_entrytmp` ( -- v1.7
"id" BIGINT NOT NULL PRIMARY KEY,
"guid" VARCHAR(760) NOT NULL,
@@ -85,7 +84,7 @@ CREATE TABLE IF NOT EXISTS `_entrytmp` ( -- v1.7
CREATE INDEX IF NOT EXISTS `_entrytmp_date_index` ON `_entrytmp` ("date");
SQL;
const SQL_CREATE_TABLE_TAGS = <<<'SQL'
$SQL_CREATE_TABLE_TAGS = <<<'SQL'
CREATE TABLE IF NOT EXISTS `_tag` ( -- v1.12
"id" SERIAL PRIMARY KEY,
"name" VARCHAR(63) UNIQUE NOT NULL,
@@ -101,12 +100,12 @@ CREATE TABLE IF NOT EXISTS `_entrytag` (
CREATE INDEX IF NOT EXISTS `_entrytag_id_entry_index` ON `_entrytag` ("id_entry");
SQL;
const SQL_INSERT_FEED = <<<'SQL'
$SQL_INSERT_FEED = <<<'SQL'
INSERT INTO `_feed` (url, category, name, website, description, ttl)
SELECT :url::VARCHAR, 1, :name, :website, :description, 86400
WHERE NOT EXISTS (SELECT id FROM `_feed` WHERE url = :url);
SQL;
const SQL_DROP_TABLES = <<<'SQL'
$SQL_DROP_TABLES = <<<'SQL'
DROP TABLE IF EXISTS `_entrytag`, `_tag`, `_entrytmp`, `_entry`, `_feed`, `_category`;
SQL;