mirror/FreshRSS
0
0
Fork 0
mirror of https://github.com/FreshRSS/FreshRSS.git synced 2026-05-24 16:25:00 -04:00
Code Issues Packages Projects Releases Wiki Activity

Feature/new archiving (#2335)

Browse Source 
* Change archiving config page layout

I've changed some wording and moved actions into a
maintenance section.

* Update purge action

Now we have more control on the purge action. The configuration allows
us to choose what to keep and what to discard in a more precise way.
At the moment, the configuration applies for all feeds.

* Add purge configuration on feed level

Now the extend purge configuration is available on feed level.
It is stored as attributes and will be used in the purge action.

* Update purge action

Now the purge action uses the feed configuration if it exists and
defaults on user configuration if not.

* Add empty option in period list

* Fix configuration warnings

* Add archiving configuration on categories

See #2369

* Add user info back

* Add explanations in UI

* Fixes for SQLite + error + misc.

* Fix invalid feed reference

* Short array syntax

Only for new code, so far

* Fix prefix error

* Query performance, default values

Work in progress

* Fix default values and confirm before leaving

Form cancel and confirm changes before leaving were broken.
And start taking advantage of the short echo syntax `<?= ?>` as we have
moved to PHP 5.4+

* More work

* Tuning SQL

* Fix MariaDB + performance issue

* SQL performance

* Fix SQLite bug

* Fix some attributes JSON encoding bugs

Especially for SQLite export/import

* More uniform, fix bugs

More uniform between global, category, feed settings

* Drop special cases for old articles during refresh

Instead will use lastSeen date with the new archiving logic.
This was generating problems anyway
https://github.com/FreshRSS/FreshRSS/issues/2154

* Draft drop index keep_history

Not needed anymore

* MySQL typo

Now properly tested with MySQL, PostgreSQL, SQLite

* More work for legacy values

Important to avoid overriding user's preference and risking deleting
data erroneously

* Fix PHP 7.3 / 7.4 warnings

@aledeg "Trying to use values of type null, bool, int, float or resource
as an array (such as $null["key"]) will now generate a notice. "
https://php.net/migration74.incompatible

* Reintroduce min articles and take care of legacy parameters

* A few changes forgotten

* Draft of migration + DROP of feed.keep_history

* Fix several errors

And give up using const for SQL to allow multiple database types (and we
cannot redefine a const)

* Add keep_min to categories + factorise archiving logic

* Legacy fix

* Fix bug yield from

* Minor: Use JSON_UNESCAPED_SLASHE for attributes

And make more uniform

* Fix sign and missing variable

* Fine tune the logic
This commit is contained in:
Alexis Degrugillier
2019-10-23 00:52:15 +02:00
committed by Alexandre Alapetite
parent d7f8883926
commit cc0db9af4f
78 changed files with 1063 additions and 278 deletions
Download Patch File Download Diff File Expand all files Collapse all files

24
app/Controllers/feedController.php
View File

@@ -267,10 +267,6 @@ class FreshRSS_feed_Controller extends Minz_ActionController {
$maxFeeds = 10;
}
// Calculate date of oldest entries we accept in DB.
$nb_month_old = max(FreshRSS_Context::$user_conf->old_entries, 1);
$date_min = time() - (3600 * 24 * 30 * $nb_month_old);
// WebSub (PubSubHubbub) support
$pubsubhubbubEnabledGeneral = FreshRSS_Context::$system_conf->pubsubhubbub_enabled;
$pshbMinAge = time() - (3600 * 24); //TODO: Make a configuration.
@@ -323,12 +319,6 @@ class FreshRSS_feed_Controller extends Minz_ActionController {
continue;
}
$feed_history = $feed->keepHistory();
if ($isNewFeed) {
$feed_history = FreshRSS_Feed::KEEP_HISTORY_INFINITE;
} elseif (FreshRSS_Feed::KEEP_HISTORY_DEFAULT === $feed_history) {
$feed_history = FreshRSS_Context::$user_conf->keep_history_default;
}
$needFeedCacheRefresh = false;
// We want chronological order and SimplePie uses reverse order.
@@ -376,15 +366,9 @@ class FreshRSS_feed_Controller extends Minz_ActionController {
}
$entryDAO->updateEntry($entry->toArray());
}
} elseif ($feed_history == 0 && $entry_date < $date_min) {
// This entry should not be added considering configuration and date.
$oldGuids[] = $entry->guid();
} else {
$id = uTimeString();
$entry->_id($id);
if ($entry_date < $date_min) {
$entry->_isRead(true); //Old article that was not in database. Probably an error, so mark as read
}
$entry->applyFilterActions();
@@ -413,17 +397,13 @@ class FreshRSS_feed_Controller extends Minz_ActionController {
$entryDAO->updateLastSeen($feed->id(), $oldGuids, $mtime);
}
if ($feed_history >= 0 && mt_rand(0, 30) === 1) {
// TODO: move this function in web cron when available (see entry::purge)
// Remove old entries once in 30.
if (mt_rand(0, 30) === 1) { // Remove old entries once in 30.
if (!$entryDAO->inTransaction()) {
$entryDAO->beginTransaction();
}
$nb = $entryDAO->cleanOldEntries($feed->id(), $date_min, max($feed_history, count($entries) + 10));
$nb = $feed->cleanOldEntries();
if ($nb > 0) {
$needFeedCacheRefresh = true;
Minz_Log::debug($nb . ' old entries cleaned in feed [' . $feed->url(false) . ']');
}
}