Heal registered sources whose map coverage went missing

The volume map summarises registered source subtrees, and only the
source's own walk fills them. An interrupted walk, a lost snapshot, or
a nested source whose volume restored around it all leave one shape: a
store holding records under a root whose map node has no children, and
nothing ever repaired it. Three live incidents in one day wore it: the
dev-tools nested source served an empty replica for days, the home
source showed a single folder, and both stayed that way across every
restart because the snapshot faithfully persisted the hole.

The discovery pass now checks every registered source on each attached
volume, restored or not: records in the store with no children in the
map dispatches the source's walk. Store evidence contradicting arena
emptiness is the trigger, so a genuinely empty source is never
rewalked. The walk dispatch is extracted from sources.track into one
shared path, so the heal honors the persisted capture policy, source
retention, and store wiring identically to an explicit track.

Verified live on titan: discovery flagged dev-tools (28,079 records,
no coverage), walked it, saved its first complete snapshot, and the
Mac's replica refetched automatically and lists the toolchain.
This commit is contained in:
Jamie Pine committed 2026-09-16 01:14:12 -07:00
1 parent 59b05a6c8d
commit 5a65d09982
2 files changed
+96 -9

No files matched your search

+33 -8
View File
@@ -174,6 +174,37 @@ pub async fn track_and_index(
)
.await;
let job_id =
dispatch_source_walk(&library, &context, id, root.clone(), whole_volume, true).await;
Ok(TrackSourceOutput {
id,
root,
volume_uuid: volume.map(|volume| volume.id),
whole_volume,
job_id,
})
}
/// Dispatch the walk and the follow-up hashing pass for a registered source.
///
/// Shared by explicit tracking and by the discovery pass's coverage heal, so
/// both honor the persisted capture policy, source retention, and the store
/// wiring the same way. `announce` distinguishes a person tracking a source
/// from a background repair.
pub(crate) async fn dispatch_source_walk(
library: &Arc<crate::library::Library>,
context: &Arc<CoreContext>,
id: Uuid,
root: PathBuf,
whole_volume: bool,
announce: bool,
) -> Option<uuid::Uuid> {
let unfiltered = context
.ephemeral_cache()
.source_config(id)
.is_some_and(|config| config.unfiltered);
// Seed the partition from its snapshot before walking over it. A partition
// that skipped restore is barred from saving over an existing snapshot, so
// tracking a root that already has one would index and then fail to persist.
@@ -198,7 +229,7 @@ pub async fn track_and_index(
IndexScope::Recursive,
whole_volume,
);
config.announce = true;
config.announce = announce;
if unfiltered {
config.rule_toggles = RuleToggles::none();
}
@@ -237,13 +268,7 @@ pub async fn track_and_index(
tracing::warn!(source = %id, %e, "could not start content identification");
}
Ok(TrackSourceOutput {
id,
root,
volume_uuid: volume.map(|volume| volume.id),
whole_volume,
job_id,
})
job_id
}
#[cfg(test)]
+63 -1
View File
@@ -130,6 +130,61 @@ pub async fn map_volume(
Ok(handle.id())
}
/// Re-walk registered sources on this volume whose map coverage is missing.
///
/// The volume map summarises registered source subtrees, and only the
/// source's own walk fills them. An interrupted walk, a lost snapshot, or a
/// nested source whose volume restored around it all leave the same shape: a
/// store that holds records under a root whose node in the map has no
/// children. Store evidence contradicting arena emptiness is the trigger, so
/// a genuinely empty source is never rewalked.
async fn heal_uncovered_sources(
library: &Arc<Library>,
context: &Arc<CoreContext>,
volume: &crate::domain::Volume,
) {
let cache = context.ephemeral_cache();
for source in cache.sources() {
if !source.root.starts_with(&volume.mount_point)
|| !source.attached
|| cache.is_indexing(&source.root)
{
continue;
}
if source.entry_count.unwrap_or(0) == 0 {
continue;
}
let covered = match cache.get_for_search(&source.root) {
Some(index) => index
.read()
.await
.list_directory(&source.root)
.is_some_and(|children| !children.is_empty()),
None => false,
};
if covered {
continue;
}
tracing::warn!(
source = %source.id,
root = %source.root.display(),
records = source.entry_count.unwrap_or(0),
"source has records but no map coverage; dispatching its walk"
);
crate::ops::sources::track::action::dispatch_source_walk(
library,
context,
source.id,
source.root.clone(),
source.root == volume.mount_point,
false,
)
.await;
}
}
/// Map every drive attached to this machine.
///
/// Called once the library is open and its sources have been walked. Search and
@@ -154,7 +209,14 @@ pub async fn map_attached_volumes(library: &Arc<Library>, context: &Arc<CoreCont
let cache = context.ephemeral_cache();
cache.track_volume(volume.id, volume.mount_point.clone());
if cache.ensure_restored(&volume.mount_point).await {
let restored = cache.ensure_restored(&volume.mount_point).await;
// Runs whether or not the volume restored: a restored snapshot can
// faithfully persist a coverage hole, which is exactly the state
// this repairs.
heal_uncovered_sources(library, context, &volume).await;
if restored {
debug!(
"{} restored from its snapshot; not walking it again",
volume.mount_point.display()