From de48a270e9815466e9a4577e2ef19d677227b8dd Mon Sep 17 00:00:00 2001 From: jokob-sk Date: Sat, 12 Sep 2026 08:33:45 +1000 Subject: [PATCH] DOCS: plugin import behavior cleanup --- docs/NOTIFICATIONS.md | 2 +- docs/PLUGINS_DEV.md | 2 +- docs/PLUGINS_DEV_DATA_CONTRACT.md | 66 +------------------------------ docs/PLUGINS_IMPORT_BEHAVIOR.md | 66 +++++++++++++++++++++++++++++++ mkdocs.yml | 1 + 5 files changed, 71 insertions(+), 66 deletions(-) create mode 100644 docs/PLUGINS_IMPORT_BEHAVIOR.md diff --git a/docs/NOTIFICATIONS.md b/docs/NOTIFICATIONS.md index 620fc328..fadad936 100755 --- a/docs/NOTIFICATIONS.md +++ b/docs/NOTIFICATIONS.md @@ -42,7 +42,7 @@ Click the **Read more in the docs.** Link at the top of each plugin to get more ### Plugin-level per-row overrides -A plugin author can also mark individual rows it reports as `quiet` via the `scanNotificationMode` data column, independent of any user-facing setting above - e.g. a bulk inventory import that shouldn't spam notifications for known-offline devices. This is a plugin-authoring concept, not something configured in the UI - see [Data contract](https://docs.netalertx.com/PLUGINS_DEV_DATA_CONTRACT#import-behavior-columns) for the full behavior (when it applies, and how it combines with the **Alert Events**/**Alert Down** device settings above when multiple plugins report the same device). +A plugin author can also mark individual rows it reports as `quiet` via the `scanNotificationMode` data column, independent of any user-facing setting above - e.g. a bulk inventory import that shouldn't spam notifications for known-offline devices. This is a plugin-authoring concept, not something configured in the UI - see [Plugin Import Behavior](https://docs.netalertx.com/PLUGINS_IMPORT_BEHAVIOR) for the full behavior (when it applies, and how it combines with the **Alert Events**/**Alert Down** device settings above when multiple plugins report the same device). ## Global settings ⚙ diff --git a/docs/PLUGINS_DEV.md b/docs/PLUGINS_DEV.md index 4c7f573f..373b50c9 100755 --- a/docs/PLUGINS_DEV.md +++ b/docs/PLUGINS_DEV.md @@ -311,7 +311,7 @@ To always map a static value (not read from plugin output): ### Import Behavior Columns (`scanCreatesDevice`, `scanNotificationMode`, `scanPresence`) -Three optional columns on `CurrentScan` control what happens once a row reaches it — see the [Data contract](PLUGINS_DEV_DATA_CONTRACT.md#import-behavior-columns) for the full contract (allowed values, defaults, downstream effects). All three default to today's behavior if never mapped, so existing plugins need no changes. +Three optional columns on `CurrentScan` control what happens once a row reaches it — see [Plugin Import Behavior](PLUGINS_IMPORT_BEHAVIOR.md) for the full contract (allowed values, defaults, downstream effects). All three default to today's behavior if never mapped, so existing plugins need no changes. Most plugins map a single static value for the whole import via `mapped_to_column_data` — e.g. an enrichment-only plugin that should never originate a new device: diff --git a/docs/PLUGINS_DEV_DATA_CONTRACT.md b/docs/PLUGINS_DEV_DATA_CONTRACT.md index 50420b45..be36efda 100644 --- a/docs/PLUGINS_DEV_DATA_CONTRACT.md +++ b/docs/PLUGINS_DEV_DATA_CONTRACT.md @@ -159,70 +159,7 @@ As the documentation might become outdated, it's good practice to check the late ### Import Behavior Columns -Three optional `CurrentScan` columns, all independent of each other, control what happens once a row reaches the table. - -| Column | Type | Default | Meaning | -|---|---|---|---| -| `scanCreatesDevice` | boolean | `1` | Whether this row can originate a *new* `Devices` entry. `0` lets an enrich-only plugin (e.g. a hostname resolver) update an already-existing device's fields without ever being able to create one. | -| `scanNotificationMode` | text (`normal` \| `quiet`) | `normal` | Whether this row's notifications are suppressed. `quiet` always suppresses the outbound email/push; whether the `Events` row itself still gets written depends on the event. **Live** (per-cycle aggregate, reclassifying a row changes future events): `New Device`, `Connected`, `Down Reconnected`, `IP Changed` — audit trail always written. `New Device` isn't gated on `scanPresence = 1` like the other three (see flowcharts below). **Frozen** (`devAlertDown`/`devAlertEvents` seeded at device creation, reclassifying later has no retroactive effect): `Device Down`, `Disconnected` — not symmetric. `Disconnected` always writes its `Events` row (`evePendingAlertEmail = 0` when quiet). `Device Down` writes **no row at all** when `devAlertDown = 0`. | -| `scanPresence` | boolean | `1` | Whether this row asserts the device is *currently online*. `0` means "identity/inventory data, no presence claim" — not "offline". A reservation, a lease record, or a static IPAM entry are typical `0` cases. | - -**Missing vs. invalid values — these behave differently, not interchangeably:** - -| Column | Column never mapped (missing) | Mapped but sent an unexpected value (invalid) | -|---|---|---| -| `scanCreatesDevice` | `1` (schema `DEFAULT`) | `CHECK (scanCreatesDevice IN (0, 1))` — anything else fails the `INSERT` outright, it does not silently fall back to `1` | -| `scanNotificationMode` | `normal` (schema `DEFAULT`) | No `CHECK` constraint — any string other than the literal `'quiet'` is treated as `normal`, since the SQL only special-cases that exact value | -| `scanPresence` | `1` (schema `DEFAULT`) | `CHECK (scanPresence IN (0, 1))` — same as `scanCreatesDevice`, invalid values fail the `INSERT`, they don't default | - -**Multiple plugins reporting the same MAC in the same scan cycle** (the normal case, not an edge case — see the `scan-pipeline` skill) resolve per column, not uniformly: `scanCreatesDevice` and `scanPresence` are most-permissive-wins (any row saying `1` wins), while `scanNotificationMode` is most-*restrictive*-wins (any row saying `quiet` suppresses the notification, even if a sibling row says `normal`) — erring toward under-notifying rather than spamming. - -**Combination matrix** — not every combination is meaningful for every plugin; pick the one that matches what your plugin actually knows: - -| `scanCreatesDevice` | `scanPresence` | Meaning | -|---|---|---| -| 1 | 1 | Normal discovery (the default) | -| 1 | 0 | Inventory/identity import — create the device, but don't claim it's online right now | -| 0 | 1 | Presence-confirming enrichment — never originate a device, but assert presence for one that exists | -| 0 | 0 | Silent enrichment — never originate a device, no presence claim either | - -`scanNotificationMode` is orthogonal to both of the above and can be combined with any row in the table (e.g. inventory import + quiet, for a fully silent bulk import of known-offline devices). - -**Decision: does this row create a device?** - -```mermaid -flowchart TD - A[Row reaches CurrentScan] --> B{scanMac blank or
null-equivalent?} - B -- yes --> Z[Never creates a device] - B -- no --> C{Any row this cycle for this
MAC has scanCreatesDevice = 1?} - C -- no, all say 0 --> Y[No device created
enrich-only] - C -- yes, at least one --> D{Devices row already
exists for this MAC?} - D -- yes --> E[No-op - existing device untouched
by this check] - D -- no --> F[New Devices row created
+ New Device event] -``` - -**Decision: is this event's notification suppressed?** - -```mermaid -flowchart TD - A[Event about to fire] --> B{Fired from a row that exists in
CurrentScan this cycle? New Device /
Connected / Down Reconnected / IP Changed} - B -- yes --> C{Live aggregate: any CurrentScan row
for this MAC says
scanNotificationMode = quiet?} - C -- yes --> S[Suppressed
evePendingAlertEmail = 0] - C -- no --> N[Notified
evePendingAlertEmail = 1] - B -- no, fired from row ABSENCE
Device Down / Disconnected --> D{Frozen device setting:
devAlertDown / devAlertEvents,
seeded at creation time} - D -- off --> S - D -- on --> N -``` - -**Worked scenarios:** - -| Scenario | `scanCreatesDevice` | `scanPresence` | `scanNotificationMode` | `scanMac` | Outcome | -|---|---|---|---|---|---| -| Normal discovery (default plugin behavior) | `1` (default) | `1` (default) | `normal` (default) | real MAC | Device created if new, notified normally, presence tracked live. | -| Enrich-only plugin (e.g. a hostname resolver) | `0` | `1` (default) | `normal` (default) | real MAC | Never originates a device; still updates an existing device's fields via `FIELD_SPECS`. If another plugin reports the same MAC with `scanCreatesDevice = 1`, the device still gets created (most-permissive-wins) — this plugin's `0` doesn't block it. | -| Bulk inventory import of known-offline devices | `1` | `0` | `quiet` | real MAC | Creates devices without claiming they're online, and without a wave of "New Device" notifications for a large batch import. | -| Presence-confirming enrichment (e.g. a DHCP lease scanner) | `0` | `1` | `normal` | real MAC | Confirms an *existing* device is online without ever being the plugin that creates it. | -| Row with no usable device identity (e.g. an object with no routable MAC available) | `0` | irrelevant | irrelevant | blank / null-equivalent | Never creates a device — but not for symmetric reasons. The blank-MAC guard blocks the whole aggregated group by its shared `scanMac` value, regardless of any individual row's `scanCreatesDevice` (even a stray `1` from an unrelated plugin sharing the same blank `scanMac` can't override it). Setting `scanCreatesDevice = 0` here is still correct practice, but on its own is only this row's vote — most-permissive-wins means a sibling row for the same `scanMac` asserting `1` would still win. The blank-MAC guard is what actually guarantees safety regardless of what other contributors do. | +Three optional `CurrentScan` columns — `scanCreatesDevice`, `scanNotificationMode`, `scanPresence` — control whether a row can create a device, whether it counts as a live presence signal, and whether its notifications are suppressed. Only relevant if your plugin maps to `mapped_to_table: "CurrentScan"`; all three default to today's behavior if never mapped. See **[Plugin Import Behavior](PLUGINS_IMPORT_BEHAVIOR.md)** for the full contract — value tables, precedence rules, decision flowcharts, and worked scenarios. ## Examples @@ -350,6 +287,7 @@ tail -f /tmp/log/app.log | grep -i "YOURPREFIX\|Plugins_Objects" ## See Also +- [Plugin Import Behavior](PLUGINS_IMPORT_BEHAVIOR.md) - `scanCreatesDevice`/`scanNotificationMode`/`scanPresence`, for plugins mapping to `CurrentScan` - [Plugin Settings System](PLUGINS_DEV_SETTINGS.md) - How to accept user input - [Data Sources](PLUGINS_DEV_DATASOURCES.md) - Different data source types - [Debugging Plugins](DEBUG_PLUGINS.md) - Troubleshooting plugin issues diff --git a/docs/PLUGINS_IMPORT_BEHAVIOR.md b/docs/PLUGINS_IMPORT_BEHAVIOR.md new file mode 100644 index 00000000..d186e3b8 --- /dev/null +++ b/docs/PLUGINS_IMPORT_BEHAVIOR.md @@ -0,0 +1,66 @@ +# Plugin Import Behavior + +Three optional `CurrentScan` columns, all independent of each other, control what happens once a row your plugin reports reaches the `CurrentScan` table: whether it can create a device, whether it counts as a live presence signal, and whether its notifications are suppressed. This only matters if your plugin maps to `mapped_to_table: "CurrentScan"` — see the [Data contract](PLUGINS_DEV_DATA_CONTRACT.md) for the base column spec these three sit alongside. + +| Column | Type | Default | Meaning | +|---|---|---|---| +| `scanCreatesDevice` | boolean | `1` | Whether this row can originate a *new* `Devices` entry. `0` lets an enrich-only plugin (e.g. a hostname resolver) update an already-existing device's fields without ever being able to create one. | +| `scanNotificationMode` | text (`normal` \| `quiet`) | `normal` | Whether this row's notifications are suppressed. `quiet` always suppresses the outbound email/push; whether the `Events` row itself still gets written depends on the event. **Live** (per-cycle aggregate, reclassifying a row changes future events): `New Device`, `Connected`, `Down Reconnected`, `IP Changed` — audit trail always written. `New Device` isn't gated on `scanPresence = 1` like the other three (see flowcharts below). **Frozen** (`devAlertDown`/`devAlertEvents` seeded at device creation, reclassifying later has no retroactive effect): `Device Down`, `Disconnected` — not symmetric. `Disconnected` always writes its `Events` row (`evePendingAlertEmail = 0` when quiet). `Device Down` writes **no row at all** when `devAlertDown = 0`. | +| `scanPresence` | boolean | `1` | Whether this row asserts the device is *currently online*. `0` means "identity/inventory data, no presence claim" — not "offline". A reservation, a lease record, or a static IPAM entry are typical `0` cases. | + +**Missing vs. invalid values — these behave differently, not interchangeably:** + +| Column | Column never mapped (missing) | Mapped but sent an unexpected value (invalid) | +|---|---|---| +| `scanCreatesDevice` | `1` (schema `DEFAULT`) | `CHECK (scanCreatesDevice IN (0, 1))` — anything else fails the `INSERT` outright, it does not silently fall back to `1` | +| `scanNotificationMode` | `normal` (schema `DEFAULT`) | No `CHECK` constraint — any string other than the literal `'quiet'` is treated as `normal`, since the SQL only special-cases that exact value | +| `scanPresence` | `1` (schema `DEFAULT`) | `CHECK (scanPresence IN (0, 1))` — same as `scanCreatesDevice`, invalid values fail the `INSERT`, they don't default | + +**Multiple plugins reporting the same MAC in the same scan cycle** (the normal case, not an edge case — see the `scan-pipeline` skill) resolve per column, not uniformly: `scanCreatesDevice` and `scanPresence` are most-permissive-wins (any row saying `1` wins), while `scanNotificationMode` is most-*restrictive*-wins (any row saying `quiet` suppresses the notification, even if a sibling row says `normal`) — erring toward under-notifying rather than spamming. + +**Combination matrix** — not every combination is meaningful for every plugin; pick the one that matches what your plugin actually knows: + +| `scanCreatesDevice` | `scanPresence` | Meaning | +|---|---|---| +| 1 | 1 | Normal discovery (the default) | +| 1 | 0 | Inventory/identity import — create the device, but don't claim it's online right now | +| 0 | 1 | Presence-confirming enrichment — never originate a device, but assert presence for one that exists | +| 0 | 0 | Silent enrichment — never originate a device, no presence claim either | + +`scanNotificationMode` is orthogonal to both of the above and can be combined with any row in the table (e.g. inventory import + quiet, for a fully silent bulk import of known-offline devices). + +**Decision: does this row create a device?** + +```mermaid +flowchart TD + A[Row reaches CurrentScan] --> B{scanMac blank or
null-equivalent?} + B -- yes --> Z[Never creates a device] + B -- no --> C{Any row this cycle for this
MAC has scanCreatesDevice = 1?} + C -- no, all say 0 --> Y[No device created
enrich-only] + C -- yes, at least one --> D{Devices row already
exists for this MAC?} + D -- yes --> E[No-op - existing device untouched
by this check] + D -- no --> F[New Devices row created
+ New Device event] +``` + +**Decision: is this event's notification suppressed?** + +```mermaid +flowchart TD + A[Event about to fire] --> B{Fired from a row that exists in
CurrentScan this cycle? New Device /
Connected / Down Reconnected / IP Changed} + B -- yes --> C{Live aggregate: any CurrentScan row
for this MAC says
scanNotificationMode = quiet?} + C -- yes --> S[Suppressed
evePendingAlertEmail = 0] + C -- no --> N[Notified
evePendingAlertEmail = 1] + B -- no, fired from row ABSENCE
Device Down / Disconnected --> D{Frozen device setting:
devAlertDown / devAlertEvents,
seeded at creation time} + D -- off --> S + D -- on --> N +``` + +**Worked scenarios:** + +| Scenario | `scanCreatesDevice` | `scanPresence` | `scanNotificationMode` | `scanMac` | Outcome | +|---|---|---|---|---|---| +| Normal discovery (default plugin behavior) | `1` (default) | `1` (default) | `normal` (default) | real MAC | Device created if new, notified normally, presence tracked live. | +| Enrich-only plugin (e.g. a hostname resolver) | `0` | `1` (default) | `normal` (default) | real MAC | Never originates a device; still updates an existing device's fields via `FIELD_SPECS`. If another plugin reports the same MAC with `scanCreatesDevice = 1`, the device still gets created (most-permissive-wins) — this plugin's `0` doesn't block it. | +| Bulk inventory import of known-offline devices | `1` | `0` | `quiet` | real MAC | Creates devices without claiming they're online, and without a wave of "New Device" notifications for a large batch import. | +| Presence-confirming enrichment (e.g. a DHCP lease scanner) | `0` | `1` | `normal` | real MAC | Confirms an *existing* device is online without ever being the plugin that creates it. | +| Row with no usable device identity (e.g. an object with no routable MAC available) | `0` | irrelevant | irrelevant | blank / null-equivalent | Never creates a device — but not for symmetric reasons. The blank-MAC guard blocks the whole aggregated group by its shared `scanMac` value, regardless of any individual row's `scanCreatesDevice` (even a stray `1` from an unrelated plugin sharing the same blank `scanMac` can't override it). Setting `scanCreatesDevice = 0` here is still correct practice, but on its own is only this row's vote — most-permissive-wins means a sibling row for the same `scanMac` asserting `1` would still win. The blank-MAC guard is what actually guarantees safety regardless of what other contributors do. | diff --git a/mkdocs.yml b/mkdocs.yml index e5da6cdc..3c1f9832 100755 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -110,6 +110,7 @@ nav: - Overview: PLUGINS_DEV.md - Quick start: PLUGINS_DEV_QUICK_START.md - Data contract: PLUGINS_DEV_DATA_CONTRACT.md + - Import behavior: PLUGINS_IMPORT_BEHAVIOR.md - Settings system: PLUGINS_DEV_SETTINGS.md - Data sources: PLUGINS_DEV_DATASOURCES.md - UI components: PLUGINS_DEV_UI_COMPONENTS.md