With "Verify SSL" working, the login request checks the Pi-hole
certificate when that setting is on. In a live test with the setting on
and Pi-hole behind a reverse proxy with an untrusted certificate, the
login failed with "Connection error during Pi-hole auth. Check
PIHOLEAPI_URL and PIHOLEAPI_PASSWORD", although URL and password were
right. requests.exceptions.SSLError is a subclass of ConnectionError, so
the certificate error fell into that branch.
Changes:
- pihole_api_auth() catches requests.exceptions.SSLError before
ConnectionError (server/plugins/pihole_api_scan/pihole_api_scan.py) and
logs: "TLS certificate check failed during Pi-hole auth: <error>. Turn
off PIHOLEAPI_VERIFY_SSL if Pi-hole uses a self-signed certificate."
- test/plugins/test_pihole_api_scan_verify_ssl.py: _login_failure() runs
pihole_api_auth() with a failing login request.
test_certificate_error_points_to_verify_ssl_setting checks the new
message (fails without this change);
test_plain_connection_error_still_points_to_url_and_password checks
that other connection errors keep the old one.
Impact:
A user who turns on "Verify SSL" while Pi-hole uses a self-signed or
otherwise untrusted certificate gets a log line that names the cause and
the setting. Other connection errors are reported as before.
main() read the setting 'PIHOLEAPI_SSL_VERIFY', but config.json defines
the function VERIFY_SSL, so the real key is PIHOLEAPI_VERIFY_SSL.
get_setting_value() returns "" for a key nobody defined, so every Pi-hole
API request got verify="": certificate checks were always off and the
InsecureRequestWarning was suppressed, even when a user had turned on
"Verify SSL". The login request sends the Pi-hole web password to
api/auth. The plugin README and docs/PIHOLE_GUIDE.md used the wrong name
too. test_plugin_conventions.py did not catch it: it only looks at
get_setting_value("X") or <literal>, and this line had no fallback.
Changes:
- test/plugins/test_plugin_conventions.py: new
test_own_setting_keys_exist_in_config. Every literal
get_setting_value('<PREFIX>_X') in a plugin's scripts must name a
setting its own config.json defines; core settings and other plugins'
keys are not checked. Before the fix it fails for exactly one of the 54
plugins: pihole_api_scan.py, get_setting_value('PIHOLEAPI_SSL_VERIFY').
- New test/plugins/test_pihole_api_scan_verify_ssl.py: runs main() with
VERIFY_SSL true and false and checks the verify= value that reaches
requests.post() for api/auth. Only the HTTP call is mocked. It fails
before the fix (verify=None).
- server/plugins/pihole_api_scan/pihole_api_scan.py: main() reads
PIHOLEAPI_VERIFY_SSL.
- server/plugins/pihole_api_scan/README.md and docs/PIHOLE_GUIDE.md name
the setting PIHOLEAPI_VERIFY_SSL.
- docs/PLUGINS_DEV.md, Conventions Checklist: read a setting under the
exact key config.json defines; names the new check.
Impact:
"Verify SSL" now works. The default stays false, so default installs
behave as before. A user who turned it on while Pi-hole uses a
self-signed certificate will now see the login fail, as the setting
description says ("Disable if you are using a self-signed certificate").
Live check against Pi-hole v6.8 behind a proxy with an untrusted
certificate: with "Verify SSL" on the fixed code refused to log in, the
old code logged in and imported 33 entries.
With PIHOLEAPI_GET_OFFLINE enabled, PIHOLEAPI imports devices it considers
offline, but every row still counted as a live sighting. The plugin
computed the online state, yet never passed it on, and its config.json
mapped nothing to CurrentScan.scanPresence, so each row got the column
default 1. An offline device imported only for its name, vendor and IP
was marked online. ADGUARDIMP had the same problem in #1813 and fixed it
the same way.
Changes:
- main() passes the online state as helpVal4 '1' or '0' to add_object()
(server/plugins/pihole_api_scan/pihole_api_scan.py).
- config.json maps helpVal4 to scanPresence, the same column definition
adguard_import uses. The GET_OFFLINE description says offline devices
are imported without being marked online; the README says the same.
- test/db_test_helpers.py: make_plugin_event_row() takes an optional
help_val4, and CREATE_CURRENT_SCAN gets the
CHECK (scanPresence IN (0, 1)) of the real schema (db_upgrade.py), so a
test sees the same constraint as production.
- New test/scan/test_piholeapi_scan_presence_mapping.py runs the real
PIHOLEAPI config.json through process_plugin_events() and checks that a
helpVal4 '0' row lands in CurrentScan with scanPresence 0.
- test/plugins/test_pihole_api_scan.py: main() writes helpVal4 '0' for an
offline device and '1' for every IP row of an online device. Both new
tests fail without this change.
Impact:
Devices imported only because of PIHOLEAPI_GET_OFFLINE keep their name,
vendor and IP data but no longer mark the device as online. Live check
against Pi-hole v6.8: 28 rows with helpVal4 '0', 5 with '1'; the absent
phone got devPresentLastScan 0 and only a New Device event. With
GET_OFFLINE disabled (the default) every imported row is online, so
nothing changes there.
Devices reported by PIHOLEAPI stayed online for about 24 hours after they
left the network (follow-up to #1436). The online check compared
PIHOLEAPI_CONSIDER_ONLINE with the freshest ips[].lastSeen of a device.
Pi-hole FTL v6.8 sets lastSeen to "now" on every network table update for
every client it still holds in memory, also when the client is not in the
ARP cache (src/database/network-table.c, add_FTL_clients_to_network_table()
calls add_netDB_network_address()). In a live run against Pi-hole v6.8 all
22 devices the old code reported as online had a lastSeen of 36 s, among
them a phone whose last DNS query was 18 h old.
Changes:
- New is_device_online(last_query, now_ts, consider_online) in
server/plugins/pihole_api_scan/pihole_api_scan.py. A device is online if
its last DNS query is at most consider_online seconds old; a lastQuery
of 0 (never queried) is always offline.
- gather_device_entries() reads the device lastQuery instead of looping
over ips[].lastSeen, and each entry carries 'isOnline'. The 'lastQuery'
field (shown as "Last Query") now holds the real lastQuery.
- PIHOLEAPI_CONSIDER_ONLINE gets a module-level default of 300, like the
other setting globals.
- The verbose line for a skipped offline device names the MAC and the
age of its last DNS query instead of dumping the whole device dict.
- config.json: CONSIDER_ONLINE is labeled "Online if last query" and its
description says the window applies to the last DNS query.
- README: lists "Time of the last DNS query" and explains that online
status comes from DNS activity, that quiet devices can show as offline,
and what to do (presence scanner on the same schedule, or a larger
CONSIDER_ONLINE when PIHOLEAPI is the only presence source).
- New test/plugins/test_pihole_api_scan.py (module stubs as in
test_adguard_import.py): the reported case, the CONSIDER_ONLINE
boundary, clock skew, a missing or zero lastQuery, a device with two
IPs. All 10 tests fail on the old code.
Impact:
A device goes offline in PIHOLEAPI CONSIDER_ONLINE seconds after its last
DNS query, not about 24 hours later. Behavior change: a device that is on
but sends no DNS query within that window is not reported as online by
PIHOLEAPI any more. Setting keys and defaults are unchanged.
Fixes found running WIFICANARY on a WiFi satellite node that pushes its
detections to a hub with no WiFi of its own.
Synced plugin objects were stored with syncHubNodeName = "decoded"
instead of the node's name. plugin.py took filename.split(".")[2], but by
the time the hub processes the file decode_and_rename_files() has renamed
it to last_result.<PREFIX>.decoded.<NodeName>.<N>.log, so index 2 is the
literal marker. The parsing now lives in
utils.plugin_utils.get_sync_hub_node_name(), next to the function that
produces these names; it keeps node names that contain dots and returns
"" for local result files.
WIFICANARY's absent_baseline_clone was evaluated per trusted_aps entry,
so with a main AP + range extender pair, a weak clone seen while only the
main AP was in range also raised absent_baseline_clone through the
extender's entry, on top of the correct evil_twin. The baseline now
counts as present if any trusted radio for that SSID is visible;
single-entry behaviour is unchanged.
Leaving WIFICANARY_IFACE blank is how such a hub ingests the node's
detections (core processes the synced files after the plugin runs), but
it logged an unconditional "not set - nothing to scan" every cycle. It is
now a verbose "hub-only mode" message, and the node + hub setup is
documented in the setting description and the plugin README.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PzCQWQVz9b2UzqLiGp2mPM
check_trusted_aps() evaluated a rogue AP once per trusted_aps entry sharing
its SSID, so the documented main-AP+extender pattern (same SSID, two
entries) produced duplicate (bssid, motor) rows for a real clone - a
problem once next_release's per-plugin identity-hash dedup guard lands in
main, since it drops a plugin's entire batch on any internal duplicate.
Fixed by deduping once in main() after collecting from all check_*
functions. Also added iw + its setcap to Dockerfile.debian and
.devcontainer/Dockerfile, which the original PR missed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011meLPKCzVpdZyAUfv5U6mm
- check_trusted_aps() was evaluating every AP sharing a protected SSID
against every trusted entry for that SSID, not just its own. A main AP
requiring a stricter accepted security set (e.g. wpa3-only) than a
separately-trusted extender (e.g. wpa2) caused the extender to be
flagged as evil_twin/absent_baseline_clone - it was being judged
against the main AP's accepted set instead of its own. Fixed by
excluding, from each trusted entry's evaluation, any BSSID that has its
own separate trusted entry for the same SSID.
- README's "iw isn't in the published image yet" section was already
stale within the same PR - this branch's own Dockerfile change adds
iw + setcap, so the image ships it. Replaced with one sentence.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Periodic iw-scan-based detection of the 6 heuristics that don't need
monitor-mode hardware (see issue #1789): pwnagotchi/Pineapple signatures,
evil-twin/open clones, baseline-AP-absent-with-clone, security downgrades,
and duplicate-SSID/different-vendor - all evaluated against a user-curated
trusted-AP baseline (WIFICANARY_trusted_aps). A detection creates a
flagged Devices entry even for BSSIDs that never associate, per the
addendum on the same issue.
- WIFICANARY_TRUSTED_SECURITY is multi-select: an observed encryption
exactly matching any selected value is accepted; otherwise it's flagged
if weaker than the strongest selected value (deliberate - comparing
against the weakest would make multi-select pointless, since anything
at/above the weakest would silently pass regardless of the rest of the
selection).
- Added a "known device turned rogue" motor: escalate_known_devices()
cross-references each detection's BSSID against the Devices table via
the new DeviceInstance.getAllByMacs(). This covers the BSSID-identity
half of the issue #1789 addendum's motor 10; the deauth/probe-source-MAC
half still needs monitor-mode data this plugin doesn't have.
- Vendor is deliberately not looked up by this plugin - any device it
creates gets devVendor filled in for free by core's own vendor_update
plugin on its next pass.
43 wificanary unit tests + 10 DeviceInstance.getAllByMacs() tests, all
test_plugin_conventions.py checks pass.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011meLPKCzVpdZyAUfv5U6mm
Maps DOCKERDISC to CurrentScan (scanMac/scanCreatesDevice/scanParentMAC/
scanLastIP) so a container on a macvlan/ipvlan network can opt into
creating or confirming its own device, parented to its Docker host.
Gated by a new DOCKERDISC_CREATE_DEV setting (default off). A container
without its own MAC (bridge/overlay/etc.) never creates a device either
way - the framework's blank-scanMac guard blocks the whole group
regardless of the setting.
Reuses the existing objectPrimaryId/extra column definitions (already
host MAC / container IP) to also feed scanParentMAC/scanLastIP, so every
promoted container is auto-parented to its host with no extra plugin
logic. Two new hidden columns (helpVal1/helpVal2) carry the per-container
scanMac/scanCreatesDevice values.
Tests: 33 -> 35, both DOCKERDISC_CREATE_DEV on/off paths asserted.
Live-verified end to end against a real built image (docker-socket-proxy
+ isolated macvlan/bridge test containers), since IMPORT_ON isn't in any
released NetAlertX image yet.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011meLPKCzVpdZyAUfv5U6mm
resolve_host_mac() returns the manually configured MAC immediately,
with no Socket Proxy /info call at all - the config.json text still
described it as a fallback used only when auto-detection fails.
Reworded both the setting's own description and the parent "Docker
hosts" description to match actual behavior.
The case-insensitivity regression test for lookup_device_mac() stubbed
DeviceInstance.getByMac() to return a fixed row regardless of input,
so it passed even without exercising real collation - functionally a
duplicate of test_lookup_device_mac_found. Replaced it with a
delegation check, and added real SQLite-backed coverage for
DeviceInstance.getByMac()'s case-insensitivity in
test/backend/test_device_instance.py. That surfaced a gap in the
shared db_test_helpers.py fixture: its Devices.devMac column was
missing the COLLATE NOCASE that the real schema declares, so it could
not have exercised this behavior. Fixed the fixture to match.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011meLPKCzVpdZyAUfv5U6mm
CodeRabbit flagged a possible case-sensitivity gap in getByMac() usage.
No functional change needed - Devices.devMac is COLLATE NOCASE at the
schema level, so getByMac()'s plain equality lookup is already
case-insensitive (that's exactly why getAllByName() has to apply it
explicitly and getByMac() doesn't - devName has no column collation).
This test guards that lookup_device_mac() doesn't do anything of its
own that would undo that.
- resolve_host_mac()/lookup_device_mac() now use the new
DeviceInstance.getAllByName()/getByMac() core methods instead of
querying Devices directly - no more direct SQL access from the plugin.
- config.json: removed the → HTML entity from a description (plain
ASCII ->, matching e.g. pihole_monitor's convention), and dropped the
partial es_es/de_de translations scattered through settings/columns
(English only now, matching e.g. rest_import) instead of leaving some
strings translated and others not.
- script.py: removed the two remaining references to
PLUGIN_DOCKERDISC_SPEC.md, a file that was never included in this PR.
- DockerHost now takes a shared run deadline instead of a per-request
timeout duration - every _get() call is capped by whatever's left of
that budget (and REQUEST_TIMEOUT_DEFAULT as an upper bound), so one
slow/hanging host can't burn the whole RUN_TIMEOUT and starve every
other configured host. config.json's hosts param now also sets
timeoutMultiplier, scaling the outer kill-timeout by host count.
- _get() validates the parsed response's shape (dict for /info, list for
/containers/json and /networks) before returning it, rejecting a
malformed/unexpected payload the same as a network failure instead of
letting a caller crash on it further down.
- resolve_host_mac()'s hostname match now detects more than one device
sharing that name and treats it as ambiguous (falls back to manual),
instead of silently picking an arbitrary one via LIMIT 1.
- README: the Socket Proxy is only reachable at 127.0.0.1:2375 under the
network_mode: host case described above it, not under normal compose
networking - fixed the doc to not imply either URL works there.
Read-only enrichment plugin, not an import/discovery plugin. For each
configured Docker host (via Docker Socket Proxy, never /var/run/docker.sock
directly), lists that host's containers under the host device's own
Device Details -> Plugins -> DOCKERDISC tab.
- Never creates a device, for either a host or a container - matches
against hosts already discovered the normal way (ARP/Nmap).
- Every container is listed (bridge/overlay included), not only
macvlan/ipvlan ones - a container only gets its own MAC/IP shown when
it has a macvlan/ipvlan network.
- Host MAC auto-detected via the Socket Proxy's /info -> Devices.devName
match, with a manual fallback.
Addresses CodeRabbit review on PR #1765 (pullrequestreview-5069337680).
pihole_monitor.py:
- last_raw is now tracked per source ({"primary": N, "secondary": M}
per device) instead of one combined value. Combining raw totals
across sources before diffing let a counter reset on one instance
silently net out against real traffic on the other - e.g. primary
+2000 (a real spike) and secondary resetting 1000->5 (-995) would
combine into a raw delta of only 1005, hiding most of the primary's
actual spike behind the secondary's unrelated restart.
- New aggregate_source_deltas(): diffs each source independently via
compute_delta(), then sums only the valid deltas. A source with no
valid delta this run (bootstrapping or just reset) contributes
nothing and doesn't block the others; each source keeps its own
reference point going forward.
- State loaded from before this change (last_raw as a plain number,
not per-source) is now tolerated instead of crashing - treated as no
prior reference point, so every source just bootstraps fresh on the
next run.
README.md:
- Fixed a self-contradicting line: a less frequent schedule means
larger per-run deltas, so PIHOLEMON_MIN_BLOCKED may need *raising*,
not lowering as it previously said.
- Corrected PIHOLEMON_HISTORY_DAYS guidance: it's a retention window,
not a detection delay. A new device becomes evaluable on its 3rd
successful run (1st anchors the counter, 2nd records the first
delta, 3rd has a baseline to compare against), not after the full
retention window.
Tests: 54 (up from 48). New coverage: aggregate_source_deltas() unit
tests including the exact dual-source reset-masking scenario, a
main()-level integration test for the same, and a regression test for
tolerating pre-per-source state. Both the reset-masking fix and the
legacy-state guard verified via mutation testing (reverted each,
confirmed the relevant tests fail, restored). 99% line+branch coverage
maintained.
References PR #1765.
Docs:
- Added PIHOLEMON to docs/PLUGINS.md and a new "Approach 4" section in
docs/PIHOLE_GUIDE.md, leading with anomaly detection (the actual
differentiator vs PIHOLEAPI) and explaining when to pick each plugin.
- README/PLUGINS.md/config.json's UI-facing description all reordered
and shortened to lead with anomaly detection instead of device
import, and to drop implementation detail that belongs in the
README, not the Settings page.
- Trimmed the "Why not extend PIHOLEAPI" README section per feedback -
useful context for a maintainer, not for an end user configuring
the plugin.
config.json / pihole_monitor.py:
- RUN defaults to "disabled", matching every other non-core plugin.
- VERIFY_SSL split into PRIMARY_VERIFY_SSL / SECONDARY_VERIFY_SSL -
each instance can be http/https independently. Settings reordered so
each *_VERIFY_SSL sits right under its matching *_PASSWORD.
- GRAPHQL_TOKEN removed; graphql_token now reads the core API_TOKEN
setting instead of a plugin-specific duplicate.
- GRAPHQL_URL replaced with a GET_OWNER boolean - the endpoint is now
derived from this app's own GRAPHQL_PORT (single source of truth)
instead of a URL the user had to keep in sync by hand.
- HISTORY_LENGTH (run count) replaced with HISTORY_DAYS (a real time
window): state now stores [timestamp, delta] samples and
trim_history() drops anything older than the window, so the
baseline means the same thing regardless of schedule - a faster
schedule adds more data points instead of shrinking the window.
- STATE_FILE moved from the log folder to dbFolderPath, so the rolling
anomaly baseline survives NetAlertX upgrades instead of being wiped
with the logs.
- netalertx_device_owner() (1 GraphQL call per device) replaced by
netalertx_device_owners() (1 call per run, batched) - avoids N
blocking round-trips on a large network.
- Fixed a zero-baseline bug: `bool(... and baseline and ...)` silently
exempted a device with an all-zero blocked-query history (0.0 is
falsy in Python) from ever being flagged, even on its first real
spike. Now checks `baseline is not None`.
- Fixed the placeholder-MAC filter: only excluded the literal "ip-::",
not Pi-hole's general "ip-<address>" placeholder pattern. Caught
downstream by is_mac() either way, but now the actual placeholder
check does what it looks like it does.
- Fixed a cumulative-counter bug: Pi-hole's /api/stats/top_clients
returns a count that's cumulative since FTL last started, not a
per-interval or daily-resetting one (confirmed against FTL's own
source and long-standing user reports that it doesn't reset at
midnight). Comparing that raw total directly against a rolling
average made any device's ordinary growing traffic look like an
escalating anomaly. compute_delta() now diffs each run's raw count
against the previous run's (state gained a per-key last_raw
reference point alongside the delta history) - None (not 0) on the
first-ever run for a device or right after a counter reset, so
those runs re-anchor the reference point instead of fabricating or
swallowing a delta.
- RUN_SCHD default changed from every 6 hours to every 5 minutes now
that the baseline window is real days, not run count, so a frequent
schedule only adds data points instead of narrowing the window; also
matches the default most other device-scanner plugins use.
- RUN_SCHD gained the same live cron-validity checkmark ARPSCAN and
other scanner plugins use (a ✓/✗ icon next to the field, validated
client-side against a regex) - reuses the existing generic
validateRegex() widget, nothing plugin-specific to build.
Tests: 48 tests (up from 37), 99% line+branch coverage. Every fix
above verified via mutation testing (deliberately broken, confirmed
the relevant test fails, then restored).
CodeRabbit follow-up on PR #1765
(https://github.com/netalertx/NetAlertX/pull/1765#discussion_r3888374014):
test_main_history_length_never_produces_empty_or_growing_unbounded only
asserted len(history) >= 1, which a mis-clamped history_length (e.g.
keeping 4 items instead of 1) would still pass unnoticed.
Replaced with test_main_history_length_clamps_and_trims_exactly,
seeding distinct ordered values and asserting the exact retained
history against each PIHOLEMON_HISTORY_LENGTH boundary. Verified it
actually catches a broken clamp: temporarily reverted the
max(1, ...) fix in pihole_monitor.py, confirmed this test fails
([] == [40]) while the rest of the suite still passes, then restored
the fix.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CHJAArRiet4GmXUsxnNLdW
Addresses 5 of the 6 actionable comments from CodeRabbit's review of
PR #1765 (netalertx/NetAlertX#1765), plus adds test coverage:
- fetch_top_blocked_clients() returns None on failure instead of {},
so a failed request can no longer be mistaken for "genuinely zero
blocked queries this run" and silently write a false 0 into a
device's rolling history baseline. main() now tracks a
stats_complete flag and skips anomaly evaluation + history
persistence entirely for a run with incomplete blocked-query data.
- fetch_top_blocked_clients() is now called with count=max_clients
(the existing PIHOLEMON_API_MAXCLIENTS setting) instead of a
hardcoded default of 50, so clients beyond the top 50 are no longer
silently dropped from anomaly detection.
- New build_ip_to_mac() derives the IP->MAC identity map from every
gathered device entry instead of from merge_device_entries()'s
by-MAC-deduplicated output, which only kept one IP per device and
silently lost a multi-IP device's other IPs (misattributing their
blocked-query traffic to a bare IP instead of the real MAC).
- PIHOLEMON_HISTORY_LENGTH is clamped to at least 1, so a negative
setting can no longer reach the history[-history_length:] slice
with a nonsensical negative-of-negative length.
- PIHOLEMON_VERIFY_SSL now defaults to true (was false, matching the
official PIHOLEAPI plugin's convention). README documents the
http:// vs https:// credentials trade-off explicitly rather than
forcing https:// - most home Pi-hole setups, including the one this
plugin targets, run over plain HTTP on a trusted LAN.
- Added test/plugins/test_pihole_monitor.py (37 tests, 99% line and
branch coverage of pihole_monitor.py per pytest-cov - only the
`if __name__ == '__main__':` entry-point guard is unreached):
auth and deauth success/failure paths, the None-sentinel-on-failure
contract, fetch_devices()'s own failure path, build_ip_to_mac()'s
multi-IP fix, gather_device_entries()'s skip branches and fake-MAC
fallback, netalertx_device_owner()'s success/failure/no-URL paths,
and main()-level coverage for source aggregation, the
stats_complete gate, the history_length boundary clamp, the
CONSIDER_ONLINE fallback, an unconfigured-sources run, and the
offline-device / invalid-MAC / unknown-IP / owner-lookup branches
together in one run.
Not addressed: CodeRabbit's suggestion to hard-reject http:// URLs in
auth(). Diverges deliberately - it would break the plugin's majority
use case (Pi-hole admin API on a trusted home LAN without TLS), which
this repo's own PIHOLEAPI plugin also targets over plain HTTP.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CHJAArRiet4GmXUsxnNLdW
jokob-sk