diff --git a/.github/skills/code-standards/SKILL.md b/.github/skills/code-standards/SKILL.md index 98c7516b..83c52d0a 100644 --- a/.github/skills/code-standards/SKILL.md +++ b/.github/skills/code-standards/SKILL.md @@ -8,10 +8,23 @@ description: NetAlertX coding standards and conventions. Use this when writing c - ask me to review before going to each next step (mention n step out of x) (AI only) - before starting, prepare implementation plan (AI only) - ask me to review it and ask any clarifying questions first -- add test creation as last step - follow repo architecture patterns - do not place in the root of /test +- add test creation as last step - follow repo architecture patterns - do not place in the root of `/test` - code has to be maintainable, no duplicate code - follow DRY principle - maintainability of code is more important than speed of implementation - code files should be less than 500 LOC for better maintainability +- DB columns must not contain underscores, use camelCase instead (e.g., deviceInstanceId, not device_instance_id) +- treat DB as temporary storage for stats, long-term configuration should be stored in the `/config` folder, the `/config` folder should allow you to restore most of your functionality (excluding historical data) +- never access DB directly from application layers, always use helper functions in `server/db/db_helper.py` and implement new functionality in handlers (e.g., `DeviceInstance` in `server/models/device_instance.py`) +- always validate and normalize MAC addresses before writing to DB (use `normalize_mac` from `plugin_helper.py`) +- all subprocess calls must set explicit timeouts +- use `timeNowUTC` from `utils.datetime_utils` for all time-related operations and DB timestamps (store all timestamps in UTC) +- use sanitizers from `server/helper.py` for user input before storing in DB +- reuse shared mocks and factories from `test/db_test_helpers.py` for tests, never redefine them locally +- use environment variables for runtime paths, never hardcode paths or use relative paths +- follow existing code style and structure, and ensure backward compatibility with existing installations when submitting PRs +- all code needs to be scalable to handle large networks with thousands of devices (10k+) without performance degradation +- no inline imports, all imports must be at the top of the file + ## File Length @@ -72,6 +85,18 @@ Use sanitizers from `server/helper.py` before storing user input. MAC addresses - Everything is already writable - If permissions needed, fix `.devcontainer/scripts/setup.sh` +## Test Helpers — No Duplicate Mocks + +Reuse shared mocks and factories from `test/db_test_helpers.py`. Never redefine `DummyDB`, `make_db`, or inline DDL in individual test files. + +```python +import sys, os +sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..")) +from db_test_helpers import make_db, DummyDB, insert_device, minutes_ago +``` + +If a helper you need doesn't exist yet, add it to `db_test_helpers.py` — not locally in the test file. + ## Path Hygiene - Use environment variables for runtime paths diff --git a/.github/workflows/mkdocs.yml b/.github/workflows/mkdocs.yml index ab1ca052..27daf1e6 100755 --- a/.github/workflows/mkdocs.yml +++ b/.github/workflows/mkdocs.yml @@ -18,12 +18,15 @@ jobs: - name: Set up Python uses: actions/setup-python@v4 with: - python-version: '3.9' + python-version: '3.11' - name: Install MkDocs run: | - pip install mkdocs mkdocs-material - pip install mkdocs-github-admonitions-plugin + pip install \ + mkdocs==1.6.0 \ + mkdocs-material==9.5.21 \ + mkdocs-github-admonitions-plugin==0.1.1 \ + mkdocs-glightbox - name: Build MkDocs run: mkdocs build diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 4f108749..a2453261 100755 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -15,6 +15,37 @@ Before opening a new issue: - [Check Common Issues & Debug Tips](https://docs.netalertx.com/DEBUG_TIPS#common-issues) - [Search Closed Issues](https://github.com/netalertx/NetAlertX/issues?q=is%3Aissue+is%3Aclosed) +--- + +## Contributing without coding knowledge + +Writing code is not the only way how you can contribute to the project. Here are some ideas how you can help out otherwise: + + - Help improving the documentation (text, typos, use cases, screenshots - all this makes things better) + - Share your favorite project on socials (see [Growth ideas](./GROWTH.md)) + - Write a blog post, or a how-to set-up guide + - Write how it helped you to achieve something fun or interesting to simplify your personal or work life + - Help with the translation effort + - Running a nightly or dev build to test for bugs before they make it to the production release + - Report bugs and features with sufficient detail and care (super important to ease everyone's maintenance burden and minimize back-and-forth) + - running a projects test suite + + +--- + +## Use of AI + +Use of AI-assisted tools is permitted, provided all generated code is reviewed, understood, and verified before submission. + +- All AI-generated code must meet the project's **quality, security, and performance standards**. +- Contributors are responsible for **fully understanding** any code they submit, regardless of how it was produced. +- Prefer **clarity and maintainability over cleverness or brevity**. Readable code is always favored over dense or obfuscated implementations. +- Follow the **DRY (Don't Repeat Yourself) principle** where appropriate, without sacrificing readability. +- Do not submit code that you cannot confidently explain or debug. + +All changes must pass the **full test suite** before opening a PR. + + --- ## Submitting Pull Requests (PRs) @@ -28,11 +59,19 @@ Please: - Provide a clear title and description for your PR - If relevant, add or update tests and documentation - For plugins, refer to the [Plugin Dev Guide](https://docs.netalertx.com/PLUGINS_DEV) +- Switch the PR to DRAFT mode if still being worked on +- Keep PRs **focused and minimal** — avoid unrelated changes in a single PR +- PRs that do not meet these guidelines may be closed without review +## Commit Messages -## Code quality +- Use clear, descriptive commit messages +- Explain *why* a change was made, not just *what* changed +- Reference related issues where applicable -- read and follow the [code-standards](/.github/skills/code-standards/SKILL.md) +## Code Quality + +- Read and follow the [code standards](/.github/skills/code-standards/SKILL.md) --- diff --git a/GROWTH.md b/GROWTH.md new file mode 100644 index 00000000..57dbca57 --- /dev/null +++ b/GROWTH.md @@ -0,0 +1,142 @@ +# 📈 NetAlertX Growth Playbook + +If you like NetAlertX and want to help it grow, this is how. + +This isn’t about spam or "growth hacks." It’s just getting the tool in front of people who already have the problem (manual network docs, blind spots, messy networks). + +--- + +## 🎯 Who this is for + +* **Homelabbers** → Proxmox, Unraid, Raspberry Pi setups +* **Self-hosters** → moving away from cloud / want local-first +* **Sysadmins / MSPs** → need visibility + fast onboarding + +--- + +## 💬 Where to post (and how) + +### Reddit (main channel) + +Reddit works *if* you don’t sound like marketing. + +**Subreddits** + +* r/homelab +* r/selfhosted +* r/sysadmin +* r/networking + +**What works** + +* "Show r/homelab" / "I built this" +* Problem → solution +* Real screenshots or short GIFs + +**Example** + +> Tired of keeping network diagrams up to date? +> I deployed something that does it automatically. + +**Avoid** + +* Over-explaining +* Buzzwords +* "Check out my project!!!" energy + +**Timing (rough guide)** + +* Tue–Thu mornings (US time) + +--- + +### Hacker News (Show HN) + +Good fit because NetAlertX is: + +* local-first +* privacy-focused +* actually useful + +**Title** + +> Show HN: NetAlertX – network documentation that updates itself + +Be ready to answer: + +* stack (Python / PHP / JS) +* how discovery works +* privacy / local scanning + +--- + +### Awesome Lists + +Low effort, long-term visibility. + +Target: + +* awesome-selfhosted +* awesome-homelab + +Keep it simple: + +> NetAlertX – automated network discovery and documentation with alerting + +--- + +## 🎥 Content / creators + +If you or someone else makes videos, these angles work: + +* **First 5 minutes** → "it already found everything" +* **Before vs after** → manual vs NetAlertX +* **Docker setup** → show how easy it is + +No need for overproduction—real usage > polished demos. + +--- + +## ⚖️ Positioning (keep it simple) + +When explaining NetAlertX, this is the mental model: + +| | Manual Docs | Basic Scanners | NetAlertX | +| ------- | ----------------- | -------------- | ------------------------- | +| Updates | Manual / outdated | On demand | **Automatic** | +| Alerts | None | Limited | **New / unknown devices** | +| History | None | None | **Device timeline** | +| Effort | High | Medium | **Set & forget** | + +--- + +## 🧑‍💻 MSP / professional angle + +If you’re using it in real environments: + +* Faster client onboarding +* Better visibility +* Less "what changed?" guessing + +That’s the value—keep it grounded. And share your experience 🙏 + +--- + +## 🛠 Easy ways to help + +* Star the repo +* Post a screenshot of your network +* Mention it when someone asks "how do you track devices?" or "how do you detect presence?" +* Write a quick "how I use it" post + +--- + +## Final note + +Authenticity matters more than reach. + +If it feels like marketing, people ignore it. +If it feels like "this solved my problem," people pay attention. + +Thanks a lot in advance! + - jokob diff --git a/README.md b/README.md index ba0777a2..7afa9ba1 100755 --- a/README.md +++ b/README.md @@ -21,13 +21,13 @@ -Centralized network visibility and continuous asset discovery. +Centralized network visibility and continuous asset discovery for homelabs, IT teams, MSPs, and distributed environments. -Monitor devices, detect change, and stay aware across distributed networks. +Monitor devices, detect change, and maintain visibility across remote sites, VLANs, branch offices, and segmented networks from a single interface. -NetAlertX provides a centralized "Source of Truth" (NSoT) for network infrastructure. Maintain a real-time inventory of every connected device, identify Shadow IT and unauthorized hardware to maintain regulatory compliance, and automate compliance workflows across distributed sites. +NetAlertX provides a centralized "Source of Truth" (NSoT) for network infrastructure. Maintain a real-time inventory of connected devices, identify Shadow IT and unauthorized hardware, support compliance initiatives, and automate operational workflows across distributed customer environments. -NetAlertX is designed to bridge the gap between simple network scanning and complex SIEM tools, providing actionable insights without the overhead. +Designed to bridge the gap between simple network scanners and complex SIEM platforms, NetAlertX delivers actionable network intelligence and centralized monitoring without the operational overhead. ## Table of Contents @@ -98,6 +98,10 @@ build your own scanners with the [Plugin system](https://docs.netalertx.com/PLUG The [workflows module](https://docs.netalertx.com/WORKFLOWS) automates IT governance by enforcing device categorization and cleanup policies. Whether you need to assign newly discovered devices to a specific Network Node, auto-group devices from a given vendor, unarchive a device if detected online, or automatically delete devices, this module provides the flexibility to tailor the automations to your needs. +### MSP & Multi-Site Monitoring + +NetAlertX enables centralized monitoring across remote sites and isolated environments through Sync Nodes for VLANs and branch offices, providing unified visibility of assets across multiple networks. It supports [NOC-style wallboard dashboards](https://docs.netalertx.com/ADVISORY_EYES_ON_GLASS/), [Prometheus metrics export](https://docs.netalertx.com/API_METRICS/), workflow automation for device governance, and distributed discovery with centralized alerting for scalable network operations. + ## Documentation @@ -126,6 +130,15 @@ Compliance & Hardening: See [Security Best Practices](https://github.com/netalertx/NetAlertX/security) for more details. +## Designed for MSPs, NOCs & Distributed Networks + +NetAlertX supports centralized monitoring across VLANs, branch offices, customer environments, isolated networks, and remote sites. + +Using [Sync Nodes](https://docs.netalertx.com/ADVISORY_MULTI_SITE_MONITORING), distributed collectors securely send device inventory and network visibility data back to a central hub, enabling unified monitoring, alerting, and asset tracking across all locations. + +This provides MSPs and NOCs with a single operational view of many independent networks, without requiring direct access or centralized scanning infrastructure. + +Common deployments include MSP wallboards, NOC dashboards, multi-site inventory monitoring, and remote office discovery. ## FAQ @@ -157,7 +170,7 @@ Check the [GitHub Issues](https://github.com/netalertx/NetAlertX/issues) for the ## Everything else -jokob-sk%2FNetAlertX | Trendshift +jokob-sk%2FNetAlertX | Trendshift ### 📧 Get notified what's new @@ -170,6 +183,7 @@ Get notified about a new release, what new functionality you can use and about b - [Fing](https://www.fing.com/) - Network scanner app for your Internet security (Commercial, Phone App, Proprietary hardware) - [NetBox](https://netboxlabs.com/) - The gold standard for Network Source of Truth (NSoT) and IPAM. - [Zabbix](https://www.zabbix.com/) or [Nagios](https://www.nagios.org/) - Strong focus on infrastructure monitoring. +- [Domotz](https://www.domotz.com/) - Commercial network monitoring and remote management platform aimed at MSPs, IT teams, and multi-site environments. - [NetAlertX](https://netalertx.com) - The streamlined, discovery-focused choice for real-time asset intelligence and noise-free alerting. ### 💙 Donations diff --git a/back/app.conf b/back/app.conf index 95fcf24f..4add3c21 100755 --- a/back/app.conf +++ b/back/app.conf @@ -30,6 +30,7 @@ REPORT_DASHBOARD_URL='update_REPORT_DASHBOARD_URL_setting' INTRNT_RUN='schedule' ARPSCAN_RUN='schedule' NSLOOKUP_RUN='before_name_updates' +DIGSCAN_RUN='before_name_updates' AVAHISCAN_RUN='before_name_updates' NBTSCAN_RUN='before_name_updates' diff --git a/docs/ADVISORY_EYES_ON_GLASS.md b/docs/ADVISORY_EYES_ON_GLASS.md index ace8f0dc..41803b76 100644 --- a/docs/ADVISORY_EYES_ON_GLASS.md +++ b/docs/ADVISORY_EYES_ON_GLASS.md @@ -4,6 +4,9 @@ For Managed Service Providers (MSPs) and Network Operations Centers (NOC), "Eyes ![filters](./img/ADVISORIES/filters.png) +> [!TIP] +> If you are using Grafana check the `/metrics` endpoint that exposes **Prometheus-compatible metrics** for NetAlertX, including aggregate device counts and per-device status. See the [Metrics API endpoint](./API_METRICS.md) documentation for details. + --- ### 1. Configure Auto-Refresh for Live Monitoring diff --git a/docs/ADVISORY_MULTI_NETWORK.md b/docs/ADVISORY_MULTI_NETWORK.md index 7a0dd094..be2cd877 100644 --- a/docs/ADVISORY_MULTI_NETWORK.md +++ b/docs/ADVISORY_MULTI_NETWORK.md @@ -8,24 +8,29 @@ Effective multi-network monitoring starts with understanding how NetAlertX "sees * **B. Plan Subnet & Scan Interfaces:** Explicitly configure each accessible segment in `SCAN_SUBNETS` with the corresponding interfaces. * **C. Remote & Inaccessible Networks:** For networks unreachable via ARP, use these strategies: * **Alternate Plugins:** Supplement discovery with [SNMPDSC](SNMPDSC) or [DHCP lease imports](https://docs.netalertx.com/PLUGINS/?h=DHCPLSS#available-plugins). -* **Centralized Multi-Tenant Management using Sync Nodes:** Run secondary NetAlertX instances on isolated networks and aggregate data using the **SYNC plugin**. +* **Sync Hub for MSP & Multi-Site Deployments:** Run secondary NetAlertX instances on isolated networks and aggregate data using the **SYNC plugin**. Use the [`SYNC_BEHAVIOR`](https://github.com/netalertx/NetAlertX/tree/main/front/plugins/sync/README.md#hub-device-write-behavior-sync_behavior) setting on the hub to control whether the hub inherits device config from nodes or manages it independently. * **Manual Entry:** For static assets where only ICMP (ping) status is needed. > [!TIP] -> Explore the [remote networks](./REMOTE_NETWORKS.md) documentation for more details on how to set up the approaches menationed above. +> Explore the [remote networks](./REMOTE_NETWORKS.md) documentation for more details on how to set up the approaches mentioned above. --- ### 2. Automating IT Asset Inventory with Workflows -[Workflows](./WORKFLOWS.md) are the "engine" of NetAlertX, reducing manual overhead as your device list grows. +[Workflows](./WORKFLOWS.md) are the "engine" of NetAlertX, reducing manual overhead as your device list grows. See some examples below: + +#### A. Logical Ownership & VLAN Tagging + +Create a workflow triggered on **Device Creation** to: -* **A. Logical Ownership & VLAN Tagging:** Create a workflow triggered on **Device Creation** to: 1. Inspect the IP/Subnet. 2. Set `devVlan` or `devOwner` custom fields automatically. +#### B. Auto-Grouping: + +Use conditional logic to categorize devices. -* **B. Auto-Grouping:** Use conditional logic to categorize devices. * *Example:* If `devLastIP == 10.10.20.*`, then `Set devLocation = "BranchOffice"`. ```json @@ -56,9 +61,9 @@ Effective multi-network monitoring starts with understanding how NetAlertX "sees ] } ``` +#### C. Sync Node Tracking - -* **C. Sync Node Tracking:** When using multiple instances, ensure all synchub nodes have a descriptive `SYNC_node_name` name to distinguish between sites. +When using multiple instances, ensure all sync hub nodes have a descriptive `SYNC_node_name` name to distinguish between sites. > [!TIP] > Always test new workflows in a "Staging" instance. A misconfigured workflow can trigger thousands of unintended updates across your database. @@ -107,6 +112,7 @@ As your environment grows, tuning the underlying engine is vital to maintain a s * **Plugin Scheduling:** Avoid "Scan Storms" by staggering plugin execution. Running intensive tasks like `NMAP` or `MASS_DNS` simultaneously can spike CPU and cause database locks. * **Database Health:** Large-scale monitoring generates massive event logs. Use the **[DBCLNP (Database Cleanup)](https://www.google.com/search?q=https://docs.netalertx.com/PLUGINS/%23dbclnp)** plugin to prune old records and keep the SQLite database performant. * **Resource Management:** For high-device counts, consider increasing the memory limit for the container and utilizing `tmpfs` for temporary files to reduce SD card/disk I/O bottlenecks. +* Enable the `DEEP_SLEEP` setting. > [!IMPORTANT] > For a deep dive into hardware requirements, database vacuuming, and specific environment variables for high-load instances, refer to the full **[Performance Optimization Guide](https://docs.netalertx.com/PERFORMANCE/)**. diff --git a/docs/ADVISORY_MULTI_SITE_MONITORING.md b/docs/ADVISORY_MULTI_SITE_MONITORING.md new file mode 100644 index 00000000..031199ca --- /dev/null +++ b/docs/ADVISORY_MULTI_SITE_MONITORING.md @@ -0,0 +1,147 @@ +# MSP & Multi-Site Monitoring with NetAlertX + +NetAlertX supports centralized monitoring across remote sites, customer environments, branch offices, VLANs, and isolated networks using distributed Sync Nodes. + +Deploy lightweight NetAlertX instances inside remote or segmented networks, then securely aggregate device inventory and network visibility data into a central hub for unified monitoring, alerting, and asset management. + +![Sync Hub Setup Diagram](https://raw.githubusercontent.com/netalertx/NetAlertX/refs/heads/main/front/plugins/sync/sync_hub.png) + +--- + +## Common MSP & Multi-Site Use Cases + +### Managed Service Providers (MSPs) + +Monitor multiple customer environments from a centralized dashboard while keeping scanning local to each customer site. + +Typical deployments include: + +* Customer branch offices +* Retail stores +* Warehouses +* Small business environments +* Segmented enterprise VLANs + +### Network Operations Centers (NOCs) + +Create centralized "Eyes on Glass" monitoring dashboards using synchronized remote collectors. + +Common NOC setups include: + +* [Wallboard dashboards](./ADVISORY_EYES_ON_GLASS.md) +* Dedicated "Down Devices" views +* Site-(node)specific monitoring filters +* [Prometheus/Grafana integrations](./API_METRICS.md) + +### Isolated or Restricted Networks + +Some environments cannot be scanned directly due to: + +* VLAN isolation +* Firewalls +* VPN segmentation +* Layer 2 limitations +* Remote WAN locations + +[Sync Nodes](./REMOTE_NETWORKS.md) solve this by running discovery locally and forwarding only inventory and monitoring data to the hub. + +--- + +# Architecture Overview + +NetAlertX supports distributed monitoring using two primary roles: + +| Role | Purpose | +| -------- | ----------------------------------------------------------- | +| **Hub** | Centralized monitoring, alerting, dashboards, and inventory | +| **Node** | Remote collector performing local network discovery | + +Each node scans its local network and synchronizes device data back to the hub. + +--- + +# Sync Modes + +NetAlertX supports both PUSH and PULL synchronization models. + +| Mode | Description | +| -------- | -------------------------------------------------- | +| **PUSH** | Nodes send inventory data directly to the hub | +| **PULL** | The hub retrieves inventory data from remote nodes | + +PUSH mode is typically recommended for MSP deployments because remote customer environments often block inbound access. + +--- + +# Device Ownership Models (`SYNC_BEHAVIOR`) in PULL mode + +The hub can operate in different synchronization ownership modes depending on your operational requirements. + +| Mode | Best For | +| -------------- | -------------------------------------------------------------------- | +| `copy-new` | MSP environments where the hub becomes the long-term source of truth | +| `carbon-copy` | Fully managed remote appliances where nodes remain authoritative | +| `hub-defaults` | Centralized inventory management with hub-defined policies | + +This flexibility allows NetAlertX to support both: + +* centrally managed environments +* distributed autonomous sites + +--- + +# Example Deployment + +## Multi-Site MSP Deployment + +```text +Customer Site A ─┐ +Customer Site B ─┼──► Central NetAlertX Hub +Customer Site C ─┘ +``` + +Each customer site runs a lightweight NetAlertX node locally. + +The central hub: + +* aggregates inventory +* handles alerting +* provides dashboards +* exports metrics +* integrates with Grafana or external systems + +--- + +# Recommended MSP Features + +For best results in multi-site environments: + +* Configure descriptive `SYNC_node_name` values +* Use Workflows to auto-tag devices by location/site +* Use predefined "Down Devices" dashboards +* Enable Prometheus metrics export +* Use UI Filters to create site-specific views +* Configure notification throttling to reduce alert fatigue + +--- + +# Related Documentation + +* [Remote Networks](./REMOTE_NETWORKS.md) +* [Sync Hub Plugin](../front/plugins/sync/README.md) +* [Workflows](./WORKFLOWS.md) +* [Metrics API](./API_METRICS.md) +* [Eyes on Glass / NOC Dashboard](./ADVISORY_EYES_ON_GLASS.md) + +--- + +# Summary + +NetAlertX enables lightweight, centralized monitoring across distributed networks without the operational overhead of traditional enterprise monitoring platforms. + +By combining distributed Sync Nodes with centralized dashboards, alerting, and workflows, NetAlertX can function as: + +* a multi-site monitoring platform +* an MSP inventory dashboard +* a lightweight NOC monitoring solution +* a centralized network visibility platform for segmented environments diff --git a/docs/API_DEVICE.md b/docs/API_DEVICE.md index 99692c3c..26c8741c 100755 --- a/docs/API_DEVICE.md +++ b/docs/API_DEVICE.md @@ -50,6 +50,10 @@ Manage a **single device** by its MAC address. Operations include retrieval, upd * **POST** `/device/` Create or update a device record. +> ⚠️ **Full-replace (PUT) semantics.** Every editable field is written on each call. Any field omitted from the payload is reset to its default (empty string or `0`). This matches how the frontend edit form works — it always sends the complete device state. +> +> To update a **single field** without affecting others, use [`POST /device//update-column`](#7-update-a-single-column) instead. + **Request Body**: ```json @@ -62,8 +66,8 @@ Manage a **single device** by its MAC address. Operations include retrieval, upd **Behavior**: -* If `createNew=true` → creates a new device -* Otherwise → updates existing device fields +* If `createNew=true` → inserts a new device row +* Otherwise → **replaces all editable fields** on the existing device **Response**: @@ -163,7 +167,13 @@ Manage a **single device** by its MAC address. Operations include retrieval, upd ## 7. Update a Single Column * **POST** `/device//update-column` - Update one specific column for a device. + Update exactly one field for a device without touching any other fields. + +> ✅ **Partial-update (PATCH) semantics.** Only the specified column is written. All other fields are left unchanged. Use this for automation, integrations, and any workflow that needs to update a single attribute. +> +> To replace all fields at once (e.g. saving from the edit form), use [`POST /device/`](#2-update-device-fields). + +Allowed `columnName` values: `devName`, `devOwner`, `devType`, `devVendor`, `devGroup`, `devLocation`, `devComments`, `devIcon`, `devFavorite`, `devAlertEvents`, `devAlertDown`, `devCanSleep`, `devSkipRepeated`, `devReqNicsOnline`, `devForceStatus`, `devParentMAC`, `devParentPort`, `devParentRelType`, `devSSID`, `devSite`, `devVlan`, `devStaticIP`, `devIsNew`, `devIsArchived`, `devCustomProps`. **Request Body**: @@ -190,6 +200,108 @@ Manage a **single device** by its MAC address. Operations include retrieval, upd --- +## 8. Lock / Unlock a Device Field + +* **POST** `/device//field/lock` + Lock a field to prevent plugin overwrites, or unlock it to allow overwrites again. + +**Request Body**: + +```json +{ + "fieldName": "devName", + "lock": true +} +``` + +| Field | Type | Required | Description | +|---|---|---|---| +| `fieldName` | string | ✅ | Field to lock/unlock (e.g. `devName`, `devVendor`) | +| `lock` | boolean | ❌ | `true` to lock, `false` to unlock (default when omitted) | + +**Response** (success): + +```json +{ + "success": true, + "fieldName": "devName", + "locked": true, + "message": "Field devName locked" +} +``` + +**Error Responses**: + +* Field does not support locking → HTTP 400 +* Unauthorized → HTTP 403 + +--- + +## 9. Unlock / Clear Device Fields (Bulk) + +* **POST** `/devices/fields/unlock` + Unlock fields (clear `LOCKED`/`USER` sources) for one device, a list of devices, or all devices. + +**Request Body**: + +```json +{ + "mac": "AA:BB:CC:DD:EE:FF", + "fields": ["devName", "devVendor"], + "clearAll": false +} +``` + +| Field | Type | Required | Description | +|---|---|---|---| +| `mac` | string or array | ❌ | Single MAC, list of MACs, or omit for all devices | +| `fields` | array of strings | ❌ | Fields to unlock. Omit to unlock all tracked fields | +| `clearAll` | boolean | ❌ | `true` clears all sources; `false` (default) clears only `LOCKED`/`USER` | + +**Response** (success): + +```json +{ + "success": true +} +``` + +**Error Responses**: + +* `fields` is not a list → HTTP 400 +* Unauthorized → HTTP 403 + +--- + +## 10. Set Device Alias + +* **POST** `/device//set-alias` + Convenience wrapper to update the device display name (`devName`). + +**Request Body**: + +```json +{ + "alias": "My Router" +} +``` + +**Response** (success): + +```json +{ + "success": true +} +``` + +**Error Responses**: + +* Missing `alias` → HTTP 400 +* Device not found → HTTP 404 +* Unauthorized → HTTP 403 + +--- + ## Example `curl` Requests **Get Device Details**: @@ -233,3 +345,30 @@ curl -X POST "http://:/device/AA:BB:CC:DD:EE:FF/update- --data '{"columnName":"devName","columnValue":"Updated Device"}' ``` +**Lock a Field**: + +```bash +curl -X POST "http://:/device/AA:BB:CC:DD:EE:FF/field/lock" \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + --data '{"fieldName":"devName","lock":true}' +``` + +**Unlock Fields (all devices)**: + +```bash +curl -X POST "http://:/devices/fields/unlock" \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + --data '{"fields":["devName","devVendor"]}' +``` + +**Set Device Alias**: + +```bash +curl -X POST "http://:/device/AA:BB:CC:DD:EE:FF/set-alias" \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + --data '{"alias":"My Router"}' +``` + diff --git a/docs/API_EVENTS.md b/docs/API_EVENTS.md index ff423c4f..d933658b 100755 --- a/docs/API_EVENTS.md +++ b/docs/API_EVENTS.md @@ -58,12 +58,12 @@ The Events API provides access to **device event logs**, allowing creation, retr "success": true, "events": [ { - "eve_MAC": "00:11:22:33:44:55", - "eve_IP": "192.168.1.10", - "eve_DateTime": "2025-08-24T12:00:00Z", - "eve_EventType": "Device Down", - "eve_AdditionalInfo": "", - "eve_PendingAlertEmail": 1 + "eveMac": "00:11:22:33:44:55", + "eveIp": "192.168.1.10", + "eveDateTime": "2025-08-24T12:00:00Z", + "eveEventType": "Device Down", + "eveAdditionalInfo": "", + "evePendingAlertEmail": 1 } ] } @@ -102,11 +102,11 @@ The Events API provides access to **device event logs**, allowing creation, retr "count": 5, "events": [ { - "eve_DateTime": "2025-12-07 12:00:00", - "eve_EventType": "New Device", - "eve_MAC": "AA:BB:CC:DD:EE:FF", - "eve_IP": "192.168.1.100", - "eve_AdditionalInfo": "Device detected" + "eveDateTime": "2025-12-07 12:00:00", + "eveEventType": "New Device", + "eveMac": "AA:BB:CC:DD:EE:FF", + "eveIp": "192.168.1.100", + "eveAdditionalInfo": "Device detected" } ] } @@ -127,9 +127,9 @@ The Events API provides access to **device event logs**, allowing creation, retr "count": 10, "events": [ { - "eve_DateTime": "2025-12-07 12:00:00", - "eve_EventType": "Device Down", - "eve_MAC": "AA:BB:CC:DD:EE:FF" + "eveDateTime": "2025-12-07 12:00:00", + "eveEventType": "Device Down", + "eveMac": "AA:BB:CC:DD:EE:FF" } ] } @@ -159,9 +159,9 @@ The Events API provides access to **device event logs**, allowing creation, retr 1. Total events in the period 2. Total sessions 3. Missing sessions -4. Voided events (`eve_EventType LIKE 'VOIDED%'`) -5. New device events (`eve_EventType LIKE 'New Device'`) -6. Device down events (`eve_EventType LIKE 'Device Down'`) +4. Voided events (`eveEventType LIKE 'VOIDED%'`) +5. New device events (`eveEventType LIKE 'New Device'`) +6. Device down events (`eveEventType LIKE 'Device Down'`) --- @@ -187,7 +187,7 @@ Event endpoints are available as **MCP Tools** for AI assistant integration: ``` * Events are stored in the **Events table** with the following fields: - `eve_MAC`, `eve_IP`, `eve_DateTime`, `eve_EventType`, `eve_AdditionalInfo`, `eve_PendingAlertEmail`. + `eveMac`, `eveIp`, `eveDateTime`, `eveEventType`, `eveAdditionalInfo`, `evePendingAlertEmail`. * Event creation automatically logs activity for debugging. diff --git a/docs/API_GRAPHQL.md b/docs/API_GRAPHQL.md index e7ccfd10..cd176f57 100755 --- a/docs/API_GRAPHQL.md +++ b/docs/API_GRAPHQL.md @@ -4,6 +4,10 @@ GraphQL queries are **read-optimized for speed**. Data may be slightly out of da * Devices * Settings +* Events +* PluginsObjects +* PluginsHistory +* PluginsEvents * Language Strings (LangStrings) ## Endpoints @@ -254,11 +258,160 @@ curl 'http://host:GRAPHQL_PORT/graphql' \ --- +## Plugin Tables (Objects, Events, History) + +Three queries expose the plugin database tables with server-side pagination, filtering, and search: + +* `pluginsObjects` — current plugin object state +* `pluginsEvents` — unprocessed plugin events +* `pluginsHistory` — historical plugin event log + +All three share the same `PluginQueryOptionsInput` and return the same `PluginEntry` shape. + +### Sample Query + +```graphql +query GetPluginObjects($options: PluginQueryOptionsInput) { + pluginsObjects(options: $options) { + dbCount + count + entries { + index plugin objectPrimaryId objectSecondaryId + dateTimeCreated dateTimeChanged + watchedValue1 watchedValue2 watchedValue3 watchedValue4 + status extra userData foreignKey + syncHubNodeName helpVal1 helpVal2 helpVal3 helpVal4 objectGuid + } + } +} +``` + +### Query Parameters (`PluginQueryOptionsInput`) + +| Parameter | Type | Description | +| ------------ | ----------------- | ------------------------------------------------------ | +| `page` | Int | Page number (1-based). | +| `limit` | Int | Rows per page (max 1000). | +| `sort` | [SortOptionsInput] | Sorting options (`field`, `order`). | +| `search` | String | Free-text search across key columns. | +| `filters` | [FilterOptionsInput] | Column-value exact-match filters. | +| `plugin` | String | Plugin prefix to scope results (e.g. `"ARPSCAN"`). | +| `foreignKey` | String | Foreign key filter (e.g. device MAC). | +| `dateFrom` | String | Start of date range filter on `dateTimeCreated`. | +| `dateTo` | String | End of date range filter on `dateTimeCreated`. | + +### Response Fields + +| Field | Type | Description | +| --------- | ------------- | ------------------------------------------------------------- | +| `dbCount` | Int | Total rows for the requested plugin (before search/filters). | +| `count` | Int | Total rows after all filters (before pagination). | +| `entries` | [PluginEntry] | Paginated list of plugin entries. | + +### `curl` Example + +```sh +curl 'http://host:GRAPHQL_PORT/graphql' \ + -X POST \ + -H 'Authorization: Bearer API_TOKEN' \ + -H 'Content-Type: application/json' \ + --data '{ + "query": "query GetPluginObjects($options: PluginQueryOptionsInput) { pluginsObjects(options: $options) { dbCount count entries { index plugin objectPrimaryId status foreignKey } } }", + "variables": { + "options": { + "plugin": "ARPSCAN", + "page": 1, + "limit": 25 + } + } + }' +``` + +### Badge Prefetch (Batched Counts) + +Use GraphQL aliases to fetch counts for all plugins in a single request: + +```graphql +query BadgeCounts { + ARPSCAN: pluginsObjects(options: {plugin: "ARPSCAN", page: 1, limit: 1}) { dbCount } + INTRNT: pluginsObjects(options: {plugin: "INTRNT", page: 1, limit: 1}) { dbCount } +} +``` + +--- + +## Events Query + +Access the Events table with server-side pagination, filtering, and search. + +### Sample Query + +```graphql +query GetEvents($options: EventQueryOptionsInput) { + events(options: $options) { + dbCount + count + entries { + eveMac + eveIp + eveDateTime + eveEventType + eveAdditionalInfo + evePendingAlertEmail + } + } +} +``` + +### Query Parameters (`EventQueryOptionsInput`) + +| Parameter | Type | Description | +| ----------- | ------------------ | ------------------------------------------------ | +| `page` | Int | Page number (1-based). | +| `limit` | Int | Rows per page (max 1000). | +| `sort` | [SortOptionsInput] | Sorting options (`field`, `order`). | +| `search` | String | Free-text search across key columns. | +| `filters` | [FilterOptionsInput] | Column-value exact-match filters. | +| `eveMac` | String | Filter by device MAC address. | +| `eventType` | String | Filter by event type (e.g. `"New Device"`). | +| `dateFrom` | String | Start of date range filter on `eveDateTime`. | +| `dateTo` | String | End of date range filter on `eveDateTime`. | + +### Response Fields + +| Field | Type | Description | +| --------- | ------------ | ------------------------------------------------------------ | +| `dbCount` | Int | Total rows in the Events table (before any filters). | +| `count` | Int | Total rows after all filters (before pagination). | +| `entries` | [EventEntry] | Paginated list of event entries. | + +### `curl` Example + +```sh +curl 'http://host:GRAPHQL_PORT/graphql' \ + -X POST \ + -H 'Authorization: Bearer API_TOKEN' \ + -H 'Content-Type: application/json' \ + --data '{ + "query": "query GetEvents($options: EventQueryOptionsInput) { events(options: $options) { dbCount count entries { eveMac eveIp eveDateTime eveEventType } } }", + "variables": { + "options": { + "eveMac": "00:11:22:33:44:55", + "page": 1, + "limit": 50 + } + } + }' +``` + +--- + ## Notes -* Device, settings, and LangStrings queries can be combined in **one request** since GraphQL supports batching. +* Device, settings, LangStrings, plugin, and event queries can be combined in **one request** since GraphQL supports batching. * The `fallback_to_en` feature ensures UI always has a value even if a translation is missing. * Data is **cached in memory** per JSON file; changes to language or plugin files will only refresh after the cache detects a file modification. * The `setOverriddenByEnv` flag helps identify setting values that are locked at container runtime. -* The schema is **read-only** — updates must be performed through other APIs or configuration management. See the other [API](API.md) endpoints for details. +* Plugin queries scope `dbCount` to the requested `plugin`/`foreignKey` so badge counts reflect per-plugin totals. +* The schema is **read-only** — updates must be performed through other APIs or configuration management. See the other [API](API.md) endpoints for details. diff --git a/docs/API_SESSIONS.md b/docs/API_SESSIONS.md index 94224aa4..879cdea2 100755 --- a/docs/API_SESSIONS.md +++ b/docs/API_SESSIONS.md @@ -106,12 +106,12 @@ curl -X DELETE "http://:/sessions/delete" \ "success": true, "sessions": [ { - "ses_MAC": "AA:BB:CC:DD:EE:FF", - "ses_Connection": "2025-08-01 10:00", - "ses_Disconnection": "2025-08-01 12:00", - "ses_Duration": "2h 0m", - "ses_IP": "192.168.1.10", - "ses_Info": "" + "sesMac": "AA:BB:CC:DD:EE:FF", + "sesDateTimeConnection": "2025-08-01 10:00", + "sesDateTimeDisconnection": "2025-08-01 12:00", + "sesDuration": "2h 0m", + "sesIp": "192.168.1.10", + "sesAdditionalInfo": "" } ] } @@ -194,12 +194,12 @@ curl -X GET "http://:/sessions/calendar?start=2025-08-0 "success": true, "sessions": [ { - "ses_MAC": "AA:BB:CC:DD:EE:FF", - "ses_Connection": "2025-08-01 10:00", - "ses_Disconnection": "2025-08-01 12:00", - "ses_Duration": "2h 0m", - "ses_IP": "192.168.1.10", - "ses_Info": "" + "sesMac": "AA:BB:CC:DD:EE:FF", + "sesDateTimeConnection": "2025-08-01 10:00", + "sesDateTimeDisconnection": "2025-08-01 12:00", + "sesDuration": "2h 0m", + "sesIp": "192.168.1.10", + "sesAdditionalInfo": "" } ] } @@ -224,15 +224,33 @@ curl -X GET "http://:/sessions/AA:BB:CC:DD:EE:FF?period * `type` → Event type (`all`, `sessions`, `missing`, `voided`, `new`, `down`) Default: `all` * `period` → Period to retrieve events (`7 days`, `1 month`, etc.) + * `page` → Page number, 1-based (default: `1`) + * `limit` → Rows per page, max 1000 (default: `100`) + * `search` → Free-text search filter across all columns + * `sortCol` → Column index to sort by, 0-based (default: `0`) + * `sortDir` → Sort direction: `asc` or `desc` (default: `desc`) **Example:** ``` - /sessions/session-events?type=all&period=7 days + /sessions/session-events?type=all&period=7 days&page=1&limit=25&sortCol=3&sortDir=desc ``` **Response:** - Returns a list of events or sessions with formatted connection, disconnection, duration, and IP information. + + ```json + { + "data": [...], + "total": 150, + "recordsFiltered": 150 + } + ``` + + | Field | Type | Description | + | ----------------- | ---- | ------------------------------------------------- | + | `data` | list | Paginated rows (each row is a list of values). | + | `total` | int | Total rows before search filter. | + | `recordsFiltered` | int | Total rows after search filter (before paging). | #### `curl` Example diff --git a/docs/API_SYNC.md b/docs/API_SYNC.md index c7569170..94cc8669 100755 --- a/docs/API_SYNC.md +++ b/docs/API_SYNC.md @@ -1,4 +1,4 @@ -# Sync API Endpoint +# Sync API Endpoint --- @@ -35,7 +35,7 @@ curl 'http://:/sync' \ --- -#### 9.2 POST `/sync` +#### 9.2 POST `/sync` The **POST** endpoint is used by nodes to **send data to the hub**. The hub expects the data as **form-encoded fields** (application/x-www-form-urlencoded or multipart/form-data). The hub then stores the data in the plugin log folder for processing. @@ -91,7 +91,7 @@ curl -X POST 'http://:/sync' \ * The `data` field contains JSON with a **`data` array**, where each element is a **device object** or **plugin data object**. * The `plugin` and `node_name` fields allow the hub to **organize and store the file correctly**. -* The data is only processed if the relevant plugins are enabled and run on the target server. +* The data is only processed if the relevant plugins are enabled and run on the target server. --- @@ -112,7 +112,7 @@ last_result..encoded...log * Both encoded and decoded files are tracked, and new submissions increment the sequence number. * If storing fails, the API returns HTTP 500 with an error message. -* The data is only processed if the relevant plugins are enabled and run on the target server. +* The data is only processed if the relevant plugins are enabled and run on the target server. --- @@ -120,6 +120,20 @@ last_result..encoded...log * **Authorization Required** – Both GET and POST require a valid API token. * **Data Integrity** – Ensure that `node_name` and `plugin` are consistent to avoid overwriting files. -* **Monitoring** – Notifications are generated whenever data is sent or received (`write_notification`), which can be used for alerting or auditing. +* **Monitoring** – An in-app log entry is written via `write_notification` whenever data is sent or received, which can be used for auditing. * **Use Case** – Typically used in multi-node deployments to consolidate device and event data on a central hub. +--- + +#### 9.4 Hub Device-Write Behavior (`SYNC_BEHAVIOR`) + +The `SYNC_BEHAVIOR` setting controls how the hub writes devices received from nodes (Mode 3 — RECEIVE). It only affects the hub. + +| Value | Default? | Writes to Devices | +|---|---|---| +| `copy-new` | ✅ | New MACs only (INSERT OR IGNORE) | +| `carbon-copy` | | All MACs every sync (UPSERT) | +| `hub-defaults` | | None — hub pipeline handles it | + +For full details and per-mode behaviour, see [SYNC plugin README — Hub Device-Write Behavior](https://github.com/netalertx/NetAlertX/tree/main/front/plugins/sync/README.md#hub-device-write-behavior-sync_behavior). + diff --git a/docs/DEBUG_PLUGINS.md b/docs/DEBUG_PLUGINS.md index 947eee0d..f84e752d 100755 --- a/docs/DEBUG_PLUGINS.md +++ b/docs/DEBUG_PLUGINS.md @@ -43,7 +43,7 @@ Input data from the plugin might cause mapping issues in specific edge cases. Lo 17:31:05 [Scheduler] run for PIHOLE: YES 17:31:05 [Plugin utils] --------------------------------------------- 17:31:05 [Plugin utils] display_name: PiHole (Device sync) -17:31:05 [Plugins] CMD: SELECT n.hwaddr AS Object_PrimaryID, {s-quote}null{s-quote} AS Object_SecondaryID, datetime() AS DateTime, na.ip AS Watched_Value1, n.lastQuery AS Watched_Value2, na.name AS Watched_Value3, n.macVendor AS Watched_Value4, {s-quote}null{s-quote} AS Extra, n.hwaddr AS ForeignKey FROM EXTERNAL_PIHOLE.Network AS n LEFT JOIN EXTERNAL_PIHOLE.Network_Addresses AS na ON na.network_id = n.id WHERE n.hwaddr NOT LIKE {s-quote}ip-%{s-quote} AND n.hwaddr is not {s-quote}00:00:00:00:00:00{s-quote} AND na.ip is not null +17:31:05 [Plugins] CMD: SELECT n.hwaddr AS objectPrimaryId, {s-quote}null{s-quote} AS objectSecondaryId, datetime() AS DateTime, na.ip AS watchedValue1, n.lastQuery AS watchedValue2, na.name AS watchedValue3, n.macVendor AS watchedValue4, {s-quote}null{s-quote} AS Extra, n.hwaddr AS ForeignKey FROM EXTERNAL_PIHOLE.Network AS n LEFT JOIN EXTERNAL_PIHOLE.Network_Addresses AS na ON na.network_id = n.id WHERE n.hwaddr NOT LIKE {s-quote}ip-%{s-quote} AND n.hwaddr is not {s-quote}00:00:00:00:00:00{s-quote} AND na.ip is not null 17:31:05 [Plugins] setTyp: subnets 17:31:05 [Plugin utils] Flattening the below array 17:31:05 ['192.168.1.0/24 --interface=eth1'] @@ -52,7 +52,7 @@ Input data from the plugin might cause mapping issues in specific edge cases. Lo 17:31:05 [Plugins] Convert to Base64: True 17:31:05 [Plugins] base64 value: b'MTkyLjE2OC4xLjAvMjQgLS1pbnRlcmZhY2U9ZXRoMQ==' 17:31:05 [Plugins] Timeout: 10 -17:31:05 [Plugins] Executing: SELECT n.hwaddr AS Object_PrimaryID, 'null' AS Object_SecondaryID, datetime() AS DateTime, na.ip AS Watched_Value1, n.lastQuery AS Watched_Value2, na.name AS Watched_Value3, n.macVendor AS Watched_Value4, 'null' AS Extra, n.hwaddr AS ForeignKey FROM EXTERNAL_PIHOLE.Network AS n LEFT JOIN EXTERNAL_PIHOLE.Network_Addresses AS na ON na.network_id = n.id WHERE n.hwaddr NOT LIKE 'ip-%' AND n.hwaddr is not '00:00:00:00:00:00' AND na.ip is not null +17:31:05 [Plugins] Executing: SELECT n.hwaddr AS objectPrimaryId, 'null' AS objectSecondaryId, datetime() AS DateTime, na.ip AS watchedValue1, n.lastQuery AS watchedValue2, na.name AS watchedValue3, n.macVendor AS watchedValue4, 'null' AS Extra, n.hwaddr AS ForeignKey FROM EXTERNAL_PIHOLE.Network AS n LEFT JOIN EXTERNAL_PIHOLE.Network_Addresses AS na ON na.network_id = n.id WHERE n.hwaddr NOT LIKE 'ip-%' AND n.hwaddr is not '00:00:00:00:00:00' AND na.ip is not null 🔻 17:31:05 [Plugins] SUCCESS, received 2 entries 17:31:05 [Plugins] sqlParam entries: [(0, 'PIHOLE', '01:01:01:01:01:01', 'null', 'null', '2023-12-25 06:31:05', '172.30.0.1', 0, 'aaaa', 'vvvvvvvvv', 'not-processed', 'null', 'null', '01:01:01:01:01:01'), (0, 'PIHOLE', '02:42:ac:1e:00:02', 'null', 'null', '2023-12-25 06:31:05', '172.30.0.2', 0, 'dddd', 'vvvvv2222', 'not-processed', 'null', 'null', '02:42:ac:1e:00:02')] diff --git a/docs/DEVICE_FIELD_LOCK.md b/docs/DEVICE_FIELD_LOCK.md index 47369c57..b68709b7 100644 --- a/docs/DEVICE_FIELD_LOCK.md +++ b/docs/DEVICE_FIELD_LOCK.md @@ -1,5 +1,7 @@ # Quick Reference Guide - Device Field Lock/Unlock System +> For how scan overwrite rules (`SET_ALWAYS`, `SET_EMPTY`) and source tracking work under the hood, see [Device Source Fields](./DEVICE_SOURCE_FIELDS.md). + ## Overview ![Field source and locks](./img/DEVICE_MANAGEMENT/field_sources_and_locks.png) diff --git a/docs/DEVICE_FILTERS.md b/docs/DEVICE_FILTERS.md new file mode 100644 index 00000000..4316cf22 --- /dev/null +++ b/docs/DEVICE_FILTERS.md @@ -0,0 +1,159 @@ +## Device List & Display Configuration + +The **Devices** page is your primary view into what NetAlertX is monitoring. If devices are missing, unexpected devices appear, or the list doesn’t look the way you expect, the issue is often related to **filters**, **display settings**, or **device visibility configuration**. + +This guide focuses on adjusting your view and troubleshooting common display-related issues. + +--- + +## I Don’t See a Device I Expect in *My Devices* + +If a device is missing from the **My Devices** list, work through these checks. + +### Check Active Filters + +The most common cause is that a filter is hiding the device. + +![Image](https://docs.netalertx.com/img/ADVISORIES/filters.png) + +![Image](https://docs.netalertx.com/img/DEVICE_MANAGEMENT/DeviceDetails_DisplaySettings.png) + +Review any active: + +* **Status filters** (Online / Offline / Down / Archived) +* **Location** filters +* **Owner / User** filters +* **Device Type** filters +* Search terms entered in the search box + +Clear filters first, then reload the page and check again. + +### Check global filters + +You can select devices of what statuses should be displayed in the My Devices view. This can be adjusted in the _Settings_ section - search for the `UI_MY_DEVICES` setting and verify that the statuses you want to show are selected. + +--- + +### Check Whether the Device Is Hidden + +Some devices may be excluded from normal views depending on configuration. + +Examples include: + +* Archived devices +* Devices marked as ignored (`NEWDEV_ignored_IPs` and `NEWDEV_ignored_MACs` settings) +* Virtual or relationship-only devices excluded from display (setting `UI_hide_rel_types`) +* Devices assigned to another user view or group + +If the device exists in the database but is intentionally hidden, it may not appear in default lists. + +--- + +### Confirm the Device Has Been Detected + +If the device has never been scanned or synced into NetAlertX yet, it won’t appear in the UI. + +Things to check: + +* Is the device currently online? +* Has the network scan already run? +* Is the correct scan source enabled? +* If using sync/import, has the sync node completed successfully? + +You can also trigger a manual scan and refresh the UI afterward. + +--- + +### Refresh the UI Cache + +Sometimes device data updates correctly in the backend but the browser view hasn’t refreshed yet. + +Try: + +* Clicking the **Reload** icon in the NetAlertX header +* Waiting for the next automatic refresh cycle +* Performing a hard browser refresh (`Ctrl+Shift+R` / `Cmd+Shift+R`) + +The built-in **Reload** action is recommended over browser refresh because it clears the application’s internal cache, otherwise cache refresh might take a couple of minutes. + +--- + +## Filtering Your Device View + +Filters help narrow large device inventories into manageable views. + +Common filtering options include: + +| Filter | Use Case | +| --------------- | ----------------------------------------------------------- | +| **Status** | Show only Online, Offline, Down, New, or Archived devices | +| **Location** | View devices from a specific site or branch | +| **Device Type** | Show only servers, network gear, clients, IoT devices, etc. | +| **Owner/User** | Limit results to a specific user or device owner | +| **Search** | Find devices by hostname, IP address, MAC address, or label | + +Filters can be combined, which is especially useful for large installations. + +Example: + +`Status = Down` + `Location = Sydney Office` + +This shows only devices currently down at that site. Available filters can be configured via the `UI_columns_filters` setting. + +--- + +## Related Display Settings + +Several UI settings affect what appears in the device list. + +### Hidden Connections / Virtual Devices + +You can hide non-essential relationships or virtual connections from the main view to reduce clutter. + +Useful when: + +* imported relationships create visual noise +* virtual devices aren’t relevant to daily monitoring +* you want a cleaner operational view + +See the `UI_hide_rel_types` setting for details. + +--- + +### Dashboard Block Visibility + +If you’re using the dashboard alongside **Devices**, UI Settings allow you to disable blocks that aren’t useful for your workflow. + +Common examples: + +* Tiles +* Presence widgets +* Summary cards +* Relationship views + +This can make the device list easier to focus on. + +To configure the above check the `UI_shown_cards`, `UI_DEV_SECTIONS` and `UI_hide_empty` settings. + +--- + +### Auto Refresh + +If devices appear stale or statuses don’t update immediately, check **UI refresh settings** (`UI_REFRESH` setting). + +A refresh interval between **60–120 seconds** is usually a good balance between responsiveness and browser performance. + +--- + +## Quick Troubleshooting Checklist + +Before digging deeper, run through this list: + +* [ ] Clear all active filters +* [ ] Search by hostname, MAC address, or IP +* [ ] Confirm the device is not archived or hidden +* [ ] Trigger or verify a recent network scan +* [ ] Use the NetAlertX **Reload** icon to refresh the UI cache +* [ ] Check related UI visibility settings + +If the device still doesn’t appear after these checks, review the scan/import logs to confirm it has been discovered successfully by NetAlertX. diff --git a/docs/DEVICE_SOURCE_FIELDS.md b/docs/DEVICE_SOURCE_FIELDS.md index 02ef5e46..7f2bdc52 100644 --- a/docs/DEVICE_SOURCE_FIELDS.md +++ b/docs/DEVICE_SOURCE_FIELDS.md @@ -1,5 +1,7 @@ # Understanding Device Source Fields and Field Updates +> For the UI guide on locking and unlocking individual fields, see [Device Field Lock/Unlock](./DEVICE_FIELD_LOCK.md). + When the system scans a network, it finds various details about devices (like names, IP addresses, and manufacturers). To ensure the data remains accurate without accidentally overwriting manual changes, the system uses a set of "Source Rules." ![Field source and locks](./img/DEVICE_MANAGEMENT/field_sources_and_locks.png) @@ -17,6 +19,8 @@ Every piece of information for a device has a **Source**. This source determines | **NEWDEV** | This value was initialized from `NEWDEV` plugin settings. | **Always** | | **(Plugin Name)** | The value was found by a specific scanner (e.g., `NBTSCAN`). | **Only if specific rules are met** | +> For how `USER` and `LOCKED` sources are set through the UI (lock/unlock buttons), see [Device Field Lock/Unlock](./DEVICE_FIELD_LOCK.md). + --- ## How Scans Update Information @@ -34,6 +38,8 @@ Some plugins are configured to be "authoritative." If a field is in the **SET_AL * The scanner will **always** overwrite the current value with the new one. * *Note: It will still never overwrite a `USER` or `LOCKED` field.* +> On large networks, enabling `SET_ALWAYS` on name-resolution fields (e.g., `devName`) widens the resolution scope and increases DNS query volume. See [Performance — Plugin Field Authority](./PERFORMANCE.md#plugin-field-authority-set_always-and-set_empty) for details. + ### 3. SET_EMPTY If a field is in the **SET_EMPTY** list: diff --git a/docs/DOCKER_INSTALLATION.md b/docs/DOCKER_INSTALLATION.md index 1c03e998..70183b10 100644 --- a/docs/DOCKER_INSTALLATION.md +++ b/docs/DOCKER_INSTALLATION.md @@ -36,7 +36,7 @@ docker run -d --rm --network=host \ > Runtime UID/GID: The image defaults to a service user `netalertx` (UID/GID 20211). A separate readonly lock owner also uses UID/GID 20211 for 004/005 immutability. You can override the runtime UID/GID at build (ARG) or run (`--user` / compose `user:`) but must align writable mounts (`/data`, `/tmp*`) and tmpfs `uid/gid` to that choice. -See alternative [docked-compose examples](https://docs.netalertx.com/DOCKER_COMPOSE). +See alternative [docker-compose examples](https://docs.netalertx.com/DOCKER_COMPOSE). ### Default ports diff --git a/docs/NOTIFICATIONS.md b/docs/NOTIFICATIONS.md index 3bd12d5a..882381e3 100755 --- a/docs/NOTIFICATIONS.md +++ b/docs/NOTIFICATIONS.md @@ -36,7 +36,7 @@ The following device properties influence notifications. You can: On almost all plugins there are 2 core settings, `_WATCH` and `_REPORT_ON`. 1. `_WATCH` specifies the columns which the app should watch. If watched columns change the device state is considered changed. This changed status is then used to decide to send out notifications based on the `_REPORT_ON` setting. -2. `_REPORT_ON` let's you specify on which events the app should notify you. This is related to the `_WATCH` setting. So if you select `watched-changed` and in `_WATCH` you only select `Watched_Value1`, then a notification is triggered if `Watched_Value1` is changed from the previous value, but no notification is send if `Watched_Value2` changes. +2. `_REPORT_ON` let's you specify on which events the app should notify you. This is related to the `_WATCH` setting. So if you select `watched-changed` and in `_WATCH` you only select `watchedValue1`, then a notification is triggered if `watchedValue1` is changed from the previous value, but no notification is send if `watchedValue2` changes. Click the **Read more in the docs.** Link at the top of each plugin to get more details on how the given plugin works. @@ -63,5 +63,3 @@ You can completely ignore detected devices globally. This could be because your 1. Ignored MACs (`NEWDEV_ignored_MACs`) - List of MACs to ignore. 2. Ignored IPs (`NEWDEV_ignored_IPs`) - List of IPs to ignore. - - diff --git a/docs/NOTIFICATION_TEMPLATES.md b/docs/NOTIFICATION_TEMPLATES.md index 1557416c..63828703 100644 --- a/docs/NOTIFICATION_TEMPLATES.md +++ b/docs/NOTIFICATION_TEMPLATES.md @@ -10,22 +10,23 @@ HTML email tables are **not affected** by these templates. 1. Go to **Settings → Notification Processing**. 2. Set a template string for the section you want to customize, e.g.: - - **Text Template: New Devices** → `{Device name} ({MAC}) - {IP}` + - **Text Template: New Devices** → `{devName} ({eveMac}) - {eveIp}` 3. Save. The next notification will use your format. **Before (default):** ``` 🆕 New devices --------- -MAC: aa:bb:cc:dd:ee:ff -Datetime: 2025-01-15 10:30:00 -IP: 192.168.1.42 -Event Type: New Device -Device name: MyPhone -Comments: +devName: MyPhone +eveMac: aa:bb:cc:dd:ee:ff +devVendor: Apple +eveIp: 192.168.1.42 +eveDateTime: 2025-01-15 10:30:00 +eveEventType: New Device +devComments: ``` -**After (with template `{Device name} ({MAC}) - {IP}`):** +**After (with template `{devName} ({eveMac}) - {eveIp}`):** ``` 🆕 New devices --------- @@ -50,7 +51,7 @@ When a template is **empty**, the section uses the original vertical `Header: Va Use `{FieldName}` to insert a value from the notification data. Field names are **case-sensitive** and must match the column names exactly. ``` -{Device name} ({MAC}) connected at {Datetime} +{devName} ({eveMac}) connected at {eveDateTime} ``` - No loops, conditionals, or nesting — just simple string replacement. @@ -58,52 +59,41 @@ Use `{FieldName}` to insert a value from the notification data. Field names are ## Variable Availability by Section -Each section has different available fields because they come from different database queries. +All four device sections (`new_devices`, `down_devices`, `down_reconnected`, `events`) share the same unified field names. -### `new_devices` and `events` - -| Variable | Description | -|----------|-------------| -| `{MAC}` | Device MAC address | -| `{Datetime}` | Event timestamp | -| `{IP}` | Device IP address | -| `{Event Type}` | Type of event (e.g. `New Device`, `Connected`) | -| `{Device name}` | Device display name | -| `{Comments}` | Device comments | - -**Example:** `{Device name} ({MAC}) - {IP} [{Event Type}]` - -### `down_devices` and `down_reconnected` +### `new_devices`, `down_devices`, `down_reconnected`, and `events` | Variable | Description | |----------|-------------| | `{devName}` | Device display name | -| `{eve_MAC}` | Device MAC address | +| `{eveMac}` | Device MAC address | | `{devVendor}` | Device vendor/manufacturer | -| `{eve_IP}` | Device IP address | -| `{eve_DateTime}` | Event timestamp | -| `{eve_EventType}` | Type of event | +| `{eveIp}` | Device IP address | +| `{eveDateTime}` | Event timestamp | +| `{eveEventType}` | Type of event (e.g. `New Device`, `Connected`, `Device Down`) | +| `{devComments}` | Device comments | -**Example:** `{devName} ({eve_MAC}) {devVendor} - went down at {eve_DateTime}` +**Example (new_devices/events):** `{devName} ({eveMac}) - {eveIp} [{eveEventType}]` + +**Example (down_devices):** `{devName} ({eveMac}) {devVendor} - went down at {eveDateTime}` + +**Example (down_reconnected):** `{devName} ({eveMac}) reconnected at {eveDateTime}` ### `plugins` | Variable | Description | |----------|-------------| -| `{Plugin}` | Plugin code name | -| `{Object_PrimaryId}` | Primary identifier of the object | -| `{Object_SecondaryId}` | Secondary identifier | -| `{DateTimeChanged}` | Timestamp of change | -| `{Watched_Value1}` | First watched value | -| `{Watched_Value2}` | Second watched value | -| `{Watched_Value3}` | Third watched value | -| `{Watched_Value4}` | Fourth watched value | -| `{Status}` | Plugin event status | +| `{plugin}` | Plugin code name | +| `{objectPrimaryId}` | Primary identifier of the object | +| `{objectSecondaryId}` | Secondary identifier | +| `{dateTimeChanged}` | Timestamp of change | +| `{watchedValue1}` | First watched value | +| `{watchedValue2}` | Second watched value | +| `{watchedValue3}` | Third watched value | +| `{watchedValue4}` | Fourth watched value | +| `{status}` | Plugin event status | -**Example:** `{Plugin}: {Object_PrimaryId} - {Status}` - -> [!NOTE] -> Field names differ between sections because they come from different SQL queries. A template configured for `new_devices` cannot use `{devName}` — that field is only available in `down_devices` and `down_reconnected`. +**Example:** `{plugin}: {objectPrimaryId} - {status}` ## Section Headers Toggle diff --git a/docs/PERFORMANCE.md b/docs/PERFORMANCE.md index 922ad923..abf779db 100755 --- a/docs/PERFORMANCE.md +++ b/docs/PERFORMANCE.md @@ -87,6 +87,7 @@ Frequent scans increase resource usage, network traffic, and database read/write * **Increase scan intervals** (`_RUN_SCHD`) on busy networks or low-end hardware. * **Increase timeouts** (`_RUN_TIMEOUT`) to avoid plugin failures. * **Reduce subnet size** – e.g., use `/24` instead of `/16` to reduce scan load. +* **Enable the deep sleep setting** (`DEEP_SLEEP`) – Lowers CPU usage by extending idle wait times between processing cycles. When enabled, scans may be delayed by up to 1 minute and the UI might become less responsive. Some plugins also include options to limit which devices are scanned. If certain plugins consistently run long, consider narrowing their scope. @@ -94,6 +95,41 @@ For example, the **ICMP plugin** allows scanning only IPs that match a specific --- +## Plugin Field Authority: `SET_ALWAYS` and `SET_EMPTY` + +Plugins can be configured to control how aggressively they overwrite existing device field values via two settings: + +| Setting | Behaviour | +|---|---| +| `_SET_ALWAYS` | Plugin always overwrites the field, as long as it can resolve a value and the field is not `USER`/`LOCKED` | +| `_SET_EMPTY` | Plugin only writes when the field is currently empty | + +Both settings accept a list of field names (e.g., `devName`, `devFQDN`). See [Name resolution](./NAME_RESOLUTION.md) and [Field locking](./DEVICE_SOURCE_FIELDS.md) docs for details. + +### Performance Impact of `SET_ALWAYS` on Name Resolution + +By default, name resolution (DIGSCAN, NBTSCAN, NSLOOKUP, AVAHISCAN) only runs against **devices that have no name yet**. This keeps DNS query volume proportional to new devices discovered, not the total inventory. + +When **any** name-resolution plugin has `devName` in its `SET_ALWAYS` list, the system additionally runs a second resolution pass against **all devices whose name is not `USER`/`LOCKED` protected**. This allows a higher-priority plugin (e.g., DIGSCAN) to overwrite names previously set by a lower-priority one (e.g., NBTSCAN). + +**Cost:** one DNS query per unprotected, already-named device per name-resolution cycle. + +| Scenario | Devices resolved per cycle | +|---|---| +| No `SET_ALWAYS` on `devName` | Only new/unknown devices | +| `SET_ALWAYS: devName` on any plugin | New/unknown devices **+** all unprotected named devices | + +> [!WARNING] +> On large installations (thousands of devices), enabling `SET_ALWAYS: devName` significantly increases DNS query volume and cycle duration. To mitigate: +> +> * Increase the scan interval of name-resolution plugins (`DIGSCAN_RUN_SCHD`, `NBTSCAN_RUN_SCHD`, etc.). +> * Mark devices whose name should never change as `USER` or `LOCKED` — they are excluded from the re-resolve pass entirely. +> * Use `SET_ALWAYS` only on the highest-priority plugin; leave lower-priority plugins without it. + +The actual number of DB rows updated is logged at `verbose` level under `[Update Device Name] SET_ALWAYS re-resolve - DB rows updated`. + +--- + ## Storing Temporary Files in Memory On devices with slower I/O, you can improve performance by storing temporary files (and optionally the database) in memory using `tmpfs`. diff --git a/docs/PLUGINS.md b/docs/PLUGINS.md index 78d93d74..e8224fd2 100755 --- a/docs/PLUGINS.md +++ b/docs/PLUGINS.md @@ -57,10 +57,12 @@ Device-detecting plugins insert values into the `CurrentScan` database table. T | `DHCPSRVS` | [dhcp_servers](https://github.com/netalertx/NetAlertX/tree/main/front/plugins/dhcp_servers/) | ♻ | DHCP servers | | | | `DIGSCAN` | [dig_scan](https://github.com/netalertx/NetAlertX/tree/main/front/plugins/dig_scan/) | 🆎 | Dig (DNS) Name resolution | | | | `FREEBOX` | [freebox](https://github.com/netalertx/NetAlertX/tree/main/front/plugins/freebox/) | 🔍/♻/🆎 | Pull data and names from Freebox/Iliadbox | | | +| `FRITZBOX` | [fritzbox](https://github.com/netalertx/NetAlertX/tree/main/front/plugins/fritzbox/) | 🔍 | Fritz!Box device scanner via TR-064 | | | | `ICMP` | [icmp_scan](https://github.com/netalertx/NetAlertX/tree/main/front/plugins/icmp_scan/) | ♻ | ICMP (ping) status checker | | | | `INTRNT` | [internet_ip](https://github.com/netalertx/NetAlertX/tree/main/front/plugins/internet_ip/) | 🔍 | Internet IP scanner | | | | `INTRSPD` | [internet_speedtest](https://github.com/netalertx/NetAlertX/tree/main/front/plugins/internet_speedtest/) | ♻ | Internet speed test | | | | `IPNEIGH` | [ipneigh](https://github.com/netalertx/NetAlertX/tree/main/front/plugins/ipneigh/) | 🔍 | Scan ARP (IPv4) and NDP (IPv6) tables | | | +| `KEALSS` | [kea_api](https://github.com/netalertx/NetAlertX/tree/main/front/plugins/kea_api/) | 🔍/🆎 | Pull lease data from the Kea DHCP API | | | | `LUCIRPC` | [luci_import](https://github.com/netalertx/NetAlertX/tree/main/front/plugins/luci_import/) | 🔍 | Import connected devices from OpenWRT | | | | `MAINT` | [maintenance](https://github.com/netalertx/NetAlertX/tree/main/front/plugins/maintenance/) | ⚙ | Maintenance of logs, etc. | | | | `MQTT` | [_publisher_mqtt](https://github.com/netalertx/NetAlertX/tree/main/front/plugins/_publisher_mqtt/) | ▶️ | MQTT for synching to Home Assistant | | | diff --git a/docs/PLUGINS_DEV.md b/docs/PLUGINS_DEV.md index 1bfaf854..57193043 100755 --- a/docs/PLUGINS_DEV.md +++ b/docs/PLUGINS_DEV.md @@ -179,13 +179,13 @@ Quick reference: | Column | Name | Required | Example | |--------|------|----------|---------| -| 0 | Object_PrimaryID | **YES** | `"device_name"` or `"192.168.1.1"` | -| 1 | Object_SecondaryID | no | `"secondary_id"` or `null` | +| 0 | objectPrimaryId | **YES** | `"device_name"` or `"192.168.1.1"` | +| 1 | objectSecondaryId | no | `"secondary_id"` or `null` | | 2 | DateTime | **YES** | `"2023-01-02 15:56:30"` | -| 3 | Watched_Value1 | **YES** | `"online"` or `"200"` | -| 4 | Watched_Value2 | no | `"ip_address"` or `null` | -| 5 | Watched_Value3 | no | `null` | -| 6 | Watched_Value4 | no | `null` | +| 3 | watchedValue1 | **YES** | `"online"` or `"200"` | +| 4 | watchedValue2 | no | `"ip_address"` or `null` | +| 5 | watchedValue3 | no | `null` | +| 6 | watchedValue4 | no | `null` | | 7 | Extra | no | `"additional data"` or `null` | | 8 | ForeignKey | no | `"aa:bb:cc:dd:ee:ff"` or `null` | @@ -243,7 +243,7 @@ Control which rows display in the UI: { "data_filters": [ { - "compare_column": "Object_PrimaryID", + "compare_column": "objectPrimaryId", "compare_operator": "==", "compare_field_id": "txtMacFilter", "compare_js_template": "'{value}'.toString()", @@ -267,7 +267,7 @@ To import plugin data into NetAlertX tables for device discovery or notification "mapped_to_table": "CurrentScan", "database_column_definitions": [ { - "column": "Object_PrimaryID", + "column": "objectPrimaryId", "mapped_to_column": "scanMac", "show": true, "type": "device_mac", @@ -302,50 +302,13 @@ Plugin results are displayed in the web interface using various component types. ### Common Display Types -**Read settings in your Python script:** - -```python -from helper import get_setting_value - -# Read a setting by code name (prefix + function) -api_url = get_setting_value('MYPLN_API_URL') -api_key = get_setting_value('MYPLN_API_KEY') -watch_columns = get_setting_value('MYPLN_WATCH') - -print(f"Connecting to {api_url}") -``` - -**Pass settings as command parameters:** - -Define `params` in config to pass settings as script arguments: - -```json -{ - "params": [ - { - "name": "api_url", - "type": "setting", - "value": "MYPLN_API_URL" - } - ] -} -``` - -Then use in `CMD`: `python3 script.py --url={api_url}` - See [PLUGINS_DEV_SETTINGS.md](PLUGINS_DEV_SETTINGS.md) for complete settings documentation, and [PLUGINS_DEV_DATASOURCES.md](PLUGINS_DEV_DATASOURCES.md) for data source details. -[screen1]: https://raw.githubusercontent.com/jokob-sk/NetAlertX/main/docs/img/plugins.png "Screen 1" -[screen2]: https://raw.githubusercontent.com/jokob-sk/NetAlertX/main/docs/img/plugins_settings.png "Screen 2" -[screen3]: https://raw.githubusercontent.com/jokob-sk/NetAlertX/main/docs/img/plugins_json_settings.png "Screen 3" -[screen4]: https://raw.githubusercontent.com/jokob-sk/NetAlertX/main/docs/img/plugins_json_ui.png "Screen 4" -[screen5]: https://raw.githubusercontent.com/jokob-sk/NetAlertX/main/docs/img/plugins_device_details.png "Screen 5" - ## Quick Reference: Key Concepts ### Plugin Output Format ``` -Object_PrimaryID|Object_SecondaryID|DateTime|Watched_Value1|Watched_Value2|Watched_Value3|Watched_Value4|Extra|ForeignKey +objectPrimaryId|objectSecondaryId|DateTime|watchedValue1|watchedValue2|watchedValue3|watchedValue4|Extra|ForeignKey ``` 9 required columns, 4 optional helpers = 13 max @@ -394,3 +357,8 @@ See: [UI Components](PLUGINS_DEV_UI_COMPONENTS.md) --- +[screen1]: https://raw.githubusercontent.com/jokob-sk/NetAlertX/main/docs/img/plugins.png "Screen 1" +[screen2]: https://raw.githubusercontent.com/jokob-sk/NetAlertX/main/docs/img/plugins_settings.png "Screen 2" +[screen3]: https://raw.githubusercontent.com/jokob-sk/NetAlertX/main/docs/img/plugins_json_settings.png "Screen 3" +[screen4]: https://raw.githubusercontent.com/jokob-sk/NetAlertX/main/docs/img/plugins_json_ui.png "Screen 4" +[screen5]: https://raw.githubusercontent.com/jokob-sk/NetAlertX/main/docs/img/plugins_device_details.png "Screen 5" \ No newline at end of file diff --git a/docs/PLUGINS_DEV_CONFIG.md b/docs/PLUGINS_DEV_CONFIG.md index bdc7efcc..69e95bdb 100755 --- a/docs/PLUGINS_DEV_CONFIG.md +++ b/docs/PLUGINS_DEV_CONFIG.md @@ -77,7 +77,7 @@ It also describes plugin output expectations and the main plugin categories. * `database_column_definitions` * `mapped_to_table` -**Example:** `Object_PrimaryID → devMAC` +**Example:** `objectPrimaryId → devMAC` --- @@ -88,9 +88,9 @@ Output values are pipe-delimited in a fixed order. #### Identifiers -* `Object_PrimaryID` and `Object_SecondaryID` uniquely identify records (for example, `MAC|IP`). +* `objectPrimaryId` and `objectSecondaryId` uniquely identify records (for example, `MAC|IP`). -#### Watched Values (`Watched_Value1–4`) +#### Watched Values (`watchedValue1–4`) * Used by the core to detect changes between runs. * Changes in these fields can trigger notifications. @@ -114,7 +114,7 @@ Output values are pipe-delimited in a fixed order. ### 7. Persistence * Parsed data is **upserted** into the database. -* Conflicts are resolved using the combined key: `Object_PrimaryID + Object_SecondaryID`. +* Conflicts are resolved using the combined key: `objectPrimaryId + objectSecondaryId`. --- diff --git a/docs/PLUGINS_DEV_DATASOURCES.md b/docs/PLUGINS_DEV_DATASOURCES.md index f6d547cd..cc3b7700 100644 --- a/docs/PLUGINS_DEV_DATASOURCES.md +++ b/docs/PLUGINS_DEV_DATASOURCES.md @@ -107,7 +107,7 @@ Query the NetAlertX SQLite database and display results. { "function": "CMD", "type": {"dataType": "string", "elements": [{"elementType": "input", "elementOptions": [], "transformers": []}]}, - "default_value": "SELECT dv.devName as Object_PrimaryID, cast(dv.devLastIP as VARCHAR(100)) || ':' || cast(SUBSTR(ns.Port, 0, INSTR(ns.Port, '/')) as VARCHAR(100)) as Object_SecondaryID, datetime() as DateTime, ns.Service as Watched_Value1, ns.State as Watched_Value2, null as Watched_Value3, null as Watched_Value4, ns.Extra as Extra, dv.devMac as ForeignKey FROM (SELECT * FROM Nmap_Scan) ns LEFT JOIN (SELECT devName, devMac, devLastIP FROM Devices) dv ON ns.MAC = dv.devMac", + "default_value": "SELECT dv.devName as objectPrimaryId, cast(dv.devLastIP as VARCHAR(100)) || ':' || cast(SUBSTR(ns.Port, 0, INSTR(ns.Port, '/')) as VARCHAR(100)) as objectSecondaryId, datetime() as DateTime, ns.Service as watchedValue1, ns.State as watchedValue2, null as watchedValue3, null as watchedValue4, ns.Extra as Extra, dv.devMac as ForeignKey FROM (SELECT * FROM Nmap_Scan) ns LEFT JOIN (SELECT devName, devMac, devLastIP FROM Devices) dv ON ns.MAC = dv.devMac", "localized": ["name"], "name": [{"language_code": "en_us", "string": "SQL to run"}], "description": [{"language_code": "en_us", "string": "This SQL query populates the plugin table"}] @@ -118,13 +118,13 @@ Query the NetAlertX SQLite database and display results. ```sql SELECT - e.EventValue as Object_PrimaryID, - d.devName as Object_SecondaryID, + e.EventValue as objectPrimaryId, + d.devName as objectSecondaryId, e.EventDateTime as DateTime, - e.EventType as Watched_Value1, - d.devLastIP as Watched_Value2, - null as Watched_Value3, - null as Watched_Value4, + e.EventType as watchedValue1, + d.devLastIP as watchedValue2, + null as watchedValue3, + null as watchedValue4, e.EventDetails as Extra, d.devMac as ForeignKey FROM @@ -181,7 +181,7 @@ Then set data source and query: ```json { "function": "CMD", - "default_value": "SELECT hwaddr as Object_PrimaryID, cast('http://' || (SELECT ip FROM EXTERNAL_PIHOLE.network_addresses WHERE network_id = id ORDER BY lastseen DESC LIMIT 1) as VARCHAR(100)) || ':' || cast(SUBSTR((SELECT name FROM EXTERNAL_PIHOLE.network_addresses WHERE network_id = id ORDER BY lastseen DESC LIMIT 1), 0, INSTR((SELECT name FROM EXTERNAL_PIHOLE.network_addresses WHERE network_id = id ORDER BY lastseen DESC LIMIT 1), '/')) as VARCHAR(100)) as Object_SecondaryID, datetime() as DateTime, macVendor as Watched_Value1, lastQuery as Watched_Value2, (SELECT name FROM EXTERNAL_PIHOLE.network_addresses WHERE network_id = id ORDER BY lastseen DESC LIMIT 1) as Watched_Value3, null as Watched_Value4, '' as Extra, hwaddr as ForeignKey FROM EXTERNAL_PIHOLE.network WHERE hwaddr NOT LIKE 'ip-%' AND hwaddr <> '00:00:00:00:00:00'", + "default_value": "SELECT hwaddr as objectPrimaryId, cast('http://' || (SELECT ip FROM EXTERNAL_PIHOLE.network_addresses WHERE network_id = id ORDER BY lastseen DESC LIMIT 1) as VARCHAR(100)) || ':' || cast(SUBSTR((SELECT name FROM EXTERNAL_PIHOLE.network_addresses WHERE network_id = id ORDER BY lastseen DESC LIMIT 1), 0, INSTR((SELECT name FROM EXTERNAL_PIHOLE.network_addresses WHERE network_id = id ORDER BY lastseen DESC LIMIT 1), '/')) as VARCHAR(100)) as objectSecondaryId, datetime() as DateTime, macVendor as watchedValue1, lastQuery as watchedValue2, (SELECT name FROM EXTERNAL_PIHOLE.network_addresses WHERE network_id = id ORDER BY lastseen DESC LIMIT 1) as watchedValue3, null as watchedValue4, '' as Extra, hwaddr as ForeignKey FROM EXTERNAL_PIHOLE.network WHERE hwaddr NOT LIKE 'ip-%' AND hwaddr <> '00:00:00:00:00:00'", "localized": ["name"], "name": [{"language_code": "en_us", "string": "SQL to run"}] } diff --git a/docs/PLUGINS_DEV_DATA_CONTRACT.md b/docs/PLUGINS_DEV_DATA_CONTRACT.md index f0c5390b..2ca03bca 100644 --- a/docs/PLUGINS_DEV_DATA_CONTRACT.md +++ b/docs/PLUGINS_DEV_DATA_CONTRACT.md @@ -15,22 +15,62 @@ Plugins communicate with NetAlertX by writing results to a **pipe-delimited log **Required Columns:** 9 (mandatory) + up to 4 optional helper columns = 13 total + +## Using `plugin_helper.py` + +The easiest way to ensure correct output is to use the [`plugin_helper.py`](../front/plugins/plugin_helper.py) library: + +```python +from plugin_helper import Plugin_Objects + +# Initialize with your plugin's prefix +plugin_objects = Plugin_Objects("YOURPREFIX") + +# Add objects +plugin_objects.add_object( + objectPrimaryId="device_id", + objectSecondaryId="192.168.1.1", + DateTime="2023-01-02 15:56:30", + watchedValue1="online", + watchedValue2=None, + watchedValue3=None, + watchedValue4=None, + Extra="Additional data", + ForeignKey="aa:bb:cc:dd:ee:ff", + helpVal1=None, + helpVal2=None, + helpVal3=None, + helpVal4=None +) + +# Write results (handles formatting, sanitization, and file creation) +plugin_objects.write_result_file() +``` + +The library automatically: + +- Validates data types +- Sanitizes string values +- Normalizes MAC addresses +- Writes to the correct file location +- Creates the file in `/tmp/log/plugins/last_result..log` + ## Column Specification > [!NOTE] -> The order of columns is **FIXED** and cannot be changed. All 9 mandatory columns must be provided. If you use any optional column (`HelpVal1`), you must supply all optional columns (`HelpVal1` through `HelpVal4`). +> The order of columns is **FIXED** and cannot be changed. All 9 mandatory columns must be provided. If you use any optional column (`helpVal1`), you must supply all optional columns (`helpVal1` through `helpVal4`). ### Mandatory Columns (0–8) | Order | Column Name | Type | Required | Description | |-------|-------------|------|----------|-------------| -| 0 | `Object_PrimaryID` | string | **YES** | The primary identifier for grouping. Examples: device MAC, hostname, service name, or any unique ID | -| 1 | `Object_SecondaryID` | string | no | Secondary identifier for relationships (e.g., IP address, port, sub-ID). Use `null` if not needed | +| 0 | `objectPrimaryId` | string | **YES** | The primary identifier for grouping. Examples: device MAC, hostname, service name, or any unique ID | +| 1 | `objectSecondaryId` | string | no | Secondary identifier for relationships (e.g., IP address, port, sub-ID). Use `null` if not needed | | 2 | `DateTime` | string | **YES** | Timestamp when the event/data was collected. Format: `YYYY-MM-DD HH:MM:SS` | -| 3 | `Watched_Value1` | string | **YES** | Primary watched value. Changes trigger notifications. Examples: IP address, status, version | -| 4 | `Watched_Value2` | string | no | Secondary watched value. Use `null` if not needed | -| 5 | `Watched_Value3` | string | no | Tertiary watched value. Use `null` if not needed | -| 6 | `Watched_Value4` | string | no | Quaternary watched value. Use `null` if not needed | +| 3 | `watchedValue1` | string | **YES** | Primary watched value. Changes trigger notifications. Examples: IP address, status, version | +| 4 | `watchedValue2` | string | no | Secondary watched value. Use `null` if not needed | +| 5 | `watchedValue3` | string | no | Tertiary watched value. Use `null` if not needed | +| 6 | `watchedValue4` | string | no | Quaternary watched value. Use `null` if not needed | | 7 | `Extra` | string | no | Any additional metadata to display in UI and notifications. Use `null` if not needed | | 8 | `ForeignKey` | string | no | Foreign key linking to parent object (usually MAC address for device relationship). Use `null` if not needed | @@ -38,10 +78,10 @@ Plugins communicate with NetAlertX by writing results to a **pipe-delimited log | Order | Column Name | Type | Required | Description | |-------|-------------|------|----------|-------------| -| 9 | `HelpVal1` | string | *conditional* | Helper value 1. If used, all help values must be supplied | -| 10 | `HelpVal2` | string | *conditional* | Helper value 2. If used, all help values must be supplied | -| 11 | `HelpVal3` | string | *conditional* | Helper value 3. If used, all help values must be supplied | -| 12 | `HelpVal4` | string | *conditional* | Helper value 4. If used, all help values must be supplied | +| 9 | `helpVal1` | string | *conditional* | Helper value 1. If used, all help values must be supplied | +| 10 | `helpVal2` | string | *conditional* | Helper value 2. If used, all help values must be supplied | +| 11 | `helpVal3` | string | *conditional* | Helper value 3. If used, all help values must be supplied | +| 12 | `helpVal4` | string | *conditional* | Helper value 4. If used, all help values must be supplied | ## Usage Guide @@ -58,15 +98,15 @@ Watched values are fields that the NetAlertX core monitors for **changes between **How to use them:** -- `Watched_Value1`: Always required; primary indicator of status/state -- `Watched_Value2–4`: Optional; use for secondary/tertiary state information +- `watchedValue1`: Always required; primary indicator of status/state +- `watchedValue2–4`: Optional; use for secondary/tertiary state information - Leave unused ones as `null` **Example:** -- Device scanner: `Watched_Value1 = "online"` or `"offline"` -- Port scanner: `Watched_Value1 = "80"` (port number), `Watched_Value2 = "open"` (state) -- Service monitor: `Watched_Value1 = "200"` (HTTP status), `Watched_Value2 = "0.45"` (response time) +- Device scanner: `watchedValue1 = "online"` or `"offline"` +- Port scanner: `watchedValue1 = "80"` (port number), `watchedValue2 = "open"` (state) +- Service monitor: `watchedValue1 = "200"` (HTTP status), `watchedValue2 = "0.45"` (response time) ### Foreign Key @@ -110,14 +150,14 @@ https://google.com|null|2023-01-02 15:56:30|200|0.7898||null|null Missing pipe ``` -❌ **Missing mandatory Watched_Value1** (column 3): +❌ **Missing mandatory watchedValue1** (column 3): ```csv https://duckduckgo.com|192.168.1.1|2023-01-02 15:56:30|null|0.9898|null|null|Best|null ↑ Must not be null ``` -❌ **Incomplete optional columns** (has HelpVal1 but missing HelpVal2–4): +❌ **Incomplete optional columns** (has helpVal1 but missing helpVal2–4): ```csv device|null|2023-01-02 15:56:30|status|null|null|null|null|null|helper1 ↑ @@ -134,50 +174,11 @@ device|null|2023-01-02 15:56:30|status|null|null|null|null|null|h1|h2|h3|h4 device|null|2023-01-02 15:56:30|status|null|null|null|null|null ``` -## Using `plugin_helper.py` - -The easiest way to ensure correct output is to use the [`plugin_helper.py`](../front/plugins/plugin_helper.py) library: - -```python -from plugin_helper import Plugin_Objects - -# Initialize with your plugin's prefix -plugin_objects = Plugin_Objects("YOURPREFIX") - -# Add objects -plugin_objects.add_object( - Object_PrimaryID="device_id", - Object_SecondaryID="192.168.1.1", - DateTime="2023-01-02 15:56:30", - Watched_Value1="online", - Watched_Value2=None, - Watched_Value3=None, - Watched_Value4=None, - Extra="Additional data", - ForeignKey="aa:bb:cc:dd:ee:ff", - HelpVal1=None, - HelpVal2=None, - HelpVal3=None, - HelpVal4=None -) - -# Write results (handles formatting, sanitization, and file creation) -plugin_objects.write_result_file() -``` - -The library automatically: - -- Validates data types -- Sanitizes string values -- Normalizes MAC addresses -- Writes to the correct file location -- Creates the file in `/tmp/log/plugins/last_result..log` - ## De-duplication The core runs **de-duplication once per hour** on the `Plugins_Objects` table: -- **Duplicate Detection Key:** Combination of `Object_PrimaryID`, `Object_SecondaryID`, `Plugin` (auto-filled from `unique_prefix`), and `UserData` +- **Duplicate Detection Key:** Combination of `objectPrimaryId`, `objectSecondaryId`, `Plugin` (auto-filled from `unique_prefix`), and `UserData` - **Resolution:** Oldest duplicate entries are removed, newest are kept - **Use Case:** Prevents duplicate notifications when the same object is detected multiple times @@ -186,6 +187,7 @@ The core runs **de-duplication once per hour** on the `Plugins_Objects` table: **Required Format:** `YYYY-MM-DD HH:MM:SS` **Examples:** + - `2023-01-02 15:56:30` ✅ - `2023-1-2 15:56:30` ❌ (missing leading zeros) - `2023-01-02T15:56:30` ❌ (wrong separator) @@ -213,9 +215,9 @@ Before writing your plugin's `script.py`, ensure: - [ ] **9 or 13 columns** in each output line (8 or 12 pipe separators) - [ ] **Mandatory columns filled:** - - Column 0: `Object_PrimaryID` (not null) + - Column 0: `objectPrimaryId` (not null) - Column 2: `DateTime` in `YYYY-MM-DD HH:MM:SS` format - - Column 3: `Watched_Value1` (not null) + - Column 3: `watchedValue1` (not null) - [ ] **Null values as literal string** `null` (not empty string or special chars) - [ ] **No extra pipes or misaligned columns** - [ ] **If using optional helpers** (columns 9–12), all 4 must be present diff --git a/docs/PLUGINS_DEV_QUICK_START.md b/docs/PLUGINS_DEV_QUICK_START.md index 933b6886..6bac4ae5 100644 --- a/docs/PLUGINS_DEV_QUICK_START.md +++ b/docs/PLUGINS_DEV_QUICK_START.md @@ -68,13 +68,13 @@ try: # Add an object to results plugin_objects.add_object( - Object_PrimaryID="example_id", - Object_SecondaryID=None, + objectPrimaryId="example_id", + objectSecondaryId=None, DateTime="2023-01-02 15:56:30", - Watched_Value1="value1", - Watched_Value2=None, - Watched_Value3=None, - Watched_Value4=None, + watchedValue1="value1", + watchedValue2=None, + watchedValue3=None, + watchedValue4=None, Extra="additional_data", ForeignKey=None ) diff --git a/docs/PLUGINS_DEV_UI_COMPONENTS.md b/docs/PLUGINS_DEV_UI_COMPONENTS.md index 776bd40e..ef2d75fd 100644 --- a/docs/PLUGINS_DEV_UI_COMPONENTS.md +++ b/docs/PLUGINS_DEV_UI_COMPONENTS.md @@ -7,6 +7,7 @@ Configure how your plugin's data is displayed in the NetAlertX web interface. Plugin results are displayed in the UI via the **Plugins page** and **Device details tabs**. You control the appearance and functionality of these displays by defining `database_column_definitions` in your plugin's `config.json`. Each column definition specifies: + - Which data field to display - How to render it (label, link, color-coded badge, etc.) - What CSS classes to apply @@ -16,7 +17,7 @@ Each column definition specifies: ```json { - "column": "Object_PrimaryID", + "column": "objectPrimaryId", "mapped_to_column": "devMac", "mapped_to_column_data": null, "css_classes": "col-sm-2", @@ -39,7 +40,7 @@ Each column definition specifies: | Property | Type | Required | Description | |----------|------|----------|-------------| -| `column` | string | **YES** | Source column name from data contract (e.g., `Object_PrimaryID`, `Watched_Value1`) | +| `column` | string | **YES** | Source column name from data contract (e.g., `objectPrimaryId`, `watchedValue1`) | | `mapped_to_column` | string | no | Target database column if mapping to a table like `CurrentScan` | | `mapped_to_column_data` | object | no | Static value to map instead of using column data | | `css_classes` | string | no | Bootstrap CSS classes for width/spacing (e.g., `"col-sm-2"`, `"col-sm-6"`) | @@ -64,7 +65,7 @@ Plain text display (read-only). ```json { - "column": "Watched_Value1", + "column": "watchedValue1", "show": true, "type": "label", "localized": ["name"], @@ -99,7 +100,7 @@ Resolves an IP address to a MAC address and creates a device link. ```json { - "column": "Object_SecondaryID", + "column": "objectSecondaryId", "show": true, "type": "device_ip", "localized": ["name"], @@ -117,7 +118,7 @@ Creates a device link with the target device's name as the link label. ```json { - "column": "Object_PrimaryID", + "column": "objectPrimaryId", "show": true, "type": "device_name_mac", "localized": ["name"], @@ -135,7 +136,7 @@ Renders as a clickable HTTP/HTTPS link. ```json { - "column": "Watched_Value1", + "column": "watchedValue1", "show": true, "type": "url", "localized": ["name"], @@ -153,7 +154,7 @@ Creates two links (HTTP and HTTPS) as lock icons for the given IP/hostname. ```json { - "column": "Object_SecondaryID", + "column": "objectSecondaryId", "show": true, "type": "url_http_https", "localized": ["name"], @@ -207,7 +208,7 @@ Color-codes values based on ranges. Useful for status codes, latency, capacity p ```json { - "column": "Watched_Value1", + "column": "watchedValue1", "show": true, "type": "threshold", "options": [ @@ -252,7 +253,7 @@ Replaces specific values with display strings or HTML. ```json { - "column": "Watched_Value2", + "column": "watchedValue2", "show": true, "type": "replace", "options": [ @@ -275,6 +276,7 @@ Replaces specific values with display strings or HTML. ``` **Output Examples:** + - `"online"` → 🟢 Online - `"offline"` → 🔴 Offline - `"idle"` → 🟡 Idle @@ -286,7 +288,7 @@ Applies a regular expression to extract/transform values. ```json { - "column": "Watched_Value1", + "column": "watchedValue1", "show": true, "type": "regex", "options": [ @@ -310,7 +312,7 @@ Evaluates JavaScript code with access to the column value (use `${value}` or `{v ```json { - "column": "Watched_Value1", + "column": "watchedValue1", "show": true, "type": "eval", "default_value": "", @@ -322,7 +324,7 @@ Evaluates JavaScript code with access to the column value (use `${value}` or `{v **Example with custom formatting:** ```json { - "column": "Watched_Value1", + "column": "watchedValue1", "show": true, "type": "eval", "options": [ @@ -347,7 +349,7 @@ You can chain multiple transformations with dot notation: ```json { - "column": "Watched_Value3", + "column": "watchedValue3", "show": true, "type": "regex.url_http_https", "options": [ @@ -376,7 +378,7 @@ Use SQL query results to populate dropdown options: ```json { - "column": "Watched_Value2", + "column": "watchedValue2", "show": true, "type": "select", "options": ["{value}"], @@ -405,7 +407,7 @@ Use plugin settings to populate options: ```json { - "column": "Watched_Value1", + "column": "watchedValue1", "show": true, "type": "select", "options": ["{value}"], @@ -439,7 +441,7 @@ To import plugin data into the device scan pipeline (for notifications, heuristi "mapped_to_table": "CurrentScan", "database_column_definitions": [ { - "column": "Object_PrimaryID", + "column": "objectPrimaryId", "mapped_to_column": "scanMac", "show": true, "type": "device_mac", @@ -447,7 +449,7 @@ To import plugin data into the device scan pipeline (for notifications, heuristi "name": [{"language_code": "en_us", "string": "MAC Address"}] }, { - "column": "Object_SecondaryID", + "column": "objectSecondaryId", "mapped_to_column": "scanLastIP", "show": true, "type": "device_ip", @@ -501,7 +503,7 @@ Control which rows are displayed based on filter conditions. Filters are applied { "data_filters": [ { - "compare_column": "Object_PrimaryID", + "compare_column": "objectPrimaryId", "compare_operator": "==", "compare_field_id": "txtMacFilter", "compare_js_template": "'{value}'.toString()", @@ -545,7 +547,7 @@ When viewing a device detail page, the `txtMacFilter` field is populated with th { "database_column_definitions": [ { - "column": "Object_PrimaryID", + "column": "objectPrimaryId", "mapped_to_column": "scanMac", "css_classes": "col-sm-2", "show": true, @@ -555,7 +557,7 @@ When viewing a device detail page, the `txtMacFilter` field is populated with th "name": [{"language_code": "en_us", "string": "MAC Address"}] }, { - "column": "Object_SecondaryID", + "column": "objectSecondaryId", "mapped_to_column": "scanLastIP", "css_classes": "col-sm-2", "show": true, @@ -574,7 +576,7 @@ When viewing a device detail page, the `txtMacFilter` field is populated with th "name": [{"language_code": "en_us", "string": "Last Seen"}] }, { - "column": "Watched_Value1", + "column": "watchedValue1", "css_classes": "col-sm-2", "show": true, "type": "threshold", @@ -589,7 +591,7 @@ When viewing a device detail page, the `txtMacFilter` field is populated with th "name": [{"language_code": "en_us", "string": "HTTP Status"}] }, { - "column": "Watched_Value2", + "column": "watchedValue2", "css_classes": "col-sm-1", "show": true, "type": "label", diff --git a/docs/REMOTE_NETWORKS.md b/docs/REMOTE_NETWORKS.md index 11696087..cb41e2dc 100755 --- a/docs/REMOTE_NETWORKS.md +++ b/docs/REMOTE_NETWORKS.md @@ -33,19 +33,22 @@ VPNs use virtual interfaces (e.g., `tun0`, `tap0`) to encapsulate traffic, bypas > **Possible workaround**: Configure the VPN to bridge networks instead of routing to enable ARP, though this depends on the VPN setup and security requirements. -# Other Workarounds +## Other Workarounds The following workarounds should work for most complex network setups. -## Supplementing Plugins +### Supplementing Plugins You can use supplementary plugins that employ alternate methods. Protocols used by the `SNMPDSC` or `DHCPLSS` plugins are widely supported on different routers and can be effective as workarounds. Check the [plugins list](./PLUGINS.md) to find a plugin that works with your router and network setup. -## Multiple NetAlertX Instances +### Multiple NetAlertX Instances If you have servers in different networks, you can set up separate NetAlertX instances on those subnets and synchronize the results into one instance using the [`SYNC` plugin](https://github.com/netalertx/NetAlertX/tree/main/front/plugins/sync). -## Manual Entry +> [!TIP] +> The [`SYNC_BEHAVIOR`](https://github.com/netalertx/NetAlertX/tree/main/front/plugins/sync/README.md#hub-device-write-behavior-sync_behavior) setting controls how the hub handles newly discovered devices from nodes - whether it inherits node config, overwrites on every sync, or applies its own NEWDEV defaults. + +### Manual Entry If you don't need to discover new devices and only need to report on their status (`online`, `offline`, `down`), you can manually enter devices and check their status using the [`ICMP` plugin](https://github.com/netalertx/NetAlertX/blob/main/front/plugins/icmp_scan/), which uses the `ping` command internally. @@ -53,7 +56,7 @@ For more information on how to add devices manually (or dummy devices), refer to To create truly dummy devices, you can use a loopback IP address (e.g., `0.0.0.0` or `127.0.0.1`) or the `Force Status` field so they appear online. -## NMAP and Fake MAC Addresses +### NMAP and Fake MAC Addresses Scanning remote networks with NMAP is possible (via the `NMAPDEV` plugin), but since it cannot retrieve the MAC address, you need to enable the `NMAPDEV_FAKE_MAC` setting. This will generate a fake MAC address based on the IP address, allowing you to track devices. However, this can lead to inconsistencies, especially if the IP address changes or a previously logged device is rediscovered. If this setting is disabled, only the IP address will be discovered, and devices with missing MAC addresses will be skipped. diff --git a/docs/SECURITY.md b/docs/SECURITY.md index 35612495..ad3aad0a 100755 --- a/docs/SECURITY.md +++ b/docs/SECURITY.md @@ -8,7 +8,7 @@ This includes (but is not limited to): - Running NetAlertX only on networks where you have legal authorization - Keeping your deployment up to date with the latest patches -> NetAlertX is not responsible for misuse, misconfiguration, or unsecure deployments. Always test and secure your setup before exposing it to the outside world. +> NetAlertX is not responsible for misuse, misconfiguration, or insecure deployments. Always test and secure your setup before exposing it to the outside world. Users interacting with the UI are treated as trusted actors within the deployment model. # 🔐 Securing Your NetAlertX Instance @@ -36,7 +36,7 @@ NetAlertX is designed to be run on **private LANs**, not the open internet. ### ✅ Tailscale (Easy VPN Alternative) -Tailscale sets up a private mesh network between your devices. It's fast to configure and ideal for NetAlertX. +Tailscale sets up a private mesh network between your devices. It's fast to configure and ideal for NetAlertX. 👉 [Get started with Tailscale](https://tailscale.com/) --- @@ -63,19 +63,19 @@ By default, NetAlertX does **not** require login. Before exposing the UI in any ## 🔥 Additional Security Measures -- **Firewall / Network Rules** +- **Firewall / Network Rules** Restrict UI/API access to trusted IPs only. -- **Limit Docker Capabilities** +- **Limit Docker Capabilities** Avoid `--privileged`. Use `--cap-add=NET_RAW` and others **only if required** by your scan method. -- **Keep NetAlertX Updated** +- **Keep NetAlertX Updated** Regular updates contain bug fixes and security patches. -- **Plugin Permissions** +- **Plugin Permissions** Disable unused plugins. Only install from trusted sources. -- **Use Read-Only API Keys** +- **Use Read-Only API Keys** When integrating NetAlertX with other tools, scope keys tightly. --- diff --git a/docs/SUBNETS.md b/docs/SUBNETS.md index ac8d0305..c1bf7ca7 100755 --- a/docs/SUBNETS.md +++ b/docs/SUBNETS.md @@ -8,8 +8,15 @@ You need to specify the network interface and the network mask. You can also con > If you don't see all expected devices run the following command in the NetAlertX container (replace the interface and ip mask): > `sudo arp-scan --interface=eth0 192.168.1.0/24` > -> If this command returns no results, the network is not accessible due to your network or firewall restrictions (Wi-Fi Extenders, VPNs and inaccessible networks). If direct scans are not possible, check the [remote networks documentation](./REMOTE_NETWORKS.md) for workarounds. - +> If this command returns no results: +> +> - ✅ If you see output like `IPv4: (none)` or `Using 0.0.0.0`: +> - The interface was not detected correctly. +> - Fix: explicitly set the interface using `--interface=`. +> +> - ❌ If the scan runs correctly but still finds no devices: +> - The network may not be accessible due to firewall, VLAN, or network restrictions (Wi-Fi extenders, VPNs, etc.). +> - If direct scans are not possible, check the [remote networks documentation](./REMOTE_NETWORKS.md) for workarounds. ## Example Values diff --git a/docs/WORKFLOWS.md b/docs/WORKFLOWS.md index 03b0879d..60d9b4d0 100755 --- a/docs/WORKFLOWS.md +++ b/docs/WORKFLOWS.md @@ -1,10 +1,10 @@ # Workflows Overview -The workflows module in allows to automate repetitive tasks, making network management more efficient. Whether you need to assign newly discovered devices to a specific Network Node, auto-group devices from a given vendor, unarchive a device if detected online, or automatically delete devices, this module provides the flexibility to tailor the automations to your needs. +The workflows module allows to automate repetitive tasks, making network management more efficient. Whether you need to assign newly discovered devices to a specific Network Node, auto-group devices from a given vendor, unarchive a device if detected online, or automatically delete devices, this module provides the flexibility to tailor the automations to your needs. ![Workflows diagram](./img/WORKFLOWS/workflows_diagram.png) -Below are a few examples that demonstrate how this module can be used to simplify network management tasks. +You can find a couple of use case examples in [Workflow Examples](WORKFLOW_EXAMPLES.md). ## Updating Workflows diff --git a/docs/WORKFLOW_EXAMPLES.md b/docs/WORKFLOW_EXAMPLES.md index a6c1b3f0..f94647f8 100755 --- a/docs/WORKFLOW_EXAMPLES.md +++ b/docs/WORKFLOW_EXAMPLES.md @@ -49,19 +49,19 @@ Sometimes devices are manually archived (e.g., no longer expected on the network ### 🔍 Explanation - - Trigger: Listens for updates to device records. - - Conditions: - - `devIsArchived` is `1` (archived). - - `devPresentLastScan` is `1` (device was detected in the latest scan). - - Action: Updates the device to set `devIsArchived` to `0` (unarchived). +* **Trigger**: Listens for updates to device records. +* **Conditions**: + + * `devIsArchived` is `1` (archived). + * `devPresentLastScan` is `1` (device was detected in the latest scan). +* **Action**: + + * Updates the device to set `devIsArchived` to `0` (unarchived). ### ✅ Result Whenever a previously archived device shows up during a network scan, it will be automatically unarchived — allowing it to reappear in your device lists and dashboards. - -Here is your updated version of **Example 2** and **Example 3**, fully aligned with the format and structure of **Example 1** for consistency and professionalism: - --- ## Example 2: Assign Device to Network Node Based on IP @@ -107,7 +107,7 @@ When new devices join your network, assigning them to the correct network node i ### 🔍 Explanation * **Trigger**: Activates when a new device is added. -* **Condition**: +* **Conditions**: * `devLastIP` contains `192.168.1.` (matches subnet). * **Action**: @@ -173,12 +173,12 @@ You may want to automatically clear out newly detected Google devices (such as C * **Trigger**: Runs on device updates. * **Conditions**: - * Vendor contains `Google`. - * Device is marked as new (`devIsNew` is `1`). + * `devVendor` contains `Google`. + * `devIsNew` is `1` (device marked as new). * **Actions**: - 1. Set `devIsNew` to `0` (mark as not new). - 2. Delete the device. + 1. Sets `devIsNew` to `0` (mark as not new). + 2. Deletes the device. ### ✅ Result diff --git a/front/appEventsCore.php b/front/appEventsCore.php index 22c4ca23..47febbbe 100755 --- a/front/appEventsCore.php +++ b/front/appEventsCore.php @@ -61,16 +61,16 @@ $(document).ready(function () { appEvents(options: $options) { count appEvents { - DateTimeCreated - AppEventProcessed - AppEventType - ObjectType - ObjectPrimaryID - ObjectSecondaryID - ObjectStatus - ObjectPlugin - ObjectGUID - GUID + dateTimeCreated + appEventProcessed + appEventType + objectType + objectPrimaryId + objectSecondaryId + objectStatus + objectPlugin + objectGuid + guid } } } @@ -128,16 +128,16 @@ $(document).ready(function () { }, columns: [ - { data: 'DateTimeCreated', title: getString('AppEvents_DateTimeCreated') }, - { data: 'AppEventProcessed', title: getString('AppEvents_AppEventProcessed') }, - { data: 'AppEventType', title: getString('AppEvents_Type') }, - { data: 'ObjectType', title: getString('AppEvents_ObjectType') }, - { data: 'ObjectPrimaryID', title: getString('AppEvents_ObjectPrimaryID') }, - { data: 'ObjectSecondaryID', title: getString('AppEvents_ObjectSecondaryID') }, - { data: 'ObjectStatus', title: getString('AppEvents_ObjectStatus') }, - { data: 'ObjectPlugin', title: getString('AppEvents_Plugin') }, - { data: 'ObjectGUID', title: 'Object GUID' }, - { data: 'GUID', title: 'Event GUID' } + { data: 'dateTimeCreated', title: getString('AppEvents_DateTimeCreated') }, + { data: 'appEventProcessed', title: getString('AppEvents_AppEventProcessed') }, + { data: 'appEventType', title: getString('AppEvents_Type') }, + { data: 'objectType', title: getString('AppEvents_ObjectType') }, + { data: 'objectPrimaryId', title: getString('AppEvents_ObjectPrimaryID') }, + { data: 'objectSecondaryId', title: getString('AppEvents_ObjectSecondaryID') }, + { data: 'objectStatus', title: getString('AppEvents_ObjectStatus') }, + { data: 'objectPlugin', title: getString('AppEvents_Plugin') }, + { data: 'objectGuid', title: 'Object GUID' }, + { data: 'guid', title: 'Event GUID' } ], columnDefs: [ diff --git a/front/css/app.css b/front/css/app.css index 302f2f94..0e6eba8b 100755 --- a/front/css/app.css +++ b/front/css/app.css @@ -34,15 +34,15 @@ h5 a[target="_blank"] { position: relative; display: inline-block; /* Needed for positioning */ - padding-right: 0.6em; /* Space for the icon */ + padding-right: 0.55em; /* Space for the icon */ } a[target="_blank"]::after { content: '↗'; position: absolute; top: 0; - right: 0; - font-size: 0.75em; + right: 0.3em; + font-size: 0.6em; line-height: 1; } @@ -1413,15 +1413,49 @@ textarea[readonly], #columnFilters { - display: flex; - flex-wrap: wrap; - gap: 10px; /* Add spacing between items */ + display: grid; + grid-template-columns: repeat(auto-fit, minmax(125px, 1fr)); + gap: 0.75em; + padding: 0.5em 0; + padding-top: 0; +} + +#columnFilters::before, +#columnFilters::after { + display: none !important; } .filter-group { - box-sizing: border-box; /* Ensure padding and borders are included in the width */ - padding: 1em; - padding-top: 0; + box-sizing: border-box; + padding: 0.4em; + margin: 0; + border-radius: 3px; + display: flex; + flex-direction: column; + align-items: stretch; + gap: 0.15em; + white-space: normal; +} + +.filter-group label { + box-sizing: border-box; + margin: 0; + border-radius: 3px; + display: flex; + flex-direction: column; + align-items: stretch; + gap: 0.15em; + white-space: normal; + padding-left: 15px; + padding-right: 0px; +} + +.filter-dropdown { + width: 100%; +} +.filter-group select { + margin-left: 15px; + padding-right: 0px; } .filter-dropdown @@ -1512,6 +1546,11 @@ textarea[readonly], height: 1.5em; } +#nextScanEta +{ + padding-left: 1.5em; +} + .info-icon-nav { top: -6px; diff --git a/front/deviceDetails.php b/front/deviceDetails.php index 6c0a1180..05067264 100755 --- a/front/deviceDetails.php +++ b/front/deviceDetails.php @@ -307,7 +307,10 @@ function updateChevrons(currentMac) { pos = refreshedList.findIndex(item => item.devMac === currentMac); if (pos === -1) { - console.error('Still not found after re-cache:', currentMac); + console.warn('Device not found in device list after re-cache — hiding navigation controls:', currentMac); + $('#txtRecord').hide(); + $('#btnPrevious').hide(); + $('#btnNext').hide(); return; } @@ -499,26 +502,9 @@ async function renderSmallBoxes() { } } -function updateDevicePageName(mac) { - let name = getDevDataByMac(mac, "devName"); - let owner = getDevDataByMac(mac, "devOwner"); - - // If data is missing, re-cache and retry once - if (mac != 'new' && (name === null|| owner === null)) { - console.warn("Device not found in cache, retrying after re-cache:", mac); - showSpinner(); - cacheDevices(true).then(() => { - hideSpinner(); - // Retry after successful cache - updateDevicePageName(mac); - }).catch((err) => { - hideSpinner(); - console.error("Failed to refresh devices:", err); - }); - return; // Exit early to avoid showing bad data - } - - // Page title - Name +// ---------------------------------------- +// Write device name/owner into page title DOM. Pure DOM side-effect, no data fetching. +function applyDevicePageTitle(mac, name, owner) { let pageTitleText; if (mac === "new") { @@ -530,12 +516,12 @@ function updateDevicePageName(mac) { ` ` + getString("Gen_create_new_device_info") ); $('#devicePageInfoPlc').show(); - } else if (!owner || name.toString().includes(owner)) { - pageTitleText = name; + } else if (!owner || (name && name.toString().includes(owner))) { + pageTitleText = encodeSpecialChars(name ?? getString("DevDetail_EveandAl_NewDevice")); $('#pageTitle').html(pageTitleText); $('#devicePageInfoPlc').hide(); } else { - pageTitleText = `${name} (${owner})`; + pageTitleText = `${encodeSpecialChars(name ?? getString("DevDetail_EveandAl_NewDevice"))} (${encodeSpecialChars(owner)})`; $('#pageTitle').html(pageTitleText); $('#devicePageInfoPlc').hide(); } @@ -544,6 +530,53 @@ function updateDevicePageName(mac) { $('title').html(pageTitleText + ' - ' + $('title').html()); } +// ---------------------------------------- +// Resolve device name/owner for the page title. +// Stage 1: localStorage cache (synchronous, fast path). +// Stage 2: one forced re-cache from table_devices.json. +// Stage 3: REST API fallback so a direct-link visit never loops. +async function updateDevicePageName(mac) { + let name = getDevDataByMac(mac, "devName"); + let owner = getDevDataByMac(mac, "devOwner"); + + // Stage 2: one re-cache attempt + if (mac !== 'new' && name === null) { + console.warn("Device not in cache, attempting re-cache:", mac); + showSpinner(); + try { + await cacheDevices(true); + } catch (err) { + console.error("Re-cache failed:", err); + } finally { + hideSpinner(); + } + name = getDevDataByMac(mac, "devName"); + owner = getDevDataByMac(mac, "devOwner"); + } + + // Stage 3: REST fallback — same endpoint renderSmallBoxes uses, always DB-direct + if (mac !== 'new' && name === null) { + console.warn("Device not found in cache after re-cache, falling back to REST API:", mac); + try { + const { apiBase, authHeader } = getAuthContext(); + const res = await fetch(`${apiBase}/device/${encodeURIComponent(mac)}`, { + headers: authHeader + }); + if (res.ok) { + const data = await res.json(); + name = data.devName ?? null; + owner = data.devOwner ?? null; + } else { + console.error("REST fallback for device name returned:", res.status); + } + } catch (err) { + console.error("REST fallback error:", err); + } + } + + applyDevicePageTitle(mac, name, owner); +} + //----------------------------------------------------------------------------------- diff --git a/front/deviceDetailsEdit.php b/front/deviceDetailsEdit.php index 542f2a59..6c70ac4b 100755 --- a/front/deviceDetailsEdit.php +++ b/front/deviceDetailsEdit.php @@ -236,6 +236,7 @@ function getDeviceData() { // console.log(setting.setKey); // console.log(fieldData); + // Additional form elements like the random MAC address button for devMac let inlineControl = ""; // handle random mac @@ -329,6 +330,11 @@ function getDeviceData() { fieldOptionsOverride = fieldDataNew; } + // XSS prevention - encode special characters for string fields, but not for arrays (like children dynamic) + // Don't move above the handle devChildrenDynamic block because it relies on the original fieldData to generate options + fieldData = encodeSpecialChars(fieldData); + // console.log(fieldData); + // Generate the input field HTML const inputFormHtml = `