mirror of
https://github.com/jokob-sk/NetAlertX.git
synced 2026-07-25 15:08:10 -04:00
feat: Implement DevicesHistory feature with triggers and history tracking
- Added db_history.py to manage DevicesHistory table and triggers for INSERT and UPDATE operations. - Created device_history_instance.py for querying and grouping DevicesHistory records. - Developed change_history.php for displaying device change history with filtering and pagination. - Introduced skel_device_details_tab_history.php for skeleton loading state in device details tab. - Added unit tests in test_device_history.py to validate trigger functionality and history management. - Implemented filter population and pagination in the change history UI.
This commit is contained in:
Jokob @NetAlertX
@@ -21,8 +21,9 @@ Ensure all work begins with **documentation-first understanding**, **PRD validat
|
||||
1. `/CONTRIBUTING.md`
|
||||
2. `/README.md`
|
||||
3. `/.github/skills/code-standards/SKILL.md`
|
||||
4. `/docs/**`
|
||||
5. Related module/code context if referenced
|
||||
4. `/.github/skills/database-patterns/SKILL.md` — read when the feature touches the `Devices` table or requires audit/history logging
|
||||
5. `/docs/**`
|
||||
6. Related module/code context if referenced
|
||||
|
||||
* Extract:
|
||||
|
||||
@@ -69,6 +70,13 @@ If anything is unclear:
|
||||
* Breaking assumptions
|
||||
* Plugin or API mismatches
|
||||
|
||||
* **If the feature writes to the `Devices` table:**
|
||||
|
||||
* Load `/.github/skills/database-patterns/SKILL.md`
|
||||
* Audit all write paths listed there — confirm which are affected
|
||||
* Determine if a SQLite trigger or Python hook is the right approach
|
||||
* Check if `*Source` fields already solve attribution requirements
|
||||
|
||||
* Clearly report inconsistencies before proceeding
|
||||
|
||||
---
|
||||
|
||||
@@ -1,16 +1,86 @@
|
||||
# Plugin development skill
|
||||
---
|
||||
name: netalertx-plugin-development
|
||||
description: Create and run NetAlertX plugins. Use this when asked to create plugin, run plugin, test plugin, or develop plugin functionality.
|
||||
---
|
||||
|
||||
1. Assess requirements
|
||||
2. Read `docs/PLUGINS_DEV.md`
|
||||
3. Confirm field mapping
|
||||
4. Ask clarifying questions
|
||||
5. code placed in `front/plugins/<new plugin name>`
|
||||
|
||||
|
||||
Plugin prefix:
|
||||
|
||||
- has to be uppercase letters only (no underscores)
|
||||
- must be unique
|
||||
- keep short but readable if possible
|
||||
# Plugin Development
|
||||
|
||||
## Expected Workflow
|
||||
|
||||
1. Read this skill and `docs/PLUGINS_DEV.md` for full context.
|
||||
2. Find or create the plugin in `front/plugins/<code_name>/`.
|
||||
3. Read the plugin's `config.json` and `script.py` to understand its functionality.
|
||||
4. Run: `python3 front/plugins/<code_name>/script.py`
|
||||
5. Retrieve the result from `/tmp/log/plugins/last_result.<PREF>.log` quickly — the backend deletes it after processing.
|
||||
|
||||
## Run a Plugin Manually
|
||||
|
||||
```bash
|
||||
python3 front/plugins/<code_name>/script.py
|
||||
```
|
||||
|
||||
Ensure `sys.path` includes `/app/front/plugins` and `/app/server` (as in the template).
|
||||
|
||||
## Plugin Structure
|
||||
|
||||
```text
|
||||
front/plugins/<code_name>/
|
||||
├── config.json # Manifest with settings
|
||||
├── script.py # Main script
|
||||
└── ...
|
||||
```
|
||||
|
||||
## Plugin Prefix Rules
|
||||
|
||||
- Uppercase letters only (no underscores, no numbers)
|
||||
- Must be unique across all plugins
|
||||
- Short but readable (e.g., `ARPSCAN`, `MQTT`, `UNIFI`)
|
||||
|
||||
## Settings Pattern
|
||||
|
||||
- `<PREF>_RUN`: execution phase
|
||||
- `<PREF>_RUN_SCHD`: cron-like schedule
|
||||
- `<PREF>_CMD`: script path
|
||||
- `<PREF>_RUN_TIMEOUT`: timeout in seconds
|
||||
- `<PREF>_WATCH`: columns to watch for changes
|
||||
|
||||
## Data Contract
|
||||
|
||||
Scripts write to `/tmp/log/plugins/last_result.<PREF>.log` using `plugin_helper.py`:
|
||||
|
||||
```python
|
||||
from plugin_helper import Plugin_Objects
|
||||
|
||||
plugin_objects = Plugin_Objects()
|
||||
plugin_objects.add_object(...) # During processing
|
||||
plugin_objects.write_result_file() # Exactly once at end
|
||||
```
|
||||
|
||||
**Important:** The backend processes and deletes the result file almost immediately. Retrieve it quickly if inspecting output.
|
||||
|
||||
## Execution Phases
|
||||
|
||||
| Phase | Trigger |
|
||||
|-------|---------|
|
||||
| `once` | Once at startup |
|
||||
| `schedule` | On cron schedule |
|
||||
| `always_after_scan` | After every scan |
|
||||
| `before_name_updates` | Before name resolution |
|
||||
| `on_new_device` | When new device detected |
|
||||
| `on_notification` | When notification triggered |
|
||||
|
||||
## Plugin Formats
|
||||
|
||||
| Format | Purpose | Phase |
|
||||
|--------|---------|-------|
|
||||
| publisher | Send notifications | `on_notification` |
|
||||
| dev scanner | Create/manage devices | `schedule` |
|
||||
| name discovery | Discover device names | `before_name_updates` |
|
||||
| importer | Import from services | `schedule` |
|
||||
| system | Core functionality | `schedule` |
|
||||
|
||||
## Starting Point
|
||||
|
||||
Copy from `front/plugins/__template` and customize. Read `docs/PLUGINS_DEV.md` for the full development guide.
|
||||
|
||||
|
||||
|
||||
@@ -3,13 +3,56 @@ name: project-navigation
|
||||
description: Reference for the NetAlertX codebase structure, key file paths, and configuration locations. Use this when exploring the codebase or looking for specific components like the backend entry point, frontend files, or database location.
|
||||
---
|
||||
|
||||
# Project Navigation & Structure
|
||||
# Project Navigation
|
||||
|
||||
## Codebase Structure & Key Paths
|
||||
## Key Paths
|
||||
|
||||
- **Source Code:** `/workspaces/NetAlertX` (mapped to `/app` in container via symlink).
|
||||
- **Backend Entry:** `server/api_server/api_server_start.py` (Flask) and `server/__main__.py`.
|
||||
- **Frontend:** `front/` (PHP/JS).
|
||||
- **Plugins:** `front/plugins/`.
|
||||
- **Config:** `/data/config/app.conf` (runtime) or `back/app.conf` (default).
|
||||
- **Database:** `/data/db/app.db` (SQLite).
|
||||
| Component | Path |
|
||||
|-----------|------|
|
||||
| Workspace root | `/workspaces/NetAlertX` |
|
||||
| Backend entry | `server/__main__.py` |
|
||||
| API server | `server/api_server/api_server_start.py` |
|
||||
| Plugin system | `server/plugin.py` |
|
||||
| Initialization | `server/initialise.py` |
|
||||
| Frontend | `front/` |
|
||||
| Frontend JS | `front/js/common.js` |
|
||||
| Frontend PHP | `front/php/server/*.php` |
|
||||
| Plugins | `front/plugins/` |
|
||||
| Plugin template | `front/plugins/__template` |
|
||||
| Database helpers | `server/db/db_helper.py` |
|
||||
| Device model | `server/models/device_instance.py` |
|
||||
| Messaging | `server/messaging/` |
|
||||
| Workflows | `server/workflows/` |
|
||||
|
||||
## Architecture
|
||||
|
||||
NetAlertX uses a frontend–backend architecture: the frontend runs on **PHP + Nginx** (see `front/`), the backend is implemented in **Python** (see `server/`), and scheduled tasks are managed by a **supercronic** scheduler.
|
||||
|
||||
## Runtime Paths
|
||||
|
||||
| Data | Path |
|
||||
|------|------|
|
||||
| Config (runtime) | `/data/config/app.conf` |
|
||||
| Config (default) | `back/app.conf` |
|
||||
| Database | `/data/db/app.db` |
|
||||
| API JSON cache | `/tmp/api/*.json` |
|
||||
| Logs | `/tmp/log/` |
|
||||
| Plugin logs | `/tmp/log/plugins/` |
|
||||
|
||||
## Environment Variables
|
||||
|
||||
Use these instead of hardcoding paths:
|
||||
- `NETALERTX_DB`
|
||||
- `NETALERTX_LOG`
|
||||
- `NETALERTX_CONFIG`
|
||||
- `NETALERTX_DATA`
|
||||
- `NETALERTX_APP`
|
||||
|
||||
## Documentation
|
||||
|
||||
| Topic | Path |
|
||||
|-------|------|
|
||||
| Plugin development | `docs/PLUGINS_DEV.md` |
|
||||
| System settings | `docs/SETTINGS_SYSTEM.md` |
|
||||
| API docs | `docs/API_*.md` |
|
||||
| Debug guides | `docs/DEBUG_*.md` |
|
||||
|
||||
412
.gemini/skills/settings/SKILL.md
Normal file
412
.gemini/skills/settings/SKILL.md
Normal file
@@ -0,0 +1,412 @@
|
||||
---
|
||||
name: nax-settings-development
|
||||
description: Best practices for creating and maintaining NetAlertX settings (config.json), including UI, validation, localization, runtime behavior, and implementation consistency.
|
||||
---
|
||||
|
||||
# NetAlertX Settings Development
|
||||
|
||||
This skill defines the conventions and best practices for adding or modifying settings in NetAlertX.
|
||||
|
||||
The goal is for every setting to be:
|
||||
|
||||
- Declarative
|
||||
- Self-documenting
|
||||
- Automatically rendered by the Settings UI
|
||||
- Fully localized
|
||||
- Backwards compatible
|
||||
- Easy to maintain
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
**Read a setting (backend):**
|
||||
```python
|
||||
from helper import get_setting_value
|
||||
value = get_setting_value('SETTING_NAME')
|
||||
```
|
||||
Never read `app.conf` directly. Always use `get_setting_value()`.
|
||||
|
||||
**Read a setting (frontend):**
|
||||
```javascript
|
||||
getSetting("SETTING_NAME")
|
||||
```
|
||||
|
||||
**Add a core setting** — use `ccd()` in `server/initialise.py`:
|
||||
```python
|
||||
ccd('SETTING_NAME', 'default_value', 'description')
|
||||
```
|
||||
|
||||
**Add a plugin setting** — define in the plugin's `config.json` under the `settings` key.
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `/data/config/app.conf` | Runtime config (source of truth) |
|
||||
| `back/app.conf` | Default config (template) |
|
||||
|
||||
Use `APP_CONF_OVERRIDE` for settings that must be set before startup.
|
||||
|
||||
---
|
||||
|
||||
# 1. Follow existing patterns
|
||||
|
||||
Before creating a new setting:
|
||||
|
||||
- Search for similar settings.
|
||||
- Reuse existing naming conventions.
|
||||
- Reuse existing setting types.
|
||||
- Reuse existing validation patterns.
|
||||
- Follow the style already used by similar plugins.
|
||||
|
||||
Avoid introducing new setting types unless absolutely necessary.
|
||||
|
||||
---
|
||||
|
||||
# 2. Setting naming
|
||||
|
||||
Setting keys use uppercase snake case.
|
||||
|
||||
Good:
|
||||
|
||||
```text
|
||||
UI_DEV_SECTIONS
|
||||
SCAN_INTERVAL
|
||||
MQTT_HOST
|
||||
```
|
||||
|
||||
Avoid:
|
||||
|
||||
```text
|
||||
uiDevSections
|
||||
scanInterval
|
||||
ScanInterval
|
||||
```
|
||||
|
||||
Names should clearly describe the purpose.
|
||||
|
||||
---
|
||||
|
||||
# 3. Categories
|
||||
|
||||
Place settings into the most appropriate category.
|
||||
|
||||
Keep related settings together.
|
||||
|
||||
Avoid creating new categories unless there is a clear need.
|
||||
|
||||
---
|
||||
|
||||
# 4. Choose the correct type
|
||||
|
||||
Use the simplest type that correctly models the value.
|
||||
|
||||
Common types include:
|
||||
|
||||
- string
|
||||
- integer
|
||||
- float
|
||||
- boolean
|
||||
- password
|
||||
- select
|
||||
- array
|
||||
- textarea
|
||||
|
||||
Avoid encoding structured JSON inside string settings.
|
||||
|
||||
---
|
||||
|
||||
# 5. Validation
|
||||
|
||||
Always define validation whenever appropriate.
|
||||
|
||||
Examples include:
|
||||
|
||||
- minimum values
|
||||
- maximum values
|
||||
- regex validation
|
||||
- allowed options
|
||||
|
||||
Reject invalid configuration rather than silently accepting it.
|
||||
|
||||
---
|
||||
|
||||
# 6. Defaults
|
||||
|
||||
Provide sensible defaults that work for a fresh installation.
|
||||
|
||||
Avoid defaults that require external services or additional configuration.
|
||||
|
||||
A default value should exist in exactly one place.
|
||||
|
||||
Use the `default_value` defined in `config.json`.
|
||||
|
||||
Do not duplicate the same default value elsewhere in Python or JavaScript.
|
||||
|
||||
---
|
||||
|
||||
# 7. Names and descriptions
|
||||
|
||||
Setting names should be concise.
|
||||
|
||||
Examples:
|
||||
|
||||
- Scan interval
|
||||
- MQTT Host
|
||||
- Hide device sections
|
||||
|
||||
Descriptions should explain:
|
||||
|
||||
- what the setting does
|
||||
- when it should be used
|
||||
- important side effects
|
||||
|
||||
Avoid implementation details.
|
||||
|
||||
Good:
|
||||
|
||||
> Interval between network scans in seconds.
|
||||
|
||||
Bad:
|
||||
|
||||
> Calls scheduler.py every X seconds.
|
||||
|
||||
---
|
||||
|
||||
# 8. Boolean settings
|
||||
|
||||
Boolean settings should read naturally.
|
||||
|
||||
Prefer verbs such as:
|
||||
|
||||
- Enable
|
||||
- Disable
|
||||
- Require
|
||||
- Allow
|
||||
- Ignore
|
||||
- Show
|
||||
- Hide
|
||||
|
||||
Examples:
|
||||
|
||||
- Enable MQTT
|
||||
- Require NICs Online
|
||||
- Hide Offline Devices
|
||||
|
||||
---
|
||||
|
||||
# 9. Select settings
|
||||
|
||||
When users must choose from predefined values, use a select setting.
|
||||
|
||||
Never require users to remember internal values.
|
||||
|
||||
Each option should have a meaningful label.
|
||||
|
||||
---
|
||||
|
||||
# 10. Array settings
|
||||
|
||||
Use arrays only when multiple independent values are expected.
|
||||
|
||||
Examples:
|
||||
|
||||
- ignored MAC addresses
|
||||
- subnet lists
|
||||
- plugin lists
|
||||
|
||||
---
|
||||
|
||||
# 11. Localization
|
||||
|
||||
Every setting must have localized language strings.
|
||||
|
||||
If the language strings are **not** defined directly inside `config.json`, they must exist in `en_us.json`.
|
||||
|
||||
For example, for:
|
||||
|
||||
```text
|
||||
UI_DEV_SECTIONS
|
||||
```
|
||||
|
||||
add:
|
||||
|
||||
```json
|
||||
"UI_DEV_SECTIONS_name": "Hide device sections",
|
||||
"UI_DEV_SECTIONS_description": "Select which UI elements to hide on the Devices page."
|
||||
```
|
||||
|
||||
When using external language files, `config.json` must reference:
|
||||
|
||||
```json
|
||||
"name": [
|
||||
{
|
||||
"string": "_GLOBAL_LANG_FILES_"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
This tells NetAlertX to resolve the display name from the language files.
|
||||
|
||||
---
|
||||
|
||||
# 12. Source of truth
|
||||
|
||||
`app.conf` is the source of truth.
|
||||
|
||||
Keep the following in mind:
|
||||
|
||||
- User-defined values always come from `app.conf`.
|
||||
- Default values from `config.json` are only used when the setting is missing from `app.conf`.
|
||||
- Never assume settings are permanently stored in the database.
|
||||
|
||||
---
|
||||
|
||||
# 13. Plugin-first settings
|
||||
|
||||
Whenever possible, define new settings inside a plugin's `config.json`.
|
||||
|
||||
Avoid adding hardcoded application settings unless there is a compelling architectural reason.
|
||||
|
||||
Plugin settings are the preferred and future-proof approach.
|
||||
|
||||
---
|
||||
|
||||
# 14. Runtime lifecycle
|
||||
|
||||
Understand how settings flow through the application.
|
||||
|
||||
```text
|
||||
config.json
|
||||
│
|
||||
│ default values
|
||||
▼
|
||||
app.conf (source of truth)
|
||||
│
|
||||
▼
|
||||
Settings database table
|
||||
│
|
||||
▼
|
||||
table_settings.json API
|
||||
│
|
||||
▼
|
||||
Frontend (getSetting())
|
||||
```
|
||||
|
||||
The database and API are runtime representations only.
|
||||
|
||||
They are regenerated from `app.conf` during initialization.
|
||||
|
||||
---
|
||||
|
||||
# 15. Access
|
||||
|
||||
Code should never read `app.conf` directly.
|
||||
|
||||
In the Frontend, always retrieve settings through:
|
||||
|
||||
```javascript
|
||||
getSetting("SETTING_NAME")
|
||||
```
|
||||
|
||||
And in the backend, always retrieve settings through:
|
||||
|
||||
```python
|
||||
get_setting_value("SETTING_NAME")
|
||||
```
|
||||
|
||||
This guarantees the value comes from the generated settings API.
|
||||
|
||||
---
|
||||
|
||||
# 16. Metadata
|
||||
|
||||
Each setting automatically has a corresponding `__metadata` entry generated in `app.conf`.
|
||||
|
||||
Do not manually create or modify metadata entries unless working on the settings framework itself.
|
||||
|
||||
---
|
||||
|
||||
# 17. Backend
|
||||
|
||||
Backend code should:
|
||||
|
||||
- use the existing configuration helpers
|
||||
- avoid duplicated parsing logic
|
||||
- gracefully handle missing values
|
||||
- avoid hardcoded defaults
|
||||
|
||||
---
|
||||
|
||||
# 18. Frontend
|
||||
|
||||
A correctly defined setting should render automatically in the existing Settings UI.
|
||||
|
||||
Avoid writing custom JavaScript unless absolutely necessary.
|
||||
|
||||
---
|
||||
|
||||
# 19. Prefer configuration over code
|
||||
|
||||
If a behaviour can reasonably be controlled through an existing setting type (boolean, select, integer, array, etc.), introduce a setting instead of hardcoding special-case logic.
|
||||
|
||||
Configuration is preferred over implementation-specific behaviour whenever practical.
|
||||
|
||||
---
|
||||
|
||||
# 20. Backwards compatibility
|
||||
|
||||
Avoid:
|
||||
|
||||
- renaming settings
|
||||
- changing data types
|
||||
- changing semantics
|
||||
|
||||
If unavoidable:
|
||||
|
||||
- provide migration logic
|
||||
- preserve compatibility where possible
|
||||
|
||||
---
|
||||
|
||||
# 21. Documentation
|
||||
|
||||
Every user-facing setting should eventually be documented.
|
||||
|
||||
Include:
|
||||
|
||||
- purpose
|
||||
- accepted values
|
||||
- default value
|
||||
- examples when useful
|
||||
|
||||
---
|
||||
|
||||
# 22. Pull request checklist
|
||||
|
||||
Before submitting a PR, verify:
|
||||
|
||||
- [ ] Setting follows existing conventions.
|
||||
- [ ] Correct category used.
|
||||
- [ ] Appropriate type selected.
|
||||
- [ ] Validation defined.
|
||||
- [ ] Sensible default provided.
|
||||
- [ ] Default defined only once.
|
||||
- [ ] Name is concise.
|
||||
- [ ] Description is clear.
|
||||
- [ ] Localization strings added.
|
||||
- [ ] `_GLOBAL_LANG_FILES_` used where appropriate.
|
||||
- [ ] Plugin-first approach followed where applicable.
|
||||
- [ ] Frontend renders automatically.
|
||||
- [ ] Backend uses existing configuration helpers.
|
||||
- [ ] No unnecessary frontend code added.
|
||||
- [ ] No duplicated parsing logic.
|
||||
- [ ] Backwards compatibility maintained.
|
||||
- [ ] Documentation updated if required.
|
||||
|
||||
---
|
||||
|
||||
# References
|
||||
|
||||
- https://docs.netalertx.com/PLUGINS_DEV_SETTINGS/
|
||||
- https://docs.netalertx.com/SETTINGS_SYSTEM/
|
||||
- https://docs.netalertx.com/PLUGINS/
|
||||
64
.gemini/skills/skills-index/SKILL.md
Normal file
64
.gemini/skills/skills-index/SKILL.md
Normal file
@@ -0,0 +1,64 @@
|
||||
---
|
||||
name: skills-index
|
||||
description: Index of all available skills across both Gemini CLI (.gemini/skills/) and GitHub Copilot (.github/skills/). Load this to find the right skill for a task, or to locate the counterpart skill in the other AI system.
|
||||
---
|
||||
|
||||
# Skills Index — Cross-Reference
|
||||
|
||||
Two AI assistants are configured for this project, each with their own skill directory:
|
||||
|
||||
- **Gemini CLI** → `.gemini/skills/`
|
||||
- **GitHub Copilot** → `.github/skills/`
|
||||
|
||||
Skills with the same purpose exist in both, sometimes under different names and with different depth. This index maps them so you can find the richer version when needed.
|
||||
|
||||
---
|
||||
|
||||
## Shared Skills (exist in both)
|
||||
|
||||
| Topic | Gemini Skill | Copilot Skill | Notes |
|
||||
|-------|-------------|--------------|-------|
|
||||
| Testing | `testing-workflow` | `testing-workflow` | Gemini version emphasises full suite preference and container detection; Copilot version covers `testFailure` tool and PYTHONPATH |
|
||||
| Settings & config | `settings` | `settings-management` | Gemini version is more comprehensive (22-point guide + PR checklist); Copilot version covers `ccd()` and `get_setting_value()` usage |
|
||||
| MCP activation | `mcp-activation` | `mcp-activation` | Gemini version covers Gemini CLI session restart; Copilot version covers VS Code window reload |
|
||||
| Project navigation | `project-navigation` | `project-navigation` | Copilot version has full path tables and env vars; Gemini version is a brief reference |
|
||||
| Plugin dev | `plugin-development` | `plugin-run-development` | Copilot version is comprehensive (data contract, phases, formats); Gemini version is a brief checklist pointing to `docs/PLUGINS_DEV.md` |
|
||||
| Devcontainer | `devcontainer-management` | `devcontainer-services` + `devcontainer-setup` + `devcontainer-configs` | Gemini combines into one (uses `docker exec`); Copilot splits into 3 focused skills |
|
||||
|
||||
---
|
||||
|
||||
## Copilot-Only Skills
|
||||
|
||||
No Gemini equivalent yet:
|
||||
|
||||
| Copilot Skill | Purpose |
|
||||
|--------------|---------|
|
||||
| `api-development` | Creating REST API endpoints |
|
||||
| `authentication` | API tokens and 401/403 debugging |
|
||||
| `code-standards` | Coding conventions and style rules |
|
||||
| `database-patterns` | Device table write paths, SQLite triggers, audit logging, `*Source` attribution |
|
||||
| `database-reset` | Wipe and regenerate the database and config |
|
||||
| `docker-build` | Build Docker images for testing or production |
|
||||
| `docker-prune` | Clean unused Docker resources (destructive — requires confirmation) |
|
||||
| `sample-data` | Load synthetic device data into the devcontainer |
|
||||
|
||||
---
|
||||
|
||||
## Gemini-Only Skills
|
||||
|
||||
No Copilot equivalent yet:
|
||||
|
||||
| Gemini Skill | Purpose |
|
||||
|-------------|---------|
|
||||
| `initiative-start` | Research methodology and structured approach for new tasks |
|
||||
|
||||
---
|
||||
|
||||
## Adding a New Skill
|
||||
|
||||
When adding a skill, create it in **both** directories to keep both AI systems current:
|
||||
|
||||
- `.gemini/skills/<name>/SKILL.md` — auto-discovered by Gemini CLI via YAML frontmatter
|
||||
- `.github/skills/<name>/SKILL.md` — add an entry to the skills table in `.github/copilot-instructions.md`
|
||||
|
||||
Keep the body content identical between both files. Only the frontmatter `name`/`description` may differ slightly to match each system's discovery heuristics.
|
||||
@@ -75,4 +75,34 @@ The retrieved token MUST be used in all subsequent API or test calls requiring a
|
||||
If tests fail with 403 Forbidden or empty tokens:
|
||||
1. Verify server is running and use the setup script (`/workspaces/NetAlertX/.devcontainer/scripts/setup.sh`) if required.
|
||||
2. Verify `app.conf` inside the container: `cat /data/config/app.conf`
|
||||
|
||||
## Check for Pre-Existing Failures
|
||||
|
||||
Before attributing failures to your changes, check what was already broken:
|
||||
|
||||
```bash
|
||||
cd /workspaces/NetAlertX; pytest test/ --tb=no -q 2>&1 | tail -20
|
||||
```
|
||||
|
||||
Do not fix pre-existing failures unless that is the explicit goal.
|
||||
|
||||
## PYTHONPATH
|
||||
|
||||
The test environment is pre-configured with:
|
||||
- `/app` — primary location where Python runs in production
|
||||
- `/app/server` — symlink to `/workspaces/NetAlertX/server`
|
||||
- `/app/front/plugins` — symlink to `/workspaces/NetAlertX/front/plugins`
|
||||
- `/workspaces/NetAlertX/test`
|
||||
- `/workspaces/NetAlertX/server`
|
||||
- `/workspaces/NetAlertX`
|
||||
|
||||
## Docker Test Image
|
||||
|
||||
If the Dockerfile or dependencies changed, rebuild the test image before running:
|
||||
|
||||
```bash
|
||||
docker buildx build -t netalertx-test .
|
||||
```
|
||||
|
||||
Takes ~30 seconds; ~90 seconds if the venv stage changed.
|
||||
3. Verify Python can read it: `python3 -c "from helper import get_setting_value; print(get_setting_value('API_TOKEN'))"`
|
||||
2
.github/copilot-instructions.md
vendored
2
.github/copilot-instructions.md
vendored
@@ -27,6 +27,7 @@ Procedural knowledge lives in `.github/skills/`. Load the appropriate skill when
|
||||
|
||||
| Task | Skill |
|
||||
|------|-------|
|
||||
| Find or cross-reference any skill | `skills-overview` |
|
||||
| Run tests, check failures | `testing-workflow` |
|
||||
| Start/stop/restart services | `devcontainer-services` |
|
||||
| Wipe database, fresh start | `database-reset` |
|
||||
@@ -42,6 +43,7 @@ Procedural knowledge lives in `.github/skills/`. Load the appropriate skill when
|
||||
| Settings and config | `settings-management` |
|
||||
| Find files and paths | `project-navigation` |
|
||||
| Coding standards | `code-standards` |
|
||||
| Devices table write paths, SQLite triggers, audit logging, `*Source` attribution | `database-patterns` |
|
||||
|
||||
## Execution Protocol
|
||||
|
||||
|
||||
142
.github/skills/database-patterns/SKILL.md
vendored
Normal file
142
.github/skills/database-patterns/SKILL.md
vendored
Normal file
@@ -0,0 +1,142 @@
|
||||
---
|
||||
name: netalertx-database-patterns
|
||||
description: NetAlertX database architecture patterns. Use this when designing features that write to the Devices table, implementing audit/history logging, or choosing between trigger-based vs Python-hook approaches.
|
||||
---
|
||||
|
||||
# Database Patterns
|
||||
|
||||
## Devices Table — Write-Path Inventory
|
||||
|
||||
Before implementing any feature that reads or writes the `Devices` table, audit ALL write paths. The table is modified from many locations — missing one path is a correctness bug.
|
||||
|
||||
**Known production write paths (as of 2026-07-04):**
|
||||
|
||||
| File | Function | Fields written |
|
||||
|---|---|---|
|
||||
| `server/models/device_instance.py` | `setDeviceData()` | All user-editable fields |
|
||||
| `server/models/device_instance.py` | `updateField()` | Any single field (workflows) |
|
||||
| `server/models/device_instance.py` | `updateDeviceColumn()` | Any single column |
|
||||
| `server/models/device_instance.py` | `deleteDevices()` etc. | DELETE operations |
|
||||
| `server/scan/device_handling.py` | `update_devices_data_from_scan()` | Scan-derived fields |
|
||||
| `server/scan/device_handling.py` | `update_vendors_from_mac()` | `devVendor`, `devVendorSource` |
|
||||
| `server/scan/device_handling.py` | Name resolution block | `devName`, `devFQDN`, `*Source` |
|
||||
| `server/scan/device_handling.py` | `update_ipv4_ipv6()` | `devPrimaryIPv4`, `devPrimaryIPv6` |
|
||||
| `server/scan/device_handling.py` | `update_icons_and_types()` | `devIcon`, `devType` |
|
||||
| `server/scan/device_handling.py` | `update_presence_from_CurrentScan()` | `devPresentLastScan` |
|
||||
| `server/scan/device_handling.py` | `update_devLastConnection_from_CurrentScan()` | `devLastConnection` |
|
||||
| `server/scan/device_handling.py` | `update_devPresentLastScan_based_on_*()` | `devPresentLastScan` |
|
||||
| `server/db/authoritative_handler.py` | `enforce_source_on_user_update()` | `*Source` columns |
|
||||
| `server/db/authoritative_handler.py` | `lock_field()` / `unlock_field()` | `*Source` columns |
|
||||
| `server/models/notification_instance.py` | `clearPendingEmailFlag()` | `devLastNotification` |
|
||||
| `front/plugins/db_cleanup/script.py` | `cleanup_database()` | DELETE operations |
|
||||
|
||||
**Key insight:** Most scan functions use `sql.executemany()` — there is no per-row Python state available. Python hooks before/after executemany require a pre-fetch+diff pattern that is expensive and error-prone.
|
||||
|
||||
---
|
||||
|
||||
## `*Source` Fields — Attribution System
|
||||
|
||||
The `FIELD_SOURCE_MAP` in `server/db/authoritative_handler.py` defines 10 fields that carry write attribution via paired `*Source` columns:
|
||||
|
||||
```python
|
||||
FIELD_SOURCE_MAP = {
|
||||
"devMac": "devMacSource",
|
||||
"devName": "devNameSource",
|
||||
"devFQDN": "devFQDNSource",
|
||||
"devLastIP": "devLastIPSource",
|
||||
"devVendor": "devVendorSource",
|
||||
"devSSID": "devSSIDSource",
|
||||
"devParentMAC": "devParentMACSource",
|
||||
"devParentPort": "devParentPortSource",
|
||||
"devParentRelType": "devParentRelTypeSource",
|
||||
"devVlan": "devVlanSource",
|
||||
}
|
||||
```
|
||||
|
||||
`*Source` values: `'USER'`, `'LOCKED'`, `'NEWDEV'`, or a plugin prefix (e.g., `'ARPSCAN'`, `'NSLOOKUP'`).
|
||||
|
||||
These fields are updated **in the same transaction** as the primary field. A SQLite `AFTER UPDATE` trigger can read `NEW.devNameSource` to obtain correct attribution without any extra context-passing.
|
||||
|
||||
**Attribution rules for features that need `changedBy`:**
|
||||
|
||||
| Field category | Attribution |
|
||||
|---|---|
|
||||
| In `FIELD_SOURCE_MAP` | `COALESCE(NULLIF(NEW.<field>Source, ''), 'system')` |
|
||||
| User-only fields (`devGroup`, `devComments`, `devFavorite`, `devOwner`, `devLocation`, etc.) | `'user:api'` — only `setDeviceData()` writes these |
|
||||
| Auto-computed fields (`devIcon`, `devType`, `devPrimaryIPv4`, `devPrimaryIPv6`) | `'system'` |
|
||||
| `*Source` fields themselves | `'system'` |
|
||||
|
||||
---
|
||||
|
||||
## Cross-Cutting Concerns — Prefer SQLite Triggers Over Python Hooks
|
||||
|
||||
When a feature needs to intercept **every write** to the `Devices` table (audit logging, computed columns, cascading logic), prefer a **SQLite `AFTER UPDATE` / `AFTER INSERT` trigger** over Python-layer hooks.
|
||||
|
||||
**Why:**
|
||||
- Triggers catch all 14+ write paths automatically, including `executemany()` bulk updates
|
||||
- Zero modifications to existing write-path functions (DRY)
|
||||
- Self-healing: future write paths are automatically covered
|
||||
- Attribution is available via `NEW.*Source` fields (see above)
|
||||
|
||||
**When Python hooks are still appropriate:**
|
||||
- The logic needs access to Python objects, settings, or services not available in SQL
|
||||
- The feature only fires from one or two known write paths
|
||||
- The logic is too complex to express in SQL (multi-table joins with app-layer business logic)
|
||||
|
||||
### Trigger Performance Pattern
|
||||
|
||||
```sql
|
||||
CREATE TRIGGER trg_example
|
||||
AFTER UPDATE ON Devices
|
||||
FOR EACH ROW
|
||||
-- Guard: short-circuit entire body when feature is disabled (zero cost)
|
||||
WHEN (SELECT CAST(setValue AS INTEGER) FROM Settings WHERE setKey = 'FEATURE_ENABLED') > 0
|
||||
BEGIN
|
||||
-- Per-field conditional insert
|
||||
INSERT INTO SomeTable (devGUID, column, oldVal, newVal, changedBy, ts)
|
||||
SELECT NEW.devGUID, 'devName', OLD.devName, NEW.devName,
|
||||
COALESCE(NULLIF(NEW.devNameSource, ''), 'system'),
|
||||
datetime('now', 'utc')
|
||||
WHERE OLD.devName IS NOT NEW.devName
|
||||
AND instr(',' || (SELECT setValue FROM Settings WHERE setKey = 'TRACKED_FIELDS') || ',', ',devName,') > 0;
|
||||
-- Repeat for each tracked field...
|
||||
END;
|
||||
```
|
||||
|
||||
**Performance:** The Settings table is tiny (~100 rows) and stays in SQLite's page cache. Per-row Settings reads inside triggers are effectively in-memory lookups. The `WHEN` guard makes the disabled state zero-cost.
|
||||
|
||||
---
|
||||
|
||||
## Snapshot vs Event-Sourced Audit Logging
|
||||
|
||||
When implementing change history, always use **event-sourced (per-field rows)** not **snapshots (full row copies)**.
|
||||
|
||||
| | Event-sourced | Snapshot |
|
||||
|---|---|---|
|
||||
| Storage | Small — only changed fields | Large — all 40+ columns every mutation |
|
||||
| Filter by field | O(log n) via index | O(n) — must diff every adjacent pair |
|
||||
| Filter by source | O(log n) via index | Not possible without diffing |
|
||||
| `changedBy` attribution | Embedded at write time | Not available without extra context |
|
||||
| Retention calculation | Simple timestamp DELETE | Same, but much higher storage |
|
||||
|
||||
At 1000 devices, 5-min scan interval, 14-day retention: snapshot storage ≈ 280 MB/day. Event-sourced storage for the same workload is typically <1 MB/day (most scans produce no tracked field changes).
|
||||
|
||||
---
|
||||
|
||||
## `DevicesHistory` Table — Reference Schema
|
||||
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS DevicesHistory (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
devGUID TEXT NOT NULL,
|
||||
timestamp DATETIME DEFAULT CURRENT_TIMESTAMP,
|
||||
changedBy TEXT NOT NULL,
|
||||
changedColumn TEXT NOT NULL,
|
||||
oldValue TEXT,
|
||||
newValue TEXT,
|
||||
FOREIGN KEY (devGUID) REFERENCES Devices(devGUID) ON DELETE CASCADE
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_devhist_guid_column ON DevicesHistory(devGUID, changedColumn);
|
||||
CREATE INDEX IF NOT EXISTS idx_devhist_timestamp ON DevicesHistory(timestamp);
|
||||
```
|
||||
5
.github/skills/mcp-activation/SKILL.md
vendored
5
.github/skills/mcp-activation/SKILL.md
vendored
@@ -7,6 +7,11 @@ description: Enables live interaction with the NetAlertX runtime. This skill con
|
||||
|
||||
This skill configures the environment to expose the Model Context Protocol (MCP) server to AI agents running inside the devcontainer.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. **Devcontainer connected:** You must be working inside the NetAlertX devcontainer.
|
||||
2. **Backend running:** The Python backend must be running — it generates `app.conf` containing the API token. Use the `devcontainer-services` skill if it's not running.
|
||||
|
||||
## Usage
|
||||
|
||||
This skill assumes you are already running within the NetAlertX devcontainer.
|
||||
|
||||
168
.github/skills/settings-management/SKILL.md
vendored
168
.github/skills/settings-management/SKILL.md
vendored
@@ -5,35 +5,175 @@ description: Manage NetAlertX configuration settings. Use this when asked to add
|
||||
|
||||
# Settings Management
|
||||
|
||||
## Reading Settings
|
||||
## Quick Reference
|
||||
|
||||
**Read a setting (backend):**
|
||||
```python
|
||||
from helper import get_setting_value
|
||||
|
||||
value = get_setting_value('SETTING_NAME')
|
||||
```
|
||||
|
||||
Never hardcode ports, secrets, or configuration values. Always use `get_setting_value()`.
|
||||
|
||||
## Adding Core Settings
|
||||
|
||||
Use `ccd()` in `server/initialise.py`:
|
||||
**Read a setting (frontend):**
|
||||
```javascript
|
||||
getSetting("SETTING_NAME")
|
||||
```
|
||||
|
||||
**Add a core setting** — use `ccd()` in `server/initialise.py`:
|
||||
```python
|
||||
ccd('SETTING_NAME', 'default_value', 'description')
|
||||
```
|
||||
|
||||
## Adding Plugin Settings
|
||||
|
||||
Define in plugin's `config.json` manifest under the settings section.
|
||||
|
||||
## Config Files
|
||||
**Add a plugin setting** — define in the plugin's `config.json` under the `settings` key.
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `/data/config/app.conf` | Runtime config (modified by app) |
|
||||
| `/data/config/app.conf` | Runtime config (source of truth, modified by app) |
|
||||
| `back/app.conf` | Default config (template) |
|
||||
|
||||
## Environment Override
|
||||
Use `APP_CONF_OVERRIDE` for settings that must be set before startup.
|
||||
|
||||
Use `APP_CONF_OVERRIDE` environment variable for settings that must be set before startup.
|
||||
---
|
||||
|
||||
# Development Guide
|
||||
|
||||
The goal is for every setting to be:
|
||||
|
||||
- Declarative and self-documenting
|
||||
- Automatically rendered by the Settings UI
|
||||
- Fully localized
|
||||
- Backwards compatible
|
||||
- Easy to maintain
|
||||
|
||||
---
|
||||
|
||||
## 1. Follow existing patterns
|
||||
|
||||
Before creating a new setting, search for similar ones. Reuse existing naming conventions, types, and validation patterns. Follow the style already used by similar plugins. Avoid introducing new setting types unless absolutely necessary.
|
||||
|
||||
---
|
||||
|
||||
## 2. Setting naming
|
||||
|
||||
Setting keys use uppercase snake case: `UI_DEV_SECTIONS`, `SCAN_INTERVAL`, `MQTT_HOST`.
|
||||
Names should clearly describe the purpose.
|
||||
|
||||
---
|
||||
|
||||
## 3. Categories
|
||||
|
||||
Place settings into the most appropriate category. Keep related settings together. Avoid creating new categories unless there is a clear need.
|
||||
|
||||
---
|
||||
|
||||
## 4. Choose the correct type
|
||||
|
||||
Use the simplest type that correctly models the value: `string`, `integer`, `float`, `boolean`, `password`, `select`, `array`, `textarea`. Avoid encoding structured JSON inside string settings.
|
||||
|
||||
---
|
||||
|
||||
## 5. Validation
|
||||
|
||||
Always define validation whenever appropriate (min/max values, regex, allowed options). Reject invalid configuration rather than silently accepting it.
|
||||
|
||||
---
|
||||
|
||||
## 6. Defaults
|
||||
|
||||
Provide sensible defaults that work for a fresh installation. The `default_value` in `config.json` is the single source of truth — do not duplicate it in Python or JavaScript.
|
||||
|
||||
---
|
||||
|
||||
## 7. Names and descriptions
|
||||
|
||||
Names should be concise. Descriptions should explain what the setting does, when to use it, and any important side effects — not implementation details.
|
||||
|
||||
---
|
||||
|
||||
## 8. Boolean settings
|
||||
|
||||
Prefer verbs: Enable, Disable, Require, Allow, Ignore, Show, Hide. Examples: *Enable MQTT*, *Require NICs Online*, *Hide Offline Devices*.
|
||||
|
||||
---
|
||||
|
||||
## 9. Select settings
|
||||
|
||||
Use when users must choose from predefined values. Each option must have a meaningful label — never require users to remember internal values.
|
||||
|
||||
---
|
||||
|
||||
## 10. Array settings
|
||||
|
||||
Use only when multiple independent values are expected (e.g., ignored MACs, subnet lists, plugin lists).
|
||||
|
||||
---
|
||||
|
||||
## 11. Localization
|
||||
|
||||
Every setting must have localized language strings. If not defined directly in `config.json`, they must exist in `en_us.json`.
|
||||
|
||||
Example for `UI_DEV_SECTIONS`:
|
||||
```json
|
||||
"UI_DEV_SECTIONS_name": "Hide device sections",
|
||||
"UI_DEV_SECTIONS_description": "Select which UI elements to hide on the Devices page."
|
||||
```
|
||||
|
||||
When using external language files, `config.json` must reference:
|
||||
```json
|
||||
"name": [{ "string": "_GLOBAL_LANG_FILES_" }]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 12. Source of truth
|
||||
|
||||
`app.conf` is the source of truth. User-defined values always come from `app.conf`; defaults from `config.json` apply only when a setting is absent. Never read `app.conf` directly — always go through `get_setting_value()`.
|
||||
|
||||
---
|
||||
|
||||
## 13. Plugin-first settings
|
||||
|
||||
Whenever possible, define new settings inside a plugin's `config.json`. Avoid adding hardcoded application settings unless there is a compelling architectural reason.
|
||||
|
||||
---
|
||||
|
||||
## 14. Runtime lifecycle
|
||||
|
||||
```text
|
||||
config.json (defaults)
|
||||
→ app.conf (source of truth)
|
||||
→ Settings database table
|
||||
→ table_settings.json API
|
||||
→ Frontend getSetting() / Backend get_setting_value()
|
||||
```
|
||||
|
||||
The database and API are runtime representations only, regenerated from `app.conf` at init.
|
||||
|
||||
---
|
||||
|
||||
## 15. Backwards compatibility
|
||||
|
||||
Avoid renaming settings, changing data types, or changing semantics. If unavoidable, provide migration logic and preserve compatibility where possible.
|
||||
|
||||
---
|
||||
|
||||
## 16. Pull request checklist
|
||||
|
||||
- [ ] Follows existing naming conventions (uppercase snake case)
|
||||
- [ ] Correct category used
|
||||
- [ ] Appropriate type selected, validation defined
|
||||
- [ ] Sensible default — defined only once (in `config.json`)
|
||||
- [ ] Name concise, description clear (no implementation details)
|
||||
- [ ] Localization strings added; `_GLOBAL_LANG_FILES_` used where appropriate
|
||||
- [ ] Plugin-first approach followed where applicable
|
||||
- [ ] Frontend renders automatically without custom JavaScript
|
||||
- [ ] Backend uses `get_setting_value()` — no direct `app.conf` reads
|
||||
- [ ] Backwards compatibility maintained
|
||||
- [ ] Documentation updated if required
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- https://docs.netalertx.com/PLUGINS_DEV_SETTINGS/
|
||||
- https://docs.netalertx.com/SETTINGS_SYSTEM/
|
||||
|
||||
64
.github/skills/skills-overview/SKILL.md
vendored
Normal file
64
.github/skills/skills-overview/SKILL.md
vendored
Normal file
@@ -0,0 +1,64 @@
|
||||
---
|
||||
name: skills-overview
|
||||
description: Index of all available skills across both GitHub Copilot (.github/skills/) and Gemini CLI (.gemini/skills/). Load this to find the right skill for a task, or to locate the counterpart skill in the other AI system.
|
||||
---
|
||||
|
||||
# Skills Index — Cross-Reference
|
||||
|
||||
Two AI assistants are configured for this project, each with their own skill directory:
|
||||
|
||||
- **GitHub Copilot** → `.github/skills/`
|
||||
- **Gemini CLI** → `.gemini/skills/`
|
||||
|
||||
Skills with the same purpose exist in both, sometimes under different names and with different depth. This index maps them so you can find the richer version when needed.
|
||||
|
||||
---
|
||||
|
||||
## Shared Skills (exist in both)
|
||||
|
||||
| Topic | Copilot Skill | Gemini Skill | Notes |
|
||||
|-------|--------------|--------------|-------|
|
||||
| Testing | `testing-workflow` | `testing-workflow` | Copilot version covers `testFailure` tool and PYTHONPATH; Gemini version emphasises full suite preference and container detection |
|
||||
| Settings & config | `settings-management` | `settings` | Gemini version is more comprehensive (22-point guide + PR checklist); Copilot version covers `ccd()` and `get_setting_value()` usage |
|
||||
| MCP activation | `mcp-activation` | `mcp-activation` | Copilot version covers VS Code window reload; Gemini version covers Gemini CLI session restart |
|
||||
| Project navigation | `project-navigation` | `project-navigation` | Copilot version has full path tables and env vars; Gemini version is a brief reference |
|
||||
| Plugin dev | `plugin-run-development` | `plugin-development` | Copilot version is comprehensive (data contract, phases, formats); Gemini version is a brief checklist pointing to `docs/PLUGINS_DEV.md` |
|
||||
| Devcontainer | `devcontainer-services` + `devcontainer-setup` + `devcontainer-configs` | `devcontainer-management` | Copilot splits into 3 focused skills; Gemini combines into one (uses `docker exec`) |
|
||||
|
||||
---
|
||||
|
||||
## Copilot-Only Skills
|
||||
|
||||
No Gemini equivalent yet:
|
||||
|
||||
| Skill | Purpose |
|
||||
|-------|---------|
|
||||
| `api-development` | Creating REST API endpoints |
|
||||
| `authentication` | API tokens and 401/403 debugging |
|
||||
| `code-standards` | Coding conventions and style rules |
|
||||
| `database-patterns` | Device table write paths, SQLite triggers, audit logging, `*Source` attribution |
|
||||
| `database-reset` | Wipe and regenerate the database and config |
|
||||
| `docker-build` | Build Docker images for testing or production |
|
||||
| `docker-prune` | Clean unused Docker resources (destructive — requires confirmation) |
|
||||
| `sample-data` | Load synthetic device data into the devcontainer |
|
||||
|
||||
---
|
||||
|
||||
## Gemini-Only Skills
|
||||
|
||||
No Copilot equivalent yet:
|
||||
|
||||
| Skill | Purpose |
|
||||
|-------|---------|
|
||||
| `initiative-start` | Research methodology and structured approach for new tasks |
|
||||
|
||||
---
|
||||
|
||||
## Adding a New Skill
|
||||
|
||||
When adding a skill, create it in **both** directories to keep both AI systems current:
|
||||
|
||||
- `.github/skills/<name>/SKILL.md` — add an entry to the skills table in `.github/copilot-instructions.md`
|
||||
- `.gemini/skills/<name>/SKILL.md` — auto-discovered by Gemini CLI via YAML frontmatter
|
||||
|
||||
Keep the body content identical between both files. Only the frontmatter `name`/`description` may differ slightly to match each system's discovery heuristics.
|
||||
@@ -23,6 +23,8 @@ TIMEZONE='Europe/Berlin'
|
||||
LOADED_PLUGINS=['ARPSCAN', 'AVAHISCAN', 'CSVBCKP','DBCLNP', 'DIGSCAN', 'INTRNT', 'MAINT', 'NEWDEV', 'NBTSCAN', 'NSLOOKUP','NTFPRCS', 'SETPWD', 'SMTP', 'SYNC', 'VNDRPDT', 'WORKFLOWS', 'UI']
|
||||
|
||||
DAYS_TO_KEEP_EVENTS=90
|
||||
DEV_HIST_DAYS=14
|
||||
DEV_HIST_TRACKED=['devMac','devName','devOwner','devType','devVendor','devFavorite','devGroup','devComments','devLastIP','devFQDN','devPrimaryIPv4','devPrimaryIPv6','devVlan','devForceStatus','devStaticIP','devScan','devAlertDown','devCanSleep','devSkipRepeated','devLocation','devIsArchived','devParentMAC','devParentPort','devParentRelType','devReqNicsOnline','devIcon','devSite','devSSID','devSyncHubNode','devSourcePlugin','devMacSource','devNameSource','devFQDNSource','devLastIPSource','devVendorSource','devSSIDSource','devParentMACSource','devParentPortSource','devParentRelTypeSource','devVlanSource','devCustomProps']
|
||||
# Used for generating links in emails. Make sure not to add a trailing slash!
|
||||
REPORT_DASHBOARD_URL='update_REPORT_DASHBOARD_URL_setting'
|
||||
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
GraphQL queries are **read-optimized for speed**. Data may be slightly out of date until the file system cache refreshes. The GraphQL endpoints allow you to access the following objects:
|
||||
|
||||
* Devices
|
||||
* Device History (change audit log)
|
||||
* Settings
|
||||
* Events
|
||||
* PluginsObjects
|
||||
@@ -415,3 +416,141 @@ curl 'http://host:GRAPHQL_PORT/graphql' \
|
||||
* Plugin queries scope `dbCount` to the requested `plugin`/`foreignKey` so badge counts reflect per-plugin totals.
|
||||
* The schema is **read-only** — updates must be performed through other APIs or configuration management. See the other [API](API.md) endpoints for details.
|
||||
|
||||
---
|
||||
|
||||
## Device History Queries
|
||||
|
||||
Device field change history is stored in `DevicesHistory` and exposed via two queries. History tracking is controlled by `DEV_HIST_DAYS` (retention window) and `DEV_HIST_TRACKED` (list of audited columns). Set `DEV_HIST_DAYS = 0` to disable tracking entirely.
|
||||
|
||||
### `deviceHistoryGrouped` — Single Device
|
||||
|
||||
Returns grouped change events for one device. Events are grouped by `(timestamp, changedBy)` so that simultaneous field changes are returned as a single object.
|
||||
|
||||
```graphql
|
||||
query DeviceHistory($devGuid: String!, $changedColumn: String, $changedBy: String, $limit: Int, $offset: Int) {
|
||||
deviceHistoryGrouped(
|
||||
devGuid: $devGuid
|
||||
changedColumn: $changedColumn
|
||||
changedBy: $changedBy
|
||||
limit: $limit
|
||||
offset: $offset
|
||||
) {
|
||||
count
|
||||
history {
|
||||
devGUID
|
||||
timestamp
|
||||
changedBy
|
||||
changes {
|
||||
changedColumn
|
||||
oldValue
|
||||
newValue
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Parameters
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|-----------------|----------|----------|----------------------------------------------------------------------|
|
||||
| `devGuid` | `String` | ✅ | GUID of the device to fetch history for |
|
||||
| `changedColumn` | `String` | No | Filter to groups containing a change to this specific column |
|
||||
| `changedBy` | `String` | No | Filter to a specific attribution source (`USER`, `ARPSCAN`, etc.) |
|
||||
| `limit` | `Int` | No | Max grouped events to return (default `50`) |
|
||||
| `offset` | `Int` | No | Grouped-event offset for pagination (default `0`) |
|
||||
|
||||
#### Response fields
|
||||
|
||||
| Field | Description |
|
||||
|----------------------------|-----------------------------------------------------------------------------|
|
||||
| `count` | Total number of grouped events matching the filters (use for pagination) |
|
||||
| `history[].devGUID` | Device GUID |
|
||||
| `history[].timestamp` | UTC timestamp of the change event |
|
||||
| `history[].changedBy` | Attribution: `USER`, plugin prefix (e.g. `ARPSCAN`), or `system` |
|
||||
| `history[].changes[].changedColumn` | Column name that changed |
|
||||
| `history[].changes[].oldValue` | Value before the change (`null` for new devices) |
|
||||
| `history[].changes[].newValue` | Value after the change |
|
||||
|
||||
#### `curl` Example
|
||||
|
||||
```sh
|
||||
curl -X POST http://localhost:20212/graphql \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer YOUR_API_TOKEN" \
|
||||
-d '{
|
||||
"query": "query($devGuid:String!,$limit:Int,$offset:Int){deviceHistoryGrouped(devGuid:$devGuid,limit:$limit,offset:$offset){count history{timestamp changedBy changes{changedColumn oldValue newValue}}}}",
|
||||
"variables": {
|
||||
"devGuid": "your-device-guid-here",
|
||||
"limit": 25,
|
||||
"offset": 0
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### `allDeviceHistoryGrouped` — All Devices (Global View)
|
||||
|
||||
Identical to `deviceHistoryGrouped` but returns changes across **all devices**. Used by the Change History page under Monitoring. Does not accept a `devGuid` parameter.
|
||||
|
||||
```graphql
|
||||
query AllHistory($changedColumn: String, $changedBy: String, $limit: Int, $offset: Int) {
|
||||
allDeviceHistoryGrouped(
|
||||
changedColumn: $changedColumn
|
||||
changedBy: $changedBy
|
||||
limit: $limit
|
||||
offset: $offset
|
||||
) {
|
||||
count
|
||||
history {
|
||||
devGUID
|
||||
timestamp
|
||||
changedBy
|
||||
changes {
|
||||
changedColumn
|
||||
oldValue
|
||||
newValue
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### `curl` Example
|
||||
|
||||
```sh
|
||||
curl -X POST http://localhost:20212/graphql \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer YOUR_API_TOKEN" \
|
||||
-d '{
|
||||
"query": "query($changedBy:String,$limit:Int){allDeviceHistoryGrouped(changedBy:$changedBy,limit:$limit){count history{devGUID timestamp changedBy changes{changedColumn oldValue newValue}}}}",
|
||||
"variables": {
|
||||
"changedBy": "USER",
|
||||
"limit": 50
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Filter Values Endpoint
|
||||
|
||||
To populate dynamic filter dropdowns, fetch available `changedBy` and `changedColumn` values via the REST endpoint:
|
||||
|
||||
```sh
|
||||
GET /devices/history/filters?devGuid=<guid> # scoped to one device
|
||||
GET /devices/history/filters # all devices
|
||||
```
|
||||
|
||||
Response:
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"data": {
|
||||
"changedBy": ["ARPSCAN", "USER", "VNDRPDT", "system"],
|
||||
"changedColumn": ["devLastIP", "devName", "devNameSource", "devVendor"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
@@ -36,6 +36,37 @@ Plugin settings are loaded dynamically from the `config.json` of individual plug
|
||||
|
||||
![Screen 1][screen1]
|
||||
|
||||
### Localization
|
||||
|
||||
Every setting must have localized language strings.
|
||||
|
||||
If the language strings are **not** defined directly inside `config.json`, they must exist in `en_us.json`.
|
||||
|
||||
For example, for:
|
||||
|
||||
```text
|
||||
UI_DEV_SECTIONS
|
||||
```
|
||||
|
||||
add:
|
||||
|
||||
```json
|
||||
"UI_DEV_SECTIONS_name": "Hide device sections",
|
||||
"UI_DEV_SECTIONS_description": "Select which UI elements to hide on the Devices page."
|
||||
```
|
||||
|
||||
When using external language files, `config.json` must reference:
|
||||
|
||||
```json
|
||||
"name": [
|
||||
{
|
||||
"string": "_GLOBAL_LANG_FILES_"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
This tells NetAlertX to resolve the display name from the language files.
|
||||
|
||||
### Settings Process flow
|
||||
|
||||
The process flow is mostly managed by the [initialise.py](/server/initialise.py) file.
|
||||
|
||||
210
front/change_history.php
Normal file
210
front/change_history.php
Normal file
@@ -0,0 +1,210 @@
|
||||
<?php
|
||||
require 'php/templates/header.php';
|
||||
?>
|
||||
|
||||
<script>
|
||||
showSpinner();
|
||||
</script>
|
||||
|
||||
<div class="content-wrapper changeHistoryPage spinnerTarget" id="changeHistoryPage">
|
||||
|
||||
<section class="content-header">
|
||||
<h1>
|
||||
<?= lang('Navigation_ChangeHistory'); ?>
|
||||
<small><?= lang('device_history_col_changes'); ?></small>
|
||||
</h1>
|
||||
</section>
|
||||
|
||||
<section class="content">
|
||||
<div class="row">
|
||||
<div class="col-xs-12">
|
||||
<div class="box">
|
||||
|
||||
<!-- Filter bar -->
|
||||
<div class="box-header with-border" style="display:flex; gap:10px; flex-wrap:wrap; align-items:center;">
|
||||
<select id="filterChangedBy" class="form-control" style="width:200px;"
|
||||
onchange="loadChangeHistory()">
|
||||
<option value=""><?= lang('device_history_col_source'); ?> - All</option>
|
||||
</select>
|
||||
<select id="filterChangedColumn" class="form-control" style="width:200px;"
|
||||
onchange="loadChangeHistory()">
|
||||
<option value=""><?= lang('device_history_col_changes'); ?> - All</option>
|
||||
</select>
|
||||
</div>
|
||||
|
||||
<!-- DataTable -->
|
||||
<div class="box-body">
|
||||
<table id="changeHistoryTable" class="table table-bordered table-striped">
|
||||
<thead>
|
||||
<tr>
|
||||
<th><?= lang('device_history_col_time'); ?></th>
|
||||
<th><?= lang('device_history_col_source'); ?></th>
|
||||
<th>Device</th>
|
||||
<th><?= lang('device_history_col_changes'); ?></th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody id="changeHistoryBody">
|
||||
<tr><td colspan="4" class="text-center">
|
||||
<i class="fa fa-spinner fa-spin"></i>
|
||||
</td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
<!-- Pagination -->
|
||||
<div class="box-footer" style="display:flex; justify-content:space-between; align-items:center;">
|
||||
<span id="changeHistoryInfo" class="text-muted"></span>
|
||||
<div>
|
||||
<button class="btn btn-default btn-sm" id="btnPrev" onclick="changePage(-1)" disabled>
|
||||
<i class="fa fa-chevron-left"></i>
|
||||
</button>
|
||||
<span id="pageLabel" style="margin:0 8px;">1</span>
|
||||
<button class="btn btn-default btn-sm" id="btnNext" onclick="changePage(1)" disabled>
|
||||
<i class="fa fa-chevron-right"></i>
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
var _chPage = 0;
|
||||
var _chLimit = 50;
|
||||
var _chTotal = 0;
|
||||
|
||||
function loadChangeHistory() {
|
||||
var changedBy = document.getElementById('filterChangedBy').value;
|
||||
var changedColumn = document.getElementById('filterChangedColumn').value;
|
||||
|
||||
var query = `
|
||||
query($changedColumn: String, $changedBy: String, $limit: Int, $offset: Int) {
|
||||
allDeviceHistoryGrouped(
|
||||
changedColumn: $changedColumn
|
||||
changedBy: $changedBy
|
||||
limit: $limit
|
||||
offset: $offset
|
||||
) {
|
||||
count
|
||||
history {
|
||||
devGUID
|
||||
timestamp
|
||||
changedBy
|
||||
changes { changedColumn oldValue newValue }
|
||||
}
|
||||
}
|
||||
}
|
||||
`;
|
||||
|
||||
var variables = {
|
||||
changedBy: changedBy || null,
|
||||
changedColumn: changedColumn || null,
|
||||
limit: _chLimit,
|
||||
offset: _chPage * _chLimit
|
||||
};
|
||||
|
||||
$.ajax({
|
||||
url: '/server/graphql',
|
||||
method: 'POST',
|
||||
contentType: 'application/json',
|
||||
headers: { 'Authorization': 'Bearer ' + getSetting('API_TOKEN') },
|
||||
data: JSON.stringify({ query: query, variables: variables }),
|
||||
success: function(resp) {
|
||||
if (!resp.data || !resp.data.allDeviceHistoryGrouped) return;
|
||||
var result = resp.data.allDeviceHistoryGrouped;
|
||||
_chTotal = result.count;
|
||||
renderTable(result.history);
|
||||
updatePagination();
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
function renderTable(groups) {
|
||||
var tbody = document.getElementById('changeHistoryBody');
|
||||
if (!groups || groups.length === 0) {
|
||||
tbody.innerHTML = '<tr><td colspan="4" class="text-center text-muted"><?= lang('device_history_empty_state'); ?></td></tr>';
|
||||
return;
|
||||
}
|
||||
var html = '';
|
||||
groups.forEach(function(g) {
|
||||
var sourceIcon = g.changedBy === 'user:api' || g.changedBy === 'USER'
|
||||
? '<i class="fa fa-user" title="' + g.changedBy + '"></i>'
|
||||
: '<i class="fa fa-cog" title="' + g.changedBy + '"></i>';
|
||||
var changesHtml = g.changes.map(function(c) {
|
||||
return '<div><code>' + escHtml(c.changedColumn) + '</code>: '
|
||||
+ '<span class="text-danger">' + escHtml(c.oldValue || '∅') + '</span>'
|
||||
+ ' → '
|
||||
+ '<span class="text-success">' + escHtml(c.newValue || '∅') + '</span>'
|
||||
+ '</div>';
|
||||
}).join('');
|
||||
html += '<tr>'
|
||||
+ '<td style="white-space:nowrap">' + escHtml(g.timestamp) + '</td>'
|
||||
+ '<td>' + sourceIcon + ' ' + escHtml(g.changedBy) + '</td>'
|
||||
+ '<td><a href="deviceDetails.php?mac=" data-guid="' + escHtml(g.devGUID) + '">' + escHtml(g.devGUID) + '</a></td>'
|
||||
+ '<td>' + changesHtml + '</td>'
|
||||
+ '</tr>';
|
||||
});
|
||||
tbody.innerHTML = html;
|
||||
}
|
||||
|
||||
function updatePagination() {
|
||||
var totalPages = Math.max(1, Math.ceil(_chTotal / _chLimit));
|
||||
document.getElementById('pageLabel').textContent = (_chPage + 1) + ' / ' + totalPages;
|
||||
document.getElementById('changeHistoryInfo').textContent =
|
||||
_chTotal + ' <?= lang('device_history_col_changes'); ?>';
|
||||
document.getElementById('btnPrev').disabled = _chPage <= 0;
|
||||
document.getElementById('btnNext').disabled = (_chPage + 1) >= totalPages;
|
||||
}
|
||||
|
||||
function changePage(delta) {
|
||||
_chPage = Math.max(0, _chPage + delta);
|
||||
loadChangeHistory();
|
||||
}
|
||||
|
||||
function escHtml(str) {
|
||||
if (str === null || str === undefined) return '';
|
||||
return String(str)
|
||||
.replace(/&/g, '&')
|
||||
.replace(/</g, '<')
|
||||
.replace(/>/g, '>')
|
||||
.replace(/"/g, '"');
|
||||
}
|
||||
|
||||
function populateFilters() {
|
||||
var query = `
|
||||
query {
|
||||
allDeviceHistoryGrouped(limit: 1) {
|
||||
history { changedBy changes { changedColumn } }
|
||||
}
|
||||
}
|
||||
`;
|
||||
// Fetch distinct filter values from the filter-values endpoint
|
||||
$.ajax({
|
||||
url: '/server/devices/history/filters',
|
||||
method: 'GET',
|
||||
headers: { 'Authorization': 'Bearer ' + getSetting('API_TOKEN') },
|
||||
success: function(resp) {
|
||||
if (!resp.data) return;
|
||||
var byEl = document.getElementById('filterChangedBy');
|
||||
var colEl = document.getElementById('filterChangedColumn');
|
||||
(resp.data.changedBy || []).forEach(function(v) {
|
||||
byEl.add(new Option(v, v));
|
||||
});
|
||||
(resp.data.changedColumn || []).forEach(function(v) {
|
||||
colEl.add(new Option(v, v));
|
||||
});
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
$(document).ready(function() {
|
||||
populateFilters();
|
||||
loadChangeHistory();
|
||||
hideSpinner();
|
||||
});
|
||||
</script>
|
||||
|
||||
<?php require 'php/templates/footer.php'; ?>
|
||||
@@ -111,6 +111,14 @@
|
||||
</span>
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a id="tabHistory" href="#panHistory" data-toggle="tab">
|
||||
<i class="fa fa-clock-rotate-left"></i>
|
||||
<span class="dev-detail-tab-name">
|
||||
<?= lang('device_history_tab_title');?>
|
||||
</span>
|
||||
</a>
|
||||
</li>
|
||||
|
||||
<div class="btn-group pull-right">
|
||||
<button type="button" class="btn btn-default" style="padding: 10px; min-width: 30px;"
|
||||
@@ -159,7 +167,46 @@
|
||||
?>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
<div class="tab-pane fade" id="panHistory">
|
||||
<?php require 'php/templates/skel_device_details_tab_history.php'; ?>
|
||||
<!-- Filter bar -->
|
||||
<div style="display:flex; gap:10px; flex-wrap:wrap; align-items:center; margin-bottom:12px;">
|
||||
<select id="histFilterBy" class="form-control" style="width:180px;"
|
||||
onchange="loadHistoryData()">
|
||||
<option value=""><?= lang('device_history_col_source');?> — All</option>
|
||||
</select>
|
||||
<select id="histFilterCol" class="form-control" style="width:180px;"
|
||||
onchange="loadHistoryData()">
|
||||
<option value=""><?= lang('device_history_col_changes');?> — All</option>
|
||||
</select>
|
||||
</div>
|
||||
<!-- Table -->
|
||||
<table class="table table-bordered table-striped table-hover">
|
||||
<thead>
|
||||
<tr>
|
||||
<th><?= lang('device_history_col_time');?></th>
|
||||
<th><?= lang('device_history_col_source');?></th>
|
||||
<th><?= lang('device_history_col_changes');?></th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody id="historyTableBody">
|
||||
<tr><td colspan="3" class="text-center"><i class="fa fa-spinner fa-spin"></i></td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<!-- Pagination -->
|
||||
<div style="display:flex; justify-content:space-between; align-items:center; margin-top:8px;">
|
||||
<span id="histInfo" class="text-muted" style="font-size:12px;"></span>
|
||||
<div>
|
||||
<button class="btn btn-default btn-xs" id="histBtnPrev" onclick="histChangePage(-1)" disabled>
|
||||
<i class="fa fa-chevron-left"></i>
|
||||
</button>
|
||||
<span id="histPageLabel" style="margin:0 6px;font-size:12px;">1</span>
|
||||
<button class="btn btn-default btn-xs" id="histBtnNext" onclick="histChangePage(1)" disabled>
|
||||
<i class="fa fa-chevron-right"></i>
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<!-- /.tab-content -->
|
||||
</div>
|
||||
<!-- /.nav-tabs-custom -->
|
||||
@@ -596,9 +643,142 @@ setTimeout(function() {
|
||||
if (typeof hideSessionsTabSkeleton === 'function') hideSessionsTabSkeleton();
|
||||
if (typeof hidePresenceTabSkeleton === 'function') hidePresenceTabSkeleton();
|
||||
if (typeof hideEventsTabSkeleton === 'function') hideEventsTabSkeleton();
|
||||
$('#skel-tab-history').hide();
|
||||
}, 15000);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Change Log tab — device-scoped history
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
var _histPage = 0;
|
||||
var _histLimit = 50;
|
||||
var _histTotal = 0;
|
||||
var _histLoaded = false;
|
||||
|
||||
function loadHistoryData() {
|
||||
var devGuid = getDevDataByMac(mac, 'devGUID');
|
||||
// Devices without a GUID (created before GUIDs were introduced) cannot be
|
||||
// queried — show a clear message instead of spinning indefinitely.
|
||||
if (devGuid === null || devGuid === undefined || devGuid === '') {
|
||||
_renderHistoryTable([]);
|
||||
return;
|
||||
}
|
||||
|
||||
var changedBy = (document.getElementById('histFilterBy') || {}).value || null;
|
||||
var changedColumn = (document.getElementById('histFilterCol') || {}).value || null;
|
||||
|
||||
var query = [
|
||||
'query($devGuid:String!,$changedBy:String,$changedColumn:String,$limit:Int,$offset:Int){',
|
||||
' deviceHistoryGrouped(devGuid:$devGuid,changedBy:$changedBy,changedColumn:$changedColumn,limit:$limit,offset:$offset){',
|
||||
' count history{timestamp changedBy changes{changedColumn oldValue newValue}}',
|
||||
' }',
|
||||
'}'
|
||||
].join('');
|
||||
|
||||
var variables = {
|
||||
devGuid: devGuid,
|
||||
changedBy: changedBy || null,
|
||||
changedColumn: changedColumn || null,
|
||||
limit: _histLimit,
|
||||
offset: _histPage * _histLimit
|
||||
};
|
||||
|
||||
$.ajax({
|
||||
url: getApiBase() + '/graphql',
|
||||
method: 'POST',
|
||||
contentType: 'application/json',
|
||||
headers: { 'Authorization': 'Bearer ' + getSetting('API_TOKEN') },
|
||||
data: JSON.stringify({ query: query, variables: variables }),
|
||||
success: function(resp) {
|
||||
if (!resp.data || !resp.data.deviceHistoryGrouped) return;
|
||||
var result = resp.data.deviceHistoryGrouped;
|
||||
_histTotal = result.count;
|
||||
_renderHistoryTable(result.history);
|
||||
_updateHistoryPagination();
|
||||
|
||||
// Populate filter dropdowns once on first load
|
||||
if (!_histLoaded) {
|
||||
_histLoaded = true;
|
||||
_populateHistoryFilters(devGuid);
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
function _populateHistoryFilters(devGuid) {
|
||||
$.ajax({
|
||||
url: getApiBase() + '/devices/history/filters?devGuid=' + encodeURIComponent(devGuid),
|
||||
method: 'GET',
|
||||
headers: { 'Authorization': 'Bearer ' + getSetting('API_TOKEN') },
|
||||
success: function(resp) {
|
||||
if (!resp.data) return;
|
||||
var byEl = document.getElementById('histFilterBy');
|
||||
var colEl = document.getElementById('histFilterCol');
|
||||
(resp.data.changedBy || []).forEach(function(v) { byEl.add(new Option(v, v)); });
|
||||
(resp.data.changedColumn || []).forEach(function(v) { colEl.add(new Option(v, v)); });
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
function _renderHistoryTable(groups) {
|
||||
var tbody = document.getElementById('historyTableBody');
|
||||
if (!tbody) return;
|
||||
// Always hide the loading skeleton once we have a result (even empty)
|
||||
$('#skel-tab-history').fadeOut(0, function() { $(this).hide(); });
|
||||
if (!groups || groups.length === 0) {
|
||||
tbody.innerHTML = '<tr><td colspan="3" class="text-center text-muted"><?= lang('device_history_empty_state');?></td></tr>';
|
||||
return;
|
||||
}
|
||||
var html = '';
|
||||
groups.forEach(function(g) {
|
||||
var iconHtml = (g.changedBy === 'user:api' || g.changedBy === 'USER')
|
||||
? '<i class="fa fa-user" title="' + _histEsc(g.changedBy) + '"></i>'
|
||||
: '<i class="fa fa-cog" title="' + _histEsc(g.changedBy) + '"></i>';
|
||||
var changesHtml = g.changes.map(function(c) {
|
||||
return '<div><code>' + _histEsc(c.changedColumn) + '</code>: '
|
||||
+ '<span class="text-danger">' + _histEsc(c.oldValue || '\u2205') + '</span>'
|
||||
+ ' \u2192 '
|
||||
+ '<span class="text-success">' + _histEsc(c.newValue || '\u2205') + '</span>'
|
||||
+ '</div>';
|
||||
}).join('');
|
||||
html += '<tr>'
|
||||
+ '<td style="white-space:nowrap;font-size:12px;">' + _histEsc(g.timestamp) + '</td>'
|
||||
+ '<td>' + iconHtml + ' ' + _histEsc(g.changedBy) + '</td>'
|
||||
+ '<td>' + changesHtml + '</td>'
|
||||
+ '</tr>';
|
||||
});
|
||||
tbody.innerHTML = html;
|
||||
}
|
||||
|
||||
function _updateHistoryPagination() {
|
||||
var totalPages = Math.max(1, Math.ceil(_histTotal / _histLimit));
|
||||
var lbl = document.getElementById('histPageLabel');
|
||||
var info = document.getElementById('histInfo');
|
||||
var prev = document.getElementById('histBtnPrev');
|
||||
var next = document.getElementById('histBtnNext');
|
||||
if (lbl) lbl.textContent = (_histPage + 1) + ' / ' + totalPages;
|
||||
if (info) info.textContent = _histTotal + ' <?= lang('device_history_col_changes');?>';
|
||||
if (prev) prev.disabled = _histPage <= 0;
|
||||
if (next) next.disabled = (_histPage + 1) >= totalPages;
|
||||
}
|
||||
|
||||
function histChangePage(delta) {
|
||||
_histPage = Math.max(0, _histPage + delta);
|
||||
loadHistoryData();
|
||||
}
|
||||
|
||||
function _histEsc(str) {
|
||||
if (str === null || str === undefined) return '';
|
||||
return String(str).replace(/&/g,'&').replace(/</g,'<').replace(/>/g,'>').replace(/"/g,'"');
|
||||
}
|
||||
|
||||
// Load history when the tab becomes visible
|
||||
$(document).on('shown.bs.tab', 'a[id="tabHistory"]', function() {
|
||||
_histPage = 0;
|
||||
loadHistoryData();
|
||||
});
|
||||
|
||||
</script>
|
||||
|
||||
|
||||
|
||||
@@ -332,14 +332,14 @@
|
||||
|
||||
<!-- Monitoring menu item -->
|
||||
|
||||
<li class=" treeview <?php if (in_array (basename($_SERVER['SCRIPT_NAME']), array('presence.php', 'report.php', 'events.php', 'userNotifications.php' ) ) ){ echo 'active menu-open'; } ?>">
|
||||
<li class=" treeview <?php if (in_array (basename($_SERVER['SCRIPT_NAME']), array('presence.php', 'report.php', 'events.php', 'userNotifications.php', 'change_history.php' ) ) ){ echo 'active menu-open'; } ?>">
|
||||
<a href="#">
|
||||
<i class="fa fa-fw fa-chart-bar"></i> <span><?= lang('Navigation_Monitoring');?></span>
|
||||
<span class="pull-right-container">
|
||||
<i class="fa fa-angle-left pull-right"></i>
|
||||
</span>
|
||||
</a>
|
||||
<ul class="treeview-menu " style="display: <?php if (in_array (basename($_SERVER['SCRIPT_NAME']), array('presence.php', 'report.php', 'events.php', 'userNotifications.php' ) ) ){ echo 'block'; } else {echo 'none';} ?>;">
|
||||
<ul class="treeview-menu " style="display: <?php if (in_array (basename($_SERVER['SCRIPT_NAME']), array('presence.php', 'report.php', 'events.php', 'userNotifications.php', 'change_history.php' ) ) ){ echo 'block'; } else {echo 'none';} ?>;">
|
||||
<li>
|
||||
<a href="presence.php"> <?= lang("Navigation_Presence");?> </a>
|
||||
</li>
|
||||
@@ -352,6 +352,9 @@
|
||||
<li>
|
||||
<a href="userNotifications.php"> <?= lang("Navigation_Notifications");?> </a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="change_history.php"> <?= lang("Navigation_ChangeHistory");?> </a>
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
</li>
|
||||
|
||||
@@ -811,5 +811,15 @@
|
||||
"settings_system_icon": "fa-solid fa-gear",
|
||||
"settings_system_label": "System",
|
||||
"settings_update_item_warning": "Update the value below. Be careful to follow the previous format. <b>Validation is not performed.</b>",
|
||||
"test_event_tooltip": "Save your changes at first before you test your settings."
|
||||
"test_event_tooltip": "Save your changes at first before you test your settings.",
|
||||
"Navigation_ChangeHistory": "Change History",
|
||||
"device_history_tab_title": "Change Log",
|
||||
"device_history_col_time": "Timestamp",
|
||||
"device_history_col_source": "Modified By",
|
||||
"device_history_col_changes": "Tracked Variations",
|
||||
"device_history_empty_state": "No property changes recorded within the configured retention window.",
|
||||
"DEV_HIST_DAYS_name": "Device History Retention (Days)",
|
||||
"DEV_HIST_DAYS_description": "Number of days to retain device field change history. Set to 0 to completely disable the auditing engine and reduce resource usage.",
|
||||
"DEV_HIST_TRACKED_name": "Device History Tracked Fields",
|
||||
"DEV_HIST_TRACKED_description": "List of Devices column names to monitor for changes. Add or remove field names to control what gets audited. Changes take effect immediately without a restart."
|
||||
}
|
||||
26
front/php/templates/skel_device_details_tab_history.php
Normal file
26
front/php/templates/skel_device_details_tab_history.php
Normal file
@@ -0,0 +1,26 @@
|
||||
<!-- Change Log Tab Skeleton =============================================== -->
|
||||
<div id="skel-tab-history" class="skel-tab-pane spinnerTarget">
|
||||
<!-- Filter row -->
|
||||
<div style="display:flex; align-items:center; gap:10px; margin-bottom:12px; padding:0 2px;">
|
||||
<span class="skel-shimmer" style="width:120px; height:28px; border-radius:4px; flex-shrink:0;"></span>
|
||||
<span class="skel-shimmer" style="width:120px; height:28px; border-radius:4px; flex-shrink:0;"></span>
|
||||
</div>
|
||||
<!-- Table -->
|
||||
<div class="skel-table-box">
|
||||
<div class="skel-table-header-row">
|
||||
<span class="skel-th skel-shimmer" style="flex:1.5"></span>
|
||||
<span class="skel-th skel-shimmer" style="flex:1.5"></span>
|
||||
<span class="skel-th skel-shimmer" style="flex:4"></span>
|
||||
</div>
|
||||
<?php for ($i = 0; $i < 6; $i++): ?>
|
||||
<div class="skel-tr">
|
||||
<span class="skel-td skel-shimmer" style="flex:1.5; max-width:16%"></span>
|
||||
<span class="skel-td skel-shimmer" style="flex:1.5; max-width:14%"></span>
|
||||
<div style="flex:4; display:flex; flex-direction:column; gap:4px; padding:4px 0;">
|
||||
<span class="skel-td skel-shimmer" style="width:<?= 40 + ($i * 7) % 30 ?>%"></span>
|
||||
<span class="skel-td skel-shimmer" style="width:<?= 30 + ($i * 11) % 25 ?>%"></span>
|
||||
</div>
|
||||
</div>
|
||||
<?php endfor; ?>
|
||||
</div>
|
||||
</div>
|
||||
@@ -13,6 +13,7 @@ from const import logPath, fullDbPath # noqa: E402 [flake8 lint suppression]
|
||||
import conf # noqa: E402 [flake8 lint suppression]
|
||||
from pytz import timezone # noqa: E402 [flake8 lint suppression]
|
||||
from database import get_temp_db_connection # noqa: E402 [flake8 lint suppression]
|
||||
from models.device_history_instance import DevicesHistoryInstance # noqa: E402 [flake8 lint suppression]
|
||||
|
||||
# Make sure the TIMEZONE for logging is correct
|
||||
conf.tz = timezone(get_setting_value("TIMEZONE"))
|
||||
@@ -34,6 +35,7 @@ def main():
|
||||
HRS_TO_KEEP_OFFDEV = int(get_setting_value("HRS_TO_KEEP_OFFDEV"))
|
||||
DAYS_TO_KEEP_EVENTS = int(get_setting_value("DAYS_TO_KEEP_EVENTS"))
|
||||
CLEAR_NEW_FLAG = get_setting_value("CLEAR_NEW_FLAG")
|
||||
DEV_HIST_DAYS = int(get_setting_value("DEV_HIST_DAYS") or 14)
|
||||
|
||||
mylog("verbose", [f"[{pluginName}] In script"])
|
||||
|
||||
@@ -45,6 +47,7 @@ def main():
|
||||
HRS_TO_KEEP_OFFDEV,
|
||||
PLUGINS_KEEP_HIST,
|
||||
CLEAR_NEW_FLAG,
|
||||
DEV_HIST_DAYS,
|
||||
)
|
||||
|
||||
mylog("verbose", [f"[{pluginName}] Cleanup complete"])
|
||||
@@ -62,6 +65,7 @@ def cleanup_database(
|
||||
HRS_TO_KEEP_OFFDEV,
|
||||
PLUGINS_KEEP_HIST,
|
||||
CLEAR_NEW_FLAG,
|
||||
DEV_HIST_DAYS=14,
|
||||
):
|
||||
"""
|
||||
Cleaning out old records from the tables that don't need to keep all data.
|
||||
@@ -156,6 +160,13 @@ def cleanup_database(
|
||||
|
||||
conn.commit()
|
||||
|
||||
# -----------------------------------------------------
|
||||
# Cleanup DevicesHistory
|
||||
if DEV_HIST_DAYS > 0:
|
||||
mylog("verbose", f"[{pluginName}] DevicesHistory: Delete rows older than {DEV_HIST_DAYS} days")
|
||||
deleted = DevicesHistoryInstance().prune_history(DEV_HIST_DAYS)
|
||||
mylog("verbose", [f"[{pluginName}] DevicesHistory deleted rows: {deleted}"])
|
||||
|
||||
# -----------------------------------------------------
|
||||
# Cleanup New Devices
|
||||
if HRS_TO_KEEP_NEWDEV != 0:
|
||||
|
||||
@@ -6,6 +6,7 @@ import os
|
||||
|
||||
from flask import Flask, redirect, request, jsonify, url_for, Response
|
||||
from models.device_instance import DeviceInstance # noqa: E402
|
||||
from models.device_history_instance import DevicesHistoryInstance # noqa: E402
|
||||
from flask_cors import CORS
|
||||
from werkzeug.exceptions import HTTPException
|
||||
|
||||
@@ -772,6 +773,20 @@ def api_devices_totals(payload=None):
|
||||
return jsonify(device_handler.getTotals())
|
||||
|
||||
|
||||
@app.route("/devices/history/filters", methods=["GET"])
|
||||
@validate_request(
|
||||
operation_id="get_device_history_filters",
|
||||
summary="Get Device History Filter Values",
|
||||
description="Return distinct changedBy and changedColumn values available in DevicesHistory. Optionally scope to a single device with ?devGuid=<guid>.",
|
||||
tags=["devices"],
|
||||
auth_callable=is_authorized
|
||||
)
|
||||
def api_devices_history_filters(payload=None):
|
||||
dev_guid = request.args.get("devGuid") or None
|
||||
filters = DevicesHistoryInstance().get_available_filter_values(devGUID=dev_guid)
|
||||
return jsonify({"success": True, "data": filters})
|
||||
|
||||
|
||||
@app.route("/devices/totals/named", methods=["GET"])
|
||||
@validate_request(
|
||||
operation_id="get_device_totals_named",
|
||||
|
||||
@@ -26,6 +26,7 @@ from .graphql_types import ( # noqa: E402 [flake8 lint suppression]
|
||||
PluginQueryOptionsInput, PluginEntry,
|
||||
PluginsObjectsResult, PluginsEventsResult, PluginsHistoryResult,
|
||||
EventQueryOptionsInput, EventEntry, EventsResult,
|
||||
DeviceHistoryChange, GroupedDeviceHistory, DeviceHistoryResult,
|
||||
)
|
||||
from .graphql_helpers import ( # noqa: E402 [flake8 lint suppression]
|
||||
mixed_type_sort_key,
|
||||
@@ -33,6 +34,7 @@ from .graphql_helpers import ( # noqa: E402 [flake8 lint suppression]
|
||||
apply_plugin_filters,
|
||||
apply_events_filters,
|
||||
)
|
||||
from models.device_history_instance import DevicesHistoryInstance # noqa: E402
|
||||
|
||||
folder = apiPath
|
||||
|
||||
@@ -41,10 +43,95 @@ _langstrings_cache = {} # caches lists per file (core JSON or plugin)
|
||||
_langstrings_cache_mtime = {} # tracks last modified times
|
||||
|
||||
|
||||
def _to_graphql_history(groups):
|
||||
"""Convert DevicesHistoryInstance grouped dicts to GraphQL ObjectType instances."""
|
||||
result = []
|
||||
for g in groups:
|
||||
changes = [
|
||||
DeviceHistoryChange(
|
||||
changedColumn=c["changedColumn"],
|
||||
oldValue=c["oldValue"],
|
||||
newValue=c["newValue"],
|
||||
)
|
||||
for c in g["changes"]
|
||||
]
|
||||
result.append(GroupedDeviceHistory(
|
||||
devGUID=g["devGUID"],
|
||||
timestamp=g["timestamp"],
|
||||
changedBy=g["changedBy"],
|
||||
changes=changes,
|
||||
))
|
||||
return result
|
||||
|
||||
|
||||
class Query(ObjectType):
|
||||
# --- DEVICES ---
|
||||
devices = Field(DeviceResult, options=PageQueryOptionsInput())
|
||||
|
||||
# --- DEVICE HISTORY ---
|
||||
deviceHistoryGrouped = Field(
|
||||
DeviceHistoryResult,
|
||||
devGuid=String(required=True, description="Device GUID to fetch history for"),
|
||||
changedColumn=String(description="Filter to groups containing this field change"),
|
||||
changedBy=String(description="Filter to a specific attribution source"),
|
||||
limit=graphene.Int(description="Max grouped events to return (default 50)"),
|
||||
offset=graphene.Int(description="Grouped-event offset for pagination (default 0)"),
|
||||
)
|
||||
|
||||
allDeviceHistoryGrouped = Field(
|
||||
DeviceHistoryResult,
|
||||
changedColumn=String(description="Filter to groups containing this field change"),
|
||||
changedBy=String(description="Filter to a specific attribution source"),
|
||||
limit=graphene.Int(description="Max grouped events to return (default 100)"),
|
||||
offset=graphene.Int(description="Grouped-event offset for pagination (default 0)"),
|
||||
)
|
||||
|
||||
def resolve_deviceHistoryGrouped(self, info, devGuid, changedColumn=None,
|
||||
changedBy=None, limit=50, offset=0):
|
||||
try:
|
||||
h = DevicesHistoryInstance()
|
||||
groups = h.get_grouped_history(
|
||||
devGUID=devGuid,
|
||||
changedColumn=changedColumn,
|
||||
changedBy=changedBy,
|
||||
limit=limit,
|
||||
offset=offset,
|
||||
)
|
||||
total = h.get_total_group_count(
|
||||
devGUID=devGuid,
|
||||
changedColumn=changedColumn,
|
||||
changedBy=changedBy,
|
||||
)
|
||||
return DeviceHistoryResult(
|
||||
history=_to_graphql_history(groups),
|
||||
count=total,
|
||||
)
|
||||
except Exception as e:
|
||||
mylog("none", f"[graphql] resolve_deviceHistoryGrouped error: {e}")
|
||||
return DeviceHistoryResult(history=[], count=0)
|
||||
|
||||
def resolve_allDeviceHistoryGrouped(self, info, changedColumn=None,
|
||||
changedBy=None, limit=100, offset=0):
|
||||
try:
|
||||
h = DevicesHistoryInstance()
|
||||
groups = h.get_all_grouped_history(
|
||||
changedColumn=changedColumn,
|
||||
changedBy=changedBy,
|
||||
limit=limit,
|
||||
offset=offset,
|
||||
)
|
||||
total = h.get_total_group_count(
|
||||
changedColumn=changedColumn,
|
||||
changedBy=changedBy,
|
||||
)
|
||||
return DeviceHistoryResult(
|
||||
history=_to_graphql_history(groups),
|
||||
count=total,
|
||||
)
|
||||
except Exception as e:
|
||||
mylog("none", f"[graphql] resolve_allDeviceHistoryGrouped error: {e}")
|
||||
return DeviceHistoryResult(history=[], count=0)
|
||||
|
||||
def resolve_devices(self, info, options=None):
|
||||
# mylog('none', f'[graphql_schema] resolve_devices: {self}')
|
||||
try:
|
||||
|
||||
@@ -259,3 +259,25 @@ class EventsResult(ObjectType):
|
||||
entries = List(EventEntry, description="Events table rows")
|
||||
count = Int(description="Filtered count (before pagination)")
|
||||
db_count = Int(description="Total rows in table before any filter")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Device History
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class DeviceHistoryChange(ObjectType):
|
||||
changedColumn = String(description="Name of the Devices column that changed")
|
||||
oldValue = String(description="Value before the change (null for new devices)")
|
||||
newValue = String(description="Value after the change")
|
||||
|
||||
|
||||
class GroupedDeviceHistory(ObjectType):
|
||||
devGUID = String(description="GUID of the device that changed")
|
||||
timestamp = String(description="UTC timestamp of the change event")
|
||||
changedBy = String(description="Attribution: USER, plugin prefix (e.g. ARPSCAN), or system")
|
||||
changes = List(DeviceHistoryChange, description="Field-level changes in this event")
|
||||
|
||||
|
||||
class DeviceHistoryResult(ObjectType):
|
||||
history = List(GroupedDeviceHistory, description="Grouped change events")
|
||||
count = Int(description="Total number of grouped events matching the filters")
|
||||
|
||||
@@ -19,6 +19,7 @@ from db.db_upgrade import (
|
||||
migrate_to_camelcase,
|
||||
migrate_timestamps_to_utc,
|
||||
)
|
||||
from db.db_history import ensure_deviceshistory_table, ensure_deviceshistory_triggers
|
||||
|
||||
|
||||
class DB:
|
||||
@@ -224,6 +225,10 @@ class DB:
|
||||
# Normalization triggers
|
||||
ensure_mac_lowercase_triggers(self.sql)
|
||||
|
||||
# Device history table + audit triggers
|
||||
ensure_deviceshistory_table(self.sql)
|
||||
ensure_deviceshistory_triggers(self.sql)
|
||||
|
||||
# commit changes
|
||||
self.commitDB()
|
||||
except Exception as e:
|
||||
@@ -234,6 +239,12 @@ class DB:
|
||||
# Init the AppEvent database table
|
||||
AppEvent_obj(self)
|
||||
|
||||
# AppEvent_obj.drop_all_triggers() wipes every trigger in the DB
|
||||
# (including trg_devhist_*) as part of its clean-start routine.
|
||||
# Re-create the device history audit triggers here so they survive.
|
||||
ensure_deviceshistory_triggers(self.sql)
|
||||
self.commitDB()
|
||||
|
||||
def get_table_as_json(self, sqlQuery, parameters=None):
|
||||
"""
|
||||
Wrapper to use the central get_table_as_json helper.
|
||||
|
||||
229
server/db/db_history.py
Normal file
229
server/db/db_history.py
Normal file
@@ -0,0 +1,229 @@
|
||||
"""
|
||||
db_history.py — DevicesHistory table/trigger management.
|
||||
|
||||
Creates and maintains:
|
||||
- DevicesHistory table + indexes
|
||||
- AFTER UPDATE trigger (trg_devhist_update)
|
||||
- AFTER INSERT trigger (trg_devhist_insert)
|
||||
|
||||
Both triggers are driven entirely by two Settings values:
|
||||
DEV_HIST_DAYS — 0 disables the engine entirely (WHEN guard)
|
||||
DEV_HIST_TRACKED — comma-separated list of field names to audit
|
||||
|
||||
Attribution uses existing *Source columns where available, falling back
|
||||
to hard-coded constants for user-only and auto-computed fields.
|
||||
"""
|
||||
|
||||
from logger import mylog
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Field → changedBy attribution mapping
|
||||
#
|
||||
# Each entry: (field_name, changedBy_sql_expression)
|
||||
# - Source-tracked fields: read NEW.<field>Source at trigger time
|
||||
# - User-only fields: hard-coded 'user:api'
|
||||
# - Auto-computed fields: hard-coded 'system'
|
||||
# - *Source fields: hard-coded 'system'
|
||||
# ---------------------------------------------------------------------------
|
||||
_HIST_FIELDS = [
|
||||
# Source-tracked fields (FIELD_SOURCE_MAP)
|
||||
("devMac", "COALESCE(NULLIF(TRIM(NEW.devMacSource), ''), 'system')"),
|
||||
("devName", "COALESCE(NULLIF(TRIM(NEW.devNameSource), ''), 'system')"),
|
||||
("devFQDN", "COALESCE(NULLIF(TRIM(NEW.devFQDNSource), ''), 'system')"),
|
||||
("devLastIP", "COALESCE(NULLIF(TRIM(NEW.devLastIPSource), ''), 'system')"),
|
||||
("devVendor", "COALESCE(NULLIF(TRIM(NEW.devVendorSource), ''), 'system')"),
|
||||
("devSSID", "COALESCE(NULLIF(TRIM(NEW.devSSIDSource), ''), 'system')"),
|
||||
("devParentMAC", "COALESCE(NULLIF(TRIM(NEW.devParentMACSource), ''), 'system')"),
|
||||
("devParentPort", "COALESCE(NULLIF(TRIM(NEW.devParentPortSource), ''), 'system')"),
|
||||
("devParentRelType", "COALESCE(NULLIF(TRIM(NEW.devParentRelTypeSource), ''), 'system')"),
|
||||
("devVlan", "COALESCE(NULLIF(TRIM(NEW.devVlanSource), ''), 'system')"),
|
||||
# User-only fields (only setDeviceData() / updateField() touch these)
|
||||
("devOwner", "'user:api'"),
|
||||
("devType", "'user:api'"),
|
||||
("devFavorite", "'user:api'"),
|
||||
("devGroup", "'user:api'"),
|
||||
("devComments", "'user:api'"),
|
||||
("devForceStatus", "'user:api'"),
|
||||
("devStaticIP", "'user:api'"),
|
||||
("devScan", "'user:api'"),
|
||||
("devAlertDown", "'user:api'"),
|
||||
("devCanSleep", "'user:api'"),
|
||||
("devSkipRepeated", "'user:api'"),
|
||||
("devLocation", "'user:api'"),
|
||||
("devIsArchived", "'user:api'"),
|
||||
("devReqNicsOnline", "'user:api'"),
|
||||
("devCustomProps", "'user:api'"),
|
||||
# Auto-computed / scan-only fields (no *Source column)
|
||||
("devPrimaryIPv4", "'system'"),
|
||||
("devPrimaryIPv6", "'system'"),
|
||||
("devIcon", "'system'"),
|
||||
("devSite", "'system'"),
|
||||
("devSyncHubNode", "'system'"),
|
||||
("devSourcePlugin", "'system'"),
|
||||
# *Source fields themselves
|
||||
("devMacSource", "'system'"),
|
||||
("devNameSource", "'system'"),
|
||||
("devFQDNSource", "'system'"),
|
||||
("devLastIPSource", "'system'"),
|
||||
("devVendorSource", "'system'"),
|
||||
("devSSIDSource", "'system'"),
|
||||
("devParentMACSource", "'system'"),
|
||||
("devParentPortSource", "'system'"),
|
||||
("devParentRelTypeSource", "'system'"),
|
||||
("devVlanSource", "'system'"),
|
||||
]
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SQL builders
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_HIST_DAYS_GUARD = (
|
||||
"(SELECT CAST(COALESCE(setValue, '0') AS INTEGER) "
|
||||
"FROM Settings WHERE setKey = 'DEV_HIST_DAYS') > 0"
|
||||
)
|
||||
|
||||
_TRACKED_CHECK = (
|
||||
"instr((SELECT COALESCE(setValue, '') FROM Settings "
|
||||
"WHERE setKey = 'DEV_HIST_TRACKED'), '''{field}''') > 0"
|
||||
)
|
||||
|
||||
|
||||
def _build_update_trigger_sql() -> str:
|
||||
"""Generate CREATE TRIGGER SQL for AFTER UPDATE on Devices."""
|
||||
blocks = []
|
||||
for field, attribution in _HIST_FIELDS:
|
||||
tracked_check = _TRACKED_CHECK.format(field=field)
|
||||
blocks.append(
|
||||
f" INSERT INTO DevicesHistory "
|
||||
f"(devGUID, changedColumn, oldValue, newValue, changedBy, timestamp)\n"
|
||||
f" SELECT NEW.devGUID, '{field}', "
|
||||
f"CAST(OLD.{field} AS TEXT), CAST(NEW.{field} AS TEXT),\n"
|
||||
f" {attribution},\n"
|
||||
f" datetime('now')\n"
|
||||
f" WHERE (OLD.{field} IS NOT NEW.{field})\n"
|
||||
f" AND COALESCE(NEW.devGUID, '') != ''\n"
|
||||
f" AND {tracked_check};"
|
||||
)
|
||||
body = "\n".join(blocks)
|
||||
return (
|
||||
"CREATE TRIGGER trg_devhist_update\n"
|
||||
"AFTER UPDATE ON Devices\n"
|
||||
"FOR EACH ROW\n"
|
||||
f"WHEN {_HIST_DAYS_GUARD}\n"
|
||||
"BEGIN\n"
|
||||
f"{body}\n"
|
||||
"END;"
|
||||
)
|
||||
|
||||
|
||||
def _build_insert_trigger_sql() -> str:
|
||||
"""Generate CREATE TRIGGER SQL for AFTER INSERT on Devices."""
|
||||
blocks = []
|
||||
for field, attribution in _HIST_FIELDS:
|
||||
tracked_check = _TRACKED_CHECK.format(field=field)
|
||||
blocks.append(
|
||||
f" INSERT INTO DevicesHistory "
|
||||
f"(devGUID, changedColumn, oldValue, newValue, changedBy, timestamp)\n"
|
||||
f" SELECT NEW.devGUID, '{field}', NULL, CAST(NEW.{field} AS TEXT),\n"
|
||||
f" {attribution},\n"
|
||||
f" datetime('now')\n"
|
||||
f" WHERE NEW.{field} IS NOT NULL\n"
|
||||
f" AND COALESCE(NEW.devGUID, '') != ''\n"
|
||||
f" AND {tracked_check};"
|
||||
)
|
||||
body = "\n".join(blocks)
|
||||
return (
|
||||
"CREATE TRIGGER trg_devhist_insert\n"
|
||||
"AFTER INSERT ON Devices\n"
|
||||
"FOR EACH ROW\n"
|
||||
f"WHEN {_HIST_DAYS_GUARD}\n"
|
||||
"BEGIN\n"
|
||||
f"{body}\n"
|
||||
"END;"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Public ensure_* functions (called from database.py initDB)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def ensure_deviceshistory_table(sql) -> bool:
|
||||
"""
|
||||
Ensures DevicesHistory table and its indexes exist.
|
||||
|
||||
Also backfills any Devices rows that have NULL/empty devGUID so that
|
||||
the history triggers can write records for them.
|
||||
|
||||
Idempotent — uses IF NOT EXISTS throughout.
|
||||
|
||||
Parameters:
|
||||
sql: database cursor (must support execute()).
|
||||
"""
|
||||
try:
|
||||
sql.execute("""
|
||||
CREATE TABLE IF NOT EXISTS DevicesHistory (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
devGUID TEXT NOT NULL,
|
||||
timestamp DATETIME DEFAULT CURRENT_TIMESTAMP,
|
||||
changedBy TEXT NOT NULL,
|
||||
changedColumn TEXT NOT NULL,
|
||||
oldValue TEXT,
|
||||
newValue TEXT,
|
||||
FOREIGN KEY (devGUID) REFERENCES Devices(devGUID) ON DELETE CASCADE
|
||||
)
|
||||
""")
|
||||
|
||||
sql.execute(
|
||||
"CREATE INDEX IF NOT EXISTS idx_devhist_guid_column "
|
||||
"ON DevicesHistory(devGUID, changedColumn)"
|
||||
)
|
||||
sql.execute(
|
||||
"CREATE INDEX IF NOT EXISTS idx_devhist_timestamp "
|
||||
"ON DevicesHistory(timestamp)"
|
||||
)
|
||||
|
||||
# Backfill any Devices rows that are missing a GUID so the history
|
||||
# triggers can record changes for them.
|
||||
sql.execute("""
|
||||
UPDATE Devices
|
||||
SET devGUID = lower(
|
||||
hex(randomblob(4)) || '-' || hex(randomblob(2)) || '-' || '4' ||
|
||||
substr(hex(randomblob(2)), 2) || '-' ||
|
||||
substr('AB89', 1 + (abs(random()) % 4), 1) ||
|
||||
substr(hex(randomblob(2)), 2) || '-' ||
|
||||
hex(randomblob(6))
|
||||
)
|
||||
WHERE devGUID IS NULL OR TRIM(devGUID) = ''
|
||||
""")
|
||||
|
||||
mylog("verbose", ["[db_history] DevicesHistory table and indexes ensured"])
|
||||
return True
|
||||
except Exception as e:
|
||||
mylog("none", [f"[db_history] ERROR ensuring DevicesHistory table: {e}"])
|
||||
return False
|
||||
|
||||
|
||||
def ensure_deviceshistory_triggers(sql) -> bool:
|
||||
"""
|
||||
Drops and recreates the AFTER UPDATE and AFTER INSERT triggers on Devices.
|
||||
|
||||
Always drops first so that trigger logic is refreshed on every upgrade.
|
||||
|
||||
Parameters:
|
||||
sql: database cursor (must support execute()).
|
||||
"""
|
||||
try:
|
||||
sql.execute("DROP TRIGGER IF EXISTS trg_devhist_update")
|
||||
sql.execute("DROP TRIGGER IF EXISTS trg_devhist_insert")
|
||||
|
||||
update_sql = _build_update_trigger_sql()
|
||||
insert_sql = _build_insert_trigger_sql()
|
||||
|
||||
sql.execute(update_sql)
|
||||
sql.execute(insert_sql)
|
||||
|
||||
mylog("verbose", ["[db_history] DevicesHistory triggers created"])
|
||||
return True
|
||||
except Exception as e:
|
||||
mylog("none", [f"[db_history] ERROR creating DevicesHistory triggers: {e}"])
|
||||
return False
|
||||
@@ -21,6 +21,7 @@ from scheduler import schedule_class
|
||||
from plugin import plugin_manager, print_plugin_info
|
||||
from utils.plugin_utils import get_plugins_configs, get_set_value_for_init
|
||||
from messaging.in_app import write_notification
|
||||
from utils.plugin_utils import list_to_csv
|
||||
|
||||
# ===============================================================================
|
||||
# Language helpers
|
||||
@@ -327,6 +328,38 @@ def importConfigs(pm, db, all_plugins):
|
||||
"[]",
|
||||
"General",
|
||||
)
|
||||
conf.DEV_HIST_DAYS = ccd(
|
||||
"DEV_HIST_DAYS",
|
||||
14,
|
||||
c_d,
|
||||
"Device history retention (days)",
|
||||
'{"dataType":"integer", "elements": [{"elementType" : "input", "elementOptions" : [{"type": "number"}] ,"transformers": []}]}',
|
||||
"[]",
|
||||
"General",
|
||||
desc="Number of days to retain device change history. Set to 0 to completely disable the auditing engine.",
|
||||
)
|
||||
_DEV_HIST_TRACKED_DEFAULT = [
|
||||
'devMac', 'devName', 'devOwner', 'devType', 'devVendor', 'devFavorite',
|
||||
'devGroup', 'devComments', 'devLastIP', 'devFQDN', 'devPrimaryIPv4',
|
||||
'devPrimaryIPv6', 'devVlan', 'devForceStatus', 'devStaticIP', 'devScan',
|
||||
'devAlertDown', 'devCanSleep', 'devSkipRepeated', 'devLocation',
|
||||
'devIsArchived', 'devParentMAC', 'devParentPort', 'devParentRelType',
|
||||
'devReqNicsOnline', 'devIcon', 'devSite', 'devSSID', 'devSyncHubNode',
|
||||
'devSourcePlugin', 'devMacSource', 'devNameSource', 'devFQDNSource',
|
||||
'devLastIPSource', 'devVendorSource', 'devSSIDSource',
|
||||
'devParentMACSource', 'devParentPortSource', 'devParentRelTypeSource',
|
||||
'devVlanSource', 'devCustomProps',
|
||||
]
|
||||
conf.DEV_HIST_TRACKED = ccd(
|
||||
"DEV_HIST_TRACKED",
|
||||
_DEV_HIST_TRACKED_DEFAULT,
|
||||
c_d,
|
||||
"Device history tracked fields",
|
||||
'{"dataType":"array","elements":[{"elementType":"select","elementHasInputValue":1,"elementOptions":[{"multiple":"true","orderable":"true"}],"transformers":[]},{"elementType":"button","elementOptions":[{"sourceSuffixes":[]},{"separator":""},{"cssClasses":"col-xs-12"},{"onClick":"selectChange(this)"},{"getStringKey":"Gen_Change"}],"transformers":[]}]}', # noqa: E501
|
||||
list_to_csv(_DEV_HIST_TRACKED_DEFAULT),
|
||||
"General",
|
||||
desc="List of Devices column names to monitor for changes. Add or remove field names to control what gets audited.",
|
||||
)
|
||||
conf.HRS_TO_KEEP_NEWDEV = ccd(
|
||||
"HRS_TO_KEEP_NEWDEV",
|
||||
0,
|
||||
|
||||
216
server/models/device_history_instance.py
Normal file
216
server/models/device_history_instance.py
Normal file
@@ -0,0 +1,216 @@
|
||||
"""
|
||||
device_history_instance.py — Query, group, and prune DevicesHistory records.
|
||||
|
||||
Writes are handled entirely by SQLite triggers (trg_devhist_update /
|
||||
trg_devhist_insert) created in db/db_history.py. This class is read/prune only.
|
||||
"""
|
||||
|
||||
from database import get_temp_db_connection
|
||||
from logger import mylog
|
||||
|
||||
|
||||
class DevicesHistoryInstance:
|
||||
|
||||
# -------------------------------------------------------------------------
|
||||
# DB helpers (mirrors DeviceInstance pattern)
|
||||
# -------------------------------------------------------------------------
|
||||
|
||||
def _fetchall(self, query, params=()):
|
||||
conn = get_temp_db_connection()
|
||||
rows = conn.execute(query, params).fetchall()
|
||||
conn.close()
|
||||
return [dict(r) for r in rows]
|
||||
|
||||
def _execute(self, query, params=()):
|
||||
conn = get_temp_db_connection()
|
||||
cur = conn.cursor()
|
||||
cur.execute(query, params)
|
||||
affected = cur.rowcount
|
||||
conn.commit()
|
||||
conn.close()
|
||||
return affected
|
||||
|
||||
# -------------------------------------------------------------------------
|
||||
# Public query API
|
||||
# -------------------------------------------------------------------------
|
||||
|
||||
def get_grouped_history(self, devGUID, changedColumn=None, changedBy=None,
|
||||
limit=50, offset=0):
|
||||
"""
|
||||
Return grouped change history for a single device.
|
||||
|
||||
Rows are grouped by (timestamp, changedBy, devGUID). Pagination is
|
||||
applied at the group level so that `limit` controls how many distinct
|
||||
change events are returned, not how many individual field rows.
|
||||
|
||||
Args:
|
||||
devGUID: Device GUID to filter on (required).
|
||||
changedColumn: Optional field-name filter — only groups that contain
|
||||
a change to this column are included.
|
||||
changedBy: Optional source filter (e.g. 'USER', 'ARPSCAN').
|
||||
limit: Max number of grouped events to return.
|
||||
offset: Number of grouped events to skip (for pagination).
|
||||
|
||||
Returns:
|
||||
list[dict] each with keys: devGUID, timestamp, changedBy, changes
|
||||
"""
|
||||
return self._query_grouped(
|
||||
devGUID=devGUID,
|
||||
changedColumn=changedColumn,
|
||||
changedBy=changedBy,
|
||||
limit=limit,
|
||||
offset=offset,
|
||||
)
|
||||
|
||||
def get_all_grouped_history(self, changedColumn=None, changedBy=None,
|
||||
limit=100, offset=0):
|
||||
"""
|
||||
Return grouped change history across all devices (global view).
|
||||
|
||||
Same behaviour as get_grouped_history but without a devGUID filter.
|
||||
"""
|
||||
return self._query_grouped(
|
||||
devGUID=None,
|
||||
changedColumn=changedColumn,
|
||||
changedBy=changedBy,
|
||||
limit=limit,
|
||||
offset=offset,
|
||||
)
|
||||
|
||||
def get_available_filter_values(self, devGUID=None):
|
||||
"""
|
||||
Return distinct changedBy and changedColumn values present in the table.
|
||||
|
||||
Used by the frontend to dynamically populate filter dropdowns.
|
||||
|
||||
Args:
|
||||
devGUID: If provided, scope to a single device.
|
||||
|
||||
Returns:
|
||||
dict with keys 'changedBy' and 'changedColumn', each a sorted list.
|
||||
"""
|
||||
where = "WHERE devGUID = ?" if devGUID else ""
|
||||
params = (devGUID,) if devGUID else ()
|
||||
|
||||
by_rows = self._fetchall(
|
||||
f"SELECT DISTINCT changedBy FROM DevicesHistory {where} ORDER BY changedBy",
|
||||
params,
|
||||
)
|
||||
col_rows = self._fetchall(
|
||||
f"SELECT DISTINCT changedColumn FROM DevicesHistory {where} ORDER BY changedColumn",
|
||||
params,
|
||||
)
|
||||
return {
|
||||
"changedBy": [r["changedBy"] for r in by_rows],
|
||||
"changedColumn": [r["changedColumn"] for r in col_rows],
|
||||
}
|
||||
|
||||
def get_total_group_count(self, devGUID=None, changedColumn=None, changedBy=None):
|
||||
"""
|
||||
Return the total number of distinct change-event groups matching the
|
||||
given filters. Used by the frontend to calculate total pages.
|
||||
"""
|
||||
clauses, params = self._build_clauses(devGUID, changedColumn, changedBy)
|
||||
where = f"WHERE {' AND '.join(clauses)}" if clauses else ""
|
||||
|
||||
rows = self._fetchall(
|
||||
f"""
|
||||
SELECT COUNT(*) AS cnt FROM (
|
||||
SELECT DISTINCT timestamp, changedBy, devGUID
|
||||
FROM DevicesHistory {where}
|
||||
)
|
||||
""",
|
||||
tuple(params),
|
||||
)
|
||||
return rows[0]["cnt"] if rows else 0
|
||||
|
||||
# -------------------------------------------------------------------------
|
||||
# Retention management
|
||||
# -------------------------------------------------------------------------
|
||||
|
||||
def prune_history(self, days):
|
||||
"""
|
||||
Delete DevicesHistory rows older than `days` days.
|
||||
|
||||
Called by the DBCLNP plugin during scheduled cleanup.
|
||||
|
||||
Args:
|
||||
days: Retention window. Rows with timestamp older than this are
|
||||
deleted.
|
||||
|
||||
Returns:
|
||||
int: Number of rows deleted.
|
||||
"""
|
||||
if days <= 0:
|
||||
mylog("verbose", ["[DevicesHistory] prune skipped — DEV_HIST_DAYS is 0"])
|
||||
return 0
|
||||
|
||||
affected = self._execute(
|
||||
"DELETE FROM DevicesHistory WHERE timestamp < datetime('now', ?)",
|
||||
(f"-{days} days",),
|
||||
)
|
||||
mylog("verbose", [f"[DevicesHistory] pruned {affected} rows older than {days} days"])
|
||||
return affected
|
||||
|
||||
# -------------------------------------------------------------------------
|
||||
# Internal helpers
|
||||
# -------------------------------------------------------------------------
|
||||
|
||||
@staticmethod
|
||||
def _build_clauses(devGUID, changedColumn, changedBy):
|
||||
clauses = []
|
||||
params = []
|
||||
if devGUID:
|
||||
clauses.append("devGUID = ?")
|
||||
params.append(devGUID)
|
||||
if changedColumn:
|
||||
clauses.append("changedColumn = ?")
|
||||
params.append(changedColumn)
|
||||
if changedBy:
|
||||
clauses.append("changedBy = ?")
|
||||
params.append(changedBy)
|
||||
return clauses, params
|
||||
|
||||
def _query_grouped(self, devGUID, changedColumn, changedBy, limit, offset):
|
||||
"""
|
||||
Core fetch-and-group logic shared by all public query methods.
|
||||
|
||||
1. Fetch matching rows ordered by timestamp DESC.
|
||||
2. Group by (timestamp, changedBy, devGUID) in Python.
|
||||
3. Apply limit/offset at the group level.
|
||||
"""
|
||||
clauses, params = self._build_clauses(devGUID, changedColumn, changedBy)
|
||||
where = f"WHERE {' AND '.join(clauses)}" if clauses else ""
|
||||
|
||||
rows = self._fetchall(
|
||||
f"""
|
||||
SELECT devGUID, timestamp, changedBy, changedColumn, oldValue, newValue
|
||||
FROM DevicesHistory
|
||||
{where}
|
||||
ORDER BY timestamp DESC, changedBy, changedColumn
|
||||
""",
|
||||
tuple(params),
|
||||
)
|
||||
|
||||
# Group by (timestamp, changedBy, devGUID) — preserving DESC order
|
||||
groups = {}
|
||||
order = []
|
||||
|
||||
for row in rows:
|
||||
key = (row["timestamp"], row["changedBy"], row["devGUID"])
|
||||
if key not in groups:
|
||||
groups[key] = {
|
||||
"devGUID": row["devGUID"],
|
||||
"timestamp": row["timestamp"],
|
||||
"changedBy": row["changedBy"],
|
||||
"changes": [],
|
||||
}
|
||||
order.append(key)
|
||||
groups[key]["changes"].append({
|
||||
"changedColumn": row["changedColumn"],
|
||||
"oldValue": row["oldValue"],
|
||||
"newValue": row["newValue"],
|
||||
})
|
||||
|
||||
grouped_list = [groups[k] for k in order]
|
||||
return grouped_list[offset: offset + limit]
|
||||
@@ -3,6 +3,7 @@ import base64
|
||||
import re
|
||||
import sqlite3
|
||||
import csv
|
||||
import uuid
|
||||
from io import StringIO
|
||||
from front.plugins.plugin_helper import is_mac, normalize_mac
|
||||
from logger import mylog
|
||||
@@ -724,7 +725,7 @@ class DeviceInstance:
|
||||
data.get("devLastConnection") or timeNowUTC(),
|
||||
data.get("devFirstConnection") or timeNowUTC(),
|
||||
data.get("devLastIP") or "",
|
||||
data.get("devGUID") or "",
|
||||
data.get("devGUID") or str(uuid.uuid4()),
|
||||
data.get("devCustomProps") or "",
|
||||
data.get("devSourcePlugin") or "DUMMY",
|
||||
data.get("devForceStatus") or "dont_force",
|
||||
|
||||
293
test/backend/test_device_history.py
Normal file
293
test/backend/test_device_history.py
Normal file
@@ -0,0 +1,293 @@
|
||||
"""
|
||||
Unit tests for the DevicesHistory feature.
|
||||
|
||||
Tests cover:
|
||||
- AFTER INSERT trigger: new device logs tracked fields with oldValue=None
|
||||
- AFTER UPDATE trigger: field change is logged with correct attribution
|
||||
- AFTER UPDATE trigger: no-op when DEV_HIST_DAYS=0
|
||||
- AFTER UPDATE trigger: field not in DEV_HIST_TRACKED is not logged
|
||||
- DevicesHistoryInstance.get_grouped_history: groups and paginates correctly
|
||||
- DevicesHistoryInstance.get_all_grouped_history: returns multi-device results
|
||||
- DevicesHistoryInstance.get_available_filter_values: distinct values returned
|
||||
- DevicesHistoryInstance.get_total_group_count: correct count
|
||||
- DevicesHistoryInstance.prune_history: deletes old rows, skips at days=0
|
||||
"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
import unittest
|
||||
import unittest.mock
|
||||
from datetime import datetime, timezone, timedelta
|
||||
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "..", "server"))
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
|
||||
|
||||
from db_test_helpers import make_history_db
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Shared fixture helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_TRACKED = "['devName', 'devVendor', 'devLastIP', 'devNameSource']"
|
||||
|
||||
_INSERT_SQL = """
|
||||
INSERT INTO Devices
|
||||
(devMac, devGUID, devName, devVendor, devLastIP, devNameSource,
|
||||
devAlertDown, devPresentLastScan, devIsNew, devIsArchived,
|
||||
devFirstConnection, devLastConnection)
|
||||
VALUES (?, ?, ?, ?, ?, ?, 0, 1, 0, 0,
|
||||
datetime('now'), datetime('now'))
|
||||
"""
|
||||
|
||||
|
||||
def _insert_device(conn, mac, guid, name="Router", vendor="Cisco",
|
||||
last_ip="192.168.1.1", name_source="ARPSCAN"):
|
||||
conn.execute(_INSERT_SQL, (mac, guid, name, vendor, last_ip, name_source))
|
||||
conn.commit()
|
||||
|
||||
|
||||
def _history_rows(conn, guid=None, column=None):
|
||||
"""Fetch DevicesHistory rows, optionally filtered."""
|
||||
clauses = []
|
||||
params = []
|
||||
if guid:
|
||||
clauses.append("devGUID = ?")
|
||||
params.append(guid)
|
||||
if column:
|
||||
clauses.append("changedColumn = ?")
|
||||
params.append(column)
|
||||
where = "WHERE " + " AND ".join(clauses) if clauses else ""
|
||||
return conn.execute(
|
||||
f"SELECT * FROM DevicesHistory {where} ORDER BY id", params
|
||||
).fetchall()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Test cases
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestInsertTrigger(unittest.TestCase):
|
||||
"""AFTER INSERT trigger fires for tracked fields on new device creation."""
|
||||
|
||||
def setUp(self):
|
||||
self.conn = make_history_db(tracked=_TRACKED)
|
||||
|
||||
def tearDown(self):
|
||||
self.conn.close()
|
||||
|
||||
def test_tracked_fields_logged_on_insert(self):
|
||||
_insert_device(self.conn, "aa:bb:cc:00:00:01", "guid-insert-1",
|
||||
name="Router", vendor="Cisco")
|
||||
rows = _history_rows(self.conn, guid="guid-insert-1")
|
||||
columns_logged = {r["changedColumn"] for r in rows}
|
||||
# devName and devVendor should always be logged (non-NULL in INSERT)
|
||||
self.assertIn("devName", columns_logged)
|
||||
self.assertIn("devVendor", columns_logged)
|
||||
|
||||
def test_old_value_is_null_on_insert(self):
|
||||
_insert_device(self.conn, "aa:bb:cc:00:00:02", "guid-insert-2")
|
||||
rows = _history_rows(self.conn, guid="guid-insert-2", column="devName")
|
||||
self.assertEqual(len(rows), 1)
|
||||
self.assertIsNone(rows[0]["oldValue"])
|
||||
self.assertEqual(rows[0]["newValue"], "Router")
|
||||
|
||||
def test_source_attribution_from_source_field(self):
|
||||
_insert_device(self.conn, "aa:bb:cc:00:00:03", "guid-insert-3",
|
||||
name_source="ARPSCAN")
|
||||
rows = _history_rows(self.conn, guid="guid-insert-3", column="devName")
|
||||
self.assertEqual(rows[0]["changedBy"], "ARPSCAN")
|
||||
|
||||
def test_untracked_field_not_logged(self):
|
||||
# devOwner is not in _TRACKED so its change must not appear
|
||||
_insert_device(self.conn, "aa:bb:cc:00:00:04", "guid-insert-4")
|
||||
rows = _history_rows(self.conn, guid="guid-insert-4")
|
||||
logged_cols = {r["changedColumn"] for r in rows}
|
||||
self.assertNotIn("devOwner", logged_cols)
|
||||
|
||||
|
||||
class TestUpdateTrigger(unittest.TestCase):
|
||||
"""AFTER UPDATE trigger fires with correct values and attribution."""
|
||||
|
||||
def setUp(self):
|
||||
self.conn = make_history_db(tracked=_TRACKED)
|
||||
_insert_device(self.conn, "aa:bb:cc:01:00:01", "guid-upd-1")
|
||||
|
||||
def tearDown(self):
|
||||
self.conn.close()
|
||||
|
||||
def test_name_change_logged(self):
|
||||
self.conn.execute(
|
||||
"UPDATE Devices SET devName='My Router', devNameSource='USER' WHERE devGUID=?",
|
||||
("guid-upd-1",)
|
||||
)
|
||||
self.conn.commit()
|
||||
rows = _history_rows(self.conn, guid="guid-upd-1", column="devName")
|
||||
# Two rows: INSERT (old=None) and UPDATE (old=Router)
|
||||
update_rows = [r for r in rows if r["oldValue"] is not None]
|
||||
self.assertEqual(len(update_rows), 1)
|
||||
self.assertEqual(update_rows[0]["oldValue"], "Router")
|
||||
self.assertEqual(update_rows[0]["newValue"], "My Router")
|
||||
self.assertEqual(update_rows[0]["changedBy"], "USER")
|
||||
|
||||
def test_plugin_attribution(self):
|
||||
self.conn.execute(
|
||||
"UPDATE Devices SET devVendor='Netgear', devVendorSource='VNDRPDT' WHERE devGUID=?",
|
||||
("guid-upd-1",)
|
||||
)
|
||||
self.conn.commit()
|
||||
rows = _history_rows(self.conn, guid="guid-upd-1", column="devVendor")
|
||||
update_rows = [r for r in rows if r["oldValue"] is not None]
|
||||
self.assertEqual(update_rows[0]["changedBy"], "VNDRPDT")
|
||||
|
||||
def test_no_change_no_row(self):
|
||||
before = len(_history_rows(self.conn, guid="guid-upd-1"))
|
||||
# Update with same value
|
||||
self.conn.execute(
|
||||
"UPDATE Devices SET devName='Router' WHERE devGUID=?",
|
||||
("guid-upd-1",)
|
||||
)
|
||||
self.conn.commit()
|
||||
after = len(_history_rows(self.conn, guid="guid-upd-1"))
|
||||
self.assertEqual(before, after)
|
||||
|
||||
|
||||
class TestDisabledGuard(unittest.TestCase):
|
||||
"""AFTER UPDATE trigger is silent when DEV_HIST_DAYS=0."""
|
||||
|
||||
def test_no_rows_when_disabled(self):
|
||||
conn = make_history_db(dev_hist_days=0, tracked=_TRACKED)
|
||||
_insert_device(conn, "aa:bb:cc:02:00:01", "guid-dis-1")
|
||||
conn.execute(
|
||||
"UPDATE Devices SET devName='Changed', devNameSource='USER' WHERE devGUID=?",
|
||||
("guid-dis-1",)
|
||||
)
|
||||
conn.commit()
|
||||
rows = _history_rows(conn)
|
||||
conn.close()
|
||||
self.assertEqual(len(rows), 0)
|
||||
|
||||
|
||||
class TestDevicesHistoryInstance(unittest.TestCase):
|
||||
"""Tests for DevicesHistoryInstance query/prune methods."""
|
||||
|
||||
def setUp(self):
|
||||
self.conn = make_history_db(tracked=_TRACKED)
|
||||
|
||||
# `device_history_instance` uses `from database import get_temp_db_connection`
|
||||
# which creates a local reference. We must patch THAT namespace, not the
|
||||
# database module attribute.
|
||||
import models.device_history_instance as _mod
|
||||
|
||||
_real = self.conn
|
||||
|
||||
class _NoClose:
|
||||
def execute(self, *a, **kw): return _real.execute(*a, **kw)
|
||||
def cursor(self): return _real.cursor()
|
||||
def commit(self): _real.commit()
|
||||
def rollback(self): _real.rollback()
|
||||
def close(self): pass # intentional no-op
|
||||
|
||||
self._patcher = unittest.mock.patch.object(
|
||||
_mod, "get_temp_db_connection", return_value=_NoClose()
|
||||
)
|
||||
self._patcher.start()
|
||||
|
||||
_insert_device(self.conn, "bb:cc:dd:00:00:01", "guid-q-1",
|
||||
name="Alpha", vendor="Cisco")
|
||||
_insert_device(self.conn, "bb:cc:dd:00:00:02", "guid-q-2",
|
||||
name="Beta", vendor="HP")
|
||||
# One user update on guid-q-1
|
||||
self.conn.execute(
|
||||
"UPDATE Devices SET devName='Alpha v2', devNameSource='USER' WHERE devGUID='guid-q-1'"
|
||||
)
|
||||
self.conn.commit()
|
||||
|
||||
from models.device_history_instance import DevicesHistoryInstance
|
||||
self.h = DevicesHistoryInstance()
|
||||
|
||||
def tearDown(self):
|
||||
self._patcher.stop()
|
||||
self.conn.close()
|
||||
|
||||
def test_get_grouped_history_returns_groups(self):
|
||||
groups = self.h.get_grouped_history("guid-q-1")
|
||||
self.assertGreater(len(groups), 0)
|
||||
for g in groups:
|
||||
self.assertEqual(g["devGUID"], "guid-q-1")
|
||||
self.assertIn("timestamp", g)
|
||||
self.assertIn("changedBy", g)
|
||||
self.assertIsInstance(g["changes"], list)
|
||||
|
||||
def test_changedcolumn_filter(self):
|
||||
groups = self.h.get_grouped_history("guid-q-1", changedColumn="devName")
|
||||
for g in groups:
|
||||
cols = [c["changedColumn"] for c in g["changes"]]
|
||||
self.assertIn("devName", cols)
|
||||
|
||||
def test_changedby_filter(self):
|
||||
groups = self.h.get_grouped_history("guid-q-1", changedBy="USER")
|
||||
self.assertGreater(len(groups), 0)
|
||||
for g in groups:
|
||||
self.assertEqual(g["changedBy"], "USER")
|
||||
|
||||
def test_pagination(self):
|
||||
all_groups = self.h.get_grouped_history("guid-q-1")
|
||||
if len(all_groups) < 2:
|
||||
self.skipTest("Not enough groups to test pagination")
|
||||
page1 = self.h.get_grouped_history("guid-q-1", limit=1, offset=0)
|
||||
page2 = self.h.get_grouped_history("guid-q-1", limit=1, offset=1)
|
||||
self.assertEqual(len(page1), 1)
|
||||
self.assertEqual(len(page2), 1)
|
||||
# Pages must represent distinct groups (same key = (timestamp, changedBy, devGUID))
|
||||
key1 = (page1[0]["timestamp"], page1[0]["changedBy"])
|
||||
key2 = (page2[0]["timestamp"], page2[0]["changedBy"])
|
||||
self.assertNotEqual(key1, key2)
|
||||
|
||||
def test_get_all_grouped_history(self):
|
||||
groups = self.h.get_all_grouped_history()
|
||||
guids = {g["devGUID"] for g in groups}
|
||||
self.assertIn("guid-q-1", guids)
|
||||
self.assertIn("guid-q-2", guids)
|
||||
|
||||
def test_get_available_filter_values(self):
|
||||
filters = self.h.get_available_filter_values("guid-q-1")
|
||||
self.assertIn("changedBy", filters)
|
||||
self.assertIn("changedColumn", filters)
|
||||
self.assertIsInstance(filters["changedBy"], list)
|
||||
self.assertIsInstance(filters["changedColumn"], list)
|
||||
self.assertGreater(len(filters["changedColumn"]), 0)
|
||||
|
||||
def test_get_total_group_count(self):
|
||||
total = self.h.get_total_group_count("guid-q-1")
|
||||
groups = self.h.get_grouped_history("guid-q-1", limit=1000)
|
||||
self.assertEqual(total, len(groups))
|
||||
|
||||
def test_prune_history_zero_skips(self):
|
||||
deleted = self.h.prune_history(0)
|
||||
self.assertEqual(deleted, 0)
|
||||
|
||||
def test_prune_history_removes_old_rows(self):
|
||||
# Manually insert an old row
|
||||
self.conn.execute(
|
||||
"""INSERT INTO DevicesHistory (devGUID, changedColumn, oldValue, newValue, changedBy, timestamp)
|
||||
VALUES ('guid-q-1', 'devName', 'old', 'new', 'system', datetime('now', '-30 days'))"""
|
||||
)
|
||||
self.conn.commit()
|
||||
old_count = self.conn.execute("SELECT COUNT(*) FROM DevicesHistory").fetchone()[0]
|
||||
deleted = self.h.prune_history(14)
|
||||
new_count = self.conn.execute("SELECT COUNT(*) FROM DevicesHistory").fetchone()[0]
|
||||
self.assertGreater(deleted, 0)
|
||||
self.assertLess(new_count, old_count)
|
||||
|
||||
def test_prune_history_keeps_recent_rows(self):
|
||||
before = self.conn.execute("SELECT COUNT(*) FROM DevicesHistory").fetchone()[0]
|
||||
# prune with 365 days — nothing should be deleted from a fresh test DB
|
||||
deleted = self.h.prune_history(365)
|
||||
after = self.conn.execute("SELECT COUNT(*) FROM DevicesHistory").fetchone()[0]
|
||||
self.assertEqual(deleted, 0)
|
||||
self.assertEqual(before, after)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -7,6 +7,7 @@ Import from any test subdirectory with:
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
|
||||
from db_test_helpers import make_db, insert_device, minutes_ago, DummyDB, down_event_macs, make_device_dict, sync_insert_devices
|
||||
from db_test_helpers import make_plugin_db, make_plugin_dict, make_plugin_event_row, seed_plugin_object, plugin_history_rows, plugin_objects_rows, PluginFakeDB
|
||||
from db_test_helpers import make_history_db
|
||||
"""
|
||||
|
||||
import sqlite3
|
||||
@@ -17,6 +18,7 @@ from datetime import datetime, timezone, timedelta
|
||||
# Make the 'server' package importable when this module is loaded directly.
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "server"))
|
||||
from db.db_upgrade import ensure_views # noqa: E402
|
||||
from db.db_history import ensure_deviceshistory_table, ensure_deviceshistory_triggers # noqa: E402
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -144,6 +146,41 @@ def make_db(sleep_minutes: int = 30) -> sqlite3.Connection:
|
||||
return conn
|
||||
|
||||
|
||||
def make_history_db(dev_hist_days: int = 14,
|
||||
tracked: str = "['devName', 'devVendor', 'devLastIP', 'devNameSource']") -> sqlite3.Connection:
|
||||
"""
|
||||
Return an in-memory SQLite connection with DevicesHistory fully set up.
|
||||
|
||||
Extends make_db() by adding the DevicesHistory table, indexes, and the
|
||||
AFTER UPDATE / AFTER INSERT triggers. Settings rows for DEV_HIST_DAYS
|
||||
and DEV_HIST_TRACKED are pre-inserted.
|
||||
|
||||
DEV_HIST_TRACKED is stored as a Python list literal string (e.g.
|
||||
\"['devName','devVendor']\") matching the settings engine and the SQLite
|
||||
trigger's instr(..., '''field''') membership check.
|
||||
|
||||
Args:
|
||||
dev_hist_days: Value for DEV_HIST_DAYS setting (0 disables tracking).
|
||||
tracked: Python list literal string for DEV_HIST_TRACKED.
|
||||
"""
|
||||
conn = make_db()
|
||||
cur = conn.cursor()
|
||||
cur.execute(
|
||||
"INSERT OR REPLACE INTO Settings (setKey, setValue) VALUES (?, ?)",
|
||||
("DEV_HIST_DAYS", str(dev_hist_days)),
|
||||
)
|
||||
cur.execute(
|
||||
"INSERT OR REPLACE INTO Settings (setKey, setValue) VALUES (?, ?)",
|
||||
("DEV_HIST_TRACKED", tracked),
|
||||
)
|
||||
conn.commit()
|
||||
ensure_deviceshistory_table(cur)
|
||||
conn.commit()
|
||||
ensure_deviceshistory_triggers(cur)
|
||||
conn.commit()
|
||||
return conn
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Time helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
Reference in New Issue
Block a user