mirror/NetAlertX
1
0
Fork 0
mirror of https://github.com/jokob-sk/NetAlertX.git synced 2026-06-14 11:26:12 -04:00
Code Issues Packages Projects Releases Wiki Activity

BE: Removal of stdout.log

Browse Source
This commit is contained in:
jokob-sk
2026-06-14 10:35:08 +10:00
parent 491225dd83
commit 76e7cc7b3d
22 changed files with 80 additions and 57 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
.devcontainer/Dockerfile
View File

@@ -89,7 +89,6 @@ ENV LOG_STDERR=${NETALERTX_LOG}/stderr.log
ENV LOG_APP_PHP_ERRORS=${NETALERTX_LOG}/app.php_errors.log
ENV LOG_EXECUTION_QUEUE=${NETALERTX_LOG}/execution_queue.log
ENV LOG_REPORT_OUTPUT_JSON=${NETALERTX_LOG}/report_output.json
ENV LOG_STDOUT=${NETALERTX_LOG}/stdout.log
ENV LOG_CRON=${NETALERTX_LOG}/cron.log
ENV LOG_NGINX_ERROR=${NETALERTX_LOG}/nginx-error.log

1
Dockerfile
View File

@@ -86,7 +86,6 @@ ENV LOG_STDERR=${NETALERTX_LOG}/stderr.log
ENV LOG_APP_PHP_ERRORS=${NETALERTX_LOG}/app.php_errors.log
ENV LOG_EXECUTION_QUEUE=${NETALERTX_LOG}/execution_queue.log
ENV LOG_REPORT_OUTPUT_JSON=${NETALERTX_LOG}/report_output.json
ENV LOG_STDOUT=${NETALERTX_LOG}/stdout.log
ENV LOG_CRON=${NETALERTX_LOG}/cron.log
ENV LOG_NGINX_ERROR=${NETALERTX_LOG}/nginx-error.log

1
Dockerfile.debian
View File

@@ -61,7 +61,6 @@ ENV LOG_STDERR=${NETALERTX_LOG}/stderr.log
ENV LOG_APP_PHP_ERRORS=${NETALERTX_LOG}/app.php_errors.log
ENV LOG_EXECUTION_QUEUE=${NETALERTX_LOG}/execution_queue.log
ENV LOG_REPORT_OUTPUT_JSON=${NETALERTX_LOG}/report_output.json
ENV LOG_STDOUT=${NETALERTX_LOG}/stdout.log
ENV LOG_CRON=${NETALERTX_LOG}/cron.log
ENV LOG_NGINX_ERROR=${NETALERTX_LOG}/nginx-error.log

3
docs/API_LOGS.md
View File

@@ -12,14 +12,13 @@ Only specific, pre-approved log files can be purged for security and stability r
**Query Parameter:**
* `file` → The name of the log file to purge (e.g., `app.log`, `stdout.log`)
* `file` → The name of the log file to purge (e.g., `app.log`)
**Allowed Files:**
```
app.log
IP_changes.log
stdout.log
stderr.log
app.php_errors.log
execution_queue.log

1
docs/API_OLD.md
View File

@@ -352,7 +352,6 @@ This API endpoint retrieves files from the `/tmp/log` folder.
| `report_output.json` | JSON format report output |
| `report_output.txt` | Text format report output |
| `stderr.log` | Logs of standard error output |
| `stdout.log` | Logs of standard output |
## API Endpoint: /config files

1
docs/DOCKER_COMPOSE.md
View File

@@ -170,6 +170,7 @@ However, if you prefer to have direct, file-level access to your configuration f
```
**After (Using a Local Folder / Bind Mount):**
Make sure to replace `/local_data_dir` with your actual path. The format is `<path_on_your_computer>:<path_inside_container>:<options>`.
```yaml

2
docs/PLUGINS_DEV.md
View File

@@ -352,7 +352,7 @@ See: [UI Components](PLUGINS_DEV_UI_COMPONENTS.md)
- **Settings Helper:** `/app/server/helper.py` - Use `get_setting_value()` in scripts
- **Example Plugins:** `/app/front/plugins/*/` - Study working implementations
- **Logs:** `/tmp/log/plugins/` - Plugin output and execution logs
- **Backend Logs:** `/tmp/log/stdout.log` - Core system logs
- **Backend Logs:** `/tmp/log/app.log` - Core system logs
---

2
docs/PLUGINS_DEV_DATASOURCES.md
View File

@@ -364,7 +364,7 @@ sqlite> SELECT ... ;
```bash
# Watch backend logs
tail -f /tmp/log/stdout.log | grep -i "data_source\|MYPREFIX"
tail -f /tmp/log/app.log | grep -i "data_source\|MYPREFIX"
```
---

2
docs/PLUGINS_DEV_DATA_CONTRACT.md
View File

@@ -245,7 +245,7 @@ cat /tmp/log/plugins/last_result.YOURPREFIX.log | awk -F'|' '{print NF}' | sort
**Check core processing in logs:**
```bash
tail -f /tmp/log/stdout.log | grep -i "YOURPREFIX\|Plugins_Objects"
tail -f /tmp/log/app.log | grep -i "YOURPREFIX\|Plugins_Objects"
```
## See Also

4
docs/PLUGINS_DEV_QUICK_START.md
View File

@@ -143,7 +143,7 @@ ls -la /tmp/log/plugins/last_result.MYPLN.log
cat /tmp/log/plugins/last_result.MYPLN.log
# Check backend logs for errors
tail -f /tmp/log/stdout.log | grep "my_plugin\|MYPLN"
tail -f /tmp/log/app.log | grep "my_plugin\|MYPLN"
```
## Next Steps
@@ -162,7 +162,7 @@ Now that you have a working basic plugin:
|-------|----------|
| "Module not found" errors | Ensure `sys.path` includes `/app/server` and `/app/front/plugins` |
| Settings not appearing | Restart backend and clear browser cache |
| Results not showing up | Check `/tmp/log/plugins/*.log` and `/tmp/log/stdout.log` for errors |
| Results not showing up | Check `/tmp/log/plugins/*.log` and `/tmp/log/app.log` for errors |
| Permission denied | Plugin runs in container, use absolute paths like `/app/front/plugins/...` |
## Resources

2
docs/PLUGINS_DEV_SETTINGS.md
View File

@@ -507,7 +507,7 @@ mylog('none', f"Setting value: {value}")
Check backend logs:
```bash
tail -f /tmp/log/stdout.log | grep -i "setting\|MYPLN"
tail -f /tmp/log/app.log | grep -i "setting\|MYPLN"
```
## See Also

11
front/php/components/logs_defaults.json
View File

@@ -70,17 +70,6 @@
"filePath": "__NETALERTX_LOG__/db_is_locked.log",
"textAreaCssClass": "logs logs-small"
},
{
"buttons": [
{
"labelStringCode": "Maint_PurgeLog",
"event": "logManage('stdout.log', 'cleanLog')"
}
],
"fileName": "stdout.log",
"filePath": "__NETALERTX_LOG__/stdout.log",
"textAreaCssClass": "logs logs-small"
},
{
"buttons": [
{

77
front/plugins/maintenance/maintenance.py
View File

@@ -2,7 +2,6 @@
import os
import sys
from collections import deque
# Register NetAlertX directories
INSTALL_PATH = os.getenv('NETALERTX_APP', '/app')
@@ -36,34 +35,49 @@ def main():
MAINT_LOG_LENGTH = int(get_setting_value('MAINT_LOG_LENGTH'))
MAINT_NOTI_LENGTH = int(get_setting_value('MAINT_NOTI_LENGTH'))
logFiles = ["app.log", "nginx-error.log", "stdout.log"]
logFiles = ["app.log", "nginx-error.log"]
# Check if set
if MAINT_LOG_LENGTH != 0:
MAX_TAIL_SIZE = 0.1 * 1024 * 1024 # 0.1 MB
for fileEntry in logFiles:
mylog('verbose', [f'[{pluginName}] Cleaning file'])
logFile = os.path.join(logPath, fileEntry)
logFile = logPath + "/" + fileEntry
if not os.path.isfile(logFile):
mylog('verbose', [f'[{pluginName}] File not found: {fileEntry}'])
continue
mylog('verbose', [f'[{pluginName}] {fileEntry} size BEFORE: {os.path.getsize(logFile)}'])
size_before = os.path.getsize(logFile)
# Using a deque to efficiently keep the last N lines
lines_to_keep = deque(maxlen=MAINT_LOG_LENGTH)
mylog('verbose', [f'[{pluginName}] {fileEntry} size BEFORE: {size_before}'])
with open(logFile, 'r') as file:
# Read lines from the file and store the last N lines
for line in file:
lines_to_keep.append(line)
try:
with open(logFile, 'w') as file:
# Write the last N lines back to the file
file.writelines(lines_to_keep)
if size_before > MAX_TAIL_SIZE:
mylog('verbose', [f'[{pluginName}] {fileEntry} size AFTER: {os.path.getsize(logFile)}'])
mylog('verbose', [f'[{pluginName}] {fileEntry} exceeds {MAX_TAIL_SIZE} bytes, truncating'])
mylog('verbose', [f'[{pluginName}] Cleanup of {fileEntry} finished'])
with open(logFile, 'r+b') as f:
f.truncate(0)
else:
lines_to_keep = tail_file(logFile, MAINT_LOG_LENGTH)
with open(logFile, 'r+b') as f:
f.seek(0)
f.truncate()
f.writelines(lines_to_keep)
size_after = os.path.getsize(logFile)
mylog('verbose', [f'[{pluginName}] {fileEntry} size AFTER: {size_after}'])
except Exception as e:
mylog('none', [f'[{pluginName}] Failed to clean {fileEntry}: {e}'])
# Check if set
if MAINT_NOTI_LENGTH != 0:
@@ -88,6 +102,37 @@ def main():
return 0
def tail_file(filepath, num_lines):
"""
Return the last num_lines lines from a file without reading the entire file.
"""
if num_lines <= 0:
return []
block_size = 8192
with open(filepath, 'rb') as f:
f.seek(0, os.SEEK_END)
file_size = f.tell()
blocks = []
lines_found = 0
position = file_size
while position > 0 and lines_found <= num_lines:
read_size = min(block_size, position)
position -= read_size
f.seek(position)
block = f.read(read_size)
blocks.append(block)
lines_found += block.count(b'\n')
data = b''.join(reversed(blocks))
return data.splitlines(keepends=True)[-num_lines:]
# ===============================================================================
# BEGIN
# ===============================================================================

2
install/debian12/start.debian12.sh
View File

@@ -112,7 +112,7 @@ fi
# Create an empty log files
# Create the execution_queue.log file if it doesn't exist
touch "${INSTALL_DIR}"/log/{app.log,execution_queue.log,app_front.log,app.php_errors.log,stderr.log,stdout.log,db_is_locked.log}
touch "${INSTALL_DIR}"/log/{app.log,execution_queue.log,app_front.log,app.php_errors.log,stderr.log,db_is_locked.log}
touch "${INSTALL_DIR}"/api/user_notifications.json
# Create plugins sub-directory if it doesn't exist in case a custom log folder is used
mkdir -p "${INSTALL_DIR}"/log/plugins

4
install/production-filesystem/services/start-backend.sh
View File

@@ -11,6 +11,6 @@ done
# Force kill if graceful shutdown failed
killall -KILL python3 &>/dev/null
echo "Starting python3 $(cat /services/config/python/backend-extra-launch-parameters 2>/dev/null) -m server > ${NETALERTX_LOG}/stdout.log 2> >(tee ${NETALERTX_LOG}/stderr.log >&2)"
read -ra EXTRA_PARAMS < <(cat /services/config/python/backend-extra-launch-parameters 2>/dev/null)
exec python3 "${EXTRA_PARAMS[@]}" -m server > "${NETALERTX_LOG}/stdout.log" 2> >(tee "${NETALERTX_LOG}/stderr.log" >&2)
echo "Starting python3 ${EXTRA_PARAMS[*]} -m server (stdout+stderr merged)"
exec python3 "${EXTRA_PARAMS[@]}" -m server 2>&1

4
install/proxmox/proxmox-install-netalertx.sh
View File

@@ -307,7 +307,7 @@ mkdir -p "${INSTALL_DIR}"/log/plugins
chown -R www-data:www-data "${INSTALL_DIR}"/log/plugins
# Create the execution_queue.log file if it doesn't exist
touch ${INSTALL_DIR}/log/{app.log,execution_queue.log,app_front.log,app.php_errors.log,stderr.log,stdout.log,db_is_locked.log}
touch ${INSTALL_DIR}/log/{app.log,execution_queue.log,app_front.log,app.php_errors.log,stderr.log,db_is_locked.log}
touch ${INSTALL_DIR}/api/user_notifications.json
chown -R www-data:www-data "${INSTALL_DIR}"/log "${INSTALL_DIR}"/api
chmod -R ug+rwX "${INSTALL_DIR}"/log "${INSTALL_DIR}"/api
@@ -319,7 +319,7 @@ chown www-data:www-data "${INSTALL_DIR}/log" "${INSTALL_DIR}/api"
mkdir -p "${INSTALL_DIR}/log/plugins"
# Create log and api files directly as the www-data user to ensure correct ownership from the start.
sudo -u www-data touch ${INSTALL_DIR}/log/{app.log,execution_queue.log,app_front.log,app.php_errors.log,stderr.log,stdout.log,db_is_locked.log}
sudo -u www-data touch ${INSTALL_DIR}/log/{app.log,execution_queue.log,app_front.log,app.php_errors.log,stderr.log,db_is_locked.log}
sudo -u www-data touch ${INSTALL_DIR}/api/user_notifications.json
# Set final permissions for all created files and directories.

2
install/ubuntu24/install.sh
View File

@@ -276,7 +276,7 @@ mount -t tmpfs -o noexec,nosuid,nodev tmpfs "${INSTALL_DIR}/api"
# Create log files if they don't exist
echo "[INSTALL] Creating log files if they don't exist"
touch "${INSTALL_DIR}"/log/{app.log,execution_queue.log,app_front.log,app.php_errors.log,stderr.log,stdout.log,db_is_locked.log}
touch "${INSTALL_DIR}"/log/{app.log,execution_queue.log,app_front.log,app.php_errors.log,stderr.log,db_is_locked.log}
touch "${INSTALL_DIR}"/api/user_notifications.json
# Create plugins sub-directory if it doesn't exist in case a custom log folder is used
mkdir -p "${INSTALL_DIR}"/log/plugins

2
server/api_server/logs_endpoint.py
View File

@@ -26,7 +26,7 @@ def clean_log(log_file):
flask.Response: JSON response with success and message keys
"""
allowed_files = [
'app.log', 'app_front.log', 'IP_changes.log', 'stdout.log', 'stderr.log',
'app.log', 'app_front.log', 'IP_changes.log', 'stderr.log',
'app.php_errors.log', 'execution_queue.log', 'db_is_locked.log'
]

2
server/api_server/mcp_endpoint.py
View File

@@ -812,7 +812,7 @@ def _list_resources() -> List[Dict[str, Any]]:
# Log files
log_files = [
("stdout.log", "Backend stdout log"),
("app.log", "Backend log"),
("stderr.log", "Backend stderr log"),
("app_front.log", "Frontend commands log"),
("app.php_errors.log", "PHP errors log")

2
server/api_server/openapi/schemas.py
View File

@@ -57,7 +57,7 @@ NOTIFICATION_LEVELS = Literal["info", "warning", "error", "alert", "interrupt"]
ALLOWED_TABLES = Literal["Devices", "Events", "Sessions", "Settings", "CurrentScan", "Online_History", "Plugins_Objects", "Plugins_History"]
ALLOWED_LOG_FILES = Literal[
"app.log", "app_front.log", "IP_changes.log", "stdout.log", "stderr.log",
"app.log", "app_front.log", "IP_changes.log", "stderr.log",
"app.php_errors.log", "execution_queue.log", "db_is_locked.log"
]

4
server/helper.py
View File

@@ -17,7 +17,7 @@ import ipaddress
import conf
from const import applicationPath, fullConfPath, fullDbPath, dbPath, confPath, apiPath
from logger import mylog, logResult
from logger import mylog
# Register NetAlertX directories using runtime configuration
INSTALL_PATH = applicationPath
@@ -70,7 +70,7 @@ def initialiseFile(pathToCheck, defaultFile):
mylog("none", ["[Setup] (" + defaultFile + ") copied over successfully to (" + pathToCheck + ")."],)
# write stdout and stderr into .log files for debugging if needed
logResult(stdout, stderr) # TO-DO should be changed to mylog
# logResult(stdout, stderr) # TO-DO should be changed to mylog
except subprocess.CalledProcessError as e:
# An error occured, handle it

7
server/logger.py
View File

@@ -146,13 +146,6 @@ def append_file_binary(file_path, input_data):
file.write(input_data)
def logResult(stdout, stderr):
if stderr is not None:
append_file_binary(logPath + "/stderr.log", stderr)
if stdout is not None:
append_file_binary(logPath + "/stdout.log", stdout)
def append_line_to_file(pPath, pText):
if sys.version_info < (3, 0):
file = io.open(pPath, mode="a", encoding="utf-8")