mirror/weewx
1
0
Fork 0
mirror of https://github.com/weewx/weewx.git synced 2026-04-18 16:46:56 -04:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'master' into development

Browse Source
This commit is contained in:
Tom Keffer
2024-01-26 16:35:12 -08:00
parent 379d99bdb7 d1ac83bc59
commit eb177de541
10 changed files with 83 additions and 154 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
DEV_NOTES.txt
View File

@@ -348,10 +348,11 @@ installations.
signing packages --------------------------------------------------------------
gpg is used to sign the deb and rpm packages. SHA1 is no longer acceptable for
signing, so use SHA256 instead. this should be the default when building on
redhat9 and later. if it is not the default, then it can be forced with a
change to the signing macro in .rpmmacros - add the line --digest-algo sha256
gpg is used to sign the deb repository and rpm packages. SHA1 is no longer
acceptable for signing, so be sure that your gpg keys and the signing command
use SHA256 instead. this should be the default when building on redhat9 and
later. if it is not the default, then it can be forced with a change to the
signing macro in .rpmmacros - add the line --digest-algo sha256
%__gpg_sign_cmd %{__gpg} \
gpg --no-verbose --no-armor \
@@ -363,10 +364,10 @@ change to the signing macro in .rpmmacros - add the line --digest-algo sha256
In the debian world, you sign the repository (specifically the file 'Release'),
not the individual .deb files. So if you need to re-sign, you re-build and
re-sign the repository; there is no need touch the individual .deb files. This
is a bit odd, since the dpkg-build has options -us and -uc that tell it to *not*
sign the .deb files, and if you do not specify -us and -uc, the .deb files are
not signed, but dpkg-build will complain.
re-sign the repository; there is no need touch the individual .deb files.
Signing is controlled by the -us and -uc options to dpkg-build. If you do not
specify those options, then dpkg-build will try to sign the .dsc, .buildinfo,
and .changes files. The .deb itself is not signed.
unit tests --------------------------------------------------------------------

16
TODO.md
View File

@@ -1,3 +1,19 @@
## 5.0.1
- mw using 'sudo weectl' on a deb/rpm install breaks permissions. new files
are owned by root:root instead of weewx:weewx. update the docs to use
"sudo -u weewx weectl ..." for deb/rpm installs, but only if running as
weewx. probably biggest problem is when manipulating database, since
confs and skins will still work when owned root:root. another example
of trying to do things the 'right' way for security ends up causing more
hassle.
- mw doing 'apt purge weewx' does the right thing if your first install was
v5. but if your first install was v4, then doing a purge will destroy
the skins. this is probably because the skins were declared as conffiles
in v4. is there a way to remove that in v5 so that v5 'fixes' the confness
of skins?
## Testing
- mw convert to pytest

14
docs_src/changes.md
View File

@@ -15,18 +15,26 @@ Kranz!
Include backwards compatible reference to `weewx.UnknownType`.
Fixed problem with installing extensions into installations that used V4 config
Fix problem with installing extensions into installations that used V4 config
files that were installed by a package installer.
Fix problem with `weectl device` when using drivers that were installed
using the extension installer. Fixes issue #918.
Allow the use of the tilde (~) prefix with `--config` options.
Fix problem that prevented daily summaries from being rebuilt if they had been
modified by using `weectl database drop-columns`.
Allow the use of the tilde (`~`) prefix with `--config` options.
Minor corrections to the Norwegian translations. Thanks to user Aslak!
PR #919.
Chinese language code changed to `zh`. Fixes issue #912.
Change Chinese language code to `zh`. Fixes issue #912.
Fix bug in redhat/suse scriptlet that incorrectly substituted `{weewx}`
instead of `weewx` in the udev rules file.
In the redhat/suse installers, use `/var/lib/weewx` as `HOME` for user `weewx`.
### 5.0.0 01/14/2024

2
makefile
View File

@@ -229,7 +229,7 @@ debian-changelog:
fi
# use dpkg-buildpackage to create the debian package
# -us -uc - skip gpg signature on .dsc and .changes
# -us -uc - skip gpg signature on .dsc, .buildinfo, and .changes
# the latest version in the debian changelog must match the packaging version
DEBARCH=all
DEBBLDDIR=$(BLDDIR)/weewx-$(VERSION)

16
pkg/debian/postinst
View File

@@ -20,10 +20,10 @@ set -e
cfgfile=/etc/weewx/weewx.conf
cfgapp=/usr/bin/weectl
WEEWX_USER=weewx
WEEWX_GROUP=weewx
WEEWX_HOME=/var/lib/weewx
WEEWX_HTML=/var/www/html/weewx
WEEWX_USER="${WEEWX_USER:-weewx}"
WEEWX_GROUP="${WEEWX_GROUP:-weewx}"
WEEWX_HOME="${WEEWX_HOME:-/var/lib/weewx}"
WEEWX_HTMLDIR="${WEEWX_HTMLDIR:-/var/www/html/weewx}"
WEEWX_USERDIR=/etc/weewx/bin/user
# insert the driver and stanza into the configuration file
@@ -398,10 +398,10 @@ setup_database_dir() {
# create the reporting directory
setup_reporting_dir() {
echo "Configuring reporting directory $WEEWX_HTML"
mkdir -p $WEEWX_HTML
chmod 2775 $WEEWX_HTML
chown -R $WEEWX_USER:$WEEWX_GROUP $WEEWX_HTML
echo "Configuring reporting directory $WEEWX_HTMLDIR"
mkdir -p $WEEWX_HTMLDIR
chmod 2775 $WEEWX_HTMLDIR
chown -R $WEEWX_USER:$WEEWX_GROUP $WEEWX_HTMLDIR
}
# set the permissions on the configuration, skins, and extensions

13
pkg/weewx.spec.in
View File

@@ -127,14 +127,15 @@ done
# if there is already a database directory, then use ownership of that to
# determine what user/group we should use for permissions and running.
# otherwise, use 'weewx' for user and group.
WEEWX_USER=weewx
WEEWX_GROUP=weewx
WEEWX_USER="${WEEWX_USER:-weewx}"
WEEWX_GROUP="${WEEWX_GROUP:-weewx}"
WEEWX_HOME="${WEEWX_HOME:-/var/lib/weewx}"
if [ -d /var/lib/weewx ]; then
WEEWX_USER=$(stat -c "%%U" /var/lib/weewx)
WEEWX_GROUP=$(stat -c "%%G" /var/lib/weewx)
fi
/usr/bin/getent group $WEEWX_GROUP || /usr/sbin/groupadd -r $WEEWX_GROUP
/usr/bin/getent passwd $WEEWX_USER || /usr/sbin/useradd -r -g $WEEWX_GROUP -M -s /sbin/nologin $WEEWX_USER
/usr/bin/getent passwd $WEEWX_USER || /usr/sbin/useradd -r -g $WEEWX_GROUP -M -d $WEEWX_HOME -s /sbin/nologin $WEEWX_USER
if [ $1 -gt 1 ]; then
# this is an upgrade
@@ -166,8 +167,8 @@ get_conf_version() {
}
# figure out which user should own everything
WEEWX_USER=weewx
WEEWX_GROUP=weewx
WEEWX_USER="${WEEWX_USER:-weewx}"
WEEWX_GROUP="${WEEWX_GROUP:-weewx}"
if [ -d /var/lib/weewx ]; then
WEEWX_USER=$(stat -c "%%U" /var/lib/weewx)
WEEWX_GROUP=$(stat -c "%%G" /var/lib/weewx)
@@ -204,7 +205,7 @@ fi
# install the udev rules
if [ -d %{udev_dir} ]; then
sed \
-e "s/GROUP=\"weewx\"/GROUP=\"{$WEEWX_GROUP}\"/" \
-e "s/GROUP=\"weewx\"/GROUP=\"${WEEWX_GROUP}\"/" \
%{dst_cfg_dir}/udev/weewx.rules > %{udev_dir}/60-weewx.rules
fi

43
src/weectllib/database_actions.py
View File

@@ -111,22 +111,37 @@ def rebuild_daily(config_dict,
t1 = time.time()
log.info("Rebuilding daily summaries in database '%s' ..." % database_name)
print("Rebuilding daily summaries in database '%s' ..." % database_name)
if dry_run:
return
msg = f"Rebuilding daily summaries in database '{database_name}' ..."
log.info(msg)
print(msg)
# Open up the database. This will create the tables necessary for the daily
# summaries if they don't already exist:
with weewx.manager.open_manager_with_config(config_dict,
db_binding, initialize=True) as dbm:
# Do the actual rebuild
nrecs, ndays = dbm.backfill_day_summary(start_d=from_d,
stop_d=to_d,
trans_days=20)
# Open the database using the Manager object, which does not use daily summaries. This allows
# us to retrieve the SQL keys without triggering an exception because of the missing daily
# summaries.
with weewx.manager.Manager.open(manager_dict['database_dict'],
manager_dict['table_name']) as db:
sqlkeys = db.sqlkeys
# From the keys, build a schema for the daily summaries that reflects what is in the database
day_summaries_schemas = [(e, 'scalar') for e in sqlkeys
if e not in ('dateTime', 'usUnits', 'interval')]
if 'windSpeed' in sqlkeys:
# For backwards compatibility, include 'wind'
day_summaries_schemas += [('wind', 'vector')]
# Replace the static schema with the one we just built:
manager_dict['schema'] = {'day_summaries': day_summaries_schemas}
# Open up the database. Use initialize=True, so the daily summary tables will be created:
with weewx.manager.open_manager(manager_dict, initialize=True) as dbm:
if dry_run:
nrecs = ndays = 0
else:
# Do the actual rebuild
nrecs, ndays = dbm.backfill_day_summary(start_d=from_d,
stop_d=to_d,
trans_days=20)
tdiff = time.time() - t1
# advise the user/log what we did
log.info("Rebuild of daily summaries in database '%s' complete." % database_name)
log.info(f"Rebuild of daily summaries in database '{database_name}' complete.")
if nrecs:
sys.stdout.flush()
# fix a bit of formatting inconsistency if less than 1000 records
@@ -140,6 +155,8 @@ def rebuild_daily(config_dict,
f"{tdiff:.2f} seconds."
print(msg)
print(f"Rebuild of daily summaries in database '{database_name}' complete.")
elif dry_run:
print("Dry run: no records processed.")
else:
print(f"Daily summaries up to date in '{database_name}'.")

62
src/weeutil/tests/test_timediff.py
View File

@@ -1,62 +0,0 @@
#
# Copyright (c) 2022 Tom Keffer <tkeffer@gmail.com>
#
# See the file LICENSE.txt for your full rights.
#
"""Test routines for weeutil.timediff."""
import time
import unittest
import weewx
from weeutil.timediff import *
start = time.mktime((2013, 3, 10, 0, 0, 0, 0, 0, -1))
record1 = {
'dateTime': start,
'outTemp': 20.0,
}
record2 = {
'dateTime': start + 100,
'outTemp': 21.0,
}
record3 = {
'dateTime': start + 400,
'outTemp': 21.0,
}
class TimeDiffTest(unittest.TestCase):
def test_simple(self):
time_diff = TimeDerivative('outTemp', 300)
self.assertIsNone(time_diff.add_record(record1))
self.assertEqual(time_diff.add_record(record2), .01)
def test_backwards(self):
time_diff = TimeDerivative('outTemp', 300)
self.assertIsNone(time_diff.add_record(record2))
with self.assertRaises(weewx.ViolatedPrecondition):
time_diff.add_record(record1)
def test_no_fwd(self):
time_diff = TimeDerivative('outTemp', 300)
self.assertIsNone(time_diff.add_record(record1))
self.assertIsNone(time_diff.add_record(record1))
def test_too_old(self):
time_diff = TimeDerivative('outTemp', 300)
self.assertIsNone(time_diff.add_record(record1))
self.assertIsNone(time_diff.add_record(record3))
def test_not_there(self):
time_diff = TimeDerivative('outTemp', 300)
with self.assertRaises(weewx.CannotCalculate):
time_diff.add_record({'dateTime': start})
if __name__ == '__main__':
unittest.main()

52
src/weeutil/timediff.py
View File

@@ -1,52 +0,0 @@
#
# Copyright (c) 2019-2021 Tom Keffer <tkeffer@gmail.com>
#
# See the file LICENSE.txt for your full rights.
#
"""Class for calculating time derivatives"""
import weewx
class TimeDerivative(object):
"""Calculate time derivative for a specific observation type."""
def __init__(self, obs_type, stale_age):
"""Initialize.
obs_type: the observation type for which the derivative will be calculated.
stale_age: Derivatives are calculated as a difference over time. This is how old the old value
can be and still be considered useful.
"""
self.obs_type = obs_type
self.stale_age = stale_age
self.old_timestamp = None
self.old_value = None
def add_record(self, record):
"""Add a new record, then return the difference in value divided by the difference in time."""
# If the type does not appear in the incoming record, then we can't calculate the derivative
if self.obs_type not in record:
raise weewx.CannotCalculate(self.obs_type)
derivative = None
if record[self.obs_type] is not None:
# We can't calculate anything if we don't have an old record
if self.old_timestamp:
# Check to make sure the incoming record is later than the retained record
if record['dateTime'] < self.old_timestamp:
raise weewx.ViolatedPrecondition("Records presented out of order (%s vs %s)"
% (record['dateTime'], self.old_timestamp))
# Calculate the time derivative only if there is a delta in time,
# and the old record is not too old.
if record['dateTime'] != self.old_timestamp \
and (record['dateTime'] - self.old_timestamp) <= self.stale_age:
# All OK.
derivative = (record[self.obs_type] - self.old_value) \
/ (record['dateTime'] - self.old_timestamp)
# Save the current values
self.old_timestamp = record['dateTime']
self.old_value = record[self.obs_type]
return derivative

2
src/weewx/manager.py
View File

@@ -164,7 +164,7 @@ class Manager(object):
'host': 'localhost',
'user': 'weewx',
'password': 'weewx-password',
'database_name' : 'weeewx',
'database_name' : 'weewx',
'driver' : 'weedb.mysql'
}