mirror/weewx
1
0
Fork 0
mirror of https://github.com/weewx/weewx.git synced 2026-04-21 01:57:00 -04:00
Code Issues Packages Projects Releases Wiki Activity

Remove wee_database and references to it

Browse Source
This commit is contained in:
Tom Keffer
2023-07-19 15:27:56 -07:00
parent 90dd6c90c7
commit 3d22889851
13 changed files with 129 additions and 1686 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
TODO.md
View File

@@ -20,7 +20,6 @@ When doing `weectl station create`, install everything under the `util` subdirec
- Allow rebuild daily summaries for only selected types.
- Check semantics of specifying a time for `--from` and/or `--to`
in `weectl database calc-missing`.
- Remove the old `wee_database` application and any references to it.
- Make sure output of `--help` matches documentation.
## Logging

1170
bin/wee_database.py
View File

File diff suppressed because it is too large Load Diff

9
bin/weectllib/database_cmd.py
View File

@@ -72,9 +72,8 @@ For example:
"""
reconfigure_description = """Create a new database using the current configuration information
found in the configuration file. This can be used to change the unit system of a
database. The new database will have the same name as the old database, with a '_new'
on the end."""
found in the configuration file. This can be used to change the unit system of a database. The new
database will have the same name as the old database, except with a '_new' on the end."""
transfer_description = """Copy a database to a new database.
The option "--dest-binding" should hold a database binding
@@ -197,7 +196,9 @@ def add_subparser(subparsers):
reconfigure_parser = action_parser.add_parser('reconfigure',
description=reconfigure_description,
usage=reconfigure_usage,
help="Reconfigure a database.",
help="Reconfigure a database, using the current "
"configuration information in the config "
"file.",
epilog=epilog)
_add_common_args(reconfigure_parser)
reconfigure_parser.set_defaults(func=reconfigure_database)

3
bin/weectllib/db_actions.py
View File

@@ -233,7 +233,8 @@ def reconfigure_database(config_path,
db_binding='wx_binding',
dry_run=False):
"""Create a new database, then populate it with the contents of an old database, but use
the current configuration options."""
the current configuration options. The reconfigure action will create a new database with the
same name as the old, except with the suffix _new attached to the end."""
config_path, config_dict, database_name = _prepare(config_path, db_binding, dry_run)
weectllib.initialize(config_dict)

223
docs_src/custom/database.md
View File

@@ -143,91 +143,83 @@ schema instead of the default.
Thereafter, WeeWX reads the schema directly from the database and your
changes will have no effect!
## Modifying an existing database
## Modify the schema of an existing database
The previous section covers the case where you do not have an existing
database, so you modify a starting schema, then use it to initialize the
database. But, what if you already have a database, and you want to
modify it, perhaps by adding a column or two? You cannot create a new
starting schema, because it is only used when the database is first
created. Here is where the tool
[`wee_database`](../../utilities/utilities.htm#wee_database_utility) can be
useful. Be sure to stop WeeWX before attempting to use it.
The previous section covers the case where you do not have an existing database,
so you modify a starting schema, then use it to initialize the database. But,
what if you already have a database, and you want to modify its schema, perhaps
by adding a column or two? Creating a new starting schema is not going to work
because it is only used when the database is first created. Here is where the
command [`weectl database`](/utilities/weectl-database) can be useful.
There are two ways to do this. Both are covered below.
1. Modify the database *in situ* by using the tool
`wee_database`. This choice works best for small changes.
2. Transfer the old database to a new one while modifying it along the
way, again by using the tool `wee_database`. This choice is
best for large modifications.
1. Modify the database *in situ*. This choice works best for small changes.
2. Reconfigure the old database to a new one while modifying it along the
way, This choice is best for large modifications.
!!! Warning
Before using the tool `wee_database`, MAKE A BACKUP FIRST!
Before using `weectl database`, MAKE A BACKUP FIRST!
### Modify the database *in situ* {#add_archive_type}
Be sure to stop `weewxd` first.
If you want to make some minor modifications to an existing database,
perhaps adding or removing a column, then this can easily be done using
the tool `wee_database` with an appropriate option. We will cover
the cases of adding, removing, and renaming a type. See the
documentation for
[`wee_database`](../../utilities/utilities.htm#wee_database_utility) for more
details.
### Modify the database *in situ*
#### Adding a type
If you want to make some minor modifications to an existing database, perhaps
adding or removing a column, then this can easily be done using the command
`weectl database` with an appropriate action. We will cover the cases of
adding, removing, and renaming a type. See the documentation for [`weectl
database`](/utilities/weectl-database) for more details.
Suppose you have an existing database and you want to add a type, such
as the type `electricity` from the example
[*Adding a second data source*](../service-engine/#Adding_2nd_source).
This can be done in one easy
step using the tool `wee_database` with the option
`--add-column`:
#### Adding a type {#add_archive_type}
Suppose you have an existing database, to which you want to add a type, such as
the type `electricity` from the example [*Adding a second data
source*](../service-engine/#Adding_2nd_source). This can be done in one easy
step using the action `weectl database add-column`:
``` shell
wee_database --add-column=electricity
weectl database add-column electricity
```
The tool not only adds `electricity` to the main archive table,
but also to the daily summaries.
The tool not only adds `electricity` to the main archive table, but also to the
daily summaries.
#### Removing a type {#remove_archive_type}
In a similar manner, the tool can remove any unneeded types from an
existing database. For example, suppose you are using the
`schemas.wview` schema, but you're pretty sure you're not going
to need to store soil moisture. You can drop the unnecessary types this
way:
In a similar manner, the tool can remove any unneeded types from an existing
database. For example, suppose you are using the `schemas.wview` schema, but
you're pretty sure you're not going to need to store soil moisture. You can drop
the unnecessary types this way:
``` shell
wee_database --drop-columns=soilMoist1,soilMoist2,soilMoist3,soilMoist4
weectl database drop-columns soilMoist1 soilMoist2 soilMoist3 soilMoist4
```
Unlike the option `--add-column`, the option
`--drop-columns` can take more than one type. This is done in
the interest of efficiency: adding new columns is easy and fast with the
SQLite database, but dropping columns requires copying the whole
database. By specifying more than one type, you can amortize the cost
over a single invocation of the utility.
Unlike the action `add-column`, the action `drop-columns` can take more than one
type. This is done in the interest of efficiency: adding new columns is easy and
fast with the SQLite database, but dropping columns requires copying the whole
database. By specifying more than one type, you can amortize the cost over a
single invocation of the utility.
!!! Warning
Dropping types from a database means *you will lose any data
associated with them!* The data cannot be recovered.
#### Renaming a type
#### Renaming a type {#rename_archive_type}
Suppose you just want to rename a type? This can be done using the
option `--to-name`. Here's an example where you rename
`soilMoist1` to `soilMoistGarden`:
Suppose you just want to rename a type? This can be done using the action
`rename-column`. Here's an example where you rename `soilMoist1` to
`soilMoistGarden`:
``` shell
wee_database --rename-column=soilMoist1 --to-name=soilMoistGarden
weectl database rename-column soilMoist1 soilMoistGarden
```
Note how the option `--rename-column` also requires option
`--to-name`, which specifies the target name.
Note how the action `rename-column` requires _two_ positional arguments:
the column being renamed, and its final name.
### Transfer database using new schema {#transfer_database_using_new_schema}
### Reconfigure database using a new schema {#reconfigure_database_using_new_schema}
If you are making major changes to your database, you may find it easier
to create a brand-new database using the schema you want, then transfer
@@ -242,9 +234,9 @@ Here is the general strategy to do this.
2. Specify this schema as the starting schema for the database.
3. Make sure you have the necessary permissions to create the new
database.
4. Use the utility
[`wee_database`](../../utilities/utilities.htm#wee_database_utility) to
create the new database and populate it with data from the old
4. Use the action
[`weectl database reconfigure`](/utilities/weectl-database/#reconfigure-a-database)
to create the new database and populate it with data from the old
database.
5. Shuffle databases around so WeeWX will use the new database.
@@ -269,23 +261,22 @@ Here are the details:
schema = user.myschema.schema
```
3. **Check permissions.** The reconfiguration utility will create a new
database with the same name as the old, except with the suffix
`_new` attached to the end. Make sure you have the necessary
permissions to do this. In particular, if you are using MySQL, you
will need `CREATE` privileges.
3. **Check permissions.** The transfer action will create a new
database with the same name as the old, except with the suffix `_new`
attached to the end. Make sure you have the necessary permissions to do
this. In particular, if you are using MySQL, you will need `CREATE`
privileges.
4. **Create and populate the new database.** Use the utility
`wee_database` with the `--reconfigure` option.
4. **Create and populate the new database.** Use the command
`weectl database` with the `reconfigure` action.
``` shell
wee_database weewx.conf --reconfigure
weectl database reconfigure
```
This will create a new database (nominally, `weewx.sdb_new`
if you are using SQLite, `weewx_new` if you are using MySQL),
using the schema found in `user.myschema.schema`, and
populate it with data from the old database.
This will create a new database (nominally, `weewx.sdb_new` if you are using
SQLite, `weewx_new` if you are using MySQL), using the schema found in
`user.myschema.schema`, and populate it with data from the old database.
5. **Shuffle the databases.** Now arrange things so WeeWX can find the
new database.
@@ -293,47 +284,46 @@ Here are the details:
!!! Warning
Make a backup of the data before doing any of the next steps!
You can either shuffle the databases around so the new database has
the same name as the old database, or edit `weewx.conf` to
use the new database name. To do the former:
You can either shuffle the databases around so the new database has the same
name as the old database, or edit `weewx.conf` to use the new database name.
To do the former:
For SQLite:
=== "SQLite"
``` shell
cd ~/weewx-data/archive
mv weewx.sdb_new weewx.sdb
```
``` shell
cd ~/weewx-data/archive
mv weewx.sdb_new weewx.sdb
```
For MySQL:
=== "MySQL"
``` shell
mysql -u <username> --password=<mypassword>
mysql> DROP DATABASE weewx; # Delete the old database
mysql> CREATE DATABASE weewx; # Create a new one with the same name
mysql> RENAME TABLE weewx_new.archive TO weewx.archive; # Rename to the nominal name
```
``` shell
mysql -u <username> --password=<mypassword>
mysql> DROP DATABASE weewx; # Delete the old database
mysql> CREATE DATABASE weewx; # Create a new one with the same name
mysql> RENAME TABLE weewx_new.archive TO weewx.archive; # Rename to the nominal name
```
6. It's worth noting that there's actually a hidden, last step:
rebuilding the daily summaries inside the new database. This will be
done automatically by WeeWX at the next startup. Alternatively, it
done automatically by `weewxd` at the next startup. Alternatively, it
can be done manually using the
[`wee_database`](../../utilities/utilities.htm#wee_database_utility)utility
and the `--rebuild-daily` option:
[`weectl database rebuild-daily`](/utilities/weectl-database/) action:
``` shell
wee_database --rebuild-daily
weectl database rebuild-daily
```
## Changing the unit system in an existing database {#Changing_the_unit_system}
Normally, data are stored in the databases using US Customary units, and
you shouldn't care; it is an "implementation detail". Data can always
be displayed using any set of units you want &mdash; the section
[*Changing unit systems*](../custom_reports/#changing-unit-systems) explains how to change the
reporting units. Nevertheless, there may be special situations where you
wish to store the data in Metric units. For example, you may need to
allow direct programmatic access to the database from another piece of
software that expects metric units.
Normally, data are stored in the databases using US Customary units, and you
shouldn't care; it is an "implementation detail". Data can always be displayed
using any set of units you want &mdash; the section [*Changing unit
systems*](/custom/custom-reports/#changing-unit-systems) explains how to change
the reporting units. Nevertheless, there may be special situations where you
wish to store the data in Metric units. For example, you may need to allow
direct programmatic access to the database from another piece of software that
expects metric units.
You should not change the database unit system midstream. That is, do
not start with one unit system then, some time later, switch to another.
@@ -343,47 +333,38 @@ WeeWX User's Guide. However, you can reconfigure the database by
copying it to a new database, performing the unit conversion along the
way. You then use this new database.
The general strategy is identical to the strategy outlined above in the
section [*Transfer database using new
schema*](#transfer_database_using_new_schema). The only difference is
that instead of specifying a new starting schema, you specify a
different database unit system. This means that instead of steps 1 and 2
above, you edit the configuration file and change option
`target_unit` in section
[`[StdConvert]`](../../usersguide/weewx-config-file/stdconvert-config/) to reflect your
choice. For example, if you are switching to metric units, the option
will look like:
The general strategy is identical to the strategy outlined above in the section
[*Reconfigure database using new
schema*](#reconfigure_database_using_new_schema). The only difference is that
instead of specifying a new starting schema, you specify a different database
unit system. This means that instead of steps 1 and 2 above, you edit the
configuration file and change option `target_unit` in section
[`[StdConvert]`](../../usersguide/weewx-config-file/stdconvert-config/) to
reflect your choice. For example, if you are switching to metric units, the
option will look like:
``` ini
[StdConvert]
target_unit = METRICWX
```
After changing `target_unit`, you then go ahead with the rest of
the steps. That is run `wee_database` with the
`--reconfigure` option, then shuffle the databases.
After changing `target_unit`, you then go ahead with the rest of the steps. That
is run the action `weectl database reconfigure`, then shuffle the databases.
## Rebuilding the daily summaries
The `wee_database` utility can also be used to rebuild the daily
The `weectl database` command can also be used to rebuild the daily
summaries:
``` shell
wee_database weewx.conf --rebuild-daily
weectl database rebuild-daily
```
In most cases this will be sufficient; however, if anomalies remain in
the daily summaries the daily summary tables may be dropped first before
rebuilding:
In most cases this will be sufficient; however, if anomalies remain in the daily
summaries the daily summary tables may be dropped first before rebuilding:
``` shell
wee_database weewx.conf --drop-daily
```
The summaries will automatically be rebuilt the next time WeeWX starts,
or they can be rebuilt with the utility:
``` shell
wee_database weewx.conf --rebuild-daily
weectl database drop-daily
```
Then try again with `weectl database rebuild-daily`.

355
docs_src/utilities/wee_database.md
View File

@@ -1,355 +0,0 @@
# wee_database
The database utility simplifies typical database maintenance operations. For
example, it can rebuild the daily summaries or check a SQLite database for
embedded strings where floats are expected.
Specify `--help` to see how it is used:
```
wee_database --help
```
```
Usage: wee_database --help
wee_database --create
wee_database --reconfigure
wee_database --transfer --dest-binding=BINDING_NAME [--dry-run]
wee_database --add-column=NAME [--type=(REAL|INTEGER)]
wee_database --rename-column=NAME --to-name=NEW_NAME
wee_database --drop-columns=NAME1,NAME2,...
wee_database --check
wee_database --update [--dry-run]
wee_database --drop-daily
wee_database --rebuild-daily [--date=YYYY-mm-dd |
[--from=YYYY-mm-dd] [--to=YYYY-mm-dd]]
[--dry-run]
wee_database --reweight [--date=YYYY-mm-dd |
[--from=YYYY-mm-dd] [--to=YYYY-mm-dd]]
[--dry-run]
wee_database --calc-missing [--date=YYYY-mm-dd |
[--from=YYYY-mm-dd[THH:MM]] [--to=YYYY-mm-dd[THH:MM]]]
wee_database --check-strings
wee_database --fix-strings [--dry-run]
Description:
Manipulate the WeeWX database. Most of these operations are handled
automatically by WeeWX, but they may be useful in special cases.
Options:
-h, --help show this help message and exit
--create Create the WeeWX database and initialize it with the
schema.
--reconfigure Create a new database using configuration information
found in the configuration file. The new database will
have the same name as the old database, with a '_new'
on the end.
--transfer Transfer the WeeWX archive from source database to
destination database.
--add-column=NAME Add new column NAME to database.
--type=TYPE New database column type (INTEGER|REAL) (option --add-
column only). Default is 'REAL'.
--rename-column=NAME Rename the column with name NAME.
--to-name=NEW_NAME New name of the column (option --rename-column only).
--drop-columns=NAME1,NAME2,...
Drop one or more columns. Names must be separated by
commas, with NO SPACES.
--check Check the calculations in the daily summary tables.
--update Update the daily summary tables if required and
recalculate the daily summary maximum windSpeed
values.
--calc-missing Calculate and store any missing derived observations.
--check-strings Check the archive table for null strings that may have
been introduced by a SQL editing program.
--fix-strings Fix any null strings in a SQLite database.
--drop-daily Drop the daily summary tables from a database.
--rebuild-daily Rebuild the daily summaries from data in the archive
table.
--reweight Recalculate the weighted sums in the daily summaries.
--config=CONFIG_FILE Use configuration file CONFIG_FILE.
--date=YYYY-mm-dd This date only (options --calc-missing and --rebuild-
daily only).
--from=YYYY-mm-dd[THH:MM]
Start with this date or date-time (options --calc-
missing and --rebuild-daily only).
--to=YYYY-mm-dd[THH:MM]
End with this date or date-time (options --calc-
missing and --rebuild-daily only).
--binding=BINDING_NAME
The data binding to use. Default is 'wx_binding'.
--dest-binding=BINDING_NAME
The destination data binding (option --transfer only).
--dry-run Print what would happen but do not do it. Default is
False.
```
### `--create`
If the database does not already exist, this action will create it and
initialize it with the schema specified in the WeeWX configuration file.
Because WeeWX does this automatically, this action is rarely needed.
```
wee_database --create
```
### `--reconfigure`
This action is useful for changing the schema or unit system in your database.
It creates a new database with the same name as the old, except with the suffix
`_new` attached at the end (nominally, `weewx.sdb_new` if you are using SQLite,
`weewx_new` if you are using MySQL). It then initializes it with the schema
specified in `weewx.conf`. Finally, it copies over the data from your old
database into the new database.
```
wee_database --reconfigure
```
See the section <a href="customizing.htm#Changing_the_unit_system">Changing
the database unit system in an existing database</a> in the Customization
Guide for step-by-step instructions that use this option.
### `--transfer`
This action is useful for moving your database from one type of database to
another, such as from SQLite to MySQL. To use it, you must have two bindings
specified in your `weewx.conf` configuration file. One will serve as the
source, the other as the destination. Specify the source binding with option
`--binding`, the destination binding with option `--dest-binding`. The
`--binding` option may be omitted in which case the default `wx-binding`
will be used.
```
wee_database --transfer --binding=source_binding --dest-binding=dest_binding
wee_database --transfer --dest-binding=dest_binding
```
See the Wiki for examples of moving data from <a href="https://github.com/weewx/weewx/wiki/Transfer%20from%20sqlite%20to%20MySQL#using-wee_database">SQLite to MySQL</a>, and from <a href="https://github.com/weewx/weewx/wiki/Transfer%20from%20MySQL%20to%20sqlite#using-wee_database">MySQL to SQLite</a>, using `wee_database`.
### `--add-column=NAME`
This action adds a new database observation type (column) to the database. If
used without the `--type` option, the type will default to `REAL`.
```
wee_database --add-column
```
Optionally, option `--type` can be used with a SQL type `REAL`, `INTEGER`,
or any other SQL column definition (such as `INTEGER DEFAULT 0`).
```
wee_database --add-column=NAME --type=TYPE
```
### `--rename-column=NAME`
Use this action to rename a database observation type (column) to a new name.
It requires the option `--to-name`.
```
wee_database --rename-column=NAME --to-name=NEW_NAME
```
For example, to rename the column `luminosity` in your schema to `illuminance`:
```
wee_database --rename-column=luminosity --to-name=illuminance
```
### `--drop-columns=NAME`
This action will drop one or more observation types (columns) from the
database. If more than one column name is given, they should be separated by
commas and <em>no spaces</em>.
It is an error to attempt to drop a non-existing column. In this case, nothing
will be done.
```
wee_database --drop-columns=NAME1,NAME2
```
!!! Note
When dropping columns from a SQLite database, the entire database must be
copied except for the dropped columns. Because this can be quite slow, if
you are dropping more than one column, it is better to do them all in one
pass. This is why option `--drop-columns` accepts more than one name.
### `--check`
This action will check the calculations in the daily summary tables as well as
checking the archive for null strings (refer to `--check-strings`). If the
daily summary tables contain summaries calculated using an old algorithm, the
user is advised to update the daily summary tables using the `--update` action.
If null strings are found the user is advised to fix them using the
`--fix-strings` action.
```
wee_database --check
```
<a id="wee_database_utility_update">FIXME</a>
### `--update`
This action updates the daily summary tables to use interval weighted
calculations as well as recalculating the `windSpeed` maximum daily values and
times. Interval weighted calculations are only applied to the daily summaries
if not previously applied. The update process is irreversible and users are
advised to backup their database before performing this action.
```
wee_database --update
```
For further information on interval weighting and recalculation of daily
`windSpeed` maximum values, see the sections <a href="upgrading.htm#change_to_daily_summaries">Changes to daily summaries</a> and <a href="upgrading.htm#recalculation_of_windspeed">Recalculation of `windSpeed` maximum values</a> in the Upgrade Guide.
### `--drop-daily`
In addition to the regular archive data, every WeeWX database also includes a
daily summary table for each observation type. Because there can be dozens of
observation types, there can be dozens of these daily summaries. It does not
happen very often, but there can be occasions when it's necessary to drop them
all and then rebuild them. Dropping them by hand would be very tedious! This
action does them all at once.
```
wee_database --drop-daily
```
### `--rebuild-daily`
This action is the inverse of action `--drop-daily` in that it rebuilds the
daily summaries from the archive data. In most cases it is not necessary to
drop the daily summary tables using the action `--drop-daily` before rebuilding
them.
The action `--rebuild-daily` accepts a number of date related options,
`--date`, `--from` and `--to` that allow selective rebuilding of the daily
summaries for one or more days rather than for the entire archive history.
These options may be useful if bogus data has been removed from the archive
covering a single day or a period of few days. The daily summaries can then
be rebuilt for this period only, resulting in a faster rebuild and detailed
low/high values and the associated times being retained for unaffected days.
The `--date` option limits the daily summary rebuild to the specified date.
The `--from` and `--to` options may be used together or individually to limit
the daily summary rebuild to a specified period. When used individually the
`--to` option limits the rebuild to the inclusive period from the earliest date
for which there is data in the database through to and including the specified
date. Similarly when used individually the `--from` option limits the rebuild
to the inclusive period from the specified date through until the last date
for which there is data in the database. When the `--from` and `--to` options
are used together the daily summary rebuild is limited to the specified
inclusive period.
```
wee_database --rebuild-daily
wee_database --rebuild-daily --date=YYYY-mm-dd
wee_database --rebuild-daily --from=YYYY-mm-dd
wee_database --rebuild-daily --to=YYYY-mm-dd
wee_database --rebuild-daily --from=YYYY-mm-dd --to=YYYY-mm-dd
```
!!! Note
Whilst the `--from` and `--to` options will accept optional hour and
minutes in the format `THH:MM`, any such hour and minute options are
ignored by the `--rebuild` action as the daily summaries represent whole
days only and it is not possible to partially rebuild a daily summary.
!!! Note
When used with the `--rebuild-daily` action the period defined by `--to`
and `--from` is inclusive and the daily summary tables will be rebuild for
the day defined by `--from` and the day defined by `--to` and all days in
between.
### `--reweight`
As an alternative to dropping and rebuilding the daily summaries, this action
simply rebuilds the weighted daily sums (used to calculate averages) from the
archive data. It does not touch the highs and lows. It is much faster than
`--rebuild-daily`, and has the advantage that the highs and lows remain
unchanged.
Other options are as in `--rebuild-daily`.
### `--calc-missing`
This action calculates derived observations for archive records in the database
and then stores the calculated observations in the database. This can be
useful if erroneous archive data is corrected or some additional observational
data is added to the archive that may alter previously calculated or missing
derived observations.
The period over which the derived observations are calculated can be limited
through use of the `--date`, `--from` and/or `--to` options. When used without
any of these options `--calc-missing` will calculate derived observations for
all archive records in the database. The `--date` option limits the calculation
of derived observations to the specified date only. The `--from` and `--to`
options can be used together to specify the start and end date-time
respectively of the period over which derived observations will be calculated.
If `--from` is used by itself the period is fom the date-time specified up to
and including the last archive record in the database.
If `--to` is used by itself the period is the first archive record in the
database through to the specified date-time.
```
wee_database --calc-missing
wee_database --calc-missing --date=YYYY-mm-dd
wee_database --calc-missing --from=YYYY-mm-dd[THH:MM]
wee_database --calc-missing --to=YYYY-mm-dd[THH:MM]
wee_database --calc-missing --from=YYYY-mm-dd[THH:MM] --to=YYYY-mm-dd[THH:MM]
```
!!! Note
When a `YYYY-mm-dd` date is used as the `--from` option the period used by
`--calc-missing` includes records after midnight at the start of
`YYYY-mm-dd`, if a `YYYY-mm-ddTHH:MM` format date-time is used as the
`--from` option the period includes records after `YYYY-mm-dd HH:MM`.
When a `YYYY-mm-dd` date is used as the `--to` option the period includes
records up to and including midnight at the end of `YYYY-mm-dd`, if a
`YYYY-mm-ddTHH:MM` format date-time is used as the `--to` option the period
includes records up to and including `YYYY-mm-dd HH:MM`. When using the
`--date` option the period is all records after midnight at the start of
`YYYY-mm-dd` up to and including midnight at the end of `YYYY-mm-dd`, in
effect the same as `--from=YYYY-mm-ddT00:00 --to=YYYY-mm-dd+1T00:00`.
!!! Note
`--calc-missing` uses the `StdWXCalculate` service to calculate missing
derived observations. The data binding used by the `StdWXCalculate` service
should normally match the data binding of the database being operated on by
`--calc-missing`. Those who use custom or additional data bindings should
take care to ensure the correct data bindings are used by both
`--calc-missing` and the `StdWXCalculate` service. Those who use the
default data binding need take no special precautions.
### `--check-strings`
Normally, all entries in the archive are numbers. However, some SQLite
database editors use a null string instead of a null value when deleting
entries. These null strings can cause problems. This action checks the
database to see if it contains any null strings.
```
wee_database --check-strings
```
### `--fix-strings`
This action will check for any null strings in a SQLite database and if found
substitute a true null value.
```
wee_database --fix-strings
```

2
docs_src/utilities/wee_import.md
View File

@@ -3199,7 +3199,7 @@ Aug 22 14:38:28 stretch12 weewx[863]: manager: unable to add record 2018-09-04 0
database. The simplicity of this process will depend on your ability to use SQL, the amount of data
imported, and whether the imported data was dispersed amongst existing. Once contaminated data have been
removed the daily summary tables will need to be rebuilt using the <span
class="code">wee_database</span> utility.
class="code">weectl database rebuild-daily</span> utility.
</li>
<li>Delete the database and start over. For SQLite, simply delete the database file. For MySQL, drop the

32
docs_src/utilities/weectl-database.md
View File

@@ -6,22 +6,6 @@ Specify `--help` to see the various actions and options:
weectl database --help
## Common options
These are options used by most of the actions.
### --config
Path to the configuration file. Default is `~/weewx-data/weewx.conf`.
### --binding
The database binding to use. Default is `wx_binding`.
### --dry-run
Show what would happen if the action was run, but do not actually make any writable changes.
## Create a new database
weectl database create
@@ -240,5 +224,21 @@ highs and lows remain unchanged.
Other options are as in `weectl database rebuild-daily`.
## Options
These are options used by most of the actions.
### --config
Path to the configuration file. Default is `~/weewx-data/weewx.conf`.
### --binding
The database binding to use. Default is `wx_binding`.
### --dry-run
Show what would happen if the action was run, but do not actually make any writable changes.

1
mkdocs.yml
View File

@@ -108,7 +108,6 @@ nav:
- weectl station: utilities/weectl-station.md
- weectl extension: utilities/weectl-extension.md
- weectl database: utilities/weectl-database.md
- wee_database: utilities/wee_database.md
- wee_debug: utilities/wee_debug.md
- wee_device: utilities/wee_device.md
- wee_import: utilities/wee_import.md

2
pkg/debian/rules
View File

@@ -17,7 +17,7 @@ DST_SYSTEMDDIR=$(DST)/etc/systemd/system
DST_CFGDIR =$(DST)/etc/weewx
# these are the entry points
ENTRIES=weewxd weectl wee_database wee_debug wee_device wee_import wee_reports
ENTRIES=weewxd weectl wee_debug wee_device wee_import wee_reports
%:
dh $@ --with python3

6
pkg/weewx.spec.in
View File

@@ -62,7 +62,7 @@
%global cfg_file %{dst_cfg_dir}/weewx.conf
%global dst_user_dir %{dst_bin_dir}/user
%define entry_points weewxd wee_config wee_database wee_debug wee_device wee_extension wee_import wee_reports
%define entry_points weewxd weectl wee_debug wee_device wee_import wee_reports
Summary: weather software
Name: weewx
@@ -213,11 +213,9 @@ rm -rf %{buildroot}
%{_sysconfdir}/default/weewx
%{dst_bin_dir}/
%{_bindir}/weewxd
%{_bindir}/wee_config
%{_bindir}/wee_database
%{_bindir}/weectl
%{_bindir}/wee_debug
%{_bindir}/wee_device
%{_bindir}/wee_extension
%{_bindir}/wee_import
%{_bindir}/wee_reports
%doc %{dst_doc_dir}/

2
pyproject.toml
View File

@@ -35,7 +35,6 @@ packages = [
{ include = "weeplot", from = "bin" },
{ include = "weeutil", from = "bin" },
{ include = "weewx", from = "bin" },
{ include = "wee_database.py", from = "bin" },
{ include = "wee_debug.py", from = "bin" },
{ include = "wee_device.py", from = "bin" },
{ include = "wee_import.py", from = "bin" },
@@ -71,7 +70,6 @@ pyserial = "^3.4"
pyusb = "^1.0.2"
[tool.poetry.scripts]
wee_database = 'wee_database:main'
wee_debug = 'wee_debug:main'
wee_device = 'wee_device:main'
wee_import = 'wee_import:main'

9
util/scripts/wee_database
View File

@@ -1,9 +0,0 @@
#!/bin/sh
app=wee_database.py
# Get the weewx location and interpreter. Default to something sane, but
# look for overrides from the system defaults.
WEEWX_BINDIR=/usr/share/weewx
WEEWX_PYTHON=python3
[ -r /etc/default/weewx ] && . /etc/default/weewx
exec "$WEEWX_PYTHON" $WEEWX_PYTHON_ARGS "$WEEWX_BINDIR/$app" "$@"