mirror of
https://github.com/weewx/weewx.git
synced 2026-04-18 16:46:56 -04:00
390 lines
14 KiB
Markdown
390 lines
14 KiB
Markdown
# Customizing the database {#archive_database}
|
|
|
|
For most users the database defaults will work just fine. However, there
|
|
may be occasions when you may want to add a new observation type to your
|
|
database, or change its unit system. This section shows you how to do
|
|
this.
|
|
|
|
Every relational database depends on a *schema* to specify which types
|
|
to include in the database. When a WeeWX database is first created, it
|
|
uses a Python version of the schema to initialize the database. However,
|
|
once the database has been created, the schema is read directly from the
|
|
database and the Python version is not used again — any changes to it
|
|
will have no effect. This means that the strategy for modifying the
|
|
schema depends on whether the database already exists.
|
|
|
|
## Specifying a schema for a new database
|
|
|
|
If the database does not exist yet, then you will want to pick an
|
|
appropriate starting schema. If it's not exactly what you want, you can
|
|
modify it to fit your needs before creating the database.
|
|
|
|
### Picking a starting schema
|
|
|
|
WeeWX gives you a choice of three different schemas to choose from when
|
|
creating a new database:
|
|
|
|
| Name | Number of<br/>observation types | Comment |
|
|
|------------------|---------------|----------------------------------------------------------|
|
|
| `schemas.wview.schema`| 49 | The original schema that came with wview. |
|
|
| `schemas.wview_extended.schema` | 111 | A version of the wview schema,<br/>which has been extended with<br/>many new types.<br/>This is the default version. |
|
|
| `schemas.wview_small.schema` | 20 | A minimalist version of the wview schema. |
|
|
|
|
|
|
For most users, the default database schema,
|
|
`schemas.wview_extended.schema`, will work just fine.
|
|
|
|
To specify which schema to use when creating a database, modify option
|
|
`schema` in section `[DataBindings]` in
|
|
`weewx.conf`. For example, suppose you wanted to use the classic
|
|
(and smaller) schema `schemas.wview.schema` instead of the
|
|
default `schemas.wview_extended.schema`. Then the section
|
|
`[DataBindings]` would look like:
|
|
|
|
``` ini hl_lines="6"
|
|
[DataBindings]
|
|
[[wx_binding]]
|
|
database = archive_sqlite
|
|
table_name = archive
|
|
manager = weewx.manager.DaySummaryManager
|
|
schema = schemas.wview.schema
|
|
```
|
|
|
|
Now, when you start WeeWX, it will use this new choice instead of the
|
|
default.
|
|
|
|
!!! Note
|
|
This only works when the database is *first created*. Thereafter,
|
|
WeeWX reads the schema directly from the database. Changing this option
|
|
will have no effect!
|
|
|
|
### Modifying a starting schema {#modify_starting_schema}
|
|
|
|
If none of the three starting schemas that come with WeeWX suits your
|
|
purposes, you can easily create your own. Just pick one of the three
|
|
schemas as a starting point, then modify it. Put the results in the
|
|
`user` subdirectory, where it will be safe from upgrades. For
|
|
example, suppose you like the `schemas.wview_small` schema, but
|
|
you need to store the type `electricity` from the example
|
|
[*Adding a second data source*](../service_engine/#Adding_2nd_source). The type
|
|
`electricity` does not appear in the schema, so you'll have to
|
|
add it before starting up WeeWX. We will call the resulting new schema
|
|
`user.myschema.schema`.
|
|
|
|
If you did a Debian install, here's how you would do this:
|
|
|
|
``` shell
|
|
# Copy the wview_small schema over to the user subdirectory and rename it myschema:
|
|
sudo cp /usr/share/weewx/schemas/wview_small.py /usr/share/weewx/user/myschema.py
|
|
|
|
# Edit it using your favorite text editor
|
|
sudo nano /usr/share/weewx/user/myschema.py
|
|
```
|
|
|
|
If you did a pip install, it can be difficult to find the starting schema because it can be
|
|
buried deep in the Python library tree. It's easier to just download from the git repository and
|
|
start with that:
|
|
|
|
``` shell
|
|
# Download the wview_small schema and rename it to myschema.py
|
|
cd ~/weewx-data/bin/user
|
|
wget https://raw.githubusercontent.com/weewx/weewx/master/bin/schemas/wview_small.py
|
|
mv wview_small.py myschema.py
|
|
|
|
# Edit it using your favorite text editor
|
|
nano myschema.py
|
|
```
|
|
|
|
In `myschema.py` change this:
|
|
|
|
``` tty
|
|
...
|
|
('windchill', 'REAL'),
|
|
('windDir', 'REAL'),
|
|
('windGust', 'REAL'),
|
|
('windGustDir', 'REAL'),
|
|
('windSpeed', 'REAL'),
|
|
]
|
|
```
|
|
|
|
to this
|
|
|
|
``` tty hl_lines="7"
|
|
...
|
|
('windchill', 'REAL'),
|
|
('windDir', 'REAL'),
|
|
('windGust', 'REAL'),
|
|
('windGustDir', 'REAL'),
|
|
('windSpeed', 'REAL'),
|
|
('electricity', 'REAL'),
|
|
]
|
|
```
|
|
|
|
The only change was the addition (==highlighted==) of
|
|
`electricity` to the list of observation names.
|
|
|
|
Now change option `schema` under `[DataBindings]` in
|
|
`weewx.conf` to use your new schema:
|
|
|
|
``` tty hl_lines="6"
|
|
[DataBindings]
|
|
[[wx_binding]]
|
|
database = archive_sqlite
|
|
table_name = archive
|
|
manager = weewx.manager.DaySummaryManager
|
|
schema = user.myschema.schema
|
|
```
|
|
|
|
Start WeeWX. When the new database is created, it will use your modified
|
|
schema instead of the default.
|
|
|
|
!!! Note
|
|
This will only work when the database is first created!
|
|
Thereafter, WeeWX reads the schema directly from the database and your
|
|
changes will have no effect!
|
|
|
|
## Modifying an existing database
|
|
|
|
The previous section covers the case where you do not have an existing
|
|
database, so you modify a starting schema, then use it to initialize the
|
|
database. But, what if you already have a database, and you want to
|
|
modify it, perhaps by adding a column or two? You cannot create a new
|
|
starting schema, because it is only used when the database is first
|
|
created. Here is where the tool
|
|
[`wee_database`](../../utilities/utilities.htm#wee_database_utility) can be
|
|
useful. Be sure to stop WeeWX before attempting to use it.
|
|
|
|
There are two ways to do this. Both are covered below.
|
|
|
|
1. Modify the database *in situ* by using the tool
|
|
`wee_database`. This choice works best for small changes.
|
|
2. Transfer the old database to a new one while modifying it along the
|
|
way, again by using the tool `wee_database`. This choice is
|
|
best for large modifications.
|
|
|
|
!!! Warning
|
|
Before using the tool `wee_database`, MAKE A BACKUP FIRST!
|
|
|
|
### Modify the database *in situ* {#add_archive_type}
|
|
|
|
If you want to make some minor modifications to an existing database,
|
|
perhaps adding or removing a column, then this can easily be done using
|
|
the tool `wee_database` with an appropriate option. We will cover
|
|
the cases of adding, removing, and renaming a type. See the
|
|
documentation for
|
|
[`wee_database`](../../utilities/utilities.htm#wee_database_utility) for more
|
|
details.
|
|
|
|
#### Adding a type
|
|
|
|
Suppose you have an existing database and you want to add a type, such
|
|
as the type `electricity` from the example
|
|
[*Adding a second data source*](../service_engine/#Adding_2nd_source).
|
|
This can be done in one easy
|
|
step using the tool `wee_database` with the option
|
|
`--add-column`:
|
|
|
|
``` shell
|
|
wee_database --add-column=electricity
|
|
```
|
|
|
|
The tool not only adds `electricity` to the main archive table,
|
|
but also to the daily summaries.
|
|
|
|
#### Removing a type {#remove_archive_type}
|
|
|
|
In a similar manner, the tool can remove any unneeded types from an
|
|
existing database. For example, suppose you are using the
|
|
`schemas.wview` schema, but you're pretty sure you're not going
|
|
to need to store soil moisture. You can drop the unnecessary types this
|
|
way:
|
|
|
|
``` shell
|
|
wee_database --drop-columns=soilMoist1,soilMoist2,soilMoist3,soilMoist4
|
|
```
|
|
|
|
Unlike the option `--add-column`, the option
|
|
`--drop-columns` can take more than one type. This is done in
|
|
the interest of efficiency: adding new columns is easy and fast with the
|
|
SQLite database, but dropping columns requires copying the whole
|
|
database. By specifying more than one type, you can amortize the cost
|
|
over a single invocation of the utility.
|
|
|
|
!!! Warning
|
|
Dropping types from a database means *you will lose any data
|
|
associated with them!* The data cannot be recovered.
|
|
|
|
#### Renaming a type
|
|
|
|
Suppose you just want to rename a type? This can be done using the
|
|
option `--to-name`. Here's an example where you rename
|
|
`soilMoist1` to `soilMoistGarden`:
|
|
|
|
``` shell
|
|
wee_database --rename-column=soilMoist1 --to-name=soilMoistGarden
|
|
```
|
|
|
|
Note how the option `--rename-column` also requires option
|
|
`--to-name`, which specifies the target name.
|
|
|
|
### Transfer database using new schema {#transfer_database_using_new_schema}
|
|
|
|
If you are making major changes to your database, you may find it easier
|
|
to create a brand-new database using the schema you want, then transfer
|
|
all data from the old database into the new one. This approach is more
|
|
work, and takes more processing time than the *in situ* strategies
|
|
outlines above, but has the advantage that it leaves behind a record of
|
|
exactly the schema you are using.
|
|
|
|
Here is the general strategy to do this.
|
|
|
|
1. Create a new schema that includes exactly the types that you want.
|
|
2. Specify this schema as the starting schema for the database.
|
|
3. Make sure you have the necessary permissions to create the new
|
|
database.
|
|
4. Use the utility
|
|
[`wee_database`](../../utilities/utilities.htm#wee_database_utility) to
|
|
create the new database and populate it with data from the old
|
|
database.
|
|
5. Shuffle databases around so WeeWX will use the new database.
|
|
|
|
Here are the details:
|
|
|
|
1. **Create a new schema.** First step is to create a new schema with
|
|
exactly the types you want. See the instructions above [*Modify a
|
|
starting schema*](#modify_starting_schema). As an example, suppose
|
|
your new schema is called `user.myschema.schema`.
|
|
|
|
2. **Set as starting schema.** Set your new schema as the starting
|
|
schema with whatever database binding you are working with
|
|
(generally, `wx_binding`). For example:
|
|
|
|
``` ini hl_lines="7"
|
|
[DataBindings]
|
|
|
|
[[wx_binding]]
|
|
database = archive_sqlite
|
|
table_name = archive
|
|
manager = weewx.manager.DaySummaryManager
|
|
schema = user.myschema.schema
|
|
```
|
|
|
|
3. **Check permissions.** The reconfiguration utility will create a new
|
|
database with the same name as the old, except with the suffix
|
|
`_new` attached to the end. Make sure you have the necessary
|
|
permissions to do this. In particular, if you are using MySQL, you
|
|
will need `CREATE` privileges.
|
|
|
|
4. **Create and populate the new database.** Use the utility
|
|
`wee_database` with the `--reconfigure` option.
|
|
|
|
``` shell
|
|
wee_database weewx.conf --reconfigure
|
|
```
|
|
|
|
This will create a new database (nominally, `weewx.sdb_new`
|
|
if you are using SQLite, `weewx_new` if you are using MySQL),
|
|
using the schema found in `user.myschema.schema`, and
|
|
populate it with data from the old database.
|
|
|
|
5. **Shuffle the databases.** Now arrange things so WeeWX can find the
|
|
new database.
|
|
|
|
!!! Warning
|
|
Make a backup of the data before doing any of the next steps!
|
|
|
|
You can either shuffle the databases around so the new database has
|
|
the same name as the old database, or edit `weewx.conf` to
|
|
use the new database name. To do the former:
|
|
|
|
For SQLite:
|
|
|
|
``` shell
|
|
cd ~/weewx-data/archive
|
|
mv weewx.sdb_new weewx.sdb
|
|
```
|
|
|
|
For MySQL:
|
|
|
|
``` shell
|
|
mysql -u <username> --password=<mypassword>
|
|
mysql> DROP DATABASE weewx; # Delete the old database
|
|
mysql> CREATE DATABASE weewx; # Create a new one with the same name
|
|
mysql> RENAME TABLE weewx_new.archive TO weewx.archive; # Rename to the nominal name
|
|
```
|
|
|
|
6. It's worth noting that there's actually a hidden, last step:
|
|
rebuilding the daily summaries inside the new database. This will be
|
|
done automatically by WeeWX at the next startup. Alternatively, it
|
|
can be done manually using the
|
|
[`wee_database`](../../utilities/utilities.htm#wee_database_utility)utility
|
|
and the `--rebuild-daily` option:
|
|
|
|
``` shell
|
|
wee_database --rebuild-daily
|
|
```
|
|
|
|
## Changing the unit system in an existing database {#Changing_the_unit_system}
|
|
|
|
Normally, data are stored in the databases using US Customary units, and
|
|
you shouldn't care; it is an "implementation detail". Data can always
|
|
be displayed using any set of units you want — the section
|
|
[*Changing unit systems*](../custom_reports/#changing-unit-systems) explains how to change the
|
|
reporting units. Nevertheless, there may be special situations where you
|
|
wish to store the data in Metric units. For example, you may need to
|
|
allow direct programmatic access to the database from another piece of
|
|
software that expects metric units.
|
|
|
|
You should not change the database unit system midstream. That is, do
|
|
not start with one unit system then, some time later, switch to another.
|
|
WeeWX cannot handle databases with mixed unit systems — see the
|
|
section [`[StdConvert]`](../../usersguide/weewx-config-file/stdconvert-config/) in the
|
|
WeeWX User's Guide. However, you can reconfigure the database by
|
|
copying it to a new database, performing the unit conversion along the
|
|
way. You then use this new database.
|
|
|
|
The general strategy is identical to the strategy outlined above in the
|
|
section [*Transfer database using new
|
|
schema*](#transfer_database_using_new_schema). The only difference is
|
|
that instead of specifying a new starting schema, you specify a
|
|
different database unit system. This means that instead of steps 1 and 2
|
|
above, you edit the configuration file and change option
|
|
`target_unit` in section
|
|
[`[StdConvert]`](../../usersguide/weewx-config-file/stdconvert-config/) to reflect your
|
|
choice. For example, if you are switching to metric units, the option
|
|
will look like:
|
|
|
|
``` ini
|
|
[StdConvert]
|
|
target_unit = METRICWX
|
|
```
|
|
|
|
After changing `target_unit`, you then go ahead with the rest of
|
|
the steps. That is run `wee_database` with the
|
|
`--reconfigure` option, then shuffle the databases.
|
|
|
|
## Rebuilding the daily summaries
|
|
|
|
The `wee_database` utility can also be used to rebuild the daily
|
|
summaries:
|
|
|
|
``` shell
|
|
wee_database weewx.conf --rebuild-daily
|
|
```
|
|
|
|
In most cases this will be sufficient; however, if anomalies remain in
|
|
the daily summaries the daily summary tables may be dropped first before
|
|
rebuilding:
|
|
|
|
``` shell
|
|
wee_database weewx.conf --drop-daily
|
|
```
|
|
|
|
The summaries will automatically be rebuilt the next time WeeWX starts,
|
|
or they can be rebuilt with the utility:
|
|
|
|
``` shell
|
|
wee_database weewx.conf --rebuild-daily
|
|
```
|
|
|