From c552b0f0dfce39b484d50e4c84fa6a7142f4a5bf Mon Sep 17 00:00:00 2001 From: Tom Keffer Date: Sun, 18 Sep 2016 08:08:09 -0700 Subject: [PATCH] Continuing to edit wee_device. --- docs/utilities.htm | 2474 +++++++++++++++++--------------------------- 1 file changed, 972 insertions(+), 1502 deletions(-) diff --git a/docs/utilities.htm b/docs/utilities.htm index 6e9d96d6..f36e661f 100644 --- a/docs/utilities.htm +++ b/docs/utilities.htm @@ -14,10 +14,8 @@
- User's Guide
- Customization Guide
- Upgrade Guide
- Utilities Guide + User's Guide
Customization Guide
Upgrade Guide
Utilities Guide
@@ -31,8 +29,7 @@
- - +
Version: 3.6.0a1 @@ -44,41 +41,30 @@ Version: 3.6.0a1

This is a reference guide to the executable utilities that come with weewx:

    -
  • - wee_config - for changing the configuration file, and configuring new device drivers; +
  • wee_config for changing the configuration + file, and configuring new device drivers;
    • -
  • - wee_database - for reconfiguring the database; +
  • wee_database for reconfiguring the + database;
    • -
  • - wee_debug - for producing debug reports for remote support; +
  • wee_debug for producing debug reports for + remote support;
    • -
  • - wee_device - for configuring your hardware; +
  • wee_device for configuring your hardware;
    • -
  • - wee_extension - for installing and removing extensions; +
  • wee_extension for installing and + removing extensions;
    • -
  • - wee_import - for importing historical data from external sources; +
  • wee_import for importing historical data + from external sources;
    • -
  • - wee_reports - for running reports without running weewx itself; +
  • wee_reports for running reports without + running weewx itself;
    • -
  • - weewxd - the main weewx program; +
  • weewxd the main weewx program;
    • -
  • - wunderfixer - for resending data missing on the Weather Underground site. +
  • wunderfixer for resending data missing on the + Weather Underground site.
@@ -88,22 +74,17 @@ Version: 3.6.0a1

wee_config

- When you install weewx for the first time, - the installation process will prompt you for the most essential - options, such as the type of hardware you are using, latitude, - longitude, or altitude. But, what if you want to change these later? - In particular, what if you want to change device drivers? You could - edit the definitive configuration file, weewx.conf, - by hand — described in detail in the User's Guide - — but it's a big file with lots of nuances. Alternatively, if - you're just changing something simple, you may be able to use the - utility wee_config. -

+ When you install weewx for the first time, the installation process will prompt + you for the most essential options, such as the type of hardware you are using, latitude, longitude, or + altitude. But, what if you want to change these later? In particular, what if you want to change device + drivers? You could edit the definitive configuration file, weewx.conf, by + hand — described in detail in the User's Guide — but it's a big + file with lots of nuances. Alternatively, if you're just changing something simple, you may be able to use + the utility wee_config.

- Before starting, it's worth running it with the --help option to see how it is used: -

+ Before starting, it's worth running it with the --help option to see how it is + used: 

wee_config --help

This results in:

Usage: wee_config --help
@@ -174,30 +155,19 @@ Options:

- The utility is pretty good about "guessing" where your configuration - file weewx.conf is, but if you've done an - unusual install, you may have to tell it explicitly. You can do this by - either putting the location directly in the command line: -

+ The utility is pretty good about "guessing" where your configuration file weewx.conf is, but if you've done an unusual install, you may have to tell it + explicitly. You can do this by either putting the location directly in the command line: 

wee_config /home/weewx/weewx.conf

- or by using option --config: -

+ or by using option --config: 

wee_config --config=/home/weewx/weewx.conf
-

Changing device drivers

- -

- The most common use for wee_config is to - change device drivers. Say that you originally installed - weewx with the Simulator. Now you've bought a - Davis Vantage and you want to switch to that. Here's how you do it. - First, if you don't know the name of the driver, you can ask - wee_config to list all the available drivers: -

+

Command --list-drivers

+

Use this command to list which device drivers are available on your system. For example:

wee_config --list-drivers
@@ -220,25 +190,44 @@ Module name Driver name Version Status weewx.drivers.ws28xx WS28xx 0.34 No module named usb -

The column Status can give you some indication of whether you are missing - any modules to use this driver. It's not completely accurate, but works for most drivers.

-

- From this list, we can see that the name of the driver for the Vantage is - weewx.drivers.vantage. Now run - wee_config, with the - --reconfigure - command, specifying that driver: +

The column Status can give you some indication of whether you are missing any + modules to use this driver. It's not completely accurate, but works for most drivers.

+ + +

Command --install

+

This command is used internally by weewx, and is not intended for the end-user.

+ +

It will install a brand new configuration file using the file DIST_CONFIG as a + template. It will then prompt the user for station parameters (such as altitude, latitude, longitude, etc.).

+ +

Command --upgrade

+

This command is used internally by weewx, and is not intended for the end-user.

+ +

This action first upgrades the existing configuration file, making any changes demanded by the latest version + of weewx. It then merges the result with the contents of DIST_CONFIG.

+ +

Command --reconfigure

+ +

+ This command is used to change station parameters, including the device driver. It is the most likely to be + used by the end-user.

+ +

Example: say that you originally installed weewx with the Simulator. Now you've + bought a Davis Vantage and you want to switch to that. Here's how you do it. First, use the command --list-drivers above to list all the available drivers.

+ +

+ From the list, you will find that the name of the driver for the Vantage is weewx.drivers.vantage. + Now run wee_config, with the --reconfigure command, + specifying that driver: 

wee_config --reconfigure --driver=weewx.drivers.vantage

- The utility will go through your previously selected options, such as - station description, latitude, longitude, altitude, etc.. With the exception - of your newly specified device driver, all your - previously selected values will be the defaults. So, all you have to do is keep hitting - enter. This is what it looked like when I switched from the simulator - to the Vantage driver: -

+ The utility will go through your previously selected options, such as station description, latitude, + longitude, altitude, etc.. With the exception of your newly specified device driver, all your + previously selected values will be the defaults. So, all you have to do is keep hitting enter. This is what + it looked like when I switched from the simulator to the Vantage driver: 

Using configuration file /home/weewx/weewx.conf
 Enter a brief description of the station, such as its location.  For example:
 Santa's Workshop, North Pole
@@ -280,33 +269,21 @@ Saved backup to /home/weewx/weewx.conf.20150430084525
 Saved configuration to /home/weewx/weewx.conf
-

- If this is too much trouble, you can specify the --no-prompt - option: -

+

If this is too much trouble, you can specify the --no-prompt option: 

wee_config --reconfigure --driver=weewx.drivers.vantage --no-prompt
-

This will accept all the defaults, including your new device - driver, without asking for any input.

+

This will accept all the defaults, including your new device driver, without asking for any input.

-

Other wee_config commands

- -

The other commands offered by wee_config, - such as --install and - --upgrade, are used by the various weewx - installers and are not generally intended for the end user.

wee_database

-

This database utility simplifies typical database maintenance operations. For example, it - can backfill the daily summaries or check a SQLite database for embedded strings where floats are - expected.

+

This database utility simplifies typical database maintenance operations. For example, it can backfill the + daily summaries or check a SQLite database for embedded strings where floats are expected.

-

Run the utility with the - --help flag to see how it is used:

+

Run the utility with the --help flag to see how it is used:

wee_database --help

This will result in an output that looks something like this:

@@ -371,68 +348,61 @@ If you are using a MySQL database it is assumed that you have the appropriate
 permissions for the requested operation.
-

Option --create-archive

-

If the database does not already exist, this option will create it and initialize it with the - schema specified in the configuration file weewx.conf. Because - weewx does this automatically, this option is rarely needed.

+

Command --create-archive

+

If the database does not already exist, this command will create it and initialize it with the schema + specified in the configuration file weewx.conf. Because weewx does this + automatically, this command is rarely needed.

-

Option --drop-daily

-

In addition to the regular archive data, every weewx database also includes a daily summary table - for each observation type. Because there can be dozens of observation types, there can be dozens - of these daily summaries. It doesn't happen very often, but there can be occasions when it's - necessary to drop them all then rebuild them. Dropping them by hand would be very tedious! - This option does them all at once.

+

Command --drop-daily

+

In addition to the regular archive data, every weewx database also includes a daily summary table for each + observation type. Because there can be dozens of observation types, there can be dozens of these daily + summaries. It doesn't happen very often, but there can be occasions when it's necessary to drop them all and + then rebuild them. Dropping them by hand would be very tedious! This command does them all at once.

-

Option --backfill-daily

-

This option is kind of the inverse of option --drop-daily in that it - rebuilds the daily summaries from the archive data.

+

Command --backfill-daily

+

This command is kind of the inverse of option --drop-daily in that it rebuilds the + daily summaries from the archive data.

-

Option --reconfigure

-

This option is useful changing the schema in your database.

+

Command --reconfigure

+

This command is useful changing the schema in your database.

-

It creates a new database with the same name as the old, - except with the suffix _new attached at - the end (nominally, weewx.sdb_new - if you are using SQLite, weewx_new if you - are using MySQL). It then initializes it - with the schema specified in weewx.conf. Finally, - it copies over the data from your old database into the new database.

+

It creates a new database with the same name as the old, except with the suffix _new attached at the end (nominally, weewx.sdb_new if you + are using SQLite, weewx_new if you are using MySQL). It then initializes it with + the schema specified in weewx.conf. Finally, it copies over the data from your old + database into the new database.

-

Option --string-check

+

Command --string-check

Normally, all entries in the archive database are pure numbers. However, some visual SQLite database editors - use a null string instead of a null value when deleting entries. These nulls strings can - crash weewx. This option checks for them and, if the option --fix - is specified, substitutes a true null value if it finds any.

+ use a null string instead of a null value when deleting entries. These nulls strings can crash weewx. This + command checks for them and, if the option --fix is specified, substitutes a true + null value if it finds any.

-

Option --transfer

-

This option is useful for moving your database from one type of database to another, such as - from SQLite to MySQL. To use it, you must have two bindings specified in your - weewx.conf configuration file. One will serve as the source, the other - as the destination. Specify the source binding with option --binding, - the destination binding with option --dest-binding.

-

See the Wiki for examples of moving data from - SQLite - to MySQL, and from - MySQL - to SQLite, - using wee_database.

+

Command --transfer

+

This command is useful for moving your database from one type of database to another, such as from SQLite to + MySQL. To use it, you must have two bindings specified in your weewx.conf + configuration file. One will serve as the source, the other as the destination. Specify the source binding + with option --binding, the destination binding with option --dest-binding. +

+

See the Wiki for examples of moving data from SQLite + to MySQL, and from MySQL + to SQLite, using wee_database.

wee_debug

-

Troubleshooting problems when running weewx - often involves analysis of a number of pieces of seemingly disparate system - and weewx related information. The - wee_debug utility gathers all this information together into - a single output to make troubleshooting easier. The - wee_debug utility is particularly useful for new - users as the output may be redirected to a file then emailed or posted to a - forum to assist in remote troubleshooting.

+

Troubleshooting problems when running weewx often involves analysis of a number of + pieces of seemingly disparate system and weewx related information. The wee_debug utility gathers all this information together into a single output to + make troubleshooting easier. The wee_debug utility is particularly useful for new + users as the output may be redirected to a file then emailed or posted to a forum to assist in remote + troubleshooting.

-

Run the utility - with the --help option to see how it is used:

+

Run the utility with the --help option to see how it is used:

wee_debug --help
@@ -467,17 +437,14 @@ before posting the information publicly.

Generating a debug report

-

Generate a debug report using the --info - option as follows:

+

Generate a debug report using the --info option as follows:

-

Warning!
- The wee_debug output includes a copy of the in use weewx - config file (usually weewx.conf) and whilst wee_debug - attempts to obfuscate any personal or sensitive information in weewx.conf, the - user should carefully check the wee_debug output for any remaining personal or - sensitive information before emailing or posting the output publicly.

+

Warning!
The wee_debug output includes a copy + of the in use weewx config file (usually weewx.conf) and + whilst wee_debug attempts to obfuscate any personal or sensitive information in + weewx.conf, the user should carefully check the wee_debug output for any remaining personal or sensitive information before + emailing or posting the output publicly.

wee_debug --info
@@ -597,26 +564,23 @@ wee_debug report successfully generated

Redirecting wee_debug output

-

By default, wee_debug output is sent to the system "standard output" - (stdout) unless the --output - option is used.

-

Option --output with no parameter sends output to the default file - /var/tmp/weewx.debug. Example:

+

By default, wee_debug output is sent to the system "standard output" (stdout) unless the --output option is used.

+

Command --output with no parameter sends output to the default file /var/tmp/weewx.debug. Example:

wee_debug --info --output
-

Option --output with a specified file will send it to that file. - Example:

+

Command --output with a specified file will send it to that file. Example:

wee_debug --info --output /home/weewx/another.debug

wee_debug verbosity

-

The depth of information included in the wee_debug - output can be changed using the --verbosity option. - The --verbosity option can be set to 0, 1 or 2 with - each higher level successively displaying more information. The default level - is 1. The information displayed for each level is:

+

The depth of information included in the wee_debug output can be changed using the + --verbosity option. The --verbosity option can be set to + 0, 1 or 2 with each higher level successively displaying more information. The default level is 1. The + information displayed for each level is:

@@ -661,8 +625,7 @@ wee_debug report successfully generated - @@ -688,29 +651,30 @@ wee_debug report successfully generated

wee_device

-

The utility wee_device can be used to configure the hardware of some - of the more popular weather stations. It can set things like rain bucket size, station archive interval, - altitude, EEPROM constants, etc. In order to do its job, it depends on optional code - being present in the hardware driver. Not all drivers have this code, so it may not work for your specific - device. If it does not, - you will have to consult your manufacturer's instructions for how to set these things through your console - or other means.

+

The utility wee_device is used to configure hardware settings, such as rain bucket + size, station archive interval, altitude, EEPROM constants, etc., on your station. In order to do its + job, it depends on optional code being present in the hardware driver. Because not all drivers have this + code, it may not work for your specific device. If it does not, you will have to consult your manufacturer's + instructions for how to set these things through your console or other means.

-

Run the utility with the --help option - to see which options are available.

+

wee_device determines station type from the station_type + option in weewx.conf. Make sure it's set correctly before attempting to use this + utility.

+ +

Because wee_device uses hardware-specific code, its options are different for every + station type. You should run it with the --help command to see how to use it for + your specific station:

wee_device --help

-

The utility requires a weewx.conf file. - If no file is specified, it will look for - weewx.conf in the standard location. If - your configuration file is in a non-standard location, specify the - path to the configuration file as the first argument. For example,

+

The utility requires a weewx.conf file. If no file is specified, it will look for + weewx.conf in the standard location. If your configuration file is in a + non-standard location, specify the path to the configuration file as the first argument. For example,

wee_device /path/to/weewx.conf --help

-

What follows is a guide to the level of support offered by wee_device for - each type of weather station:

+

What follows is a guide to how to use wee_device for each type of weather station, + as well as its level of support:

  • AcuRite
  • CC3000
    • @@ -729,37 +693,31 @@ wee_debug report successfully generated

    AcuRite

    -

    The wee_device utility cannot - configure AcuRite stations.

    +

    The wee_device utility cannot configure AcuRite stations.

    -

    The AcuRite console must be set to USB mode 3 or 4. Communication - via USB is disabled in modes 1 and 2. Mode 4 is more reliable than - mode 3; mode 3 enables logging of data, mode 4 does not. When the - console is logging it frequently causes USB communication problems.

    +

    The AcuRite console must be set to USB mode 3 or 4. Communication via USB is disabled in modes 1 and 2. Mode + 4 is more reliable than mode 3; mode 3 enables logging of data, mode 4 does not. When the console is logging + it frequently causes USB communication problems.

    -

    The wind speed updates every 18 seconds. The wind direction updates - every 30 seconds. Other sensors update every 60 seconds.

    +

    The wind speed updates every 18 seconds. The wind direction updates every 30 seconds. Other sensors update + every 60 seconds.

    The AcuRite stations do not record wind gusts.

    -

    The station emits partial packets, which may confuse some online - services

    +

    The station emits partial packets, which may confuse some online services

    CC3000

    -

    The CC3000 data logger may be configured to return data in - METRIC or ENGLISH units as selected by option - --set-units. These are then mapped - to the weewx unit groups METRICWX or US, respectively.

    +

    The CC3000 data logger may be configured to return data in METRIC or ENGLISH units as selected by option + --set-units. These are then mapped to the weewx unit groups METRICWX or US, + respectively.

    The CC3000 data logger stores 2MB of records.

    -

    The CC3000 driver supports catchup on startup, but it does not - support hardware record generation.

    +

    The CC3000 driver supports catchup on startup, but it does not support hardware record generation.

    -

    Invoking wee_device with - the --help option will produce +

    Invoking wee_device with the --help option will produce something like this:

    CC3000 driver version 0.8
@@ -782,10 +740,9 @@ Options:
 
 Mutating actions will request confirmation before proceeding.
    -

    Station information

    +

    Command --info

    -

    Display the station settings with the --info - option.

    +

    Display the station settings with the --info option.

    wee_device --info

    This will result in something like:

    firmware: Rainwise CC-3000 Version: 1.3 Build 006 Sep 04 2013
@@ -794,39 +751,30 @@ units: ENGLISH
 memory: 251372 bytes, 4334 records, 12%
 interval: 1
    -

    Changing the archive interval

    +

    Command --set-interval

    -

    CC3000 loggers ship from the factory with an archive interval - of 1 minutes (60 seconds). To change the station's - interval to 5 minutes, do the following:

    +

    CC3000 loggers ship from the factory with an archive interval of 1 minutes (60 seconds). To change the + station's interval to 5 minutes, do the following:

    wee_device --set-interval=5

    FineOffsetUSB

    -

    The station clock can only be set manually via buttons on the - console, or (if the station supports it) by WWVB radio. The - FineOffsetUSB driver ignores the station clock since it cannot be - trusted.

    +

    The station clock can only be set manually via buttons on the console, or (if the station supports it) by + WWVB radio. The FineOffsetUSB driver ignores the station clock since it cannot be trusted.

    -

    The station reads data from the sensors every 48 seconds. - The 30xx stations read UV data every 60 seconds.

    +

    The station reads data from the sensors every 48 seconds. The 30xx stations read UV data every 60 + seconds.

    -

    The 10xx and 20xx stations can save up to 4080 historical readings. - That is about 85 days of data with the default recording interval of - 30 minutes, or about 14 days with a recording interval of 5 minutes. - The 30xx stations can save up to 3264 historical readings.

    +

    The 10xx and 20xx stations can save up to 4080 historical readings. That is about 85 days of data with the + default recording interval of 30 minutes, or about 14 days with a recording interval of 5 minutes. The 30xx + stations can save up to 3264 historical readings.

    -

    When weewx starts up it will attempt - to download all records from the console since the last record in - the archive database.

    +

    When weewx starts up it will attempt to download all records from the console since + the last record in the archive database.

    -

    The wee_device utility can be used to - configure the station hardware. When the - station_type is - FineOffsetUSB, - the --help option will produce output +

    Invoking wee_device with the --help option will produce something like this:

    FineOffsetUSB driver version 1.7
@@ -853,10 +801,10 @@ Options:
 --format=FORMAT      format for output, one of raw, table, or dict
 
 Mutating actions will request confirmation before proceeding.
    -

    Station information

    -

    Display the station settings with the --info - option.

    +

    Command --info

    + +

    Display the station settings with the --info option.

    wee_device --info

    This will result in something like:

    Fine Offset station settings:
@@ -886,56 +834,48 @@ Mutating actions will request confirmation before proceeding.
    version: 255 wind_coef: None wind_mult: 0 -

    Highlighted values can be modified with the wee_device utility.

    -

    Changing the archive interval

    +

    Highlighted values can be modified using the various wee_device + commands below.

    -

    Fine Offset stations ship from the factory with an archive interval - (read_period) of 30 minutes (1800 seconds). To change the station's - interval to 5 minutes, do the following:

    +

    Command --set-interval

    + +

    Fine Offset stations ship from the factory with an archive interval (read_period) of 30 minutes (1800 + seconds). To change the station's interval to 5 minutes, do the following:

    wee_device --set-interval=5

    -

    Dumping the console memory

    +

    Command --history

    -

    Fine Offset stations store records in a circular buffer — once the buffer fills, the oldest records - are - replaced by newer records. The 1080 and 2080 consoles store up to 4080 records. The 3080 consoles store - up to - 3264 records. The data_count indicates how many records are in memory. - The read_period indicates the number of minutes between records. wee_device +

    Fine Offset stations store records in a circular buffer — once the buffer fills, the oldest records are + replaced by newer records. The 1080 and 2080 consoles store up to 4080 records. The 3080 consoles store up + to 3264 records. The data_count indicates how many records are in memory. The + read_period indicates the number of minutes between records. wee_device can display these records in space-delimited, raw bytes, or dictionary format.

    For example, to display the most recent 30 records from the console memory:

    wee_device --history=30
    + +

    Command --clear-memory

    To clear the console memory:

    wee_device --clear-memory
    -

    Checking the USB quality

    +

    Command --check-usb

    -

    Use the wee_device utility to test the quality of the USB connection between - computer - and console. Poor quality USB cables, under-powered USB hubs, and other devices on the bus can interfere - with - communication.

    +

    This command can test the quality of the USB connection between the computer and console. Poor quality USB + cables, under-powered USB hubs, and other devices on the bus can interfere with communication.

    To test the quality of the USB connection to the console:

    wee_device --check-usb
    -

    Let the utility run for at least a few minutes, or possibly an hour or two. It is not unusual to see a - few bad - reads in an hour, but if you see many bad reads within a few minutes, consider replacing the USB cable, - USB hub, - or removing other devices from the bus.

    +

    Let the utility run for at least a few minutes, or possibly an hour or two. It is not unusual to see a few + bad reads in an hour, but if you see many bad reads within a few minutes, consider replacing the USB cable, + USB hub, or removing other devices from the bus.

    +

    Polling mode and the polling interval

    -

    When reading 'live' data, weewx can read as fast as possible, or at a - user-defined - period. This is controlled by the polling_mode parameter. -

    +

    When reading 'live' data, weewx can read as fast as possible, or at a user-defined + period. This is controlled by the polling_mode parameter.

system load averages
driver config extract from weewx config file (usually weewx.conf) + driver config extract from weewx config file (usually weewx.conf)
@@ -950,11 +890,9 @@ Mutating actions will request confirmation before proceeding. polling_mode = ADAPTIVE @@ -966,13 +904,11 @@ Mutating actions will request confirmation before proceeding.
Polling modes for Fine Offset stations
-

In this mode, weewx reads data from the station as often as - possible, but - at intervals that avoid communication between the console and the sensors. Nominally this - results in - reading data every 48 seconds.

+

In this mode, weewx reads data from the station as often as possible, + but at intervals that avoid communication between the console and the sensors. Nominally this + results in reading data every 48 seconds.

In this mode, weewx reads data from the station every polling_interval - seconds.

+ class='code'>polling_interval seconds.

The console reads the sensors every 48 seconds (60 seconds for UV), so setting the polling_interval to a value less than 48 will result in duplicate - readings. -

+ readings.

@@ -980,27 +916,19 @@ Mutating actions will request confirmation before proceeding.

TE923

-

Some station models will recognize up to 5 remote - temperature/humidity sensor units. Use the hardware switch in each - sensor unit to identify sensors. Use the - map in - weewx.conf to associate each sensor with a - database field. -

+

Some station models will recognize up to 5 remote temperature/humidity sensor units. Use the hardware switch + in each sensor unit to identify sensors. Use the map in weewx.conf + to associate each sensor with a database field.

-

The station has either 208 or 3442 history records, depending on - the model. That is just over a day (208 records) or about 23 days - (3442 records) with an archive interval of 5 minutes.

+

The station has either 208 or 3442 history records, depending on the model. That is just over a day (208 + records) or about 23 days (3442 records) with an archive interval of 5 minutes.

-

The TE923 driver will read history records from the station when weewx - starts up, but it does not support hardware record generation.

+

The TE923 driver will read history records from the station when weewx starts up, but it does not support + hardware record generation.

-

The wee_device utility can be used to - configure the station hardware. When the - station_type is - TE932, - the --help option will produce output - something like this:

+

The wee_device utility can be used to configure the station hardware. When the + station_type is TE932, the --help option will produce output something like this:

Using configuration file /home/weewx/weewx.conf
 Using TE923 driver version 0.16 (weewx.drivers.te923)
@@ -1039,10 +967,9 @@ Options:
 Be sure to stop weewx first before using. Mutating actions will request
 confirmation before proceeding.
-

Station information

+

Command --info

-

Use the --info option to display the - station configuration:

+

Use the --info option to display the station configuration:

wee_device --info

This will result in something like:

Querying the station for the configuration...
@@ -1063,10 +990,9 @@ confirmation before proceeding.
version_uv: 20 version_wind: 38 -

Display sensor status

+

Command --current

-

Use the --current option to display the - current status of each sensor:

+

Use the --current option to display the current status of each sensor:

wee_device --current

This will result in something like:

Querying the station for current weather data...
@@ -1112,41 +1038,36 @@ confirmation before proceeding.
windspeed: None windspeed_state: invalid -

Changing the archive interval

+

Command --set-interval

-

TE923 stations ship from the factory with an archive interval of 1 - hour (3600 seconds). To change the station's - interval to 5 minutes (300 seconds), do the following:

+

TE923 stations ship from the factory with an archive interval of 1 hour (3600 seconds). To change the + station's interval to 5 minutes (300 seconds), do the following:

wee_device --set-interval=300

-

Dumping the logger memory

+

Command --history

-

wee_device can display the records from the - logger in tabular or dictionary format.

+

wee_device can display the records from the logger in tabular or dictionary format. +

-

For example, to display the most recent 30 records:

+

For example, to display the most recent 30 records in dictionary format:

wee_device --history=30 --format dict

Ultimeter

-

The wee_device utility cannot - configure Ultimeter stations.

+

The wee_device utility cannot configure Ultimeter stations.

-

The Ultimeter driver operates the Ultimeter in Data Logger Mode, - which results in sensor readings every 1/2 second or so.

+

The Ultimeter driver operates the Ultimeter in Data Logger Mode, which results in sensor readings every 1/2 + second or so.

-

The Ultimeter driver ignores the maximum, minimum, and average - values recorded by the station.

+

The Ultimeter driver ignores the maximum, minimum, and average values recorded by the station.

Vantage

-

When the station_type is - Vantage, - the --help option will produce output - something like this:

+

When the station_type is Vantage, the --help + option will produce output something like this:

Using configuration file /home/weewx/weewx.conf
 Using Vantage driver version 3.0 (weewx.drivers.vantage)
@@ -1212,10 +1133,9 @@ Options:
 Be sure to stop weewx first before using. Mutating actions will request
 confirmation before proceeding.
-

Station information

+

Command --info

-

Use the --info option to display the - current EEPROM settings:

+

Use the --info option to display the current EEPROM settings: 

wee_device --info

This will result in something like:

Davis Vantage EEPROM settings:
@@ -1284,180 +1204,164 @@ confirmation before proceeding.
Outside Temperature: +0.0 F Outside Humidity: +0% Extra Temperature 1: +0.0 F -

The console version number is available only on consoles with firmware - dates after about 2006.

+

The console version number is available only on consoles with firmware dates after about 2006.

-

Highlighted values can be changed using this utility.

+

Highlighted values can be modified using the various wee_device + commands below.

-

Time zone

+

Command --set-tz-code

-

To set the time zone code to Central European Time (code 20):

+

This command can be used to change the time zone. Consult the Vantage manual for the code that corresponds to + your time zone.

+ +

You can set either the time zone code or the time zone offset, but not both.

+ +

For example, to set the time zone code to Central European Time (code 20):

wee_device --set-tz-code=20
-

You can set either the time zone code or the - time zone offset, but not both.

-

Archive interval

+

Command --set-interval

-

Valid archive intervals for the Davis Vantage stations are 60, - 300, 600, 900, 1800, 3600, and 7200 seconds. However, if you are - ftp'ing lots of files to a server, setting it to 60 seconds - may not give enough time to have them all uploaded before the next - archive record is due. If this is the case, you should pick an - archive interval of at least 300 seconds, or trim the number of - files you are using.

+

Use this command to change the archive interval of the internal logger. Valid intervals are 60, 300, 600, + 900, 1800, 3600, and 7200 seconds. However, if you are ftp'ing lots of files to a server, setting it to + 60 seconds may not give enough time to have them all uploaded before the next archive record is due. If this + is the case, you should pick an archive interval of at least 300 seconds, or trim the number of files you + are using.

-

To change the archive interval to 10 minutes (600 seconds):

+

An archive interval of five minute (300 seconds) works well for the Vantage stations. Because of the large + amount of onboard memory they carry, going to a larger interval does not have any real advantages.

+ +

Example: to change the archive interval to 10 minutes (600 seconds):

wee_device --set-interval=600
-

I have found that a five minute (300 seconds) archive interval works - well for the Vantage stations. Because of the large amount of onboard - memory they carry, going to a larger interval does not have any real - advantages.

-

Rain bucket type

+

Command --set-altitude

+

Use this command to set the console's stored altitude. The altitude must be in feet. Example:

+ 
wee_device --set-altitude=700
-

Normally, this is set by Davis, but if you have replaced your bucket - with a different kind, you might want to reconfigure. For example, to - change to a 0.1 mm bucket (bucket code "2"), use the - following:

+

Command --set-bucket

+ +

Normally, this is set by Davis, but if you have replaced your bucket with a different kind, you might want to + reconfigure. For example, to change to a 0.1 mm bucket (bucket code "2"), use the following:

wee_device --set-bucket=2
-

Additional sensors

+

Command --set-rain-year-start

+

The Davis Vantage series allows the start of the rain year to be something other than 1 January. For + example, to set it to 1 October:

+ 
wee_device --set-rain-year-start=10
-

If you have additional sensors for your Vantage station, you - can configure them using your console. However, if you have - a Davis - Weather Envoy receiver, it will not have a console! As an alternative, - the weewx - utility wee_device lets - you do this from the command line.

+

Command --set-transmitter-type

+ +

If you have additional sensors for your Vantage station, you can configure them using your console. However, + if you have a Davis + Weather Envoy receiver, it will not have a console! As an alternative, wee_device can do this using the command --set-transmitter-type. +

For example, to add an extra temperature sensor to channel 3, do the following:

wee_device --set-transmitter-type=3,1,2
-

This says to turn on channel 3, set its type to 1 ("Temperature only"), and have it show up in the - database as extraTemp2. Here's another example, this time for a combined - temperature / humidity sensor:

+

This says to turn on channel 3, set its type to 1 ("Temperature only"), and have it show up in the database + as extraTemp2. Here's another example, this time for a combined temperature / + humidity sensor:

wee_device --set-transmitter-type=5,3,2,4
-

This will add the combined sensor to channel 5, set its type to 3 ("Temperature and humidity"), - and it will show up in the database - as extraTemp2 and extraHumid4.

+

This will add the combined sensor to channel 5, set its type to 3 ("Temperature and humidity"), and it will + show up in the database as extraTemp2 and extraHumid4. +

The --help option will give you the code for each sensor type.

-

Setting offsets

+

Command --set-offset

-

The Davis instruments can correct sensor errors by adding - an offset to their emitted values. This is particularly - useful for Southern Hemisphere users. Davis fits the wind vane - to the Integrated Sensor Suite (ISS) in a position optimized for - Northern Hemisphere users, who face the solar panel to the - south. Users south of the equator must orient the ISS's solar - panel to the north to get maximal insolation, resulting in a - 180° error in the wind direction. The solution is to add a - 180° offset correction. You can do this with the following - command:

+

The Davis instruments can correct sensor errors by adding an offset to their emitted values. This is + particularly useful for Southern Hemisphere users. Davis fits the wind vane to the Integrated Sensor Suite + (ISS) in a position optimized for Northern Hemisphere users, who face the solar panel to the south. Users + south of the equator must orient the ISS's solar panel to the north to get maximal insolation, resulting in + a 180° error in the wind direction. The solution is to add a 180° offset correction. You can do this + with the following command:

wee_device --set-offset=windDir,180
-

Dumping the logger memory

+

Command --dump

-

Generally, weewx downloads only new archive - records from the on-board logger in the Vantage. However, occasionally the - memory in the Vantage will get corrupted, making this impossible. See the - troubleshooting section below Weewx - generates HTML pages, but it does not update them. The - fix involves clearing the memory but, unfortunately, this means you may - lose any data which might have accumulated in the logger memory, but not - yet downloaded. By using the --dump command - before clearing the memory, you might be able to save these data. - Stop weewx first, then

+

Generally, weewx downloads only new archive records from the on-board logger in the + Vantage. However, occasionally the memory in the Vantage will get corrupted, making this impossible. See the + troubleshooting section Weewx generates HTML pages, but + it does not update them. The fix involves clearing the memory but, unfortunately, this means + you may lose any data which might have accumulated in the logger memory, but not yet downloaded. By using + the --dump command before clearing the memory, you might be able to save these + data. Stop weewx first, then

wee_device --dump
-

This will dump all data archived in the Vantage memory directly to the - database, without regard to whether or not they have been seen before. - Because the command dumps all data, it may result in many - duplicate primary key errors. These can be ignored.

+

This will dump all data archived in the Vantage memory directly to the database, without regard to whether or + not they have been seen before. Because the command dumps all data, it may result in many duplicate + primary key errors. These can be ignored.

+ +

Command --logger-summary

+

This command is useful for debugging the console logger. It will scan the logger memory, recording the timestamp + in each page and index slot to the file FILE.

+

Example:

+ 
wee_device --logger-summary=/var/tmp/summary.txt

WMR100

-

The wee_device utility cannot - configure WMR100 stations.

+

The wee_device utility cannot configure WMR100 stations.

-

The station emits partial packets, which may confuse some online - services.

+

The station emits partial packets, which may confuse some online services.

WMR200

-

The wee_device utility cannot - configure WMR200 stations.

+

The wee_device utility cannot configure WMR200 stations.

-

The station emits partial packets, which may confuse some online - services.

+

The station emits partial packets, which may confuse some online services.

-

When weewx starts up it will attempt to - download all records from the console since the last record in the - archive database. -

+

When weewx starts up it will attempt to download all records from the console since + the last record in the archive database.

WMR300

-

The wee_device utility cannot - configure WMR300 stations.

+

The wee_device utility cannot configure WMR300 stations.

-

The station emits partial packets, which may confuse some online - services.

+

The station emits partial packets, which may confuse some online services.

-

When weewx starts up it will attempt to - download all records from the console since the last record in the - archive database. This can take a couple of hours, depending on the - number of records in the logger and the speed of the computer and disk. -

+

When weewx starts up it will attempt to download all records from the console since + the last record in the archive database. This can take a couple of hours, depending on the number of records + in the logger and the speed of the computer and disk.

WMR9x8

-

The wee_device utility cannot - configure WMR9x8 stations.

+

The wee_device utility cannot configure WMR9x8 stations.

-

The station includes a data logger, but the driver does not read - records from the station.

+

The station includes a data logger, but the driver does not read records from the station.

-

The station emits partial packets, which may confuse some online - services.

+

The station emits partial packets, which may confuse some online services.

WS1

-

The wee_device utility cannot - configure WS1 stations.

+

The wee_device utility cannot configure WS1 stations.

The WS1 stations produce data every 1/2 second or so.

WS23xx

-

The hardware interface is a serial port, but USB-serial converters - can be used with computers that have no serial port. Beware that - not every type of USB-serial converter will work. Converters based - on ATEN UC-232A chipset are known to work.

+

The hardware interface is a serial port, but USB-serial converters can be used with computers that have no + serial port. Beware that not every type of USB-serial converter will work. Converters based on ATEN UC-232A + chipset are known to work.

The station does not record wind gust or wind gust direction.

The hardware calculates windchill and dewpoint.

-

The station has 175 history records. That is just over 7 days - of data with the factory default history recording interval of 60 - minutes, or about 14 hours with a recording interval of 5 minutes.

+

The station has 175 history records. That is just over 7 days of data with the factory default history + recording interval of 60 minutes, or about 14 hours with a recording interval of 5 minutes.

-

When weewx starts up it will attempt - to download all records from the console since the last record in - the archive database.

+

When weewx starts up it will attempt to download all records from the console since + the last record in the archive database.

-

When the station_type is - WS23xx, - the --help option will produce output - something like this:

+

When the station_type is WS23xx, the --help + option will produce output something like this:

WS23xx driver version 0.21
 Usage: wee_device [config_file] [options] [--debug] [--help]
@@ -1480,8 +1384,7 @@ Mutating actions will request confirmation before proceeding.

Station information

-

Display the station settings with the --info - option.

+

Display the station settings with the --info option.

wee_device --info

This will result in something like:

buzzer: 0
@@ -1510,24 +1413,20 @@ in humidity: 48.0
 
         
Changing the archive interval

 
-        
WS23xx stations ship from the factory with an archive interval of 60
-            minutes (3600 seconds). To change the station's interval to 5 minutes,
-            do the following:

+        
WS23xx stations ship from the factory with an archive interval of 60 minutes (3600 seconds). To change the
+            station's interval to 5 minutes, do the following:

 
         
wee_device --set-interval=5

 
-        
Warning!

-            Changing the recording interval will clear the station memory.

+        
Warning!
 Changing the recording interval will clear the station memory.
+        

 
         
Dumping the console memory

 
-        
WS23xx stations store records in a circular buffer - once the
-            buffer fills, the oldest records are replaced by newer records. The
-            console stores up to 175 records.
-            The history number of records indicates how
-            many records are in memory. The
-            history interval indicates the number of
-            minutes between records.

+        
WS23xx stations store records in a circular buffer - once the buffer fills, the oldest records are replaced
+            by newer records. The console stores up to 175 records. The history number of records indicates how many records are in memory. The history interval indicates the number of minutes between records.

 
         
For example, to display the latest 30 records from the console memory:

         
wee_device --history=30

@@ -1537,29 +1436,22 @@ in humidity: 48.0
 
         
WS28xx

 
-        
weewx communicates with a USB transceiver,
-            which communicates with the station console, which in turn communicates
-            with the sensors. The transceiver and console must be paired and
+        
weewx communicates with a USB transceiver, which communicates with the station
+            console, which in turn communicates with the sensors. The transceiver and console must be paired and
             synchronized.

 
-        
The station has 1797 history records. That is just over 6 days
-            of data with an archive interval of 5 minutes.

+        
The station has 1797 history records. That is just over 6 days of data with an archive interval of 5
+            minutes.

 
-        
When weewx starts up it will attempt to
-            download all records from the console since the last record in the
-            archive database.
-        

+        
When weewx starts up it will attempt to download all records from the console since
+            the last record in the archive database. 

 
-        
The WS28xx driver sets the station archive interval to
-            5 minutes.

+        
The WS28xx driver sets the station archive interval to 5 minutes.

 
-        
The WS28xx driver does not support hardware archive record
-            generation.

+        
The WS28xx driver does not support hardware archive record generation.

 
-        
When the station_type is
-            WS28xx,
-            the --help option will produce output
-            something like this:

+        
When the station_type is WS28xx, the --help
+            option will produce output something like this:

 
     
WS28xx driver version 0.33
 Usage: wee_device [config_file] [options] [--debug] [--help]
@@ -1583,74 +1475,55 @@ Mutating actions will request confirmation before proceeding.

 
         
Pairing

 
-        
The console and transceiver must be paired. Pairing ensures that your
-            transceiver is talking to your console, not your neighbor's console.
-            Pairing should only have to be done once, although you might have to
-            pair again after power cycling the console, for example after you replace
-            the batteries.

+        
The console and transceiver must be paired. Pairing ensures that your transceiver is talking to your console,
+            not your neighbor's console. Pairing should only have to be done once, although you might have to pair again
+            after power cycling the console, for example after you replace the batteries.

 
         
There are two ways to pair the console and the transceiver:

         
    
-            
  1. The Weewx way. Be sure that
-                weewx is not running.
-                Run the configuration utility,
-                press and hold the [v] button on the console until you see
-                'PC' in the display, then release the button. If the console pairs
-                with the transceiver, 'PC' will go away within a second or two.
+            
  2. The Weewx way. Be sure that weewx is not running. Run the
+                configuration utility, press and hold the [v] button on the console until you see 'PC' in the display,
+                then release the button. If the console pairs with the transceiver, 'PC' will go away within a second or
+                two.
             
    wee_device --pair
 
 Pairing transceiver with console...
 Press and hold the [v] key until "PC" appears (attempt 1 of 3)
 Transceiver is paired to console
    
             
    3. 
-            
  3. The HeavyWeather way. Follow the pairing
-                instructions that came with the station. You will have to run
-                HeavyWeather on a windows computer with the USB transceiver. After
-                HeavyWeather indicates the devices are paired, put the USB transceiver
-                in your weewx computer and start weewx.
-                Do not power cycle the station console or you will have to start
-                over.
+            
  4. The HeavyWeather way. Follow the pairing instructions that came with the station. You
+                will have to run HeavyWeather on a windows computer with the USB transceiver. After HeavyWeather
+                indicates the devices are paired, put the USB transceiver in your weewx
+                computer and start weewx. Do not power cycle the station console or you will have to start over.
             
    5. 
         

 
-        
If the console does not pair, you will see messages in the log such as
-            this:

+        
If the console does not pair, you will see messages in the log such as this:

         
ws28xx: RFComm: message from console contains unknown device ID (id=165a resp=80 req=6)

         
Either approach to pairing may require multiple attempts.

 
         
Synchronizing

 
-        
After pairing, the transceiver and console must be synchronized in
-            order to communicate. Synchronization will happen automatically at the
-            top of each hour, or you can force synchronization by pressing the [SET]
-            button momentarily. Do not press and hold the [SET] button - that
-            modifies the console alarms.

+        
After pairing, the transceiver and console must be synchronized in order to communicate. Synchronization will
+            happen automatically at the top of each hour, or you can force synchronization by pressing the [SET] button
+            momentarily. Do not press and hold the [SET] button - that modifies the console alarms.

 
-        
When the transceiver and console are synchronized, you will see lots of
-            'ws28xx: RFComm' messages in the log when debug=1.
-            When the
-            devices are
-            not synchronized, you will see messages like this about every 10
-            minutes:

+        
When the transceiver and console are synchronized, you will see lots of 'ws28xx: RFComm' messages in the log when debug=1. When the
+            devices are not synchronized, you will see messages like this about every 10 minutes:

         
Nov  7 19:12:17 raspi weewx[2335]: ws28xx: MainThread: no contact with console

-        
If you see this, or if you see an extended gap in the weather data
-            in the weewx plots, press momentarily the [SET] button, or wait until
-            the top of the hour.

+        
If you see this, or if you see an extended gap in the weather data in the weewx
+            plots, press momentarily the [SET] button, or wait until the top of the hour.

 
-        
When the transceiver has not received new data for awhile, you will see
-            messages like this in the log:

+        
When the transceiver has not received new data for awhile, you will see messages like this in the log:

         
Nov  7 19:12:17 raspi weewx[2335]: ws28xx: MainThread: no new weather data

-        
If you see 'no new weather data' messages with the
-            'no contact with console' messages,
-            it simply means that the transceiver has not been able to synchronize
-            with the console. If you see only the 'no new weather data' messages,
-            then the sensors are not communicating with the console, or the console
-            may be defective.

+        
If you see 'no new weather data' messages with the 'no contact with console' messages, it simply means that
+            the transceiver has not been able to synchronize with the console. If you see only the 'no new weather data'
+            messages, then the sensors are not communicating with the console, or the console may be defective.

 
         
Station information

 
-        
Display the station settings with the --info
-            option.

+        
Display the station settings with the --info option.

 
         
wee_device --info

 
@@ -1697,10 +1570,9 @@ wind_gust_max_time: None

Alarms

-

When an alarm goes off, communication with the transceiver stops. - The WS28xx driver clears all alarms in the station. It is better to - create alarms in weewx, and the weewx alarms - can do much more than the console alarms anyway.

+

When an alarm goes off, communication with the transceiver stops. The WS28xx driver clears all alarms in the + station. It is better to create alarms in weewx, and the weewx alarms can do much + more than the console alarms anyway.

@@ -1708,11 +1580,8 @@ wind_gust_max_time: None

wee_extension

- The utility wee_extension is used to add and - remove extensions. It's worth running with the - --help - option to see how it is used: -

+ The utility wee_extension is used to add and remove extensions. It's worth running + with the --help option to see how it is used: 

wee_extension --help
@@ -1772,31 +1641,24 @@ wee_extension --install basic.tar.gz
  • Add/Remove directories in skins
  • Add/Remove directories in user
    • -

    The utility makes a copy of any file or directory that it - modifies or replaces. When installing, it creates a directory called - installer in the - user directory. The contents of the - installer directory are - used to enumerate and uninstall extensions.

    +

    The utility makes a copy of any file or directory that it modifies or replaces. When installing, it creates a + directory called installer in the user directory. The + contents of the installer directory are used to enumerate and uninstall + extensions.

    Installing an extension

    - Let's try installing a simple extension, - cmon, - used to monitor your computer. -

    + Let's try installing a simple extension, cmon, used to monitor your computer.

    - First download it. You can either do this from the link given in the - wiki, or by using wget (which you may have - to install): -

    + First download it. You can either do this from the link given in the wiki, or by using wget + (which you may have to install): 

    wget -P /var/tmp http://lancet.mit.edu/mwall/projects/weather/releases/weewx-cmon-0.7.tgz

    - This will put the tarfile weewx-cmon-0.7.tgz - in the directory /var/tmp. + This will put the tarfile weewx-cmon-0.7.tgz in the directory /var/tmp.

    Now install the extension:

    @@ -1810,33 +1672,26 @@ Finished installing extension '/var/tmp/weewx-cmon-0.7.tgz'

    The installer has done a number of things for you:

      -
    1. It put a new skin, cmon, in the - skins subdirectory; +
    2. It put a new skin, cmon, in the skins subdirectory;
      3. -
    3. It put a new service, - user.cmon.ComputerMonitor, - in the list of services to be run by - weewx; +
    4. It put a new service, user.cmon.ComputerMonitor, in the list of services to be + run by weewx;
      5. -
    5. It defined a new database, cmon_sqlite, - and a binding, cmon_binding, +
    6. It defined a new database, cmon_sqlite, and a binding, cmon_binding, to that database;
      7. -
    7. It added a top-level "stanza" - [ComputerMonitor] - to your configuration file weewx.conf, - that specifies the data binding cmon - is to use. +
    8. It added a top-level "stanza" [ComputerMonitor] to your configuration file + weewx.conf, that specifies the data binding cmon is + to use.
      9. -
    9. And, finally, it saved the details of how the extension was - installed so you can remove it later, should you choose to do so. +
    10. And, finally, it saved the details of how the extension was installed so you can remove it later, should + you choose to do so.

    Listing installed extensions

    -

    The utility wee_extension can tell you which - extensions you have installed:

    +

    The utility wee_extension can tell you which extensions you have installed:

    wee_extension --list
 Extension Name   Version  Description
 cmon             0.7      Collect and display computer health indicators
    @@ -1855,27 +1710,22 @@ No extensions installed

    wee_import

    -

    Some weewx users will have historical data from - another source (e.g., other weather station software or a manually compiled - file) which they wish to import into weewx. - Such data can, depending upon the source, be imported - using the wee_import utility. This section - details the use of the wee_import utility. -

    +

    Some weewx users will have historical data from another source (e.g., other weather + station software or a manually compiled file) which they wish to import into weewx. + Such data can, depending upon the source, be imported using the wee_import + utility. This section details the use of the wee_import utility.

    -

    The wee_import utility supports importing - observational data from the following sources:

    +

    The wee_import utility supports importing observational data from the following + sources:

    • a single Comma Separated Values (CSV) format file
      • -
    • the historical observations of a Weather Underground personal - weather station +
    • the historical observations of a Weather Underground personal weather station
    • one or more Cumulus monthly log files
    -

    Before starting, it's worth running the utility with the - --help flag to see how - wee_import is used:

    +

    Before starting, it's worth running the utility with the --help flag to + see how wee_import is used:

    wee_import --help
    @@ -1920,40 +1770,34 @@ the daily summaries using the wee_database utility.

    Command line options

    -

    The wee_import command line options are - described in more detail below:

    +

    The wee_import command line options are described in more detail below:

    --config

    -

    The utility is pretty good at "guessing" where your configuration file - weewx.conf is, but if you've done an unusual - install, you may have to tell it explicitly. You can do this by using - the --config option:

    +

    The utility is pretty good at "guessing" where your configuration file weewx.conf + is, but if you've done an unusual install, you may have to tell it explicitly. You can do this by using the + --config option:

    wee_import --config=/this/folder/weewx.conf --import-config=/folder/import.conf

    --import-config

    -

    wee_import uses a secondary configuration file to - store various import parameters. The - --import-config command line option - is mandatory for all imports. Example import configuration files for - each type of import supported by wee_import - are provided in the util/import folder. These - example files are best used by making a copy of the applicable example - file in a working directory and then modifying the duplicate file to - suit your needs. The --import-config - command line option is used as follows:

    +

    wee_import uses a secondary configuration file to store various import parameters. + The --import-config command line option is mandatory for all imports. + Example import configuration files for each type of import supported by wee_import + are provided in the util/import folder. These example files are best used by + making a copy of the applicable example file in a working directory and then modifying the duplicate file to + suit your needs. The --import-config command line option is used as + follows:

    wee_import --import-config=/folder/import.conf

    --dry-run

    -

    The inclusion of the --dry-run command - line option will cause the import to proceed but no actual data will be - saved to the database. This is a useful option to use when first +

    The inclusion of the --dry-run command line option will cause the import + to proceed but no actual data will be saved to the database. This is a useful option to use when first importing data.

    wee_import --import-config=/folder/import.conf --dry-run
@@ -1961,18 +1805,14 @@ the daily summaries using the wee_database utility.

    --date

    -

    The date-time range of records to be imported can be specified by use of - the --date command line option. The - --date command line option can - specify a single date, as single date-time, a date range or a date-time - range. The date format used is YYYY/MM/DD and - the date-time format YYYY/MM/DD HH:MM. A range - is specified by separating two date or date-time formats by a hyphen, - e.g., '2015/12/1-2015/12/30'. Note that the - date-time or date-time range string must be enclosed in single or double - quotation marks. The effect of the different - --date command line option values - is shown in the following table:

    +

    The date-time range of records to be imported can be specified by use of the --date + command line option. The --date command line option can specify a single + date, as single date-time, a date range or a date-time range. The date format used is YYYY/MM/DD + and the date-time format YYYY/MM/DD HH:MM. A range is specified by separating two + date or date-time formats by a hyphen, e.g., '2015/12/1-2015/12/30'. Note that the + date-time or date-time range string must be enclosed in single or double quotation marks. The effect of the + different --date command line option values is shown in the following + table:

    @@ -2010,78 +1850,59 @@ the daily summaries using the wee_database utility.

    - Note
    If the --date - command line option is omitted the default is to import all available - records when importing from a CSV or Cumulus source or to import today's - records only when importing from Weather Underground. -

    + Note
    If the --date command line option is omitted the default + is to import all available records when importing from a CSV or Cumulus source or to import today's records + only when importing from Weather Underground.

    --log

    -

    The --log option controls the - wee_import log output. Omitting the option - will result in wee_import log output being - sent to the weewx log file (nominally the - system log, refer to - Monitoring weewx - and - Where to find things - to find it). wee_import log output can be - disabled by using --log=-. The - --log command line option is used as - follows: -

    +

    The --log option controls the wee_import log + output. Omitting the option will result in wee_import log output being sent to the + weewx log file (nominally the system log, refer to Monitoring weewx and Where to find things to find it). wee_import log output can be disabled by using --log=-. + The --log command line option is used as follows: 

    wee_import --import-config=/folder/import.conf --log=-

    --verbose

    -

    Inclusion of the --verbose command - line option will cause additional information to be printed during - wee_import execution.

    +

    Inclusion of the --verbose command line option will cause additional + information to be printed during wee_import execution.

    wee_import --import-config=/folder/import.conf --verbose

    Importing from CSV files

    -

    wee_import can import data from - a single CSV file. The CSV source file must be +

    wee_import can import data from a single CSV file. The CSV source file must be structured as follows:

      -
    • The file must have a header row consisting of a comma separated - list of field names. The field names can be any valid string as long - as each field name is unique within the list. There is no - requirement for the field names to be in any particular order as - long as the same order is used for the observations on each row in - the file. These field names will be mapped to - weewx field names in the - [CSV] section of the import configuration - file. +
    • The file must have a header row consisting of a comma separated list of field names. The field names can + be any valid string as long as each field name is unique within the list. There is no requirement for + the field names to be in any particular order as long as the same order is used for the observations on + each row in the file. These field names will be mapped to weewx field names in + the [CSV] section of the import configuration file.
      • -
    • Observation data for a given date-time must be listed on a - single line with individual fields separated by a comma. The fields - must be in the same order as the field names in the header row. +
    • Observation data for a given date-time must be listed on a single line with individual fields separated + by a comma. The fields must be in the same order as the field names in the header row.
      • -
    • Blank fields are represented by the use of white space or no - space only between commas. +
    • Blank fields are represented by the use of white space or no space only between commas.
      • -
    • There must a field that represents the date-time of the - observations on each line. This date-time field must be either a - Unix epoch timestamp or any date-time format that can be - represented using - Python - strptime() format codes. +
    • There must a field that represents the date-time of the observations on each line. This date-time field + must be either a Unix epoch timestamp or any date-time format that can be represented using Python + strptime() format codes.
    -

    A CSV file suitable for import by wee_import - may look like this:

    +

    A CSV file suitable for import by wee_import may look like this:

    Time,Temp,Dewpoint,Press,WindDir,WindSpeed,WindGust,Hum,dailyrain,SolarRad
 2016-02-01 00:00:00,13.5,9.2,1005.6,359,0.0,0.0,75,12.0,0.00
@@ -2097,50 +1918,36 @@ the daily summaries using the wee_database utility.

    weewx archive fields populated during a CSV import

    -

    The weewx archive fields populated during a - CSV import depend on the CSV-to-weewx field - mappings specified in [CSV] section in the - import configuration file. If a valid field mapping exists, the - weewx field exists in the - weewx archive table schema and provided the - mapped CSV field contains valid data, then the corresponding - weewx field will populated. Note that the CSV - import is the only import supported by - wee_import that allows any - weewx archive field to be populated. -

    +

    The weewx archive fields populated during a CSV import depend on the CSV-to-weewx field mappings specified in [CSV] section in the + import configuration file. If a valid field mapping exists, the weewx field exists + in the weewx archive table schema and provided the mapped CSV field contains valid + data, then the corresponding weewx field will populated. Note that the CSV import + is the only import supported by wee_import that allows any weewx + archive field to be populated.

    - Note
    The use of the - calc_missing option in the import - configuration file may result in a - number of derived fields being calculated from the imported data. If - these derived fields exist in the in-use database schema they will be - saved to the database as well. -

    + Note
    The use of the calc_missing option in the import configuration file may result in a + number of derived fields being calculated from the imported data. If these derived fields exist in the + in-use database schema they will be saved to the database as well.

    Importing observations from a CSV file

    To import observations from a CSV file:

      -
    1. Ensure the source data file is in a folder accessible by the machine - that will run wee_import. For the - purposes of these instructions the source data file - data.csv located in the - /var/tmp folder will be used. +
    2. Ensure the source data file is in a folder accessible by the machine that will run wee_import. + For the purposes of these instructions the source data file data.csv located + in the /var/tmp folder will be used.
      3. -
    3. Make a backup of the - weewx database in case the import - should go awry. +
    4. Make a backup of the weewx database in case the import should go awry.
      5. -
    5. Create an import configuration file. - In this case we will make a copy of the example CSV - import configuration file and save it as - csv.conf in the - /var/tmp directory: +
    6. Create an import configuration file. In this case we will make a copy of the example CSV import + configuration file and save it as csv.conf in the /var/tmp directory: 
      7. $ cp /home/weewx/util/import/csv-example.conf /var/tmp/csv.conf
@@ -2152,64 +1959,51 @@ the daily summaries using the wee_database utility.
      source = CSV
      -
    7. Confirm that the following options in the [CSV] - section are set: +
    8. Confirm that the following options in the [CSV] section are set:
        9. -
      • file. - The full path and file name of the file containing the CSV - formatted data to be imported. +
      • file. The full path and file name + of the file containing the CSV formatted data to be imported.
        • -
      • interval. - Determines how the weewx interval - field is derived. +
      • interval. Determines how the + weewx interval field is derived.
        • -
      • qc. Determines - whether quality control checks are performed on the imported - data. +
      • qc. Determines whether quality + control checks are performed on the imported data.
        • -
      • calc_missing. - Determines whether missing derived observations will be - calculated from the imported data. +
      • calc_missing. Determines + whether missing derived observations will be calculated from the imported data.
        • -
      • tranche. The - number of records written to the - weewx database in each transaction. +
      • tranche. The number of records + written to the weewx database in each transaction.
        • -
      • UV_sensor. - Whether a UV sensor was installed when the source data was - produced. +
      • UV_sensor. Whether a UV sensor was + installed when the source data was produced.
        • -
      • solar_sensor. - Whether a solar radiation sensor was installed when the - source data was produced. +
      • solar_sensor. Whether a solar + radiation sensor was installed when the source data was produced.
      • raw_datetime_format. - The format of the imported date time field. + href="#csv_raw_datetime_format">raw_datetime_format. The format of the + imported date time field.
        • -
      • rain. Determines - how the weewx rain field is - derived. +
      • rain. Determines how the weewx rain field is derived.
      • wind_direction. - Determines how imported wind direction fields are - interpreted. + Determines how imported wind direction fields are interpreted.
        • -
      • [[Map]]. Defines - the mapping between imported data fields and - weewx archive fields. Also defines - the units of measure for each imported field. +
      • [[Map]]. Defines the mapping + between imported data fields and weewx archive fields. Also defines the + units of measure for each imported field.
      -
    9. When first importing data it is prudent to do a dry run import - before any data are actually imported. A dry run import will perform - all steps of the import without actually writing imported data to - the weewx database. In addition, - consideration should be given to any additional command line options - such as --date. +
    10. When first importing data it is prudent to do a dry run import before any data are actually imported. A + dry run import will perform all steps of the import without actually writing imported data to the weewx database. In addition, consideration should be given to any additional + command line options such as --date.

      11. To perform a dry run enter the following command:

      @@ -2229,34 +2023,25 @@ Records processed: 70685; Unique records: 70685; Last timestamp: 2010-09-04 04:2 Finished dry run import. 70685 records were processed and 70685 unique records would have been imported. -

      The output includes details about the data source, its - destination and some other details on how the data will - be processed. The import will then - be performed but no data will be written to the - weewx database. Upon completion a - brief summary of the records processed is provided. -

      +

      The output includes details about the data source, its destination and some other details on how the data + will be processed. The import will then be performed but no data will be written to the weewx database. Upon completion a brief summary of the records processed is + provided.

      - Note
      As the weewx database is - not altered when the --dry-run command - line option is used, wee_import log output - is suspended during a dry run import. In effect, the use of - --dry-run is equivalent to - --dry-run --log=-. During a dry run import - the only wee_import output is that - displayed on stdout. -

      + Note
      As the weewx database is not altered when the --dry-run command line option is used, wee_import log + output is suspended during a dry run import. In effect, the use of --dry-run + is equivalent to --dry-run --log=-. During a dry run import the only wee_import output is that displayed on stdout.

      -
    11. Once the dry run results are satisfactory the data can be - imported using the following command: +
    12. Once the dry run results are satisfactory the data can be imported using the following command: 
      13. wee_import --import-config=/var/tmp/csv.conf
      -

      This will result in a short preamble similar to that from - the dry run. At the end - of the preamble there will be a prompt:

      +

      This will result in a short preamble similar to that from the dry run. At the end of the preamble there + will be a prompt:

      Starting wee_import...
 A CSV import from source file '/var/tmp/data.csv' has been requested.
@@ -2269,76 +2054,61 @@ Proceeding will save all imported records in the weewx archive.
 Are you sure you want to proceed (y/n)?
      -
    13. If the import parameters - are acceptable enter y to proceed with the - import or n to abort the import. - If the import is confirmed then the source - data will be imported, processed and saved in the - weewx database. Information on the +
    14. If the import parameters are acceptable enter y to proceed with the import or + n to abort the import. If the import is confirmed then the source data will be + imported, processed and saved in the weewx database. Information on the progress of the import will be displayed similar to the following: 
      15. Records processed: 250; Unique records: 250; Last timestamp: 2010-02-09 19:25:00 AEST (1265707500)
      -

      The line commencing with Records processed - should update as records are imported with progress information on - number of records processed, number of unique records imported and - the date time of the latest record processed. When the import is - complete a brief summary is displayed similar to the following:

      +

      The line commencing with Records processed should update as records are + imported with progress information on number of records processed, number of unique records imported and + the date time of the latest record processed. When the import is complete a brief summary is displayed + similar to the following:

      Records processed: 70685; Unique records: 70685; Last timestamp: 2010-09-04 04:20:00 AEST (1283538000)
 Finished import. 70685 raw records resulted in 70685 unique records being processed in 276.63 seconds.
 Those records with a timestamp already in the archive will not have been imported.
 Confirm successful import in the weewx log file.
      -
    15. Whilst wee_import will advise of the - number of records processed and the number of unique records found, - wee_import does know how many, if any, of - the imported records were successfully saved to the database. The - user should look carefully through the - weewx log file covering the - wee_import session and take note of any - records that were not imported. The most common reason for imported - records not being saved to the database is because a record with - that timestamp already exists in the database, in such cases - something similar to following will be found in the log: +
    16. Whilst wee_import will advise of the number of records processed and the + number of unique records found, wee_import does know how many, if any, of the + imported records were successfully saved to the database. The user should look carefully through the + weewx log file covering the wee_import session and + take note of any records that were not imported. The most common reason for imported records not being + saved to the database is because a record with that timestamp already exists in the database, in such + cases something similar to following will be found in the log: 
      17.  Aug 22 14:38:28 jessie2 weewx[863]: manager: unable to add record 2010-09-04 04:20:00 AEST (1283538000) to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime
      -

      In such cases the user should take note of the timestamp of the - record(s) concerned and make a decision about whether to delete the - pre-existing record and re-import the record or retain the - pre-existing record.

      +

      In such cases the user should take note of the timestamp of the record(s) concerned and make a decision + about whether to delete the pre-existing record and re-import the record or retain the pre-existing + record.

    Importing from Weather Underground

    -

    wee_import can import data from - the daily history of a Weather Undeground PWS. A Weather Underground - daily history provides weather station observations received by Weather - Underground for the PWS concerned on a day by day basis. As such, the - data is analogous to the weewx archive table. - When wee_import imports data from a Weather - Underground daily history each day is considered a 'period'. - wee_import processes one period at a time in - chronological order (oldest to newest) and provides import summary data - on a per period basis. -

    +

    wee_import can import data from the daily history of a Weather Undeground PWS. A + Weather Underground daily history provides weather station observations received by Weather Underground for + the PWS concerned on a day by day basis. As such, the data is analogous to the weewx archive table. When wee_import imports data from + a Weather Underground daily history each day is considered a 'period'. wee_import + processes one period at a time in chronological order (oldest to newest) and provides import summary data on + a per period basis.

    weewx archive fields populated during a Weather Underground import

    -

    A Weather Underground import will populate - weewx archive fields as follows:

    +

    A Weather Underground import will populate weewx archive fields as follows:

      -
    • Provided data exists for each field in the Weather Underground - PWS daily history, the following weewx - archive fields will be directly populated by imported data: +
    • Provided data exists for each field in the Weather Underground PWS daily history, the following weewx archive fields will be directly populated by imported data:
      • dateTime
        • @@ -2354,21 +2124,15 @@ Aug 22 14:38:28 jessie2 weewx[863]: manager: unable to add record 2010-09-04 04:

      - Note
      If an appropriate field does not exist in - the Weather Underground daily history then the corresponding - weewx archive field will be set - to None/NULL. - For example, if there is no solar radiation sensor - then radiation will be NULL, - or if outHumidity was never - uploaded to Weather Undeground then - outHumidity will be NULL. -

      + Note
      If an appropriate field does not exist in the Weather Underground daily history then + the corresponding weewx archive field will be set to None/NULL. + For example, if there is no solar radiation sensor then radiation will be + NULL, or if outHumidity was never uploaded to Weather Undeground then + outHumidity will be NULL.

      • -
    • The following weewx archive fields - will be populated from other settings or configuration - options: +
    • The following weewx archive fields will be populated from other settings or + configuration options:
        • @@ -2376,11 +2140,9 @@ Aug 22 14:38:28 jessie2 weewx[863]: manager: unable to add record 2010-09-04 04:
      • usUnits
      -
    • The following weewx archive fields - will be populated with values derived from the imported data - provided calc_missing = True is - included in the [WU] section of the - import configuration file and the field exists in the +
    • The following weewx archive fields will be populated with values derived from + the imported data provided calc_missing = True is included in the [WU] section of the import configuration file and the field exists in the in-use weewx archive table schema.
      • @@ -2394,40 +2156,30 @@ Aug 22 14:38:28 jessie2 weewx[863]: manager: unable to add record 2010-09-04 04:

    - Note
    If - calc_missing = False is included - in the [WU] section of the import - configuration file being used then all of the above fields - will be set to None/NULL. - The default setting of the - calc_missing option is - True + Note
    If calc_missing = False is included in the [WU] + section of the import configuration file being used then all of the above fields will be set to None/NULL. The default setting of the calc_missing + option is True

    Importing observations from Weather Underground

    -

    To import observations from the daily history of a Weather Underground - PWS:

    +

    To import observations from the daily history of a Weather Underground PWS:

      -
    1. Obtain the weather station ID of the Weather Underground PWS from - which data is to be imported. The station ID will be a sequence of - numbers and upper case letters that is usually 11 or 12 characters - in length. For the purposes of these instructions a weather station - ID of ISTATION123 will be used. +
    2. Obtain the weather station ID of the Weather Underground PWS from which data is to be imported. The + station ID will be a sequence of numbers and upper case letters that is usually 11 or 12 characters in + length. For the purposes of these instructions a weather station ID of ISTATION123 will be used.
      3. -
    3. Make a backup of the - weewx database in case the - import should go awry. +
    4. Make a backup of the weewx database in case the import should go awry.
      5. -
    5. Create an import configuration file. In this case - we will make a copy of the example Weather Underground import - configuration file and save it as - wu.conf in the - /var/tmp directory: +
    6. Create an import configuration file. In this case we will make a copy of the example Weather Underground + import configuration file and save it as wu.conf in the /var/tmp + directory: 
      $ cp /home/weewx/util/import/wu-example.conf /var/tmp/wu.conf
      @@ -2438,55 +2190,42 @@ Aug 22 14:38:28 jessie2 weewx[863]: manager: unable to add record 2010-09-04 04: 
      source = WU
      -
    7. Confirm that the following options in the - [WU] section are correctly set: +
    8. Confirm that the following options in the [WU] section are correctly set:
        9. -
      • station_id. - The 11 or 12 character weather station ID of the Weather - Underground PWS that will be the source of the imported +
      • station_id. The 11 or 12 + character weather station ID of the Weather Underground PWS that will be the source of the imported data.
        • -
      • interval. - Determines how the weewx interval - field is derived. +
      • interval. Determines how the + weewx interval field is derived.
        • -
      • qc. Determines - whether quality control checks are performed on the imported - data. +
      • qc. Determines whether quality + control checks are performed on the imported data.

        • - Note
        As Weather Underground imports at times - contain nonsense values, particularly for fields for - which no data were uploaded to Weather Underground by the - PWS, the use of quality control checks on imported data - can prevent these nonsense values from being imported - and contaminating the weewx - database. -

        + Note
        As Weather Underground imports at times contain nonsense values, particularly for + fields for which no data were uploaded to Weather Underground by the PWS, the use of quality control + checks on imported data can prevent these nonsense values from being imported and contaminating the + weewx database.

        -
      • calc_missing. - Determines whether missing derived observations will be - calculated from the imported data. +
      • calc_missing. Determines + whether missing derived observations will be calculated from the imported data.
        • -
      • tranche. The - number of records written to the - weewx database in each transaction. +
      • tranche. The number of records + written to the weewx database in each transaction.
      • wind_direction. - Determines how imported wind direction fields are - interpreted. + Determines how imported wind direction fields are interpreted.
      -
    9. When first importing data it is prudent to do a dry run import - before any data are actually imported. A dry run import will perform - all steps of the import without actually writing imported data to - the weewx database. In addition, - consideration should be given to any additional command line options - to be used such as --date. +
    10. When first importing data it is prudent to do a dry run import before any data are actually imported. A + dry run import will perform all steps of the import without actually writing imported data to the weewx database. In addition, consideration should be given to any additional + command line options to be used such as --date.

      11. To perform a dry run enter the following command:

      @@ -2494,40 +2233,26 @@ Aug 22 14:38:28 jessie2 weewx[863]: manager: unable to add record 2010-09-04 04:

      - Note
      If the --date command - line option is omitted or a date (not date-time) range is used - during a Weather Underground import, then the current days - history data will be imported. This includes records timestamped - from 00:00 (inclusive) at the start - of the day up to but NOT including the - 00:00 record at the end of the last - day. As the timestamped record refers to observations of the - previous interval, such an import actually includes one record - with observations from the previous day (the - 00:00 record at the start of the day). - Whilst this will not present a problem for - wee_import as any records being - imported with a timestamp that already exists in the - weewx database are ignored, the user - may wish to use the --date option - with a suitable date-time range to precisely control which - records are imported. -

      + Note
      If the --date command line option is omitted or a date (not + date-time) range is used during a Weather Underground import, then the current days history data will be + imported. This includes records timestamped from 00:00 (inclusive) at the + start of the day up to but NOT including the 00:00 record at the end of the + last day. As the timestamped record refers to observations of the previous interval, such an import + actually includes one record with observations from the previous day (the 00:00 record at the start of the day). Whilst this will not present a problem + for wee_import as any records being imported with a timestamp that already + exists in the weewx database are ignored, the user may wish to use the --date option with a suitable date-time range to precisely control which records + are imported.

      - Note
      wee_import obtains - Weather Underground daily history data one day at a time via a - HTTP request and as such the import of large time spans of data - may take some time. Such imports may be best handled as a series - of imports of smaller time spans. -

      + Note
      wee_import obtains Weather Underground daily history data one + day at a time via a HTTP request and as such the import of large time spans of data may take some time. + Such imports may be best handled as a series of imports of smaller time spans.

      -

      This will result in a short preamble with details - on the data source, its destination and some other details on - how the data will be processed. The import will then be - performed but no data will written to the - weewx database. -

      +

      This will result in a short preamble with details on the data source, its destination and some other + details on how the data will be processed. The import will then be performed but no data will written to + the weewx database.

      The output should be similar to:

      Starting wee_import...
@@ -2552,26 +2277,21 @@ Finished dry run import. 607 records were processed and 607 unique records would

      - Note
      As the weewx database is - not altered when the --dry-run command - line option is used, wee_import log output - is suspended during a dry run import. In effect, the use of - --dry-run is equivalent to - --dry-run --log=-. During a dry run import - the only wee_import output is that - displayed on stdout(console). + Note
      As the weewx database is not altered when the --dry-run command line option is used, wee_import log + output is suspended during a dry run import. In effect, the use of --dry-run + is equivalent to --dry-run --log=-. During a dry run import the only wee_import output is that displayed on stdout(console).

      -
    11. Once the dry run results are satisfactory the source data can be - imported using the following command: +
    12. Once the dry run results are satisfactory the source data can be imported using the following command: 
      13. wee_import --import-config=/var/tmp/wu.conf --date="2016/01/20 22:30-2016/01/23 06:00"
      -

      This will result in a short preamble similar to that of - a dry run. At the end of the preamble there will be a - prompt:

      +

      This will result in a short preamble similar to that of a dry run. At the end of the preamble there will + be a prompt:

      Starting wee_import...
 Observation history for Weather Underground station 'ISTATION123' will be imported.
@@ -2587,19 +2307,13 @@ Are you sure you want to proceed (y/n)?

      - Note
      wee_import obtains - Weather Underground daily history data one day at a time via a HTTP - request and as such the import of large time spans of data may take - some time. Such imports may be best handled as a series of imports - of smaller time spans. -

      + Note
      wee_import obtains Weather Underground daily history data one + day at a time via a HTTP request and as such the import of large time spans of data may take some time. + Such imports may be best handled as a series of imports of smaller time spans.

      -
    13. If the import parameters - are acceptable enter y to proceed with the - import or n to abort the import. - If the import is confirmed, the source - data will be imported, processed and saved in the - weewx database. Information on the +
    14. If the import parameters are acceptable enter y to proceed with the import or + n to abort the import. If the import is confirmed, the source data will be + imported, processed and saved in the weewx database. Information on the progress of the import will be displayed similar to the following:
      15. @@ -2609,13 +2323,11 @@ Period 2 ... Records processed: 286; Unique records: 286; Last timestamp: 2016-01-21 23:55:00 AEST (1453384500) -

      The line commencing with Records processed - should update as records are imported with progress information on - number of records processed, number of unique records imported and - the date time of the latest record processed. If the import spans - multiple days then a new Period line is - created for each day. When the import is complete a brief summary - is displayed similar to the following:

      +

      The line commencing with Records processed should update as records are + imported with progress information on number of records processed, number of unique records imported and + the date time of the latest record processed. If the import spans multiple days then a new Period line is created for each day. When the import is complete a brief + summary is displayed similar to the following:

      Finished import. 607 raw records resulted in 607 unique records being processed in 80.94 seconds.
 Those records with a timestamp already in the archive will not have been imported.
@@ -2623,68 +2335,53 @@ Confirm successful import in the weewx log file.

      - Note
      It is not unusual to see a Weather Underground - import return a different number of records for the same import - performed at different times. If importing the current day this - could be because an additional record may have been added between - wee_import runs. For periods before today, - this behaviour appears to be a vagary of Weather Underground. The - only solution appears to be to repeat the import with the same - --date command line option setting and - observe whether the missing records are imported. Repeating the - import will not adversely affect any existing data as records with - timestamps that are already in the weewx - archive will be ignored. It may; however, generated many - UNIQUE constraint failed: archive.dateTime - messages in the weewx log. -

      + Note
      It is not unusual to see a Weather Underground import return a different number of + records for the same import performed at different times. If importing the current day this could be + because an additional record may have been added between wee_import runs. For + periods before today, this behaviour appears to be a vagary of Weather Underground. The only solution + appears to be to repeat the import with the same --date command line option + setting and observe whether the missing records are imported. Repeating the import will not adversely + affect any existing data as records with timestamps that are already in the weewx archive will be ignored. It may; however, generated many UNIQUE constraint failed: archive.dateTime messages in the weewx + log.

      -
    15. Whilst wee_import will advise of the - number of records processed and the number of unique records found, - wee_import does know how many, if any, of - the imported records were successfully saved to the database. The - user should look carefully through the - weewx log file covering the - wee_import session and take note of any - records that were not imported. The most common reason for imported - records not being saved to the database is because a record with - that timestamp already exists in the database, in such cases - something similar to following will be found in the log: +
    16. Whilst wee_import will advise of the number of records processed and the + number of unique records found, wee_import does know how many, if any, of the + imported records were successfully saved to the database. The user should look carefully through the + weewx log file covering the wee_import session and + take note of any records that were not imported. The most common reason for imported records not being + saved to the database is because a record with that timestamp already exists in the database, in such + cases something similar to following will be found in the log: 
      17.  Aug 22 14:38:28 jessie2 weewx[863]: manager: unable to add record 2010-09-04 04:20:00 AEST (1283538000) to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime
      -

      In such cases the user should take note of the timestamp of the - record(s) concerned and make a decision about whether to delete the - pre-existing record and re-import the record or retain the - pre-existing record.

      +

      In such cases the user should take note of the timestamp of the record(s) concerned and make a decision + about whether to delete the pre-existing record and re-import the record or retain the pre-existing + record.

    Importing from Cumulus

    -

    wee_import can import observational data from - the one or more Cumulus monthly log files. A Cumulus monthly log file - records weather station observations for a single month. These files are - accumulated over time and can be considered analogous to the - weewx archive table. When - wee_import imports data from the Cumulus - monthly log files each log file is considered a 'period'. - wee_import processes one period at a time in - chronological order (oldest to newest) and provides import summary - data on a per period basis.

    +

    wee_import can import observational data from the one or more Cumulus monthly log + files. A Cumulus monthly log file records weather station observations for a single month. These files are + accumulated over time and can be considered analogous to the weewx archive table. + When wee_import imports data from the Cumulus monthly log files each log file is + considered a 'period'. wee_import processes one period at a time in chronological + order (oldest to newest) and provides import summary data on a per period basis.

    weewx archive fields populated during a Cumulus monthly log import

    -

    A Cumulus monthly log file import will populate the - weewx archive fields as follows:

    +

    A Cumulus monthly log file import will populate the weewx archive fields as + follows:

      -
    • Provided data exists for each field in the Cumulus monthly - logs, the following weewx - archive fields will be directly populated by imported data: +
    • Provided data exists for each field in the Cumulus monthly logs, the following weewx archive fields will be directly populated by imported data:
      • dateTime
        • @@ -2696,8 +2393,7 @@ Aug 22 14:38:28 jessie2 weewx[863]: manager: unable to add record 2010-09-04 04:
      • outHumidity
      • outTemp
      • radiation
        • -
      • rain (Cumulus 1.9.4 or - later) +
      • rain (Cumulus 1.9.4 or later)
      • rainRate
      • UV
        • @@ -2708,15 +2404,12 @@ Aug 22 14:38:28 jessie2 weewx[863]: manager: unable to add record 2010-09-04 04:

      - Note
      If a field in the Cumulus monthly log file - has no data then the corresponding - weewx archive field will be set - to None/NULL. -

      + Note
      If a field in the Cumulus monthly log file has no data then the corresponding weewx archive field will be set to None/NULL.

      • -
    • The following weewx archive fields - will be populated from other settings or configuration options: +
    • The following weewx archive fields will be populated from other settings or + configuration options:
        • @@ -2724,13 +2417,10 @@ Aug 22 14:38:28 jessie2 weewx[863]: manager: unable to add record 2010-09-04 04:
      • usUnits
      -
    • The following weewx archive fields - will be populated with values derived from the imported data - provided calc_missing = True is - included in the [Cumulus] section of - the import configuration file being used and the field exists - in the in-use weewx archive table - schema. +
    • The following weewx archive fields will be populated with values derived from + the imported data provided calc_missing = True is included in the [Cumulus] section of the import configuration file being used and the field + exists in the in-use weewx archive table schema.
        • @@ -2740,14 +2430,10 @@ Aug 22 14:38:28 jessie2 weewx[863]: manager: unable to add record 2010-09-04 04:

      - Note
      If - calc_missing = False is included - in the [WU] section of the import - configuration file being used then all of the above fields - will be set to None/NULL. - The default setting of the - calc_missing option is - True + Note
      If calc_missing = False is included in the [WU] + section of the import configuration file being used then all of the above fields will be set to None/NULL. The default setting of the calc_missing + option is True

    @@ -2756,23 +2442,18 @@ Aug 22 14:38:28 jessie2 weewx[863]: manager: unable to add record 2010-09-04 04:

    To import observations from one or more Cumulus monthly log files:

      -
    1. Ensure the Cumulus monthly log file(s) to be used for the import - are located in a directory accessible by the machine that will run - wee_import. For the purposes of these - instructions, there are nine monthly logs files covering the - period November 2015 to July 2016, inclusive, located in the - /var/tmp/cumulus folder. +
    2. Ensure the Cumulus monthly log file(s) to be used for the import are located in a directory accessible + by the machine that will run wee_import. For the purposes of these + instructions, there are nine monthly logs files covering the period November 2015 to July 2016, + inclusive, located in the /var/tmp/cumulus folder.
      3. -
    3. Make a backup of the - weewx database in case the - import should go awry. +
    4. Make a backup of the weewx database in case the import should go awry.
      5. -
    5. Create an import configuration file. In this case - we will make a copy of the example Cumulus import configuration file - and save it as cumulus.conf in the - /var/tmp directory: +
    6. Create an import configuration file. In this case we will make a copy of the example Cumulus import + configuration file and save it as cumulus.conf in the /var/tmp + directory: 
      $ cp /home/weewx/util/import/cumulus-example.conf /var/tmp/cumulus.conf
      @@ -2782,70 +2463,56 @@ Aug 22 14:38:28 jessie2 weewx[863]: manager: unable to add record 2010-09-04 04: 
      source = Cumulus
      -
    7. Confirm that the following options in the - [Cumulus] section are correctly set: +
    8. Confirm that the following options in the [Cumulus] section are correctly set:
        9. -
      • directory. - The full path to the directory containing the Cumulus - monthly log files to be used as the source of the imported +
      • directory. The full path + to the directory containing the Cumulus monthly log files to be used as the source of the imported data.
        • -
      • interval. - Determines how the weewx interval - field is derived. +
      • interval. Determines how + the weewx interval field is derived.
        • -
      • qc. Determines - whether quality control checks are performed on the imported - data. +
      • qc. Determines whether quality + control checks are performed on the imported data.
      • calc_missing. - Determines whether missing derived observations will be - calculated from the imported data. + Determines whether missing derived observations will be calculated from the imported data.
        • -
      • delimiter. The - field delimiter used in the Cumulus monthly log files. +
      • delimiter. The field + delimiter used in the Cumulus monthly log files.
        • -
      • decimal. The - decimal point character used in the Cumulus monthly log - files. +
      • decimal. The decimal point + character used in the Cumulus monthly log files.
        • -
      • tranche. The - number of records written to the - weewx database in each transaction. +
      • tranche. The number of + records written to the weewx database in each transaction.
        • -
      • UV_sensor. - Whether a UV sensor was installed when the source data was - produced. +
      • UV_sensor. Whether a UV sensor + was installed when the source data was produced.
        • -
      • solar_sensor. - Whether a solar radiation sensor was installed when the - source data was produced. +
      • solar_sensor. Whether a + solar radiation sensor was installed when the source data was produced.
        • -
      • [[Units]]. - Defines the units used in the Cumulus monthly log files. +
      • [[Units]]. Defines the units + used in the Cumulus monthly log files.
      -
    9. When first importing data it is prudent to do a dry run import - before any data are actually imported. A dry run import will perform - all steps of the import without actually writing imported data to - the weewx database. In addition, - consideration should be given to any additional command line options - to be used such as --date. +
    10. When first importing data it is prudent to do a dry run import before any data are actually imported. A + dry run import will perform all steps of the import without actually writing imported data to the weewx database. In addition, consideration should be given to any additional + command line options to be used such as --date.

      11. To perform a dry run enter the following command:

      wee_import --import-config=/var/tmp/cumulus.conf --dry-run
      -

      This will result in a short preamble with - details on the data source, its destination and some other - details on how the data will be processed. The import will then - be performed but no data will be written to the - weewx database. -

      +

      This will result in a short preamble with details on the data source, its destination and some other + details on how the data will be processed. The import will then be performed but no data will be written + to the weewx database.

      The output should be similar to:

      Starting wee_import...
@@ -2878,19 +2545,16 @@ Finished dry run import. 37648 records were processed and 37620 unique records w

      - Note
      The nine periods correspond to the nine monthly log - files used for this import. -

      + Note
      The nine periods correspond to the nine monthly log files used for this import.

      -
    11. Once the dry run results are satisfactory the data can be - imported using the following command: +
    12. Once the dry run results are satisfactory the data can be imported using the following command: 
      13. wee_import --import-config=/var/tmp/cumulus.conf
      -

      This will result in a preamble similar to that of a dry run. - At the end of the preamble there will be a prompt:

      +

      This will result in a preamble similar to that of a dry run. At the end of the preamble there will be a + prompt:

      Starting wee_import...
 Cumulus monthly log files in the '/var/tmp/cumulus' directory will be imported
@@ -2903,14 +2567,11 @@ Proceeding will save all imported records in the weewx archive.
 Are you sure you want to proceed (y/n)?
      -

      If there is more than one Cumulus monthly log file then - wee_import will provide summary - information on a per period basis during the import. In addition, - if the --date command line option is used - then source data that falls outside the date or date range - specified with the --date command line - option is ignored. In such cases the preamble may look similar - to:

      +

      If there is more than one Cumulus monthly log file then wee_import will provide + summary information on a per period basis during the import. In addition, if the --date + command line option is used then source data that falls outside the date or date range specified with + the --date command line option is ignored. In such cases the preamble may look + similar to:

      Starting wee_import...
 Cumulus monthly log files in the '/var/tmp/cumulus' directory will be imported
@@ -2927,21 +2588,17 @@ Proceeding will save all imported records in the weewx archive.
 Are you sure you want to proceed (y/n)?
      -
    13. If the import parameters are acceptable enter - y to proceed with the - import or n to abort the import. - If the import is confirmed, the source - data will be imported, processed and saved in the - weewx database. Information on the +
    14. If the import parameters are acceptable enter y to proceed with the import or + n to abort the import. If the import is confirmed, the source data will be + imported, processed and saved in the weewx database. Information on the progress of the import will be displayed similar to the following: 
      15. Records processed: 1599; Unique records: 1599; Last timestamp: 2016-02-24 00:00:00 AEST (1456236000)
      -

      Again if there is more than one Cumulus monthly log file and if the - --date command line option is used then - the progress information may instead look similar to:

      +

      Again if there is more than one Cumulus monthly log file and if the --date + command line option is used then the progress information may instead look similar to:

      Period 4 ...
 Records processed: 2521; Unique records: 2521; Last timestamp: 2016-03-01 09:40:00 AEST (1456789200)
@@ -2951,371 +2608,261 @@ Period 6 ...
 Records processed: 3238; Unique records: 3232; Last timestamp: 2016-04-24 00:00:00 AEST (1461420000)
      -

      The line commencing with Records processed - should update as records are imported with progress information on - number of records processed, number of unique records imported and - the date time of the latest record processed. If the import spans - multiple months (ie multiple monthly log files) then a new - Period line is created for each month. - When the import is complete a brief summary is displayed similar to - the following:

      +

      The line commencing with Records processed should update as records are + imported with progress information on number of records processed, number of unique records imported and + the date time of the latest record processed. If the import spans multiple months (ie multiple monthly + log files) then a new Period line is created for each month. When the import + is complete a brief summary is displayed similar to the following:

      Finished import. 37648 raw records resulted in 37620 unique records being processed in 93.70 seconds.
 Those records with a timestamp already in the archive will not have been imported.
 Confirm successful import in the weewx log file.
      -
    15. Whilst wee_import will advise of the - number of records processed and the number of unique records found, - wee_import does know how many, if any, of - the imported records were successfully saved to the database. The - user should look carefully through the - weewx log file covering the - wee_import session and take note of any - records that were not imported. The most common reason for imported - records not being saved to the database is because a record with - that timestamp already exists in the database, in such cases - something similar to following will be found in the log: +
    16. Whilst wee_import will advise of the number of records processed and the + number of unique records found, wee_import does know how many, if any, of the + imported records were successfully saved to the database. The user should look carefully through the + weewx log file covering the wee_import session and + take note of any records that were not imported. The most common reason for imported records not being + saved to the database is because a record with that timestamp already exists in the database, in such + cases something similar to following will be found in the log: 
      17.  Aug 22 14:38:28 jessie2 weewx[863]: manager: unable to add record 2010-09-04 04:20:00 AEST (1283538000) to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime
      -

      In such cases take note of the timestamp of the - record(s) concerned and make a decision about whether to delete the - pre-existing record and re-import the record or retain the - pre-existing record.

      +

      In such cases take note of the timestamp of the record(s) concerned and make a decision about whether to + delete the pre-existing record and re-import the record or retain the pre-existing record.

    Dealing with import failures

    Sometimes bad things happen during an import.

    -

    If errors were encountered, or if you suspect that the - weewx database has been - contaminated with incorrect data, here are some things - you can try to fix things up. -

    +

    If errors were encountered, or if you suspect that the weewx database has been + contaminated with incorrect data, here are some things you can try to fix things up.

      -
    • Manually delete the contaminated data. Use SQL - commands to manipulate the data in the - weewx archive database. - The simplicity of this process will depend on your - ability to use SQL, the amount of data imported, and - whether the imported data was dispersed amongst - existing. Once contaminated data have been removed the - daily summary tables will need to be dropped and - rebuilt using the - wee_database utility. +
    • Manually delete the contaminated data. Use SQL commands to manipulate the data in the weewx archive database. The simplicity of this process will depend on your + ability to use SQL, the amount of data imported, and whether the imported data was dispersed amongst + existing. Once contaminated data have been removed the daily summary tables will need to be dropped and + rebuilt using the wee_database utility.
      • -
    • Delete the database and start over. For SQLite, - simply delete the database file. For MySQL, drop - the database. Then try the import again. +
    • Delete the database and start over. For SQLite, simply delete the database file. For MySQL, drop the + database. Then try the import again.
      • -
    • If the above steps are not appropriate then the - database should be restored from backup. You did - make a backup before starting the import? +
    • If the above steps are not appropriate then the database should be restored from backup. You did make a + backup before starting the import?

    The import configuration file

    wee_import requires a second configuration file, the import configuration file, in - addition to the standard weewx configuration file. The import configuration file - specifies the import type and various options associated with - each type of import. The import configuration file is specified at the - wee_import command line using the mandatory - --import-config command line option. How the - user constructs the import configuration file is up to the user; - however, the recommended method is to copy one of the example import - configuration files located in the util/import - folder, modify the configuration options in the newly copied file to - suit the import to be performed and then use this file as the import - configuration file. -

    + addition to the standard weewx configuration file. The import configuration file specifies the import type + and various options associated with each type of import. The import configuration file is specified at the + wee_import command line using the mandatory --import-config command line option. How the user constructs the import + configuration file is up to the user; however, the recommended method is to copy one of the example import + configuration files located in the util/import folder, modify the configuration + options in the newly copied file to suit the import to be performed and then use this file as the import + configuration file.

    -

    Following is the definitive guide to the options available in the import - configuration file. Default values are provided for a number of options, - meaning that if they are not listed in the import configuration file at - all wee_import will pick sensible values. - When the documentation below gives a "default value" this is what it - means. What follows is organized by the different sections of the import - configuration file. -

    +

    Following is the definitive guide to the options available in the import configuration file. Default values + are provided for a number of options, meaning that if they are not listed in the import configuration file + at all wee_import will pick sensible values. When the documentation below gives a + "default value" this is what it means. What follows is organized by the different sections of the import + configuration file.

    General

    source

    -

    The source option determines the type of import - to be performed by wee_import. The option - must be set to one of the following: -

    +

    The source option determines the type of import to be performed by wee_import. The option must be set to one of the following:

      -
    • CSV to import from a single CSV format - file. +
    • CSV to import from a single CSV format file.
      • -
    • WU to import from a Weather - Underground PWS daily history. +
    • WU to import from a Weather Underground PWS daily history.
      • -
    • Cumulus to import from one or more - Cumulus monthly log files. +
    • Cumulus to import from one or more Cumulus monthly log files.

    There is no default.

    [CSV]

    -

    The [CSV] section contains the - options relating to the import of observational data from a CSV format - file. -

    +

    The [CSV] section contains the options relating to the import of + observational data from a CSV format file.

    file

    -

    The file containing the CSV format data to be used as the source during - the import. Include full path and filename. There is no default. -

    +

    The file containing the CSV format data to be used as the source during the import. Include full path and + filename. There is no default.

    interval

    -

    Determines how the time interval (weewx - archive table field interval) between - successive observations is derived. The interval can be derived by one - of three methods: -

    +

    Determines how the time interval (weewx archive table field interval) + between successive observations is derived. The interval can be derived by one of three methods:

      -
    • The interval can be calculated as the time, rounded to the - nearest minute, between the date-time of successive records. - This method is suitable when the data was recorded at fixed - intervals and there are NO missing records in the source data. - Use of this method when there are missing records in the source - data can compromise the integrity of the - weewx statistical data. Select this +
    • The interval can be calculated as the time, rounded to the nearest minute, between the date-time of + successive records. This method is suitable when the data was recorded at fixed intervals and there are + NO missing records in the source data. Use of this method when there are missing records in the source + data can compromise the integrity of the weewx statistical data. Select this method by setting interval = derive.
      • -
    • The interval can be set to the same value as the - archive_interval setting under - [StdArchive] in - weewx.conf. This setting is useful if - the data was recorded at fixed intervals but there are some - missing records and the fixed interval is the same as the - archive_interval setting under - [StdArchive] in - weewx.conf. Select this method by - setting interval = conf. +
    • The interval can be set to the same value as the archive_interval setting + under [StdArchive] in weewx.conf. This setting is + useful if the data was recorded at fixed intervals but there are some missing records and the fixed + interval is the same as the archive_interval setting under [StdArchive] + in weewx.conf. Select this method by setting interval = conf.
      • -
    • The interval can be set to a fixed number of minutes. This - setting is useful if the source data was recorded at fixed - intervals but there are some missing records and the fixed - interval is different to the - archive_interval setting under - [StdArchive] in - weewx.conf. Select this method by - setting interval = x where - x is an integer number of minutes. +
    • The interval can be set to a fixed number of minutes. This setting is useful if the source data was + recorded at fixed intervals but there are some missing records and the fixed interval is different to + the archive_interval setting under [StdArchive] in + weewx.conf. Select this method by setting interval = x where x is an integer number of + minutes.
    -

    The default value is derive. If the CSV source - data records are equally spaced in time, but some records are missing, - then better result may be achieved using conf - or a fixed interval setting. -

    +

    The default value is derive. If the CSV source data records are equally spaced in + time, but some records are missing, then better result may be achieved using conf + or a fixed interval setting.

    qc

    -

    Determines whether simple quality control checks are applied to imported - data. Setting qc = True will result in - wee_import applying the - weewx StdQC minimum - and maximum checks to any imported observations. - wee_import quality control checks use the same - configuration settings, and operate in the same manner, as the - StdQC service. For example, for min/max QC, if - an observation falls - outside of the quality control range for that observation, then the - observation will be set to None. The user will - be alerted through a short message similar to:

    +

    Determines whether simple quality control checks are applied to imported data. Setting qc = True + will result in wee_import applying the weewx StdQC minimum and maximum checks to any imported observations. wee_import quality control checks use the same configuration settings, and + operate in the same manner, as the StdQC + service. For example, for min/max QC, if an observation falls outside of the quality control range for that + observation, then the observation will be set to None. The user will be alerted + through a short message similar to:

    2016-01-12 10:00:00 AEST (1452556800) record value 'outTemp' 194.34 outside limits (0.0, 120.0)
    -

    As derived observations are calculated after the quality control check is applied, derived - observations are not subject to quality control checks. Setting - qc = False will result in - wee_import not applying quality control checks - to imported data. The default is True. -

    +

    As derived observations are calculated after the quality control check is applied, derived observations are + not subject to quality control checks. Setting qc = False will result in wee_import not applying quality control checks to imported data. The default is + True.

    calc_missing

    -

    Determines whether any missing derived observations will be calculated - from the imported data. Setting - calc_missing = True will result in - wee_import using the weewx - StdWXCalculate service to calculate any - missing derived observations from the imported data. Setting - calc_missing = False will result in - weewx leaving any missing derived observations - as None. The observations that - StdWXCalculate can calculate are listed in the - [StdWXCalculate] section - of the User's Guide. The default is - True. -

    +

    Determines whether any missing derived observations will be calculated from the imported data. Setting calc_missing = True will result in wee_import using the + weewx StdWXCalculate service to calculate any missing + derived observations from the imported data. Setting calc_missing = False will + result in weewx leaving any missing derived observations as None. + The observations that StdWXCalculate can calculate are listed in the [StdWXCalculate] section of the User's + Guide. The default is True.

    tranche

    -

    To speed up database operations imported records are committed to - database in groups of records rather than individually. The size of the - group is set by the tranche parameter. - Increasing the tranche parameter may result - in a slight speed increase but at the expense of increased memory usage. - Decreasing the tranche parameter will result - in less memory usage but at the expense of more frequent database access - and likely increased time to import. The default is - 250 which should suit most users. -

    +

    To speed up database operations imported records are committed to database in groups of records rather than + individually. The size of the group is set by the tranche parameter. Increasing + the tranche parameter may result in a slight speed increase but at the expense of + increased memory usage. Decreasing the tranche parameter will result in less + memory usage but at the expense of more frequent database access and likely increased time to import. The + default is 250 which should suit most users.

    UV_sensor

    -

    weewx records a - None/NULL for UV when no UV sensor is - installed, whereas some weather station software records a value of 0 - for UV index when there is no UV sensor installed. The - UV_sensor parameter enables - wee_import to distinguish between the case - where a UV sensor is present and the UV index is 0 and the case where - no UV sensor is present and UV index is 0. - UV_sensor = False should be used when no UV - sensor was used in producing the source data. - UV_sensor = False will result in - None/Null being recorded in the - weewx archive field - UV irrespective of any UV observations in the - source data. UV_sensor = True should be used - when a UV sensor was used in producing the source data. - UV_sensor = True will result in UV - observations in the source data being stored in the - weewx archive field - UV. The default is - True. -

    +

    weewx records a None/NULL for UV when no UV sensor is + installed, whereas some weather station software records a value of 0 for UV index when there is no UV + sensor installed. The UV_sensor parameter enables wee_import to distinguish between the case where a UV sensor is present and the + UV index is 0 and the case where no UV sensor is present and UV index is 0. UV_sensor = False + should be used when no UV sensor was used in producing the source data. UV_sensor = False will result in None/Null being + recorded in the weewx archive field UV irrespective of + any UV observations in the source data. UV_sensor = True should be used when a UV + sensor was used in producing the source data. UV_sensor = True will result in UV + observations in the source data being stored in the weewx archive field UV. The default is True.

    solar_sensor

    -

    weewx records a - None/NULL when no solar radiation sensor is - installed, whereas some weather station software records a value of 0 - for solar radiation when there is no solar radiation sensor installed. - The solar_sensor parameter enables - wee_import to distinguish between the case - where a solar radiation sensor is present and solar radiation is 0 and - the case where no solar radiation sensor is present and solar radiation - is 0. solar_sensor = False should be used - when no solar radiation sensor was used in producing the source data. - solar_sensor = False will result in - None/Null being recorded in the - weewx archive field - radiation irrespective of any solar radiation - observations in the source data. - solar_sensor = True should be used when a - solar radiation sensor was used in producing the source data. - solar_sensor = True will result in solar - radiation observations in the source data being stored in the - weewx archive field - radiation. The default is - True. -

    +

    weewx records a None/NULL when no solar radiation sensor + is installed, whereas some weather station software records a value of 0 for solar radiation when there is + no solar radiation sensor installed. The solar_sensor parameter enables wee_import to distinguish between the case where a solar radiation sensor is + present and solar radiation is 0 and the case where no solar radiation sensor is present and solar radiation + is 0. solar_sensor = False should be used when no solar radiation sensor was used + in producing the source data. solar_sensor = False will result in None/Null being recorded in the weewx archive field + radiation irrespective of any solar radiation observations in the source data. + solar_sensor = True should be used when a solar radiation sensor was used in + producing the source data. solar_sensor = True will result in solar radiation + observations in the source data being stored in the weewx archive field radiation. The default is True.

    raw_datetime_format

    -

    weewx records each record with a unique unix - epoch timestamp, whereas many weather station applications or web - sources export observational data with a human readable date-time. - This human readable date-time is interpreted according to the format - set by the raw_datetime_format - option. This option consists of - Python - strptime() format codes - and literal characters to represent the date-time data being - imported. For example, if the source data uses the format - 23 January 2015 15:34 then the appropriate setting for - raw_datetime_format would be - %d %B %Y %H:%M, 9:25:00 12/28/16 would use - %H:%M:%S %m/%d/%y. If the source data provides - a unix epoch timestamp as the date-time field then the unix epoch - timestamp is used directly and the - raw_datetime_format option is - ignored. The default is %Y-%m-%d %H:%M:%S. -

    +

    weewx records each record with a unique unix epoch timestamp, whereas many weather + station applications or web sources export observational data with a human readable date-time. This human + readable date-time is interpreted according to the format set by the raw_datetime_format + option. This option consists of Python + strptime() format codes and literal characters to represent the date-time data being imported. + For example, if the source data uses the format 23 January 2015 15:34 then the appropriate setting for raw_datetime_format would be %d %B %Y %H:%M, + 9:25:00 12/28/16 would use %H:%M:%S %m/%d/%y. If the source data provides a unix + epoch timestamp as the date-time field then the unix epoch timestamp is used directly and the raw_datetime_format option is ignored. The default is %Y-%m-%d %H:%M:%S.

    rain

    -

    weewx records rainfall as the amount of rain - in the preceding archive period, so for a 5 minute archive period the - rain field in each archive record would contain the total rain that - fell in the previous 5 minutes. Many weather station applications - provide a daily or yearly total. wee_import - can derive the weewx - rain field in one of two ways: -

    +

    weewx records rainfall as the amount of rain in the preceding archive period, so + for a 5 minute archive period the rain field in each archive record would contain the total rain that fell + in the previous 5 minutes. Many weather station applications provide a daily or yearly total. wee_import can derive the weewx rain + field in one of two ways:

      -
    • If the imported rain data is a running total then - wee_import can derive the - weewx rain field from successive - totals. For this method use - rain = cumulative. +
    • If the imported rain data is a running total then wee_import can derive the + weewx rain field from successive totals. For this method use rain = cumulative.
      • -
    • If the imported rain data is a discrete value per date-time - period then rain = discrete +
    • If the imported rain data is a discrete value per date-time period then rain = discrete should be used.

    - Note
    wee_import only supports - cumulative rainfall data that resets on a midnight boundary, cumulative - rainfall data that resets at some other time; e.g., 9am, is not supported. - In such cases the rainfall data will need to be converted to either - reset on a midnight boundary or a discrete value per date-time period - and rain = discrete used. The former - may be possible by selecting another rainfall field (if available) in - the source data, otherwise it will require manual manipulation of the - source data. -

    + Note
    wee_import only supports cumulative rainfall data that resets on a + midnight boundary, cumulative rainfall data that resets at some other time; e.g., 9am, is not supported. In + such cases the rainfall data will need to be converted to either reset on a midnight boundary or a discrete + value per date-time period and rain = discrete used. The former may be + possible by selecting another rainfall field (if available) in the source data, otherwise it will require + manual manipulation of the source data.

    wind_direction

    -

    weewx records wind direction in degrees as a - number from 0 to 360 inclusive (no wind direction is recorded as - None/NULL), whereas some data sources may - provide wind direction as number over a different range (e.g., -180 to - +180) or may use a particular value when there is no wind direction - (e.g., 0 may represent no wind direction and 360 may represent a northerly - wind, or -9999 (or some similar clearly invalid number) to represent - there being no wind direction). wee_import - handles such variations in data by defining a range over which imported - wind direction values are accepted. Any value outside of this range is - treated as there being no wind direction and is recorded as - None/NULL. Any value inside the range is - normalised to the range 0 to 360 inclusive (e.g., -180 would be normalised - to 180). The wind_direction option - consists of two comma separated numbers of the format lower, upper where - lower and upper are inclusive. The operation of the - wind_direction option is best - illustrated through the following table: -

    +

    weewx records wind direction in degrees as a number from 0 to 360 inclusive (no + wind direction is recorded as None/NULL), whereas some data sources may provide + wind direction as number over a different range (e.g., -180 to +180) or may use a particular value when + there is no wind direction (e.g., 0 may represent no wind direction and 360 may represent a northerly wind, + or -9999 (or some similar clearly invalid number) to represent there being no wind direction). wee_import handles such variations in data by defining a range over which + imported wind direction values are accepted. Any value outside of this range is treated as there being no + wind direction and is recorded as None/NULL. Any value inside the range is + normalised to the range 0 to 360 inclusive (e.g., -180 would be normalised to 180). The wind_direction option consists of two comma separated numbers of the + format lower, upper where lower and upper are inclusive. The operation of the wind_direction + option is best illustrated through the following table:

    @@ -3414,265 +2961,192 @@ Aug 22 14:38:28 jessie2 weewx[863]: manager: unable to add record 2010-09-04 04:
    -

    The default is 0, 360. -

    +

    The default is 0, 360.

    [[Map]]

    -

    The [[Map]] stanza defines the mapping - from the source data fields to weewx archive - fields. The map consists of one row per field using the format: -

    +

    The [[Map]] stanza defines the mapping from the source data fields to + weewx archive fields. The map consists of one row per field using the format: 

    weewx_archive_field_name = csv_field_name, weewx_unit_name
    -

    Where weewx_archive_field_name is a - database field name in the weewx archive table - schema, csv_field_name is the name - of a field from the CSV file and - weewx_unit_name is the - weewx unit name of the units used by - csv_field_name. This mapping allows - wee_import to take a source data field, do - the appropriate unit conversion and store the resulting value in the - appropriate weewx archive field. A mapping is - not required for every weewx archive field - (e.g., the source may not provide inside temperature so no - inTemp field mapping is required) and neither - does every CSV field need to be included in a mapping (e.g., the source - data field monthrain may have no use if the - source data field dayrain provides the data - for the weewx archive - rain field). Unused field mapping lines will - not be used and may be omitted. -

    +

    Where weewx_archive_field_name is a database field name in the weewx archive table schema, csv_field_name is the + name of a field from the CSV file and weewx_unit_name is the weewx unit name of the units used by csv_field_name. This mapping allows wee_import to take a source data field, do the appropriate unit conversion and + store the resulting value in the appropriate weewx archive field. A mapping is not + required for every weewx archive field (e.g., the source may not provide inside + temperature so no inTemp field mapping is required) and neither does every CSV + field need to be included in a mapping (e.g., the source data field monthrain may + have no use if the source data field dayrain provides the data for the weewx archive rain field). Unused field mapping lines + will not be used and may be omitted.

    - Note
    Any weewx archive fields that - are derived (e.g., dewpoint) and for which there - is no field mapping may be calculated during import by use of the - calc_missing option in the - [CSV] section of the import configuration - file. -

    + Note
    Any weewx archive fields that are derived (e.g., dewpoint) and for which there is no field mapping may be calculated during import by + use of the calc_missing option in the [CSV] + section of the import configuration file.

    [WU]

    -

    The [WU] section contains the - options relating to the import of observational data from a Weather - Underground PWS daily history. -

    +

    The [WU] section contains the options relating to the import of + observational data from a Weather Underground PWS daily history.

    station_id

    -

    The Weather Underground weather station ID of the PWS from which the - daily history will be imported. There is no default. -

    +

    The Weather Underground weather station ID of the PWS from which the daily history will be imported. There is + no default.

    interval

    -

    Determines how the time interval (weewx - database field interval) between successive - observations is determined. This option is identical in operation to the - CSV interval option but applies to - Weather Underground imports only. As Weather Underground often skips - observation records when responding to a daily history query, the use of - interval = derive may give incorrect - or inconsistent interval values. Better results may be obtained by using - interval = conf if the current - weewx installation has the same - archive_interval as the Weather Underground - data, or by using interval = x where - x is the time interval in minutes - used to upload the Weather Underground data. The most appropriate - setting will depend on the completeness and (time) accuracy of the - Weather Underground data being imported. -

    +

    Determines how the time interval (weewx database field interval) between successive observations is determined. This option is identical in + operation to the CSV interval option but applies to Weather Underground + imports only. As Weather Underground often skips observation records when responding to a daily history + query, the use of interval = derive may give incorrect or inconsistent + interval values. Better results may be obtained by using interval = conf + if the current weewx installation has the same archive_interval as the Weather Underground data, or by using interval = x where x is the time + interval in minutes used to upload the Weather Underground data. The most appropriate setting will depend on + the completeness and (time) accuracy of the Weather Underground data being imported.

    The default is derive.

    qc

    -

    Determines whether simple quality control checks are applied to imported - data. This option is identical in operation to the CSV - qc option but applies to Weather - Underground imports only. As Weather Underground imports at times - contain nonsense values, particularly for fields for which no data was - uploaded to Weather Underground by the PWS, the use of quality control - checks on imported data can prevent these nonsense values from being - imported and contaminating the weewx database. - The default is True. -

    +

    Determines whether simple quality control checks are applied to imported data. This option is identical in + operation to the CSV qc option but applies to Weather Underground imports + only. As Weather Underground imports at times contain nonsense values, particularly for fields for which no + data was uploaded to Weather Underground by the PWS, the use of quality control checks on imported data can + prevent these nonsense values from being imported and contaminating the weewx + database. The default is True.

    calc_missing

    -

    Determines whether any missing derived observations will be calculated - from the imported data. This option is identical in operation to the - CSV calc_missing option but - applies to Weather Underground imports only. The default is - True. -

    +

    Determines whether any missing derived observations will be calculated from the imported data. This option is + identical in operation to the CSV calc_missing option but applies + to Weather Underground imports only. The default is True.

    tranche

    -

    The number of records written to the weewx - database in each transaction. This option is identical in operation to - the CSV tranche option but applies - to Weather Underground imports only. The default is - 250 which should suit most users. -

    +

    The number of records written to the weewx database in each transaction. This + option is identical in operation to the CSV tranche option but applies + to Weather Underground imports only. The default is 250 which should suit most + users.

    wind_direction

    -

    Determines the range of acceptable wind direction values in degrees. - This option is identical in operation to the CSV - wind_direction option but - applies to Weather Underground imports only. The default is - 0, 360 which should suit most users. -

    +

    Determines the range of acceptable wind direction values in degrees. This option is identical in operation to + the CSV wind_direction option but applies to Weather Underground + imports only. The default is 0, 360 which should suit most users.

    [Cumulus]

    -

    The [Cumulus] section contains the - options relating to the import of observational data from Cumulus - monthly log files. -

    +

    The [Cumulus] section contains the options relating to the import of + observational data from Cumulus monthly log files.

    directory

    -

    The full path to the directory containing the Cumulus monthly log files - to be imported. Do not include a trailing /. There is no default. -

    +

    The full path to the directory containing the Cumulus monthly log files to be imported. Do not include a + trailing /. There is no default.

    interval

    -

    Determines how the time interval (weewx - database field interval) between successive - observations is determined. This option is identical in operation to - the CSV interval option but - applies to Cumulus monthly log file imports only. As Cumulus monthly - log files can, at times, have missing entries, the use of - interval = derive may give incorrect - or inconsistent interval values. Better results may be obtained by - using interval = conf if the - archive_interval for the current - weewx installation is the same as the Cumulus - 'data log interval' setting used to generate the Cumulus monthly log - files, or by using interval = x where - x is the time interval in minutes - used as the Cumulus 'data log interval' setting. The most appropriate - setting will depend on the completeness and (time) accuracy of the - Cumulus data being imported. -

    +

    Determines how the time interval (weewx database field interval) between successive observations is determined. This option is identical in + operation to the CSV interval option but applies to Cumulus monthly log + file imports only. As Cumulus monthly log files can, at times, have missing entries, the use of interval = derive may give incorrect or inconsistent interval values. + Better results may be obtained by using interval = conf if the archive_interval for the current weewx + installation is the same as the Cumulus 'data log interval' setting used to generate the Cumulus monthly log + files, or by using interval = x where x is the time interval in minutes used as the Cumulus 'data log + interval' setting. The most appropriate setting will depend on the completeness and (time) accuracy of the + Cumulus data being imported.

    The default is derive.

    qc

    -

    Determines whether simple quality control checks are applied to imported - data. This option is identical in operation to the CSV - qc option but applies to Cumulus imports - only. The default is True. -

    +

    Determines whether simple quality control checks are applied to imported data. This option is identical in + operation to the CSV qc option but applies to Cumulus imports only. The + default is True.

    calc_missing

    -

    Determines whether any missing derived observations will be calculated - from the imported data. This option is identical in operation to the - CSV calc_missing option but - applies to Cumulus imports only. The default is - True. -

    +

    Determines whether any missing derived observations will be calculated from the imported data. This option is + identical in operation to the CSV calc_missing option but applies + to Cumulus imports only. The default is True.

    delimiter

    -

    The character used as the field delimiter in the Cumulus monthly log - file. A comma is frequently used but it may be another character - depending on the settings on the machine that produced the Cumulus - monthly log files. This parameter must be included in quotation marks. - Default is ','. +

    The character used as the field delimiter in the Cumulus monthly log file. A comma is frequently used but it + may be another character depending on the settings on the machine that produced the Cumulus monthly log + files. This parameter must be included in quotation marks. Default is ','.

    decimal

    -

    The character used as the decimal point in the Cumulus monthly log files - A full stop is frequently used but it may be another character depending - on the settings on the machine that produced the Cumulus monthly log - files. This parameter must be included in quotation marks. Default is - '.'. +

    The character used as the decimal point in the Cumulus monthly log files A full stop is frequently used but + it may be another character depending on the settings on the machine that produced the Cumulus monthly log + files. This parameter must be included in quotation marks. Default is '.'.

    tranche

    -

    The number of records are written to the weewx - database in each transaction. This option is identical in operation to - the CSV tranche option but applies - to Cumulus monthly log file imports only. The default is - 250 which should suit most users. -

    +

    The number of records are written to the weewx database in each transaction. This + option is identical in operation to the CSV tranche option but applies + to Cumulus monthly log file imports only. The default is 250 which should + suit most users.

    UV_sensor

    -

    The UV_sensor parameter enables - wee_import to distinguish between the case - where a UV sensor is present and the UV index is 0 and the case where - no UV sensor is present and UV index is 0. This option is identical in - operation to the CSV UV_sensor option - but applies to Cumulus monthly log file imports only. The default is - True. -

    +

    The UV_sensor parameter enables wee_import to + distinguish between the case where a UV sensor is present and the UV index is 0 and the case where no UV + sensor is present and UV index is 0. This option is identical in operation to the CSV UV_sensor + option but applies to Cumulus monthly log file imports only. The default is True.

    solar_sensor

    -

    The solar_sensor parameter enables - wee_import to distinguish between the case - where a solar radiation sensor is present and the solar radiation is 0 - and the case where no solar radiation sensor is present and solar - radiation is 0. This option is identical in operation to - the CSV solar_sensor option but - applies to Cumulus monthly log file imports only. The default is - True. -

    +

    The solar_sensor parameter enables wee_import to + distinguish between the case where a solar radiation sensor is present and the solar radiation is 0 and the + case where no solar radiation sensor is present and solar radiation is 0. This option is identical in + operation to the CSV solar_sensor option but applies to Cumulus monthly + log file imports only. The default is True.

    [[Units]]

    -

    The [[Units]] stanza defines the units - used in the Cumulus monthly log files. Units settings are required for - temperature, - pressure, rain and - speed. The format for each setting is: -

    +

    The [[Units]] stanza defines the units used in the Cumulus monthly log + files. Units settings are required for temperature, pressure, rain and speed. + The format for each setting is: 

    obs_type = weewx_unit_name
    -

    Where obs_type is one of - temperature, - pressure, rain or - speed and - weewx_unit_name is the - weewx unit name of the units used by that - particular obs_type. As Cumulus supports a - different suite of possible units only a subset of the available - weewx unit names can be used for some - settings. -

    +

    Where obs_type is one of temperature, pressure, + rain or speed and weewx_unit_name is the weewx unit name of the units + used by that particular obs_type. As Cumulus supports a different suite of + possible units only a subset of the available weewx unit names can be used for + some settings.

    wee_reports

    -

    In normal operation, weewx generates - reports on each archive interval, when new data arrive. The - reports utility is used to generate reports on demand. It uses - the same configuration file that - weewx uses. -

    +

    In normal operation, weewx generates reports on each archive interval, when new + data arrive. The reports utility is used to generate reports on demand. It uses the same configuration file + that weewx uses.

    -

    Run the utility with the --help - option to see how it is used: -

    +

    Run the utility with the --help option to see how it is used: 

    wee_reports --help
    @@ -3694,14 +3168,10 @@ Options:

    weewxd

    -

    The weewxd application is the heart - of weewx. It can be run directly, - or in the background as a daemon. -

    +

    The weewxd application is the heart of weewx. It can be + run directly, or in the background as a daemon.

    -

    Run with the --help option to see - how it is used: -

    +

    Run with the --help option to see how it is used: 

    weewxd --help