This is a reference guide to the executable utilities that come with weewx:
- wee_config - for changing the configuration file, and configuring new device drivers; + for changing the configuration file, and configuring new device drivers; +
- wee_database - for reconfiguring the database; + for reconfiguring the database; +
- wee_debug - for producing debug reports for remote support; + for producing debug reports for remote support; +
- wee_device - for configuring your hardware; + for configuring your hardware; +
- wee_extension - for installing and removing extensions; + for installing and removing extensions; +
- wee_import - for importing historical data from external sources; + for importing historical data from external sources; +
- wee_reports - for running reports without running weewx itself; + for running reports without running weewx itself; +
- weewxd - the main weewx program; + the main weewx program; +
- wunderfixer - for resending data missing on the Weather Underground site. + for resending data missing on the Weather Underground site. +
wee_database
-The database utility simplifies operations typically made to the - database. For example, it can backfill the daily summaries - or check a SQLite database for embedded strings where floats are +
This database utility simplifies typical database maintenance operations. For example, it + can backfill the daily summaries or check a SQLite database for embedded strings where floats are expected.
Run the utility with the @@ -362,6 +371,51 @@ If you are using a MySQL database it is assumed that you have the appropriate permissions for the requested operation. +
Option --create-archive
+If the database does not already exist, this option will create it and initialize it with the + schema specified in the configuration file weewx.conf. Because + weewx does this automatically, this option is rarely needed.
+ +Option --drop-daily
+In addition to the regular archive data, every weewx database also includes a daily summary table + for each observation type. Because there can be dozens of observation types, there can be dozens + of these daily summaries. It doesn't happen very often, but there can be occasions when it's + necessary to drop them all then rebuild them. Dropping them by hand would be very tedious! + This option does them all at once.
+ +Option --backfill-daily
+This option is kind of the inverse of option --drop-daily in that it + rebuilds the daily summaries from the archive data.
+ +Option --reconfigure
+This option is useful changing the schema in your database.
+ +It creates a new database with the same name as the old, + except with the suffix _new attached at + the end (nominally, weewx.sdb_new + if you are using SQLite, weewx_new if you + are using MySQL). It then initializes it + with the schema specified in weewx.conf. Finally, + it copies over the data from your old database into the new database.
+ +Option --string-check
+Normally, all entries in the archive database are pure numbers. However, some visual SQLite database editors + use a null string instead of a null value when deleting entries. These nulls strings can + crash weewx. This option checks for them and, if the option --fix + is specified, substitutes a true null value if it finds any.
+ +Option --transfer
+This option is useful for moving your database from one type of database to another, such as + from SQLite to MySQL. To use it, you must have two bindings specified in your + weewx.conf configuration file. One will serve as the source, the other + as the destination. Specify the source binding with option --binding, + the destination binding with option --dest-binding.
+See the Wiki for examples of moving data from + SQLite + to MySQL, and from + MySQL + to SQLite, + using wee_database.
@@ -371,13 +425,13 @@ permissions for the requested operation.Troubleshooting problems when running weewx often involves analysis of a number of pieces of seemingly disparate system and weewx related information. The - wee_debug utility gathers this information into + wee_debug utility gathers all this information together into a single output to make troubleshooting easier. The wee_debug utility is particularly useful for new - users as the output may be redirected to file then emailed or posted to a + users as the output may be redirected to a file then emailed or posted to a forum to assist in remote troubleshooting.
-Before using wee_debug, it's worth running it +
Run the utility with the --help option to see how it is used:
wee_debug --help@@ -411,23 +465,23 @@ should thoroughly check the generated output for personal/private information before posting the information publicly. -
Generating a debug report
+Generating a debug report
-Generate a debug report using the --info - option as follows:
+Generate a debug report using the --info + option as follows:
-Warning!
- The wee_debug output includes a copy of the in use weewx
- config file (usually weewx.conf) and whilst wee_debug
- attempts to obfuscate any personal or sensitive information in weewx.conf, the
- user should carefully check the wee_debug output for any remaining personal or
- sensitive information before emailing or posting the output publicly.
Warning!
+ The wee_debug output includes a copy of the in use weewx
+ config file (usually weewx.conf) and whilst wee_debug
+ attempts to obfuscate any personal or sensitive information in weewx.conf, the
+ user should carefully check the wee_debug output for any remaining personal or
+ sensitive information before emailing or posting the output publicly.
wee_debug --info+
wee_debug --info-
This results in:
+This results in:
Using verbosity=1, displaying most info wee_debug output will be sent to stdout(console) @@ -541,164 +595,161 @@ Parsed and obfuscated weewx.conf wee_debug report successfully generated-
Redirecting wee_debug output
+Redirecting wee_debug output
-wee_debug output is sent by default to - stdout unless the optional --output - option is used. --output with no parameter will - send the wee_debug output to the default file - /var/tmp/weewx.debug. Usage is as follows:
+By default, wee_debug output is sent to the system "standard output" + (stdout) unless the --output + option is used.
+Option --output with no parameter sends output to the default file + /var/tmp/weewx.debug. Example:
-wee_debug --info --output+
wee_debug --info --output-
output can be sent to a file other than the default by including the - path and file name as a parameter after --output:
+Option --output with a specified file will send it to that file. + Example:
-wee_debug --info --output /home/weewx/another.debug+
wee_debug --info --output /home/weewx/another.debug-
wee_debug verbosity
+wee_debug verbosity
-The depth of information included in the wee_debug - output can be changed using the --verbosity option. - The --verbosity option can be set to 0, 1 or 2 with - each higher level successively displaying more information. The default level - is 1. The information displayed for each level is:
+The depth of information included in the wee_debug + output can be changed using the --verbosity option. + The --verbosity option can be set to 0, 1 or 2 with + each higher level successively displaying more information. The default level + is 1. The information displayed for each level is:
-| Level | -Included Information | -
| --verbosity 0 | -path and name of weewx config file used (usually weewx.conf) - | -
| name of weewx database binding used | -|
| operating system version | -|
| weewx version number | -|
| weewx station type and driver name | -|
| summary of currently installed extensions | -|
| summary of weewx archive | -|
| parsed and obfusated weewx config file (usually weewx.conf) - | -|
| --verbosity 1 | -as per --verbosity 0 | -
| cpu info summary | -|
| system load averages | -|
| driver config extract from weewx config file (usually weewx.conf) - | -|
| summary of databases configured in weewx config file (usually weewx.conf) - | -|
| --verbosity 2 | -as per --verbosity 1 | -
| list of supported SQL keys | -|
| list of supported observation types | -
| Level | +Included Information | +
| --verbosity 0 | +path and name of weewx config file used (usually weewx.conf) + | +
| name of weewx database binding used | +|
| operating system version | +|
| weewx version number | +|
| weewx station type and driver name | +|
| summary of currently installed extensions | +|
| summary of weewx archive | +|
| parsed and obfusated weewx config file (usually weewx.conf) + | +|
| --verbosity 1 | +as per --verbosity 0 | +
| cpu info summary | +|
| system load averages | +|
| driver config extract from weewx config file (usually weewx.conf) + | +|
| summary of databases configured in weewx config file (usually weewx.conf) + | +|
| --verbosity 2 | +as per --verbosity 1 | +
| list of supported SQL keys | +|
| list of supported observation types | +
wee_device
+wee_device
-This section describes how to configure some of the more popular - station hardware.
+The utility wee_device can be used to configure the hardware of some + of the more popular weather stations. It can set things like rain bucket size, station archive interval, + altitude, EEPROM constants, etc. In order to do its job, it depends on optional code + being present in the hardware driver. Not all drivers have this code, so it may not work for your specific + device. If it does not, + you will have to consult your manufacturer's instructions for how to set these things through your console + or other means.
-Some stations can be configured using the - wee_device utility supplied with weewx. - This utility uses code in the hardware-specific driver to set EEPROM - constants, read station memory, set the station archive interval, set - the altitude, configure rain bucket types, and many other options, - depending on the hardware.
+Run the utility with the --help option + to see which options are available.
-Note that some stations cannot be configured by software at all, - and some stations are only partly configurable by software.
+wee_device --help
-Run the utility with the --help option - to see which options are available.
+The utility requires a weewx.conf file. + If no file is specified, it will look for + weewx.conf in the standard location. If + your configuration file is in a non-standard location, specify the + path to the configuration file as the first argument. For example,
-wee_device --help
+wee_device /path/to/weewx.conf --help
-The utility requires a weewx.conf file. - If no file is specified, it will look for - weewx.conf in the standard location. If - your configuration file is in a non-standard location, specify the - path to the configuration file as the first argument. For example,
+What follows is a guide to the level of support offered by wee_device for + each type of weather station.
-wee_device /path/to/weewx.conf --help
+AcuRite
+ +The wee_device utility cannot + configure AcuRite stations.
+ +The AcuRite console must be set to USB mode 3 or 4. Communication + via USB is disabled in modes 1 and 2. Mode 4 is more reliable than + mode 3; mode 3 enables logging of data, mode 4 does not. When the + console is logging it frequently causes USB communication problems.
+ +The wind speed updates every 18 seconds. The wind direction updates + every 30 seconds. Other sensors update every 60 seconds.
+ +The AcuRite stations do not record wind gusts.
+ +The station emits partial packets, which may confuse some online + services
-AcuRite
+CC3000
-The wee_device utility cannot - configure AcuRite stations.
+The CC3000 data logger may be configured to return data in + METRIC or US units. These are not the same groups of METRIC and US + units as defined within weewx. However, the + CC3000 driver will convert to the appropriate units for the + weewx configuration.
-The AcuRite console must be set to USB mode 3 or 4. Communication - via USB is disabled in modes 1 and 2. Mode 4 is more reliable than - mode 3; mode 3 enables logging of data, mode 4 does not. When the - console is logging it frequently causes USB communication problems.
+The CC3000 data logger stores 2MB of records.
-The wind speed updates every 18 seconds. The wind direction updates - every 30 seconds. Other sensors update every 60 seconds.
+The CC3000 driver supports catchup on startup, but it does not + support hardware record generation.
-The AcuRite stations do not record wind gusts.
- -The station emits partial packets, which may confuse some online - services
- - -CC3000
- -The CC3000 data logger may be configured to return data in - METRIC or US units. These are not the same groups of METRIC and US - units as defined within weewx. However, the - CC3000 driver will convert to the appropriate units for the - weewx configuration.
- -The CC3000 data logger stores 2MB of records.
- -The CC3000 driver supports catchup on startup, but it does not - support hardware record generation.
- -The wee_device utility can be used to - configure the station hardware. When the - station_type is - CC3000, - the --help option will produce output - something like this:
+The wee_device utility can be used to + configure the station hardware. When the + station_type is + CC3000, + the --help option will produce output + something like this:
CC3000 driver version 0.8 Usage: wee_device [config_file] [options] [--debug] [--help] @@ -720,52 +771,52 @@ Options: Mutating actions will request confirmation before proceeding.-
Station information
+Station information
-Display the station settings with the --info - option.
-wee_device --info-
This will result in something like:
+Display the station settings with the --info + option.
+wee_device --info+
This will result in something like:
firmware: Rainwise CC-3000 Version: 1.3 Build 006 Sep 04 2013 time: 2014/06/02 08:22:17 units: ENGLISH memory: 251372 bytes, 4334 records, 12% interval: 1-
Changing the archive interval
+Changing the archive interval
-CC3000 loggers ship from the factory with an archive interval - of 1 minutes (60 seconds). To change the station's - interval to 5 minutes, do the following:
+CC3000 loggers ship from the factory with an archive interval + of 1 minutes (60 seconds). To change the station's + interval to 5 minutes, do the following:
-wee_device --set-interval=5
+wee_device --set-interval=5
-FineOffsetUSB
+FineOffsetUSB
-The station clock can only be set manually via buttons on the - console, or (if the station supports it) by WWVB radio. The - FineOffsetUSB driver ignores the station clock since it cannot be - trusted.
+The station clock can only be set manually via buttons on the + console, or (if the station supports it) by WWVB radio. The + FineOffsetUSB driver ignores the station clock since it cannot be + trusted.
-The station reads data from the sensors every 48 seconds. - The 30xx stations read UV data every 60 seconds.
+The station reads data from the sensors every 48 seconds. + The 30xx stations read UV data every 60 seconds.
-The 10xx and 20xx stations can save up to 4080 historical readings. - That is about 85 days of data with the default recording interval of - 30 minutes, or about 14 days with a recording interval of 5 minutes. - The 30xx stations can save up to 3264 historical readings.
+The 10xx and 20xx stations can save up to 4080 historical readings. + That is about 85 days of data with the default recording interval of + 30 minutes, or about 14 days with a recording interval of 5 minutes. + The 30xx stations can save up to 3264 historical readings.
-When weewx starts up it will attempt - to download all records from the console since the last record in - the archive database.
+When weewx starts up it will attempt + to download all records from the console since the last record in + the archive database.
-The wee_device utility can be used to - configure the station hardware. When the - station_type is - FineOffsetUSB, - the --help option will produce output - something like this:
+The wee_device utility can be used to + configure the station hardware. When the + station_type is + FineOffsetUSB, + the --help option will produce output + something like this:
FineOffsetUSB driver version 1.7 Usage: wee_device [config_file] [options] [--debug] [--help] @@ -791,12 +842,12 @@ Options: --format=FORMAT format for output, one of raw, table, or dict Mutating actions will request confirmation before proceeding.-
Station information
+Station information
-Display the station settings with the --info - option.
-wee_device --info-
This will result in something like:
+Display the station settings with the --info + option.
+wee_device --info+
This will result in something like:
Fine Offset station settings:
local time: 2013.02.11 18:34:28 CET
polling_mode: ADAPTIVE
@@ -824,121 +875,121 @@ Mutating actions will request confirmation before proceeding.
version: 255
wind_coef: None
wind_mult: 0
- Highlighted values can be modified with the wee_device utility.
+Highlighted values can be modified with the wee_device utility.
-Changing the archive interval
+Changing the archive interval
-Fine Offset stations ship from the factory with an archive interval - (read_period) of 30 minutes (1800 seconds). To change the station's - interval to 5 minutes, do the following:
+Fine Offset stations ship from the factory with an archive interval + (read_period) of 30 minutes (1800 seconds). To change the station's + interval to 5 minutes, do the following:
-wee_device --set-interval=5
+wee_device --set-interval=5
-Dumping the console memory
+Dumping the console memory
-Fine Offset stations store records in a circular buffer — once the buffer fills, the oldest records - are - replaced by newer records. The 1080 and 2080 consoles store up to 4080 records. The 3080 consoles store - up to - 3264 records. The data_count indicates how many records are in memory. - The read_period indicates the number of minutes between records. wee_device - can display these records in space-delimited, raw bytes, or dictionary format.
+Fine Offset stations store records in a circular buffer — once the buffer fills, the oldest records + are + replaced by newer records. The 1080 and 2080 consoles store up to 4080 records. The 3080 consoles store + up to + 3264 records. The data_count indicates how many records are in memory. + The read_period indicates the number of minutes between records. wee_device + can display these records in space-delimited, raw bytes, or dictionary format.
-For example, to display the most recent 30 records from the console memory:
-wee_device --history=30-
To clear the console memory:
-wee_device --clear-memory+
For example, to display the most recent 30 records from the console memory:
+wee_device --history=30+
To clear the console memory:
+wee_device --clear-memory-
Checking the USB quality
+Checking the USB quality
-Use the wee_device utility to test the quality of the USB connection between - computer - and console. Poor quality USB cables, under-powered USB hubs, and other devices on the bus can interfere - with - communication.
+Use the wee_device utility to test the quality of the USB connection between + computer + and console. Poor quality USB cables, under-powered USB hubs, and other devices on the bus can interfere + with + communication.
-To test the quality of the USB connection to the console:
-wee_device --check-usb-
Let the utility run for at least a few minutes, or possibly an hour or two. It is not unusual to see a - few bad - reads in an hour, but if you see many bad reads within a few minutes, consider replacing the USB cable, - USB hub, - or removing other devices from the bus.
+To test the quality of the USB connection to the console:
+wee_device --check-usb+
Let the utility run for at least a few minutes, or possibly an hour or two. It is not unusual to see a + few bad + reads in an hour, but if you see many bad reads within a few minutes, consider replacing the USB cable, + USB hub, + or removing other devices from the bus.
-Polling mode and the polling interval
+Polling mode and the polling interval
-When reading 'live' data, weewx can read as fast as possible, or at a - user-defined - period. This is controlled by the polling_mode parameter. -
-| Mode | -weewx.conf | -Notes | -||||||||||||
| ADAPTIVE | -
+ When reading 'live' data, weewx can read as fast as possible, or at a + user-defined + period. This is controlled by the polling_mode parameter. + +
The console reads the sensors every 48 seconds (60 seconds for UV), so setting the polling_interval to a value less than 48 will result in duplicate + readings. + + |
+
TE923
+TE923
-Some station models will recognize up to 5 remote - temperature/humidity sensor units. Use the hardware switch in each - sensor unit to identify sensors. Use the - map in - weewx.conf to associate each sensor with a - database field. -
+Some station models will recognize up to 5 remote + temperature/humidity sensor units. Use the hardware switch in each + sensor unit to identify sensors. Use the + map in + weewx.conf to associate each sensor with a + database field. +
-The station has either 208 or 3442 history records, depending on - the model. That is just over a day (208 records) or about 23 days - (3442 records) with an archive interval of 5 minutes.
+The station has either 208 or 3442 history records, depending on + the model. That is just over a day (208 records) or about 23 days + (3442 records) with an archive interval of 5 minutes.
-The TE923 driver will read history records from the station when weewx - starts up, but it does not support hardware record generation.
+The TE923 driver will read history records from the station when weewx + starts up, but it does not support hardware record generation.
-The wee_device utility can be used to - configure the station hardware. When the - station_type is - TE932, - the --help option will produce output - something like this:
+The wee_device utility can be used to + configure the station hardware. When the + station_type is + TE932, + the --help option will produce output + something like this:
Using configuration file /home/weewx/weewx.conf Using TE923 driver version 0.16 (weewx.drivers.te923) @@ -977,12 +1028,12 @@ Options: Be sure to stop weewx first before using. Mutating actions will request confirmation before proceeding.-
Station information
+Station information
-Use the --info option to display the - station configuration:
-wee_device --info-
This will result in something like:
+Use the --info option to display the + station configuration:
+wee_device --info+
This will result in something like:
Querying the station for the configuration...
altitude: 16
bat_1: True
@@ -1001,12 +1052,12 @@ confirmation before proceeding.
version_uv: 20
version_wind: 38
- Display sensor status
+Display sensor status
-Use the --current option to display the - current status of each sensor:
-wee_device --current-
This will result in something like:
+Use the --current option to display the + current status of each sensor:
+wee_device --current+
This will result in something like:
Querying the station for current weather data...
dateTime: 1454615168
forecast: 5
@@ -1050,41 +1101,41 @@ confirmation before proceeding.
windspeed: None
windspeed_state: invalid
- Changing the archive interval
+Changing the archive interval
-TE923 stations ship from the factory with an archive interval of 1 - hour (3600 seconds). To change the station's - interval to 5 minutes (300 seconds), do the following:
+TE923 stations ship from the factory with an archive interval of 1 + hour (3600 seconds). To change the station's + interval to 5 minutes (300 seconds), do the following:
-wee_device --set-interval=300
+wee_device --set-interval=300
-Dumping the logger memory
+Dumping the logger memory
-wee_device can display the records from the - logger in tabular or dictionary format.
+wee_device can display the records from the + logger in tabular or dictionary format.
-For example, to display the most recent 30 records:
-wee_device --history=30 --format dict+
For example, to display the most recent 30 records:
+wee_device --history=30 --format dict-
Ultimeter
+Ultimeter
-The wee_device utility cannot - configure Ultimeter stations.
+The wee_device utility cannot + configure Ultimeter stations.
-The Ultimeter driver operates the Ultimeter in Data Logger Mode, - which results in sensor readings every 1/2 second or so.
+The Ultimeter driver operates the Ultimeter in Data Logger Mode, + which results in sensor readings every 1/2 second or so.
-The Ultimeter driver ignores the maximum, minimum, and average - values recorded by the station.
+The Ultimeter driver ignores the maximum, minimum, and average + values recorded by the station.
-Vantage
+Vantage
-When the station_type is - Vantage, - the --help option will produce output - something like this:
+When the station_type is + Vantage, + the --help option will produce output + something like this:
Using configuration file /home/weewx/weewx.conf Using Vantage driver version 3.0 (weewx.drivers.vantage) @@ -1150,12 +1201,12 @@ Options: Be sure to stop weewx first before using. Mutating actions will request confirmation before proceeding.-
Station information
+Station information
-Use the --info option to display the - current EEPROM settings:
-wee_device --info-
This will result in something like:
+Use the --info option to display the + current EEPROM settings:
+wee_device --info+
This will result in something like:
Davis Vantage EEPROM settings:
CONSOLE TYPE: VantagePro2
@@ -1222,180 +1273,180 @@ confirmation before proceeding.
Outside Temperature: +0.0 F
Outside Humidity: +0%
Extra Temperature 1: +0.0 F
- The console version number is available only on consoles with firmware - dates after about 2006.
+The console version number is available only on consoles with firmware + dates after about 2006.
-Highlighted values can be changed using this utility.
+Highlighted values can be changed using this utility.
-Time zone
+Time zone
-To set the time zone code to Central European Time (code 20):
-wee_device --set-tz-code=20-
You can set either the time zone code or the - time zone offset, but not both.
+To set the time zone code to Central European Time (code 20):
+wee_device --set-tz-code=20+
You can set either the time zone code or the + time zone offset, but not both.
-Archive interval
+Archive interval
-Valid archive intervals for the Davis Vantage stations are 60, - 300, 600, 900, 1800, 3600, and 7200 seconds. However, if you are - ftp'ing lots of files to a server, setting it to 60 seconds - may not give enough time to have them all uploaded before the next - archive record is due. If this is the case, you should pick an - archive interval of at least 300 seconds, or trim the number of - files you are using.
+Valid archive intervals for the Davis Vantage stations are 60, + 300, 600, 900, 1800, 3600, and 7200 seconds. However, if you are + ftp'ing lots of files to a server, setting it to 60 seconds + may not give enough time to have them all uploaded before the next + archive record is due. If this is the case, you should pick an + archive interval of at least 300 seconds, or trim the number of + files you are using.
-To change the archive interval to 10 minutes (600 seconds):
-wee_device --set-interval=600-
I have found that a five minute (300 seconds) archive interval works - well for the Vantage stations. Because of the large amount of onboard - memory they carry, going to a larger interval does not have any real - advantages.
+To change the archive interval to 10 minutes (600 seconds):
+wee_device --set-interval=600+
I have found that a five minute (300 seconds) archive interval works + well for the Vantage stations. Because of the large amount of onboard + memory they carry, going to a larger interval does not have any real + advantages.
-Rain bucket type
+Rain bucket type
-Normally, this is set by Davis, but if you have replaced your bucket - with a different kind, you might want to reconfigure. For example, to - change to a 0.1 mm bucket (bucket code "2"), use the - following:
-wee_device --set-bucket=2+
Normally, this is set by Davis, but if you have replaced your bucket + with a different kind, you might want to reconfigure. For example, to + change to a 0.1 mm bucket (bucket code "2"), use the + following:
+wee_device --set-bucket=2-
Additional sensors
+Additional sensors
-If you have additional sensors for your Vantage station, you - can configure them using your console. However, if you have - a Davis - Weather Envoy receiver, it will not have a console! As an alternative, - the weewx - utility wee_device lets - you do this from the command line.
+If you have additional sensors for your Vantage station, you + can configure them using your console. However, if you have + a Davis + Weather Envoy receiver, it will not have a console! As an alternative, + the weewx + utility wee_device lets + you do this from the command line.
-For example, to add an extra temperature sensor to channel 3, do the following:
-wee_device --set-transmitter-type=3,1,2-
This says to turn on channel 3, set its type to 1 ("Temperature only"), and have it show up in the - database as extraTemp2. Here's another example, this time for a combined - temperature / humidity sensor:
-wee_device --set-transmitter-type=5,3,2,4-
This will add the combined sensor to channel 5, set its type to 3 ("Temperature and humidity"), - and it will show up in the database - as extraTemp2 and extraHumid4.
+For example, to add an extra temperature sensor to channel 3, do the following:
+wee_device --set-transmitter-type=3,1,2+
This says to turn on channel 3, set its type to 1 ("Temperature only"), and have it show up in the + database as extraTemp2. Here's another example, this time for a combined + temperature / humidity sensor:
+wee_device --set-transmitter-type=5,3,2,4+
This will add the combined sensor to channel 5, set its type to 3 ("Temperature and humidity"), + and it will show up in the database + as extraTemp2 and extraHumid4.
-The --help option will give you the code for each sensor type.
+The --help option will give you the code for each sensor type.
-Setting offsets
+Setting offsets
-The Davis instruments can correct sensor errors by adding - an offset to their emitted values. This is particularly - useful for Southern Hemisphere users. Davis fits the wind vane - to the Integrated Sensor Suite (ISS) in a position optimized for - Northern Hemisphere users, who face the solar panel to the - south. Users south of the equator must orient the ISS's solar - panel to the north to get maximal insolation, resulting in a - 180° error in the wind direction. The solution is to add a - 180° offset correction. You can do this with the following - command:
+The Davis instruments can correct sensor errors by adding + an offset to their emitted values. This is particularly + useful for Southern Hemisphere users. Davis fits the wind vane + to the Integrated Sensor Suite (ISS) in a position optimized for + Northern Hemisphere users, who face the solar panel to the + south. Users south of the equator must orient the ISS's solar + panel to the north to get maximal insolation, resulting in a + 180° error in the wind direction. The solution is to add a + 180° offset correction. You can do this with the following + command:
-wee_device --set-offset=windDir,180+
wee_device --set-offset=windDir,180-
Dumping the logger memory
+Dumping the logger memory
-Generally, weewx downloads only new archive - records from the on-board logger in the Vantage. However, occasionally the - memory in the Vantage will get corrupted, making this impossible. See the - troubleshooting section below Weewx - generates HTML pages, but it does not update them. The - fix involves clearing the memory but, unfortunately, this means you may - lose any data which might have accumulated in the logger memory, but not - yet downloaded. By using the --dump command - before clearing the memory, you might be able to save these data. - Stop weewx first, then
-wee_device --dump-
This will dump all data archived in the Vantage memory directly to the - database, without regard to whether or not they have been seen before. - Because the command dumps all data, it may result in many - duplicate primary key errors. These can be ignored.
+Generally, weewx downloads only new archive + records from the on-board logger in the Vantage. However, occasionally the + memory in the Vantage will get corrupted, making this impossible. See the + troubleshooting section below Weewx + generates HTML pages, but it does not update them. The + fix involves clearing the memory but, unfortunately, this means you may + lose any data which might have accumulated in the logger memory, but not + yet downloaded. By using the --dump command + before clearing the memory, you might be able to save these data. + Stop weewx first, then
+wee_device --dump+
This will dump all data archived in the Vantage memory directly to the + database, without regard to whether or not they have been seen before. + Because the command dumps all data, it may result in many + duplicate primary key errors. These can be ignored.
-WMR100
+WMR100
-The wee_device utility cannot - configure WMR100 stations.
+The wee_device utility cannot + configure WMR100 stations.
-The station emits partial packets, which may confuse some online - services.
+The station emits partial packets, which may confuse some online + services.
-WMR200
+WMR200
-The wee_device utility cannot - configure WMR200 stations.
+The wee_device utility cannot + configure WMR200 stations.
-The station emits partial packets, which may confuse some online - services.
+The station emits partial packets, which may confuse some online + services.
-When weewx starts up it will attempt to - download all records from the console since the last record in the - archive database. -
+When weewx starts up it will attempt to + download all records from the console since the last record in the + archive database. +
-WMR300
+WMR300
-The wee_device utility cannot - configure WMR300 stations.
+The wee_device utility cannot + configure WMR300 stations.
-The station emits partial packets, which may confuse some online - services.
+The station emits partial packets, which may confuse some online + services.
-When weewx starts up it will attempt to - download all records from the console since the last record in the - archive database. This can take a couple of hours, depending on the - number of records in the logger and the speed of the computer and disk. -
+When weewx starts up it will attempt to + download all records from the console since the last record in the + archive database. This can take a couple of hours, depending on the + number of records in the logger and the speed of the computer and disk. +
-WMR9x8
+WMR9x8
-The wee_device utility cannot - configure WMR9x8 stations.
+The wee_device utility cannot + configure WMR9x8 stations.
-The station includes a data logger, but the driver does not read - records from the station.
+The station includes a data logger, but the driver does not read + records from the station.
-The station emits partial packets, which may confuse some online - services.
+The station emits partial packets, which may confuse some online + services.
-WS1
+WS1
-The wee_device utility cannot - configure WS1 stations.
+The wee_device utility cannot + configure WS1 stations.
-The WS1 stations produce data every 1/2 second or so.
+The WS1 stations produce data every 1/2 second or so.
-WS23xx
+WS23xx
-The hardware interface is a serial port, but USB-serial converters - can be used with computers that have no serial port. Beware that - not every type of USB-serial converter will work. Converters based - on ATEN UC-232A chipset are known to work.
+The hardware interface is a serial port, but USB-serial converters + can be used with computers that have no serial port. Beware that + not every type of USB-serial converter will work. Converters based + on ATEN UC-232A chipset are known to work.
-The station does not record wind gust or wind gust direction.
+The station does not record wind gust or wind gust direction.
-The hardware calculates windchill and dewpoint.
+The hardware calculates windchill and dewpoint.
-The station has 175 history records. That is just over 7 days - of data with the factory default history recording interval of 60 - minutes, or about 14 hours with a recording interval of 5 minutes.
+The station has 175 history records. That is just over 7 days + of data with the factory default history recording interval of 60 + minutes, or about 14 hours with a recording interval of 5 minutes.
-When weewx starts up it will attempt - to download all records from the console since the last record in - the archive database.
+When weewx starts up it will attempt + to download all records from the console since the last record in + the archive database.
-When the station_type is - WS23xx, - the --help option will produce output - something like this:
+When the station_type is + WS23xx, + the --help option will produce output + something like this:
WS23xx driver version 0.21 Usage: wee_device [config_file] [options] [--debug] [--help] @@ -1416,12 +1467,12 @@ Options: Mutating actions will request confirmation before proceeding.-
Station information
+Station information
-Display the station settings with the --info - option.
-wee_device --info-
This will result in something like:
+Display the station settings with the --info + option.
+wee_device --info+
This will result in something like:
buzzer: 0 connection time till connect: 1.5 connection type: 15 @@ -1446,58 +1497,58 @@ icon alarm active: 0 in humidity: 48.0 ...-
Changing the archive interval
+Changing the archive interval
-WS23xx stations ship from the factory with an archive interval of 60 - minutes (3600 seconds). To change the station's interval to 5 minutes, - do the following:
+WS23xx stations ship from the factory with an archive interval of 60 + minutes (3600 seconds). To change the station's interval to 5 minutes, + do the following:
-wee_device --set-interval=5
+wee_device --set-interval=5
-Warning!
- Changing the recording interval will clear the station memory.
Warning!
+ Changing the recording interval will clear the station memory.
Dumping the console memory
+Dumping the console memory
-WS23xx stations store records in a circular buffer - once the - buffer fills, the oldest records are replaced by newer records. The - console stores up to 175 records. - The history number of records indicates how - many records are in memory. The - history interval indicates the number of - minutes between records.
+WS23xx stations store records in a circular buffer - once the + buffer fills, the oldest records are replaced by newer records. The + console stores up to 175 records. + The history number of records indicates how + many records are in memory. The + history interval indicates the number of + minutes between records.
-For example, to display the latest 30 records from the console memory:
-wee_device --history=30-
To clear the console memory:
-wee_device --clear-memory+
For example, to display the latest 30 records from the console memory:
+wee_device --history=30+
To clear the console memory:
+wee_device --clear-memory-
WS28xx
+WS28xx
-weewx communicates with a USB transceiver, - which communicates with the station console, which in turn communicates - with the sensors. The transceiver and console must be paired and - synchronized.
+weewx communicates with a USB transceiver, + which communicates with the station console, which in turn communicates + with the sensors. The transceiver and console must be paired and + synchronized.
-The station has 1797 history records. That is just over 6 days - of data with an archive interval of 5 minutes.
+The station has 1797 history records. That is just over 6 days + of data with an archive interval of 5 minutes.
-When weewx starts up it will attempt to - download all records from the console since the last record in the - archive database. -
+When weewx starts up it will attempt to + download all records from the console since the last record in the + archive database. +
-The WS28xx driver sets the station archive interval to - 5 minutes.
+The WS28xx driver sets the station archive interval to + 5 minutes.
-The WS28xx driver does not support hardware archive record - generation.
+The WS28xx driver does not support hardware archive record + generation.
-When the station_type is - WS28xx, - the --help option will produce output - something like this:
+When the station_type is + WS28xx, + the --help option will produce output + something like this:
WS28xx driver version 0.33 Usage: wee_device [config_file] [options] [--debug] [--help] @@ -1519,80 +1570,80 @@ Options: Mutating actions will request confirmation before proceeding.-
Pairing
+Pairing
-The console and transceiver must be paired. Pairing ensures that your - transceiver is talking to your console, not your neighbor's console. - Pairing should only have to be done once, although you might have to - pair again after power cycling the console, for example after you replace - the batteries.
+The console and transceiver must be paired. Pairing ensures that your + transceiver is talking to your console, not your neighbor's console. + Pairing should only have to be done once, although you might have to + pair again after power cycling the console, for example after you replace + the batteries.
-There are two ways to pair the console and the transceiver:
--
-
- The Weewx way. Be sure that
- weewx is not running.
- Run the configuration utility,
- press and hold the [v] button on the console until you see
- 'PC' in the display, then release the button. If the console pairs
- with the transceiver, 'PC' will go away within a second or two.
+
There are two ways to pair the console and the transceiver:
+-
+
- The Weewx way. Be sure that
+ weewx is not running.
+ Run the configuration utility,
+ press and hold the [v] button on the console until you see
+ 'PC' in the display, then release the button. If the console pairs
+ with the transceiver, 'PC' will go away within a second or two.
wee_device --pair Pairing transceiver with console... Press and hold the [v] key until "PC" appears (attempt 1 of 3) Transceiver is paired to console-
- - The HeavyWeather way. Follow the pairing - instructions that came with the station. You will have to run - HeavyWeather on a windows computer with the USB transceiver. After - HeavyWeather indicates the devices are paired, put the USB transceiver - in your weewx computer and start weewx. - Do not power cycle the station console or you will have to start - over. - -
+ - The Weewx way. Be sure that
+ weewx is not running.
+ Run the configuration utility,
+ press and hold the [v] button on the console until you see
+ 'PC' in the display, then release the button. If the console pairs
+ with the transceiver, 'PC' will go away within a second or two.
- The HeavyWeather way. Follow the pairing + instructions that came with the station. You will have to run + HeavyWeather on a windows computer with the USB transceiver. After + HeavyWeather indicates the devices are paired, put the USB transceiver + in your weewx computer and start weewx. + Do not power cycle the station console or you will have to start + over. + +
If the console does not pair, you will see messages in the log such as - this:
-ws28xx: RFComm: message from console contains unknown device ID (id=165a resp=80 req=6)-
Either approach to pairing may require multiple attempts.
+If the console does not pair, you will see messages in the log such as + this:
+ws28xx: RFComm: message from console contains unknown device ID (id=165a resp=80 req=6)+
Either approach to pairing may require multiple attempts.
-Synchronizing
+Synchronizing
-After pairing, the transceiver and console must be synchronized in - order to communicate. Synchronization will happen automatically at the - top of each hour, or you can force synchronization by pressing the [SET] - button momentarily. Do not press and hold the [SET] button - that - modifies the console alarms.
+After pairing, the transceiver and console must be synchronized in + order to communicate. Synchronization will happen automatically at the + top of each hour, or you can force synchronization by pressing the [SET] + button momentarily. Do not press and hold the [SET] button - that + modifies the console alarms.
-When the transceiver and console are synchronized, you will see lots of - 'ws28xx: RFComm' messages in the log when debug=1. - When the - devices are - not synchronized, you will see messages like this about every 10 - minutes:
-Nov 7 19:12:17 raspi weewx[2335]: ws28xx: MainThread: no contact with console-
If you see this, or if you see an extended gap in the weather data - in the weewx plots, press momentarily the [SET] button, or wait until - the top of the hour.
+When the transceiver and console are synchronized, you will see lots of + 'ws28xx: RFComm' messages in the log when debug=1. + When the + devices are + not synchronized, you will see messages like this about every 10 + minutes:
+Nov 7 19:12:17 raspi weewx[2335]: ws28xx: MainThread: no contact with console+
If you see this, or if you see an extended gap in the weather data + in the weewx plots, press momentarily the [SET] button, or wait until + the top of the hour.
-When the transceiver has not received new data for awhile, you will see - messages like this in the log:
-Nov 7 19:12:17 raspi weewx[2335]: ws28xx: MainThread: no new weather data-
If you see 'no new weather data' messages with the - 'no contact with console' messages, - it simply means that the transceiver has not been able to synchronize - with the console. If you see only the 'no new weather data' messages, - then the sensors are not communicating with the console, or the console - may be defective.
+When the transceiver has not received new data for awhile, you will see + messages like this in the log:
+Nov 7 19:12:17 raspi weewx[2335]: ws28xx: MainThread: no new weather data+
If you see 'no new weather data' messages with the + 'no contact with console' messages, + it simply means that the transceiver has not been able to synchronize + with the console. If you see only the 'no new weather data' messages, + then the sensors are not communicating with the console, or the console + may be defective.
-Station information
+Station information
-Display the station settings with the --info - option.
+Display the station settings with the --info + option.
-wee_device --info
+wee_device --info
-This will result in something like:
+This will result in something like:
alarm_flags_other: 0 alarm_flags_wind_dir: 0 @@ -1633,28 +1684,28 @@ threshold_weather: 3 wind_gust_max: 12.874765625 wind_gust_max_time: None-
Alarms
+Alarms
-When an alarm goes off, communication with the transceiver stops. - The WS28xx driver clears all alarms in the station. It is better to - create alarms in weewx, and the weewx alarms - can do much more than the console alarms anyway.
+When an alarm goes off, communication with the transceiver stops. + The WS28xx driver clears all alarms in the station. It is better to + create alarms in weewx, and the weewx alarms + can do much more than the console alarms anyway.
- + -wee_extension
+wee_extension
-- The utility wee_extension is used to add and - remove extensions. It's worth running with the - --help - option to see how it is used: -
++ The utility wee_extension is used to add and + remove extensions. It's worth running with the + --help + option to see how it is used: +
-wee_extension --help+
wee_extension --help-
This results in:
+This results in:
Usage: wee_extension --help @@ -1691,53 +1742,53 @@ Options: --dry-run Print what would happen but do not do it. --verbosity=N How much status to display, 0-3-
To install an extension:
+To install an extension:
wee_extension --install extensions/basic wee_extension --install basic.tar.gz-
To uninstall an extension:
+To uninstall an extension:
-wee_extension --uninstall basic
+wee_extension --uninstall basic
-To list installed extensions:
+To list installed extensions:
-wee_extension --list
+wee_extension --list
-The utility will make only the following changes:
--
-
- Modifications to weewx.conf -
- Add/Remove directories in skins -
- Add/Remove directories in user -
The utility makes a copy of any file or directory that it - modifies or replaces. When installing, it creates a directory called - installer in the - user directory. The contents of the - installer directory are - used to enumerate and uninstall extensions.
+The utility will make only the following changes:
+-
+
- Modifications to weewx.conf +
- Add/Remove directories in skins +
- Add/Remove directories in user +
The utility makes a copy of any file or directory that it + modifies or replaces. When installing, it creates a directory called + installer in the + user directory. The contents of the + installer directory are + used to enumerate and uninstall extensions.
-Installing an extension
+Installing an extension
-- Let's try installing a simple extension, - cmon, - used to monitor your computer. -
++ Let's try installing a simple extension, + cmon, + used to monitor your computer. +
-- First download it. You can either do this from the link given in the - wiki, or by using wget (which you may have - to install): -
-wget -P /var/tmp http://lancet.mit.edu/mwall/projects/weather/releases/weewx-cmon-0.7.tgz+
+ First download it. You can either do this from the link given in the + wiki, or by using wget (which you may have + to install): +
+wget -P /var/tmp http://lancet.mit.edu/mwall/projects/weather/releases/weewx-cmon-0.7.tgz-
- This will put the tarfile weewx-cmon-0.7.tgz - in the directory /var/tmp. -
++ This will put the tarfile weewx-cmon-0.7.tgz + in the directory /var/tmp. +
-Now install the extension:
+Now install the extension:
wee_extension --install=/var/tmp/weewx-cmon-0.7.tgz
Request to install '/var/tmp/weewx-cmon-0.7.tgz'
Extracting from tarball /var/tmp/weewx-cmon-0.7.tgz
@@ -1746,78 +1797,78 @@ Saving installer file to /home/weewx/bin/weecfg/user/installer/cmon
Saved configuration dictionary. Backup copy at /home/weewx/weewx.conf.20150430130322
Finished installing extension '/var/tmp/weewx-cmon-0.7.tgz'
- The installer has done a number of things for you:
--
-
- It put a new skin, cmon, in the - skins subdirectory; - -
- It put a new service, - user.cmon.ComputerMonitor, - in the list of services to be run by - weewx; - -
- It defined a new database, cmon_sqlite, - and a binding, cmon_binding, - to that database; - -
- It added a top-level "stanza" - [ComputerMonitor] - to your configuration file weewx.conf, - that specifies the data binding cmon - is to use. - -
- And, finally, it saved the details of how the extension was - installed so you can remove it later, should you choose to do so. - -
The installer has done a number of things for you:
+-
+
- It put a new skin, cmon, in the + skins subdirectory; + +
- It put a new service, + user.cmon.ComputerMonitor, + in the list of services to be run by + weewx; + +
- It defined a new database, cmon_sqlite, + and a binding, cmon_binding, + to that database; + +
- It added a top-level "stanza" + [ComputerMonitor] + to your configuration file weewx.conf, + that specifies the data binding cmon + is to use. + +
- And, finally, it saved the details of how the extension was + installed so you can remove it later, should you choose to do so. + +
Listing installed extensions
+Listing installed extensions
-The utility wee_extension can tell you which - extensions you have installed:
+The utility wee_extension can tell you which + extensions you have installed:
wee_extension --list
Extension Name Version Description
cmon 0.7 Collect and display computer health indicators
- You can see it lists the extension we just installed, cmon.
+You can see it lists the extension we just installed, cmon.
-Removing extensions
+Removing extensions
-You can remove an extension using the same tool:
+You can remove an extension using the same tool:
wee_extension --uninstall=cmon
wee_extension --list
No extensions installed
-
+
- wee_import
+wee_import
-Some weewx users will have historical data from - another source (e.g., other weather station software or a manually compiled - file) which they wish to import into weewx. - Such data can, depending upon the source, be imported - using the wee_import utility. This section - details the use of the wee_import utility. -
+Some weewx users will have historical data from + another source (e.g., other weather station software or a manually compiled + file) which they wish to import into weewx. + Such data can, depending upon the source, be imported + using the wee_import utility. This section + details the use of the wee_import utility. +
-The wee_import utility supports importing - observational data from the following sources:
--
-
- a single Comma Separated Values (CSV) format file -
- the historical observations of a Weather Underground personal - weather station - -
- one or more Cumulus monthly log files -
The wee_import utility supports importing + observational data from the following sources:
+-
+
- a single Comma Separated Values (CSV) format file +
- the historical observations of a Weather Underground personal + weather station + +
- one or more Cumulus monthly log files +
Before starting, it's worth running the utility with the - --help flag to see how - wee_import is used:
+Before starting, it's worth running the utility with the + --help flag to see how + wee_import is used:
-wee_import --help+
wee_import --help-
This will result in an output that looks something like this:
+This will result in an output that looks something like this:
Usage: wee_import --help
wee_import --version
@@ -1856,170 +1907,170 @@ archive. Daily summaries are updated as each archive record is
imported so there should be no need to separately drop and rebuild
the daily summaries using the wee_database utility.
- Command line options
+Command line options
-The wee_import command line options are - described in more detail below:
+The wee_import command line options are + described in more detail below:
---config
+--config
-The utility is pretty good at "guessing" where your configuration file - weewx.conf is, but if you've done an unusual - install, you may have to tell it explicitly. You can do this by using - the --config option:
+The utility is pretty good at "guessing" where your configuration file + weewx.conf is, but if you've done an unusual + install, you may have to tell it explicitly. You can do this by using + the --config option:
wee_import --config=/this/folder/weewx.conf --import-config=/folder/import.conf-
--import-config
+--import-config
-wee_import uses a secondary configuration file to - store various import parameters. The - --import-config command line option - is mandatory for all imports. Example import configuration files for - each type of import supported by wee_import - are provided in the util/import folder. These - example files are best used by making a copy of the applicable example - file in a working directory and then modifying the duplicate file to - suit your needs. The --import-config - command line option is used as follows:
+wee_import uses a secondary configuration file to + store various import parameters. The + --import-config command line option + is mandatory for all imports. Example import configuration files for + each type of import supported by wee_import + are provided in the util/import folder. These + example files are best used by making a copy of the applicable example + file in a working directory and then modifying the duplicate file to + suit your needs. The --import-config + command line option is used as follows:
wee_import --import-config=/folder/import.conf-
--dry-run
+--dry-run
-The inclusion of the --dry-run command - line option will cause the import to proceed but no actual data will be - saved to the database. This is a useful option to use when first - importing data.
+The inclusion of the --dry-run command + line option will cause the import to proceed but no actual data will be + saved to the database. This is a useful option to use when first + importing data.
wee_import --import-config=/folder/import.conf --dry-run-
--date
+--date
-The date-time range of records to be imported can be specified by use of - the --date command line option. The - --date command line option can - specify a single date, as single date-time, a date range or a date-time - range. The date format used is YYYY/MM/DD and - the date-time format YYYY/MM/DD HH:MM. A range - is specified by separating two date or date-time formats by a hyphen, - e.g., '2015/12/1-2015/12/30'. Note that the - date-time or date-time range string must be enclosed in single or double - quotation marks. The effect of the different - --date command line option values - is shown in the following table:
+The date-time range of records to be imported can be specified by use of + the --date command line option. The + --date command line option can + specify a single date, as single date-time, a date range or a date-time + range. The date format used is YYYY/MM/DD and + the date-time format YYYY/MM/DD HH:MM. A range + is specified by separating two date or date-time formats by a hyphen, + e.g., '2015/12/1-2015/12/30'. Note that the + date-time or date-time range string must be enclosed in single or double + quotation marks. The effect of the different + --date command line option values + is shown in the following table:
-| command line option | -Records imported for a CSV or Cumulus import | -Records imported for a Weather Underground import | -
| omitted (i.e., the default) |
- All available records | -Todays records only | -
| --date 2015/12/22 | -All records from 2015/12/22 00:00 (inclusive) to 2015/12/23 00:00 (exclusive) | -All records from 2015/12/22 00:00 (inclusive) to 2015/12/23 00:00 (exclusive) | -
| --date "2015/12/22 22:30" | -The record timestamped 2015/12/22 22:30 only | -The record timestamped 2015/12/22 22:30 only | -
| --date 2015/12/22-2016/02/20 | -All records from 2015/12/22 00:00 (inclusive) to 2016/2/21 00:00 (exclusive) | -All records from 2015/12/22 00:00 (inclusive) to 2016/2/21 00:00 (exclusive) | -
| --date "2016/3/18 15:29-2016/6/20 22:00" | -All records from 2016/3/18 15:29 (inclusive) to 2016/6/20 22:00 (exclusive) | -All records from 2016/3/18 15:29 (inclusive) to 2016/6/20 22:00 (exclusive) | -
| command line option | +Records imported for a CSV or Cumulus import | +Records imported for a Weather Underground import | +
| omitted (i.e., the default) |
+ All available records | +Todays records only | +
| --date 2015/12/22 | +All records from 2015/12/22 00:00 (inclusive) to 2015/12/23 00:00 (exclusive) | +All records from 2015/12/22 00:00 (inclusive) to 2015/12/23 00:00 (exclusive) | +
| --date "2015/12/22 22:30" | +The record timestamped 2015/12/22 22:30 only | +The record timestamped 2015/12/22 22:30 only | +
| --date 2015/12/22-2016/02/20 | +All records from 2015/12/22 00:00 (inclusive) to 2016/2/21 00:00 (exclusive) | +All records from 2015/12/22 00:00 (inclusive) to 2016/2/21 00:00 (exclusive) | +
| --date "2016/3/18 15:29-2016/6/20 22:00" | +All records from 2016/3/18 15:29 (inclusive) to 2016/6/20 22:00 (exclusive) | +All records from 2016/3/18 15:29 (inclusive) to 2016/6/20 22:00 (exclusive) | +
- Note
If the --date
- command line option is omitted the default is to import all available
- records when importing from a CSV or Cumulus source or to import today's
- records only when importing from Weather Underground.
-
+ Note
If the --date
+ command line option is omitted the default is to import all available
+ records when importing from a CSV or Cumulus source or to import today's
+ records only when importing from Weather Underground.
+
--log
+--log
-The --log option controls the - wee_import log output. Omitting the option - will result in wee_import log output being - sent to the weewx log file (nominally the - system log, refer to - Monitoring weewx - and - Where to find things - to find it). wee_import log output can be - disabled by using --log=-. The - --log command line option is used as - follows: -
+The --log option controls the + wee_import log output. Omitting the option + will result in wee_import log output being + sent to the weewx log file (nominally the + system log, refer to + Monitoring weewx + and + Where to find things + to find it). wee_import log output can be + disabled by using --log=-. The + --log command line option is used as + follows: +
wee_import --import-config=/folder/import.conf --log=--
--verbose
+--verbose
-Inclusion of the --verbose command - line option will cause additional information to be printed during - wee_import execution.
+Inclusion of the --verbose command + line option will cause additional information to be printed during + wee_import execution.
wee_import --import-config=/folder/import.conf --verbose-
Importing from CSV files
+Importing from CSV files
-wee_import can import data from - a single CSV file. The CSV source file must be - structured as follows:
+wee_import can import data from + a single CSV file. The CSV source file must be + structured as follows:
--
-
- The file must have a header row consisting of a comma separated - list of field names. The field names can be any valid string as long - as each field name is unique within the list. There is no - requirement for the field names to be in any particular order as - long as the same order is used for the observations on each row in - the file. These field names will be mapped to - weewx field names in the - [CSV] section of the import configuration - file. - +
- The file must have a header row consisting of a comma separated + list of field names. The field names can be any valid string as long + as each field name is unique within the list. There is no + requirement for the field names to be in any particular order as + long as the same order is used for the observations on each row in + the file. These field names will be mapped to + weewx field names in the + [CSV] section of the import configuration + file. + -
- Observation data for a given date-time must be listed on a - single line with individual fields separated by a comma. The fields - must be in the same order as the field names in the header row. - +
- Observation data for a given date-time must be listed on a + single line with individual fields separated by a comma. The fields + must be in the same order as the field names in the header row. + -
- Blank fields are represented by the use of white space or no - space only between commas. - +
- Blank fields are represented by the use of white space or no + space only between commas. + -
- There must a field that represents the date-time of the - observations on each line. This date-time field must be either a - Unix epoch timestamp or any date-time format that can be - represented using - Python - strptime() format codes. - -
- There must a field that represents the date-time of the + observations on each line. This date-time field must be either a + Unix epoch timestamp or any date-time format that can be + represented using + Python + strptime() format codes. + +
-
+
A CSV file suitable for import by wee_import - may look like this:
+A CSV file suitable for import by wee_import + may look like this:
Time,Temp,Dewpoint,Press,WindDir,WindSpeed,WindGust,Hum,dailyrain,SolarRad 2016-02-01 00:00:00,13.5,9.2,1005.6,359,0.0,0.0,75,12.0,0.00 @@ -2033,125 +2084,127 @@ the daily summaries using the wee_database utility.2016-02-01 00:45:00,12.9,9.8,1005.3,22,1.6,3.2,81,23.0,0.00 2016-02-01 00:50:00,12.9,9.7,1005.3,22,1.6,1.6,81,23.0,0.00 -
weewx archive fields populated during a CSV import
+weewx archive fields populated during a CSV import
-The weewx archive fields populated during a - CSV import depend on the CSV-to-weewx field - mappings specified in [CSV] section in the - import configuration file. If a valid field mapping exists, the - weewx field exists in the - weewx archive table schema and provided the - mapped CSV field contains valid data, then the corresponding - weewx field will populated. Note that the CSV - import is the only import supported by - wee_import that allows any - weewx archive field to be populated. -
+The weewx archive fields populated during a + CSV import depend on the CSV-to-weewx field + mappings specified in [CSV] section in the + import configuration file. If a valid field mapping exists, the + weewx field exists in the + weewx archive table schema and provided the + mapped CSV field contains valid data, then the corresponding + weewx field will populated. Note that the CSV + import is the only import supported by + wee_import that allows any + weewx archive field to be populated. +
-
- Note
The use of the
- calc_missing option in the import
- configuration file may result in a
- number of derived fields being calculated from the imported data. If
- these derived fields exist in the in-use database schema they will be
- saved to the database as well.
-
+ Note
The use of the
+ calc_missing option in the import
+ configuration file may result in a
+ number of derived fields being calculated from the imported data. If
+ these derived fields exist in the in-use database schema they will be
+ saved to the database as well.
+
Importing observations from a CSV file
+Importing observations from a CSV file
-To import observations from a CSV file:
+To import observations from a CSV file:
--
-
- Ensure the source data file is in a folder accessible by the machine - that will run wee_import. For the - purposes of these instructions the source data file - data.csv located in the - /var/tmp folder will be used. - +
- Ensure the source data file is in a folder accessible by the machine + that will run wee_import. For the + purposes of these instructions the source data file + data.csv located in the + /var/tmp folder will be used. + -
- Make a backup of the - weewx database in case the import - should go awry. - +
- Make a backup of the + weewx database in case the import + should go awry. + -
- Create an import configuration file. - In this case we will make a copy of the example CSV - import configuration file and save it as - csv.conf in the - /var/tmp directory: - +
- Create an import configuration file. + In this case we will make a copy of the example CSV + import configuration file and save it as + csv.conf in the + /var/tmp directory: +
- Confirm that the source option is set to CSV: +
- Confirm that the source option is set to + CSV: + + +
- Confirm that the following options in the [CSV] + section are set: + + +
- file. + The full path and file name of the file containing the CSV + formatted data to be imported. - -
- Confirm that the following options in the [CSV] - section are set: +
- interval. + Determines how the weewx interval + field is derived. - -
- file. - The full path and file name of the file containing the CSV - formatted data to be imported. - -
- interval. - Determines how the weewx interval - field is derived. - -
- qc. Determines - whether quality control checks are performed on the imported - data. - -
- calc_missing. - Determines whether missing derived observations will be - calculated from the imported data. - -
- tranche. The - number of records written to the - weewx database in each transaction. - -
- UV_sensor. - Whether a UV sensor was installed when the source data was - produced. - -
- solar_sensor. - Whether a solar radiation sensor was installed when the - source data was produced. - -
- raw_datetime_format. - The format of the imported date time field. - -
- rain. Determines - how the weewx rain field is - derived. - -
- wind_direction. - Determines how imported wind direction fields are - interpreted. - -
- [[Map]]. Defines - the mapping between imported data fields and - weewx archive fields. Also defines - the units of measure for each imported field. - -
- When first importing data it is prudent to do a dry run import - before any data are actually imported. A dry run import will perform - all steps of the import without actually writing imported data to - the weewx database. In addition, - consideration should be given to any additional command line options - such as --date. +
- qc. Determines + whether quality control checks are performed on the imported + data. -
- calc_missing. + Determines whether missing derived observations will be + calculated from the imported data. + +
- tranche. The + number of records written to the + weewx database in each transaction. + +
- UV_sensor. + Whether a UV sensor was installed when the source data was + produced. + +
- solar_sensor. + Whether a solar radiation sensor was installed when the + source data was produced. + +
- raw_datetime_format. + The format of the imported date time field. + +
- rain. Determines + how the weewx rain field is + derived. + +
- wind_direction. + Determines how imported wind direction fields are + interpreted. + +
- [[Map]]. Defines + the mapping between imported data fields and + weewx archive fields. Also defines + the units of measure for each imported field. + + + +
- When first importing data it is prudent to do a dry run import + before any data are actually imported. A dry run import will perform + all steps of the import without actually writing imported data to + the weewx database. In addition, + consideration should be given to any additional command line options + such as --date. + +
- Once the dry run results are satisfactory the data can be - imported using the following command: - +
- Once the dry run results are satisfactory the data can be + imported using the following command: + -
- If the import parameters - are acceptable enter y to proceed with the - import or n to abort the import. - If the import is confirmed then the source - data will be imported, processed and saved in the - weewx database. Information on the - progress of the import will be displayed similar to the following: - +
- If the import parameters + are acceptable enter y to proceed with the + import or n to abort the import. + If the import is confirmed then the source + data will be imported, processed and saved in the + weewx database. Information on the + progress of the import will be displayed similar to the following: +
- Whilst wee_import will advise of the - number of records processed and the number of unique records found, - wee_import does know how many, if any, of - the imported records were successfully saved to the database. The - user should look carefully through the - weewx log file covering the - wee_import session and take note of any - records that were not imported. The most common reason for imported - records not being saved to the database is because a record with - that timestamp already exists in the database, in such cases - something similar to following will be found in the log: - +
- Whilst wee_import will advise of the + number of records processed and the number of unique records found, + wee_import does know how many, if any, of + the imported records were successfully saved to the database. The + user should look carefully through the + weewx log file covering the + wee_import session and take note of any + records that were not imported. The most common reason for imported + records not being saved to the database is because a record with + that timestamp already exists in the database, in such cases + something similar to following will be found in the log: +
-
+
$ cp /home/weewx/util/import/csv-example.conf /var/tmp/csv.conf-
source = CSV+ +
-
+
source = CSV- -
-
-
To perform a dry run enter the following command:
+To perform a dry run enter the following command:
wee_import --import-config=/var/tmp/csv.conf --dry-run-
The output should be something like this:
+The output should be something like this:
Starting wee_import... A CSV import from source file '/var/tmp/data.csv' has been requested. @@ -2165,34 +2218,34 @@ Records processed: 70685; Unique records: 70685; Last timestamp: 2010-09-04 04:2 Finished dry run import. 70685 records were processed and 70685 unique records would have been imported.-
The output includes details about the data source, its - destination and some other details on how the data will - be processed. The import will then - be performed but no data will be written to the - weewx database. Upon completion a - brief summary of the records processed is provided. -
+The output includes details about the data source, its + destination and some other details on how the data will + be processed. The import will then + be performed but no data will be written to the + weewx database. Upon completion a + brief summary of the records processed is provided. +
-
- Note
As the weewx database is
- not altered when the --dry-run command
- line option is used, wee_import log output
- is suspended during a dry run import. In effect, the use of
- --dry-run is equivalent to
- --dry-run --log=-. During a dry run import
- the only wee_import output is that
- displayed on stdout.
-
+ Note
As the weewx database is
+ not altered when the --dry-run command
+ line option is used, wee_import log output
+ is suspended during a dry run import. In effect, the use of
+ --dry-run is equivalent to
+ --dry-run --log=-. During a dry run import
+ the only wee_import output is that
+ displayed on stdout.
+
wee_import --import-config=/var/tmp/csv.conf+
wee_import --import-config=/var/tmp/csv.conf-
This will result in a short preamble similar to that from - the dry run. At the end - of the preamble there will be a prompt:
+This will result in a short preamble similar to that from + the dry run. At the end + of the preamble there will be a prompt:
Starting wee_import... A CSV import from source file '/var/tmp/data.csv' has been requested. @@ -2205,265 +2258,266 @@ Proceeding will save all imported records in the weewx archive. Are you sure you want to proceed (y/n)?-
Records processed: 250; Unique records: 250; Last timestamp: 2010-02-09 19:25:00 AEST (1265707500)-
The line commencing with Records processed - should update as records are imported with progress information on - number of records processed, number of unique records imported and - the date time of the latest record processed. When the import is - complete a brief summary is displayed similar to the following:
+The line commencing with Records processed + should update as records are imported with progress information on + number of records processed, number of unique records imported and + the date time of the latest record processed. When the import is + complete a brief summary is displayed similar to the following:
Records processed: 70685; Unique records: 70685; Last timestamp: 2010-09-04 04:20:00 AEST (1283538000) Finished import. 70685 raw records resulted in 70685 unique records being processed in 276.63 seconds. Those records with a timestamp already in the archive will not have been imported. Confirm successful import in the weewx log file.-
Aug 22 14:38:28 jessie2 weewx[863]: manager: unable to add record 2010-09-04 04:20:00 AEST (1283538000) to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime-
In such cases the user should take note of the timestamp of the - record(s) concerned and make a decision about whether to delete the - pre-existing record and re-import the record or retain the - pre-existing record.
- -In such cases the user should take note of the timestamp of the + record(s) concerned and make a decision about whether to delete the + pre-existing record and re-import the record or retain the + pre-existing record.
-Importing from Weather Underground
+wee_import can import data from - the daily history of a Weather Undeground PWS. A Weather Underground - daily history provides weather station observations received by Weather - Underground for the PWS concerned on a day by day basis. As such, the - data is analogous to the weewx archive table. - When wee_import imports data from a Weather - Underground daily history each day is considered a 'period'. - wee_import processes one period at a time in - chronological order (oldest to newest) and provides import summary data - on a per period basis. -
+Importing from Weather Underground
-weewx archive fields populated during a Weather Underground import
+wee_import can import data from + the daily history of a Weather Undeground PWS. A Weather Underground + daily history provides weather station observations received by Weather + Underground for the PWS concerned on a day by day basis. As such, the + data is analogous to the weewx archive table. + When wee_import imports data from a Weather + Underground daily history each day is considered a 'period'. + wee_import processes one period at a time in + chronological order (oldest to newest) and provides import summary data + on a per period basis. +
-A Weather Underground import will populate - weewx archive fields as follows:
+weewx archive fields populated during a Weather Underground import
--
-
- Provided data exists for each field in the Weather Underground
- PWS daily history, the following weewx
- archive fields will be directly populated by imported data:
+
A Weather Underground import will populate + weewx archive fields as follows:
--
-
- dateTime -
- barometer -
- dewpoint -
- outHumidity -
- outTemp -
- radiation -
- rain -
- windDir -
- windGust -
- windSpeed -
- Note
-
If an appropriate field does not exist in - the Weather Underground daily history then the corresponding - weewx archive field will be set - to None/NULL. - For example, if there is no solar radiation sensor - then radiation will be NULL, - or if outHumidity was never - uploaded to Weather Undeground then - outHumidity will be NULL. -
-
- - The following weewx archive fields - will be populated from other settings or configuration - options: - +
- Provided data exists for each field in the Weather Underground
+ PWS daily history, the following weewx
+ archive fields will be directly populated by imported data:
-
-
- interval -
- usUnits -
- The following weewx archive fields - will be populated with values derived from the imported data - provided calc_missing = True is - included in the [WU] section of the - import configuration file and the field exists in the - in-use weewx archive table schema. - - -
- altimeter -
- ET -
- heatindex -
- pressure -
- rainRate -
- windchill +
- dateTime +
- barometer +
- dewpoint +
- outHumidity +
- outTemp +
- radiation +
- rain +
- windDir +
- windGust +
- windSpeed
- The following weewx archive fields + will be populated from other settings or configuration + options: + + +
- interval +
- usUnits
- The following weewx archive fields + will be populated with values derived from the imported data + provided calc_missing = True is + included in the [WU] section of the + import configuration file and the field exists in the + in-use weewx archive table schema. + -
- altimeter +
- ET +
- heatindex +
- pressure +
- rainRate +
- windchill +
- Obtain the weather station ID of the Weather Underground PWS from - which data is to be imported. The station ID will be a sequence of - numbers and upper case letters that is usually 11 or 12 characters - in length. For the purposes of these instructions a weather station - ID of ISTATION123 will be used. - +
- Make a backup of the - weewx database in case the - import should go awry. - +
- Create an import configuration file. In this case
- we will make a copy of the example Weather Underground import
- configuration file and save it as
- wu.conf in the
- /var/tmp directory:
+
To import observations from the daily history of a Weather Underground + PWS:
+ +-
+
- Obtain the weather station ID of the Weather Underground PWS from + which data is to be imported. The station ID will be a sequence of + numbers and upper case letters that is usually 11 or 12 characters + in length. For the purposes of these instructions a weather station + ID of ISTATION123 will be used. + + +
- Make a backup of the + weewx database in case the + import should go awry. + + +
- Create an import configuration file. In this case
+ we will make a copy of the example Weather Underground import
+ configuration file and save it as
+ wu.conf in the
+ /var/tmp directory:
$ cp /home/weewx/util/import/wu-example.conf /var/tmp/wu.conf
- - Confirm that the source option is set to WU +
- Confirm that the source option is set to + WU + + +
- Confirm that the following options in the + [WU] section are correctly set: + + +
- station_id. + The 11 or 12 character weather station ID of the Weather + Underground PWS that will be the source of the imported + data. + +
- interval. + Determines how the weewx interval + field is derived. + +
- qc. Determines + whether quality control checks are performed on the imported + data. -
- Confirm that the following options in the - [WU] section are correctly set: +
- calc_missing. + Determines whether missing derived observations will be + calculated from the imported data. - -
- station_id. - The 11 or 12 character weather station ID of the Weather - Underground PWS that will be the source of the imported - data. - -
- interval. - Determines how the weewx interval - field is derived. - -
- qc. Determines - whether quality control checks are performed on the imported - data. - - -
- calc_missing. - Determines whether missing derived observations will be - calculated from the imported data. - -
- tranche. The - number of records written to the - weewx database in each transaction. - -
- wind_direction. - Determines how imported wind direction fields are - interpreted. - -
- When first importing data it is prudent to do a dry run import - before any data are actually imported. A dry run import will perform - all steps of the import without actually writing imported data to - the weewx database. In addition, - consideration should be given to any additional command line options - to be used such as --date. +
- tranche. The + number of records written to the + weewx database in each transaction. -
- wind_direction. + Determines how imported wind direction fields are + interpreted. + +
source = WU
+ +-
+
source = WU
++ Note
-
As Weather Underground imports at times + contain nonsense values, particularly for fields for + which no data were uploaded to Weather Underground by the + PWS, the use of quality control checks on imported data + can prevent these nonsense values from being imported + and contaminating the weewx + database. +-
-
- Note
- -
As Weather Underground imports at times - contain nonsense values, particularly for fields for - which no data were uploaded to Weather Underground by the - PWS, the use of quality control checks on imported data - can prevent these nonsense values from being imported - and contaminating the weewx - database. -To perform a dry run enter the following command:
+
-
+
-
-
- Note
If
- calc_missing = False is included
- in the [WU] section of the import
- configuration file being used then all of the above fields
- will be set to None/NULL.
- The default setting of the
- calc_missing option is
- True
+ Note
If an appropriate field does not exist in
+ the Weather Underground daily history then the corresponding
+ weewx archive field will be set
+ to None/NULL.
+ For example, if there is no solar radiation sensor
+ then radiation will be NULL,
+ or if outHumidity was never
+ uploaded to Weather Undeground then
+ outHumidity will be NULL.
-
+
Importing observations from Weather Underground
+To import observations from the daily history of a Weather Underground - PWS:
+-
+
-
-
+ Note
If
+ calc_missing = False is included
+ in the [WU] section of the import
+ configuration file being used then all of the above fields
+ will be set to None/NULL.
+ The default setting of the
+ calc_missing option is
+ True
+
Importing observations from Weather Underground
-To perform a dry run enter the following command:
wee_import --import-config=/var/tmp/wu.conf --date="2016/01/20 22:30-2016/01/23 06:00" --dry-run-
- Note
If the --date command
- line option is omitted or a date (not date-time) range is used
- during a Weather Underground import, then the current days
- history data will be imported. This includes records timestamped
- from 00:00 (inclusive) at the start
- of the day up to but NOT including the
- 00:00 record at the end of the last
- day. As the timestamped record refers to observations of the
- previous interval, such an import actually includes one record
- with observations from the previous day (the
- 00:00 record at the start of the day).
- Whilst this will not present a problem for
- wee_import as any records being
- imported with a timestamp that already exists in the
- weewx database are ignored, the user
- may wish to use the --date option
- with a suitable date-time range to precisely control which
- records are imported.
-
+ Note
If the --date command
+ line option is omitted or a date (not date-time) range is used
+ during a Weather Underground import, then the current days
+ history data will be imported. This includes records timestamped
+ from 00:00 (inclusive) at the start
+ of the day up to but NOT including the
+ 00:00 record at the end of the last
+ day. As the timestamped record refers to observations of the
+ previous interval, such an import actually includes one record
+ with observations from the previous day (the
+ 00:00 record at the start of the day).
+ Whilst this will not present a problem for
+ wee_import as any records being
+ imported with a timestamp that already exists in the
+ weewx database are ignored, the user
+ may wish to use the --date option
+ with a suitable date-time range to precisely control which
+ records are imported.
+
- Note
wee_import obtains
- Weather Underground daily history data one day at a time via a
- HTTP request and as such the import of large time spans of data
- may take some time. Such imports may be best handled as a series
- of imports of smaller time spans.
-
+ Note
wee_import obtains
+ Weather Underground daily history data one day at a time via a
+ HTTP request and as such the import of large time spans of data
+ may take some time. Such imports may be best handled as a series
+ of imports of smaller time spans.
+
This will result in a short preamble with details - on the data source, its destination and some other details on - how the data will be processed. The import will then be - performed but no data will written to the - weewx database. -
-The output should be similar to:
+This will result in a short preamble with details + on the data source, its destination and some other details on + how the data will be processed. The import will then be + performed but no data will written to the + weewx database. +
+The output should be similar to:
Starting wee_import... Observation history for Weather Underground station 'ISTATION123' will be imported. @@ -2486,27 +2540,27 @@ Records processed: 62; Unique records: 62; Last timestamp: 2016-01-23 05:55:00 A Finished dry run import. 607 records were processed and 607 unique records would have been imported.-
- Note
As the weewx database is
- not altered when the --dry-run command
- line option is used, wee_import log output
- is suspended during a dry run import. In effect, the use of
- --dry-run is equivalent to
- --dry-run --log=-. During a dry run import
- the only wee_import output is that
- displayed on stdout(console).
-
+ Note
As the weewx database is
+ not altered when the --dry-run command
+ line option is used, wee_import log output
+ is suspended during a dry run import. In effect, the use of
+ --dry-run is equivalent to
+ --dry-run --log=-. During a dry run import
+ the only wee_import output is that
+ displayed on stdout(console).
+
wee_import --import-config=/var/tmp/wu.conf --date="2016/01/20 22:30-2016/01/23 06:00"- -
This will result in a short preamble similar to that of - a dry run. At the end of the preamble there will be a - prompt:
+ +This will result in a short preamble similar to that of + a dry run. At the end of the preamble there will be a + prompt:
Starting wee_import... Observation history for Weather Underground station 'ISTATION123' will be imported. @@ -2521,22 +2575,22 @@ Proceeding will save all imported records in the weewx archive. Are you sure you want to proceed (y/n)?-
- Note
wee_import obtains
- Weather Underground daily history data one day at a time via a HTTP
- request and as such the import of large time spans of data may take
- some time. Such imports may be best handled as a series of imports
- of smaller time spans.
-
+ Note
wee_import obtains
+ Weather Underground daily history data one day at a time via a HTTP
+ request and as such the import of large time spans of data may take
+ some time. Such imports may be best handled as a series of imports
+ of smaller time spans.
+
Period 1 ... Records processed: 18; Unique records: 18; Last timestamp: 2016-01-20 23:55:00 AEST (1453298100) @@ -2544,244 +2598,244 @@ Period 2 ... Records processed: 286; Unique records: 286; Last timestamp: 2016-01-21 23:55:00 AEST (1453384500)-
The line commencing with Records processed - should update as records are imported with progress information on - number of records processed, number of unique records imported and - the date time of the latest record processed. If the import spans - multiple days then a new Period line is - created for each day. When the import is complete a brief summary - is displayed similar to the following:
+The line commencing with Records processed + should update as records are imported with progress information on + number of records processed, number of unique records imported and + the date time of the latest record processed. If the import spans + multiple days then a new Period line is + created for each day. When the import is complete a brief summary + is displayed similar to the following:
Finished import. 607 raw records resulted in 607 unique records being processed in 80.94 seconds. Those records with a timestamp already in the archive will not have been imported. Confirm successful import in the weewx log file.-
- Note
It is not unusual to see a Weather Underground
- import return a different number of records for the same import
- performed at different times. If importing the current day this
- could be because an additional record may have been added between
- wee_import runs. For periods before today,
- this behaviour appears to be a vagary of Weather Underground. The
- only solution appears to be to repeat the import with the same
- --date command line option setting and
- observe whether the missing records are imported. Repeating the
- import will not adversely affect any existing data as records with
- timestamps that are already in the weewx
- archive will be ignored. It may; however, generated many
- UNIQUE constraint failed: archive.dateTime
- messages in the weewx log.
-
+ Note
It is not unusual to see a Weather Underground
+ import return a different number of records for the same import
+ performed at different times. If importing the current day this
+ could be because an additional record may have been added between
+ wee_import runs. For periods before today,
+ this behaviour appears to be a vagary of Weather Underground. The
+ only solution appears to be to repeat the import with the same
+ --date command line option setting and
+ observe whether the missing records are imported. Repeating the
+ import will not adversely affect any existing data as records with
+ timestamps that are already in the weewx
+ archive will be ignored. It may; however, generated many
+ UNIQUE constraint failed: archive.dateTime
+ messages in the weewx log.
+
Aug 22 14:38:28 jessie2 weewx[863]: manager: unable to add record 2010-09-04 04:20:00 AEST (1283538000) to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime-
In such cases the user should take note of the timestamp of the - record(s) concerned and make a decision about whether to delete the - pre-existing record and re-import the record or retain the - pre-existing record.
+In such cases the user should take note of the timestamp of the + record(s) concerned and make a decision about whether to delete the + pre-existing record and re-import the record or retain the + pre-existing record.
- + -Importing from Cumulus
+Importing from Cumulus
-wee_import can import observational data from - the one or more Cumulus monthly log files. A Cumulus monthly log file - records weather station observations for a single month. These files are - accumulated over time and can be considered analogous to the - weewx archive table. When - wee_import imports data from the Cumulus - monthly log files each log file is considered a 'period'. - wee_import processes one period at a time in - chronological order (oldest to newest) and provides import summary - data on a per period basis.
+wee_import can import observational data from + the one or more Cumulus monthly log files. A Cumulus monthly log file + records weather station observations for a single month. These files are + accumulated over time and can be considered analogous to the + weewx archive table. When + wee_import imports data from the Cumulus + monthly log files each log file is considered a 'period'. + wee_import processes one period at a time in + chronological order (oldest to newest) and provides import summary + data on a per period basis.
-weewx archive fields populated during a Cumulus monthly log import
+weewx archive fields populated during a Cumulus monthly log import
-A Cumulus monthly log file import will populate the - weewx archive fields as follows:
+A Cumulus monthly log file import will populate the + weewx archive fields as follows:
--
-
- Provided data exists for each field in the Cumulus monthly
- logs, the following weewx
- archive fields will be directly populated by imported data:
-
-
-
-
- dateTime -
- barometer -
- dewpoint -
- heatindex -
- inHumidity -
- inTemp -
- outHumidity -
- outTemp -
- radiation -
- rain (Cumulus 1.9.4 or - later) - -
- rainRate -
- UV -
- windDir -
- windGust -
- windSpeed -
- windchill -
- Note
-
If a field in the Cumulus monthly log file - has no data then the corresponding - weewx archive field will be set - to None/NULL. -
-
- - The following weewx archive fields - will be populated from other settings or configuration options: - +
- Provided data exists for each field in the Cumulus monthly
+ logs, the following weewx
+ archive fields will be directly populated by imported data:
-
-
- interval -
- usUnits -
- The following weewx archive fields - will be populated with values derived from the imported data - provided calc_missing = True is - included in the [Cumulus] section of - the import configuration file being used and the field exists - in the in-use weewx archive table - schema. - - -
- altimeter -
- ET -
- pressure +
- dateTime +
- barometer +
- dewpoint +
- heatindex +
- inHumidity +
- inTemp +
- outHumidity +
- outTemp +
- radiation +
- rain (Cumulus 1.9.4 or + later) + +
- rainRate +
- UV +
- windDir +
- windGust +
- windSpeed +
- windchill
- The following weewx archive fields + will be populated from other settings or configuration options: + + +
- interval +
- usUnits
- The following weewx archive fields + will be populated with values derived from the imported data + provided calc_missing = True is + included in the [Cumulus] section of + the import configuration file being used and the field exists + in the in-use weewx archive table + schema. + -
- altimeter +
- ET +
- pressure +
- Ensure the Cumulus monthly log file(s) to be used for the import - are located in a directory accessible by the machine that will run - wee_import. For the purposes of these - instructions, there are nine monthly logs files covering the - period November 2015 to July 2016, inclusive, located in the - /var/tmp/cumulus folder. - +
- Make a backup of the - weewx database in case the - import should go awry. - +
- Create an import configuration file. In this case
- we will make a copy of the example Cumulus import configuration file
- and save it as cumulus.conf in the
- /var/tmp directory:
+
To import observations from one or more Cumulus monthly log files:
+ +-
+
- Ensure the Cumulus monthly log file(s) to be used for the import + are located in a directory accessible by the machine that will run + wee_import. For the purposes of these + instructions, there are nine monthly logs files covering the + period November 2015 to July 2016, inclusive, located in the + /var/tmp/cumulus folder. + + +
- Make a backup of the + weewx database in case the + import should go awry. + + +
- Create an import configuration file. In this case
+ we will make a copy of the example Cumulus import configuration file
+ and save it as cumulus.conf in the
+ /var/tmp directory:
$ cp /home/weewx/util/import/cumulus-example.conf /var/tmp/cumulus.conf
- - Confirm that the source option is set to Cumulus: +
- Confirm that the source option is set to Cumulus: + + +
- Confirm that the following options in the + [Cumulus] section are correctly set: + + +
- directory. + The full path to the directory containing the Cumulus + monthly log files to be used as the source of the imported + data. - -
- Confirm that the following options in the - [Cumulus] section are correctly set: +
- interval. + Determines how the weewx interval + field is derived. - -
- directory. - The full path to the directory containing the Cumulus - monthly log files to be used as the source of the imported - data. - -
- interval. - Determines how the weewx interval - field is derived. - -
- qc. Determines - whether quality control checks are performed on the imported - data. - -
- calc_missing. - Determines whether missing derived observations will be - calculated from the imported data. - -
- delimiter. The - field delimiter used in the Cumulus monthly log files. - -
- decimal. The - decimal point character used in the Cumulus monthly log - files. - -
- tranche. The - number of records written to the - weewx database in each transaction. - -
- UV_sensor. - Whether a UV sensor was installed when the source data was - produced. - -
- solar_sensor. - Whether a solar radiation sensor was installed when the - source data was produced. - -
- [[Units]]. - Defines the units used in the Cumulus monthly log files. - -
- When first importing data it is prudent to do a dry run import - before any data are actually imported. A dry run import will perform - all steps of the import without actually writing imported data to - the weewx database. In addition, - consideration should be given to any additional command line options - to be used such as --date. +
- qc. Determines + whether quality control checks are performed on the imported + data. -
- calc_missing. + Determines whether missing derived observations will be + calculated from the imported data. + +
- delimiter. The + field delimiter used in the Cumulus monthly log files. + +
- decimal. The + decimal point character used in the Cumulus monthly log + files. + +
- tranche. The + number of records written to the + weewx database in each transaction. + +
- UV_sensor. + Whether a UV sensor was installed when the source data was + produced. + +
- solar_sensor. + Whether a solar radiation sensor was installed when the + source data was produced. + +
- [[Units]]. + Defines the units used in the Cumulus monthly log files. + +
source = Cumulus
+ +-
+
source = Cumulus
- --
-
To perform a dry run enter the following command:
+
-
+
-
-
- Note
If
- calc_missing = False is included
- in the [WU] section of the import
- configuration file being used then all of the above fields
- will be set to None/NULL.
- The default setting of the
- calc_missing option is
- True
+ Note
If a field in the Cumulus monthly log file
+ has no data then the corresponding
+ weewx archive field will be set
+ to None/NULL.
-
+
Importing observations from Cumulus monthly log files
+To import observations from one or more Cumulus monthly log files:
+-
+
-
-
+ Note
If
+ calc_missing = False is included
+ in the [WU] section of the import
+ configuration file being used then all of the above fields
+ will be set to None/NULL.
+ The default setting of the
+ calc_missing option is
+ True
+
Importing observations from Cumulus monthly log files
-To perform a dry run enter the following command:
wee_import --import-config=/var/tmp/cumulus.conf --dry-run-
This will result in a short preamble with - details on the data source, its destination and some other - details on how the data will be processed. The import will then - be performed but no data will be written to the - weewx database. -
-The output should be similar to:
+This will result in a short preamble with + details on the data source, its destination and some other + details on how the data will be processed. The import will then + be performed but no data will be written to the + weewx database. +
+The output should be similar to:
Starting wee_import... Cumulus monthly log files in the '/var/tmp/cumulus' directory will be imported @@ -2812,20 +2866,20 @@ Records processed: 3544; Unique records: 3543; Last timestamp: 2016-07-26 17:00: Finished dry run import. 37648 records were processed and 37620 unique records would have been imported.-
- Note
The nine periods correspond to the nine monthly log
- files used for this import.
-
+ Note
The nine periods correspond to the nine monthly log
+ files used for this import.
+
wee_import --import-config=/var/tmp/cumulus.conf-
This will result in a preamble similar to that of a dry run. - At the end of the preamble there will be a prompt:
+This will result in a preamble similar to that of a dry run. + At the end of the preamble there will be a prompt:
Starting wee_import... Cumulus monthly log files in the '/var/tmp/cumulus' directory will be imported @@ -2838,14 +2892,14 @@ Proceeding will save all imported records in the weewx archive. Are you sure you want to proceed (y/n)?-
If there is more than one Cumulus monthly log file then - wee_import will provide summary - information on a per period basis during the import. In addition, - if the --date command line option is used - then source data that falls outside the date or date range - specified with the --date command line - option is ignored. In such cases the preamble may look similar - to:
+If there is more than one Cumulus monthly log file then + wee_import will provide summary + information on a per period basis during the import. In addition, + if the --date command line option is used + then source data that falls outside the date or date range + specified with the --date command line + option is ignored. In such cases the preamble may look similar + to:
Starting wee_import... Cumulus monthly log files in the '/var/tmp/cumulus' directory will be imported @@ -2862,21 +2916,21 @@ Proceeding will save all imported records in the weewx archive. Are you sure you want to proceed (y/n)?-
Records processed: 1599; Unique records: 1599; Last timestamp: 2016-02-24 00:00:00 AEST (1456236000)-
Again if there is more than one Cumulus monthly log file and if the - --date command line option is used then - the progress information may instead look similar to:
+Again if there is more than one Cumulus monthly log file and if the + --date command line option is used then + the progress information may instead look similar to:
Period 4 ... Records processed: 2521; Unique records: 2521; Last timestamp: 2016-03-01 09:40:00 AEST (1456789200) @@ -2886,730 +2940,732 @@ Period 6 ... Records processed: 3238; Unique records: 3232; Last timestamp: 2016-04-24 00:00:00 AEST (1461420000)-
The line commencing with Records processed - should update as records are imported with progress information on - number of records processed, number of unique records imported and - the date time of the latest record processed. If the import spans - multiple months (ie multiple monthly log files) then a new - Period line is created for each month. - When the import is complete a brief summary is displayed similar to - the following:
+The line commencing with Records processed + should update as records are imported with progress information on + number of records processed, number of unique records imported and + the date time of the latest record processed. If the import spans + multiple months (ie multiple monthly log files) then a new + Period line is created for each month. + When the import is complete a brief summary is displayed similar to + the following:
Finished import. 37648 raw records resulted in 37620 unique records being processed in 93.70 seconds. Those records with a timestamp already in the archive will not have been imported. Confirm successful import in the weewx log file.-
Aug 22 14:38:28 jessie2 weewx[863]: manager: unable to add record 2010-09-04 04:20:00 AEST (1283538000) to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime-
In such cases take note of the timestamp of the - record(s) concerned and make a decision about whether to delete the - pre-existing record and re-import the record or retain the - pre-existing record.
- +In such cases take note of the timestamp of the + record(s) concerned and make a decision about whether to delete the + pre-existing record and re-import the record or retain the + pre-existing record.
+ -Dealing with import failures
+Dealing with import failures
-Sometimes bad things happen during an import.
+Sometimes bad things happen during an import.
-If errors were encountered, or if you suspect that the - weewx database has been - contaminated with incorrect data, here are some things - you can try to fix things up. -
+If errors were encountered, or if you suspect that the + weewx database has been + contaminated with incorrect data, here are some things + you can try to fix things up. +
--
-
- Manually delete the contaminated data. Use SQL - commands to manipulate the data in the - weewx archive database. - The simplicity of this process will depend on your - ability to use SQL, the amount of data imported, and - whether the imported data was dispersed amongst - existing. Once contaminated data have been removed the - daily summary tables will need to be dropped and - rebuilt using the - wee_database utility. - +
- Manually delete the contaminated data. Use SQL + commands to manipulate the data in the + weewx archive database. + The simplicity of this process will depend on your + ability to use SQL, the amount of data imported, and + whether the imported data was dispersed amongst + existing. Once contaminated data have been removed the + daily summary tables will need to be dropped and + rebuilt using the + wee_database utility. + -
- Delete the database and start over. For SQLite, - simply delete the database file. For MySQL, drop - the database. Then try the import again. - +
- Delete the database and start over. For SQLite, + simply delete the database file. For MySQL, drop + the database. Then try the import again. + -
- If the above steps are not appropriate then the - database should be restored from backup. You did - make a backup before starting the import? - -
- If the above steps are not appropriate then the + database should be restored from backup. You did + make a backup before starting the import? + +
-
+
The import configuration file
+The import configuration file
-wee_import requires a second configuration file, the import configuration file, in addition to the standard weewx configuration file. The import configuration file - specifies the import type and various options associated with - each type of import. The import configuration file is specified at the - wee_import command line using the mandatory - --import-config command line option. How the - user constructs the import configuration file is up to the user; - however, the recommended method is to copy one of the example import - configuration files located in the util/import - folder, modify the configuration options in the newly copied file to - suit the import to be performed and then use this file as the import - configuration file. -
+wee_import requires a second configuration file, the import configuration file, in + addition to the standard weewx configuration file. The import configuration file + specifies the import type and various options associated with + each type of import. The import configuration file is specified at the + wee_import command line using the mandatory + --import-config command line option. How the + user constructs the import configuration file is up to the user; + however, the recommended method is to copy one of the example import + configuration files located in the util/import + folder, modify the configuration options in the newly copied file to + suit the import to be performed and then use this file as the import + configuration file. +
-Following is the definitive guide to the options available in the import - configuration file. Default values are provided for a number of options, - meaning that if they are not listed in the import configuration file at - all wee_import will pick sensible values. - When the documentation below gives a "default value" this is what it - means. What follows is organized by the different sections of the import - configuration file. -
+Following is the definitive guide to the options available in the import + configuration file. Default values are provided for a number of options, + meaning that if they are not listed in the import configuration file at + all wee_import will pick sensible values. + When the documentation below gives a "default value" this is what it + means. What follows is organized by the different sections of the import + configuration file. +
-General
+General
-source
+source
-The source option determines the type of import - to be performed by wee_import. The option - must be set to one of the following: -
+The source option determines the type of import + to be performed by wee_import. The option + must be set to one of the following: +
--
-
- CSV to import from a single CSV format - file. - -
- WU to import from a Weather - Underground PWS daily history. - -
- Cumulus to import from one or more - Cumulus monthly log files. - -
There is no default.
- -[CSV]
- -The [CSV] section contains the - options relating to the import of observational data from a CSV format +
-
+
- CSV to import from a single CSV format file. - + +
- WU to import from a Weather + Underground PWS daily history. + +
- Cumulus to import from one or more + Cumulus monthly log files. + +
There is no default.
-file
+[CSV]
-The file containing the CSV format data to be used as the source during - the import. Include full path and filename. There is no default. -
+The [CSV] section contains the + options relating to the import of observational data from a CSV format + file. +
-interval
+file
-Determines how the time interval (weewx - archive table field interval) between - successive observations is derived. The interval can be derived by one - of three methods: -
+The file containing the CSV format data to be used as the source during + the import. Include full path and filename. There is no default. +
--
-
- The interval can be calculated as the time, rounded to the - nearest minute, between the date-time of successive records. - This method is suitable when the data was recorded at fixed - intervals and there are NO missing records in the source data. - Use of this method when there are missing records in the source - data can compromise the integrity of the - weewx statistical data. Select this - method by setting interval = derive. - -
- The interval can be set to the same value as the - archive_interval setting under - [StdArchive] in - weewx.conf. This setting is useful if - the data was recorded at fixed intervals but there are some - missing records and the fixed interval is the same as the - archive_interval setting under - [StdArchive] in - weewx.conf. Select this method by - setting interval = conf. - -
- The interval can be set to a fixed number of minutes. This - setting is useful if the source data was recorded at fixed - intervals but there are some missing records and the fixed - interval is different to the - archive_interval setting under - [StdArchive] in - weewx.conf. Select this method by - setting interval = x where - x is an integer number of minutes. - -
interval
-The default value is derive. If the CSV source - data records are equally spaced in time, but some records are missing, - then better result may be achieved using conf - or a fixed interval setting. -
+Determines how the time interval (weewx + archive table field interval) between + successive observations is derived. The interval can be derived by one + of three methods: +
-qc
+-
+
- The interval can be calculated as the time, rounded to the + nearest minute, between the date-time of successive records. + This method is suitable when the data was recorded at fixed + intervals and there are NO missing records in the source data. + Use of this method when there are missing records in the source + data can compromise the integrity of the + weewx statistical data. Select this + method by setting interval = derive. + +
- The interval can be set to the same value as the + archive_interval setting under + [StdArchive] in + weewx.conf. This setting is useful if + the data was recorded at fixed intervals but there are some + missing records and the fixed interval is the same as the + archive_interval setting under + [StdArchive] in + weewx.conf. Select this method by + setting interval = conf. + +
- The interval can be set to a fixed number of minutes. This + setting is useful if the source data was recorded at fixed + intervals but there are some missing records and the fixed + interval is different to the + archive_interval setting under + [StdArchive] in + weewx.conf. Select this method by + setting interval = x where + x is an integer number of minutes. + +
Determines whether simple quality control checks are applied to imported - data. Setting qc = True will result in - wee_import applying the - weewx StdQC minimum - and maximum checks to any imported observations. - wee_import quality control checks use the same - configuration settings, and operate in the same manner, as the - StdQC service. For example, for min/max QC, if an observation falls - outside of the quality control range for that observation, then the - observation will be set to None. The user will - be alerted through a short message similar to:
+The default value is derive. If the CSV source + data records are equally spaced in time, but some records are missing, + then better result may be achieved using conf + or a fixed interval setting. +
+ +qc
+ +Determines whether simple quality control checks are applied to imported + data. Setting qc = True will result in + wee_import applying the + weewx StdQC minimum + and maximum checks to any imported observations. + wee_import quality control checks use the same + configuration settings, and operate in the same manner, as the + StdQC service. For example, for min/max QC, if + an observation falls + outside of the quality control range for that observation, then the + observation will be set to None. The user will + be alerted through a short message similar to:
2016-01-12 10:00:00 AEST (1452556800) record value 'outTemp' 194.34 outside limits (0.0, 120.0)-
As derived observations are calculated after the quality control check is applied, derived - observations are not subject to quality control checks. Setting - qc = False will result in - wee_import not applying quality control checks - to imported data. The default is True. -
+As derived observations are calculated after the quality control check is applied, derived + observations are not subject to quality control checks. Setting + qc = False will result in + wee_import not applying quality control checks + to imported data. The default is True. +
-calc_missing
+calc_missing
-Determines whether any missing derived observations will be calculated - from the imported data. Setting - calc_missing = True will result in - wee_import using the weewx - StdWXCalculate service to calculate any - missing derived observations from the imported data. Setting - calc_missing = False will result in - weewx leaving any missing derived observations - as None. The observations that - StdWXCalculate can calculate are listed in the - [StdWXCalculate] section - of the User's Guide. The default is - True. -
+Determines whether any missing derived observations will be calculated + from the imported data. Setting + calc_missing = True will result in + wee_import using the weewx + StdWXCalculate service to calculate any + missing derived observations from the imported data. Setting + calc_missing = False will result in + weewx leaving any missing derived observations + as None. The observations that + StdWXCalculate can calculate are listed in the + [StdWXCalculate] section + of the User's Guide. The default is + True. +
-tranche
+tranche
-To speed up database operations imported records are committed to - database in groups of records rather than individually. The size of the - group is set by the tranche parameter. - Increasing the tranche parameter may result - in a slight speed increase but at the expense of increased memory usage. - Decreasing the tranche parameter will result - in less memory usage but at the expense of more frequent database access - and likely increased time to import. The default is - 250 which should suit most users. -
+To speed up database operations imported records are committed to + database in groups of records rather than individually. The size of the + group is set by the tranche parameter. + Increasing the tranche parameter may result + in a slight speed increase but at the expense of increased memory usage. + Decreasing the tranche parameter will result + in less memory usage but at the expense of more frequent database access + and likely increased time to import. The default is + 250 which should suit most users. +
-UV_sensor
+UV_sensor
-weewx records a - None/NULL for UV when no UV sensor is - installed, whereas some weather station software records a value of 0 - for UV index when there is no UV sensor installed. The - UV_sensor parameter enables - wee_import to distinguish between the case - where a UV sensor is present and the UV index is 0 and the case where - no UV sensor is present and UV index is 0. - UV_sensor = False should be used when no UV - sensor was used in producing the source data. - UV_sensor = False will result in - None/Null being recorded in the - weewx archive field - UV irrespective of any UV observations in the - source data. UV_sensor = True should be used - when a UV sensor was used in producing the source data. - UV_sensor = True will result in UV - observations in the source data being stored in the - weewx archive field - UV. The default is - True. -
+weewx records a + None/NULL for UV when no UV sensor is + installed, whereas some weather station software records a value of 0 + for UV index when there is no UV sensor installed. The + UV_sensor parameter enables + wee_import to distinguish between the case + where a UV sensor is present and the UV index is 0 and the case where + no UV sensor is present and UV index is 0. + UV_sensor = False should be used when no UV + sensor was used in producing the source data. + UV_sensor = False will result in + None/Null being recorded in the + weewx archive field + UV irrespective of any UV observations in the + source data. UV_sensor = True should be used + when a UV sensor was used in producing the source data. + UV_sensor = True will result in UV + observations in the source data being stored in the + weewx archive field + UV. The default is + True. +
-solar_sensor
+solar_sensor
-weewx records a - None/NULL when no solar radiation sensor is - installed, whereas some weather station software records a value of 0 - for solar radiation when there is no solar radiation sensor installed. - The solar_sensor parameter enables - wee_import to distinguish between the case - where a solar radiation sensor is present and solar radiation is 0 and - the case where no solar radiation sensor is present and solar radiation - is 0. solar_sensor = False should be used - when no solar radiation sensor was used in producing the source data. - solar_sensor = False will result in - None/Null being recorded in the - weewx archive field - radiation irrespective of any solar radiation - observations in the source data. - solar_sensor = True should be used when a - solar radiation sensor was used in producing the source data. - solar_sensor = True will result in solar - radiation observations in the source data being stored in the - weewx archive field - radiation. The default is - True. -
+weewx records a + None/NULL when no solar radiation sensor is + installed, whereas some weather station software records a value of 0 + for solar radiation when there is no solar radiation sensor installed. + The solar_sensor parameter enables + wee_import to distinguish between the case + where a solar radiation sensor is present and solar radiation is 0 and + the case where no solar radiation sensor is present and solar radiation + is 0. solar_sensor = False should be used + when no solar radiation sensor was used in producing the source data. + solar_sensor = False will result in + None/Null being recorded in the + weewx archive field + radiation irrespective of any solar radiation + observations in the source data. + solar_sensor = True should be used when a + solar radiation sensor was used in producing the source data. + solar_sensor = True will result in solar + radiation observations in the source data being stored in the + weewx archive field + radiation. The default is + True. +
-raw_datetime_format
+raw_datetime_format
-weewx records each record with a unique unix - epoch timestamp, whereas many weather station applications or web - sources export observational data with a human readable date-time. - This human readable date-time is interpreted according to the format - set by the raw_datetime_format - option. This option consists of - Python - strptime() format codes - and literal characters to represent the date-time data being - imported. For example, if the source data uses the format - 23 January 2015 15:34 then the appropriate setting for - raw_datetime_format would be - %d %B %Y %H:%M, 9:25:00 12/28/16 would use - %H:%M:%S %m/%d/%y. If the source data provides - a unix epoch timestamp as the date-time field then the unix epoch - timestamp is used directly and the - raw_datetime_format option is - ignored. The default is %Y-%m-%d %H:%M:%S. -
+weewx records each record with a unique unix + epoch timestamp, whereas many weather station applications or web + sources export observational data with a human readable date-time. + This human readable date-time is interpreted according to the format + set by the raw_datetime_format + option. This option consists of + Python + strptime() format codes + and literal characters to represent the date-time data being + imported. For example, if the source data uses the format + 23 January 2015 15:34 then the appropriate setting for + raw_datetime_format would be + %d %B %Y %H:%M, 9:25:00 12/28/16 would use + %H:%M:%S %m/%d/%y. If the source data provides + a unix epoch timestamp as the date-time field then the unix epoch + timestamp is used directly and the + raw_datetime_format option is + ignored. The default is %Y-%m-%d %H:%M:%S. +
-rain
+rain
-weewx records rainfall as the amount of rain - in the preceding archive period, so for a 5 minute archive period the - rain field in each archive record would contain the total rain that - fell in the previous 5 minutes. Many weather station applications - provide a daily or yearly total. wee_import - can derive the weewx - rain field in one of two ways: -
+weewx records rainfall as the amount of rain + in the preceding archive period, so for a 5 minute archive period the + rain field in each archive record would contain the total rain that + fell in the previous 5 minutes. Many weather station applications + provide a daily or yearly total. wee_import + can derive the weewx + rain field in one of two ways: +
--
-
- If the imported rain data is a running total then - wee_import can derive the - weewx rain field from successive - totals. For this method use - rain = cumulative. - +
- If the imported rain data is a running total then + wee_import can derive the + weewx rain field from successive + totals. For this method use + rain = cumulative. + -
- If the imported rain data is a discrete value per date-time - period then rain = discrete - should be used. - -
- If the imported rain data is a discrete value per date-time + period then rain = discrete + should be used. + +
-
+
- Note
wee_import only supports
- cumulative rainfall data that resets on a midnight boundary, cumulative
- rainfall data that resets at some other time; e.g., 9am, is not supported.
- In such cases the rainfall data will need to be converted to either
- reset on a midnight boundary or a discrete value per date-time period
- and rain = discrete used. The former
- may be possible by selecting another rainfall field (if available) in
- the source data, otherwise it will require manual manipulation of the
- source data.
-
+ Note
wee_import only supports
+ cumulative rainfall data that resets on a midnight boundary, cumulative
+ rainfall data that resets at some other time; e.g., 9am, is not supported.
+ In such cases the rainfall data will need to be converted to either
+ reset on a midnight boundary or a discrete value per date-time period
+ and rain = discrete used. The former
+ may be possible by selecting another rainfall field (if available) in
+ the source data, otherwise it will require manual manipulation of the
+ source data.
+
wind_direction
+wind_direction
-weewx records wind direction in degrees as a - number from 0 to 360 inclusive (no wind direction is recorded as - None/NULL), whereas some data sources may - provide wind direction as number over a different range (e.g., -180 to - +180) or may use a particular value when there is no wind direction - (e.g., 0 may represent no wind direction and 360 may represent a northerly - wind, or -9999 (or some similar clearly invalid number) to represent - there being no wind direction). wee_import - handles such variations in data by defining a range over which imported - wind direction values are accepted. Any value outside of this range is - treated as there being no wind direction and is recorded as - None/NULL. Any value inside the range is - normalised to the range 0 to 360 inclusive (e.g., -180 would be normalised - to 180). The wind_direction option - consists of two comma separated numbers of the format lower, upper where - lower and upper are inclusive. The operation of the - wind_direction option is best - illustrated through the following table: -
+weewx records wind direction in degrees as a + number from 0 to 360 inclusive (no wind direction is recorded as + None/NULL), whereas some data sources may + provide wind direction as number over a different range (e.g., -180 to + +180) or may use a particular value when there is no wind direction + (e.g., 0 may represent no wind direction and 360 may represent a northerly + wind, or -9999 (or some similar clearly invalid number) to represent + there being no wind direction). wee_import + handles such variations in data by defining a range over which imported + wind direction values are accepted. Any value outside of this range is + treated as there being no wind direction and is recorded as + None/NULL. Any value inside the range is + normalised to the range 0 to 360 inclusive (e.g., -180 would be normalised + to 180). The wind_direction option + consists of two comma separated numbers of the format lower, upper where + lower and upper are inclusive. The operation of the + wind_direction option is best + illustrated through the following table: +
-| wind_direction option setting | -Source data wind direction value | -Imported wind direction value | -
| 0, 360 | -0 | -0 | -
| 160 | -160 | -|
| 360 | -360 | -|
| 500 | -None/NULL | -|
| -45 | -None/NULL/td> - | |
| -9999 | -None/NULL | -|
| No data | -None/NULL | -|
| -360, 360 | -0 | -0 | -
| 160 | -160 | -|
| 360 | -360 | -|
| 500 | -None/NULL | -|
| -45 | -315 | -|
| -9999 | -None/NULL | -|
| No data | -None/NULL | -|
| -180, 180 | -0 | -0 | -
| 160 | -160 | -|
| 360 | -None/NULL | -|
| 500 | -None/NULL | -|
| -45 | -315 | -|
| -9999 | -None/NULL | -|
| No data | -None/NULL | -
| wind_direction option setting | +Source data wind direction value | +Imported wind direction value | +
| 0, 360 | +0 | +0 | +
| 160 | +160 | +|
| 360 | +360 | +|
| 500 | +None/NULL | +|
| -45 | +None/NULL/td> + | |
| -9999 | +None/NULL | +|
| No data | +None/NULL | +|
| -360, 360 | +0 | +0 | +
| 160 | +160 | +|
| 360 | +360 | +|
| 500 | +None/NULL | +|
| -45 | +315 | +|
| -9999 | +None/NULL | +|
| No data | +None/NULL | +|
| -180, 180 | +0 | +0 | +
| 160 | +160 | +|
| 360 | +None/NULL | +|
| 500 | +None/NULL | +|
| -45 | +315 | +|
| -9999 | +None/NULL | +|
| No data | +None/NULL | +
The default is 0, 360. -
+The default is 0, 360. +
-[[Map]]
+[[Map]]
-The [[Map]] stanza defines the mapping - from the source data fields to weewx archive - fields. The map consists of one row per field using the format: -
+The [[Map]] stanza defines the mapping + from the source data fields to weewx archive + fields. The map consists of one row per field using the format: +
weewx_archive_field_name = csv_field_name, weewx_unit_name-
Where weewx_archive_field_name is a - database field name in the weewx archive table - schema, csv_field_name is the name - of a field from the CSV file and - weewx_unit_name is the - weewx unit name of the units used by - csv_field_name. This mapping allows - wee_import to take a source data field, do - the appropriate unit conversion and store the resulting value in the - appropriate weewx archive field. A mapping is - not required for every weewx archive field - (e.g., the source may not provide inside temperature so no - inTemp field mapping is required) and neither - does every CSV field need to be included in a mapping (e.g., the source - data field monthrain may have no use if the - source data field dayrain provides the data - for the weewx archive - rain field). Unused field mapping lines will - not be used and may be omitted. -
+Where weewx_archive_field_name is a + database field name in the weewx archive table + schema, csv_field_name is the name + of a field from the CSV file and + weewx_unit_name is the + weewx unit name of the units used by + csv_field_name. This mapping allows + wee_import to take a source data field, do + the appropriate unit conversion and store the resulting value in the + appropriate weewx archive field. A mapping is + not required for every weewx archive field + (e.g., the source may not provide inside temperature so no + inTemp field mapping is required) and neither + does every CSV field need to be included in a mapping (e.g., the source + data field monthrain may have no use if the + source data field dayrain provides the data + for the weewx archive + rain field). Unused field mapping lines will + not be used and may be omitted. +
-
- Note
Any weewx archive fields that
- are derived (e.g., dewpoint) and for which there
- is no field mapping may be calculated during import by use of the
- calc_missing option in the
- [CSV] section of the import configuration
- file.
-
+ Note
Any weewx archive fields that
+ are derived (e.g., dewpoint) and for which there
+ is no field mapping may be calculated during import by use of the
+ calc_missing option in the
+ [CSV] section of the import configuration
+ file.
+
[WU]
+[WU]
-The [WU] section contains the - options relating to the import of observational data from a Weather - Underground PWS daily history. -
+The [WU] section contains the + options relating to the import of observational data from a Weather + Underground PWS daily history. +
-station_id
+station_id
-The Weather Underground weather station ID of the PWS from which the - daily history will be imported. There is no default. -
+The Weather Underground weather station ID of the PWS from which the + daily history will be imported. There is no default. +
-interval
+interval
-Determines how the time interval (weewx - database field interval) between successive - observations is determined. This option is identical in operation to the - CSV interval option but applies to - Weather Underground imports only. As Weather Underground often skips - observation records when responding to a daily history query, the use of - interval = derive may give incorrect - or inconsistent interval values. Better results may be obtained by using - interval = conf if the current - weewx installation has the same - archive_interval as the Weather Underground - data, or by using interval = x where - x is the time interval in minutes - used to upload the Weather Underground data. The most appropriate - setting will depend on the completeness and (time) accuracy of the - Weather Underground data being imported. -
+Determines how the time interval (weewx + database field interval) between successive + observations is determined. This option is identical in operation to the + CSV interval option but applies to + Weather Underground imports only. As Weather Underground often skips + observation records when responding to a daily history query, the use of + interval = derive may give incorrect + or inconsistent interval values. Better results may be obtained by using + interval = conf if the current + weewx installation has the same + archive_interval as the Weather Underground + data, or by using interval = x where + x is the time interval in minutes + used to upload the Weather Underground data. The most appropriate + setting will depend on the completeness and (time) accuracy of the + Weather Underground data being imported. +
-The default is derive.
+The default is derive.
-qc
+qc
-Determines whether simple quality control checks are applied to imported - data. This option is identical in operation to the CSV - qc option but applies to Weather - Underground imports only. As Weather Underground imports at times - contain nonsense values, particularly for fields for which no data was - uploaded to Weather Underground by the PWS, the use of quality control - checks on imported data can prevent these nonsense values from being - imported and contaminating the weewx database. - The default is True. -
+Determines whether simple quality control checks are applied to imported + data. This option is identical in operation to the CSV + qc option but applies to Weather + Underground imports only. As Weather Underground imports at times + contain nonsense values, particularly for fields for which no data was + uploaded to Weather Underground by the PWS, the use of quality control + checks on imported data can prevent these nonsense values from being + imported and contaminating the weewx database. + The default is True. +
-calc_missing
+calc_missing
-Determines whether any missing derived observations will be calculated - from the imported data. This option is identical in operation to the - CSV calc_missing option but - applies to Weather Underground imports only. The default is - True. -
+Determines whether any missing derived observations will be calculated + from the imported data. This option is identical in operation to the + CSV calc_missing option but + applies to Weather Underground imports only. The default is + True. +
-tranche
+tranche
-The number of records written to the weewx - database in each transaction. This option is identical in operation to - the CSV tranche option but applies - to Weather Underground imports only. The default is - 250 which should suit most users. -
+The number of records written to the weewx + database in each transaction. This option is identical in operation to + the CSV tranche option but applies + to Weather Underground imports only. The default is + 250 which should suit most users. +
-wind_direction
+wind_direction
-Determines the range of acceptable wind direction values in degrees. - This option is identical in operation to the CSV - wind_direction option but - applies to Weather Underground imports only. The default is - 0, 360 which should suit most users. -
+Determines the range of acceptable wind direction values in degrees. + This option is identical in operation to the CSV + wind_direction option but + applies to Weather Underground imports only. The default is + 0, 360 which should suit most users. +
-[Cumulus]
+[Cumulus]
-The [Cumulus] section contains the - options relating to the import of observational data from Cumulus - monthly log files. -
+The [Cumulus] section contains the + options relating to the import of observational data from Cumulus + monthly log files. +
-directory
+directory
-The full path to the directory containing the Cumulus monthly log files - to be imported. Do not include a trailing /. There is no default. -
+The full path to the directory containing the Cumulus monthly log files + to be imported. Do not include a trailing /. There is no default. +
-interval
+interval
-Determines how the time interval (weewx - database field interval) between successive - observations is determined. This option is identical in operation to - the CSV interval option but - applies to Cumulus monthly log file imports only. As Cumulus monthly - log files can, at times, have missing entries, the use of - interval = derive may give incorrect - or inconsistent interval values. Better results may be obtained by - using interval = conf if the - archive_interval for the current - weewx installation is the same as the Cumulus - 'data log interval' setting used to generate the Cumulus monthly log - files, or by using interval = x where - x is the time interval in minutes - used as the Cumulus 'data log interval' setting. The most appropriate - setting will depend on the completeness and (time) accuracy of the - Cumulus data being imported. -
+Determines how the time interval (weewx + database field interval) between successive + observations is determined. This option is identical in operation to + the CSV interval option but + applies to Cumulus monthly log file imports only. As Cumulus monthly + log files can, at times, have missing entries, the use of + interval = derive may give incorrect + or inconsistent interval values. Better results may be obtained by + using interval = conf if the + archive_interval for the current + weewx installation is the same as the Cumulus + 'data log interval' setting used to generate the Cumulus monthly log + files, or by using interval = x where + x is the time interval in minutes + used as the Cumulus 'data log interval' setting. The most appropriate + setting will depend on the completeness and (time) accuracy of the + Cumulus data being imported. +
-The default is derive.
+The default is derive.
-qc
+qc
-Determines whether simple quality control checks are applied to imported - data. This option is identical in operation to the CSV - qc option but applies to Cumulus imports - only. The default is True. -
+Determines whether simple quality control checks are applied to imported + data. This option is identical in operation to the CSV + qc option but applies to Cumulus imports + only. The default is True. +
-calc_missing
+calc_missing
-Determines whether any missing derived observations will be calculated - from the imported data. This option is identical in operation to the - CSV calc_missing option but - applies to Cumulus imports only. The default is - True. -
+Determines whether any missing derived observations will be calculated + from the imported data. This option is identical in operation to the + CSV calc_missing option but + applies to Cumulus imports only. The default is + True. +
-delimiter
+delimiter
-The character used as the field delimiter in the Cumulus monthly log - file. A comma is frequently used but it may be another character - depending on the settings on the machine that produced the Cumulus - monthly log files. This parameter must be included in quotation marks. - Default is ','. -
+The character used as the field delimiter in the Cumulus monthly log + file. A comma is frequently used but it may be another character + depending on the settings on the machine that produced the Cumulus + monthly log files. This parameter must be included in quotation marks. + Default is ','. +
-decimal
+decimal
-The character used as the decimal point in the Cumulus monthly log files - A full stop is frequently used but it may be another character depending - on the settings on the machine that produced the Cumulus monthly log - files. This parameter must be included in quotation marks. Default is - '.'. -
+The character used as the decimal point in the Cumulus monthly log files + A full stop is frequently used but it may be another character depending + on the settings on the machine that produced the Cumulus monthly log + files. This parameter must be included in quotation marks. Default is + '.'. +
-tranche
+tranche
-The number of records are written to the weewx - database in each transaction. This option is identical in operation to - the CSV tranche option but applies - to Cumulus monthly log file imports only. The default is - 250 which should suit most users. -
+The number of records are written to the weewx + database in each transaction. This option is identical in operation to + the CSV tranche option but applies + to Cumulus monthly log file imports only. The default is + 250 which should suit most users. +
-UV_sensor
+UV_sensor
-The UV_sensor parameter enables - wee_import to distinguish between the case - where a UV sensor is present and the UV index is 0 and the case where - no UV sensor is present and UV index is 0. This option is identical in - operation to the CSV UV_sensor option - but applies to Cumulus monthly log file imports only. The default is - True. -
+The UV_sensor parameter enables + wee_import to distinguish between the case + where a UV sensor is present and the UV index is 0 and the case where + no UV sensor is present and UV index is 0. This option is identical in + operation to the CSV UV_sensor option + but applies to Cumulus monthly log file imports only. The default is + True. +
-solar_sensor
+solar_sensor
-The solar_sensor parameter enables - wee_import to distinguish between the case - where a solar radiation sensor is present and the solar radiation is 0 - and the case where no solar radiation sensor is present and solar - radiation is 0. This option is identical in operation to - the CSV solar_sensor option but - applies to Cumulus monthly log file imports only. The default is - True. -
+The solar_sensor parameter enables + wee_import to distinguish between the case + where a solar radiation sensor is present and the solar radiation is 0 + and the case where no solar radiation sensor is present and solar + radiation is 0. This option is identical in operation to + the CSV solar_sensor option but + applies to Cumulus monthly log file imports only. The default is + True. +
-[[Units]]
+[[Units]]
-The [[Units]] stanza defines the units - used in the Cumulus monthly log files. Units settings are required for - temperature, - pressure, rain and - speed. The format for each setting is: -
+The [[Units]] stanza defines the units + used in the Cumulus monthly log files. Units settings are required for + temperature, + pressure, rain and + speed. The format for each setting is: +
obs_type = weewx_unit_name-
Where obs_type is one of - temperature, - pressure, rain or - speed and - weewx_unit_name is the - weewx unit name of the units used by that - particular obs_type. As Cumulus supports a - different suite of possible units only a subset of the available - weewx unit names can be used for some - settings. -
+Where obs_type is one of + temperature, + pressure, rain or + speed and + weewx_unit_name is the + weewx unit name of the units used by that + particular obs_type. As Cumulus supports a + different suite of possible units only a subset of the available + weewx unit names can be used for some + settings. +
- + -wee_reports
-In normal operation, weewx generates - reports on each archive interval, when new data arrive. The - reports utility is used to generate reports on demand. It uses - the same configuration file that - weewx uses. -
+wee_reports
+In normal operation, weewx generates + reports on each archive interval, when new data arrive. The + reports utility is used to generate reports on demand. It uses + the same configuration file that + weewx uses. +
-Run the utility with the --help - option to see how it is used: -
+Run the utility with the --help + option to see how it is used: +
-wee_reports --help+
wee_reports --help-
This results in something like this:
+This results in something like this:
Usage: wee_reports: [config_file] [timestamp] [--config=CONFIG_FILE] [--help] @@ -3622,23 +3678,23 @@ Options: --config=CONFIG_FILE Use the configuration file CONFIG_FILE- + -
weewxd
+weewxd
-The weewxd application is the heart - of weewx. It can be run directly, - or in the background as a daemon. -
+The weewxd application is the heart + of weewx. It can be run directly, + or in the background as a daemon. +
-Run with the --help option to see - how it is used: -
+Run with the --help option to see + how it is used: +
-weewxd --help+
weewxd --help-
This results in output something like:
+This results in output something like:
Usage: weewxd --help
weewxd --version
@@ -3665,13 +3721,13 @@ Options:
Label to use in syslog entries
-
+
-