mirror of
https://github.com/weewx/weewx.git
synced 2026-04-20 17:46:58 -04:00
* Got test suites working again. * Updated TODO * indicate specific firmware for cc3000 * clarify rainwise firmware * Further refinement - max and maxtime now updated - reworked the progress function, now a method in the class Have left __main__ code in that has been used for testing windSpeed recalculation fix * indicate when calibrations are ignored * Now emits the barometer trend in LOOP packets as field 'trendIcon' Vantage only. * Updated TODO guide, reflecting this morning's phone call. * Presses on, despite database error * Simplified patching by moving metadata code to DaySummaryManager * Got rid of _getVersion() Can now get it from _read_metadata * Remove interval weighting fix from weewx startup Refer TODO.txt * Update TODO.txt * added write timeout to ultimeter driver * better logging when cc3000 loses contact with sensors * document some of the channel, sensor, and logger specifics for the wmr100/wmr200 hardware * Removed the p word, patch.py is now database.py, rejigged wee_database as per skype and followup emails - progress function for fixes included as a method in base class that can be overridden - much changing of logging to give consistent results/output - believe I retained Tom's recent changes to patch.py (commit4acf752) * Picks daily summary weights on the basis of database version * Rewrote the backfill routine (again) * wee_database now uses new version of backfill_day_summary * Fixed problem that prevented cold startup * Revised to reflect latest wee_database incantation * Modified weighting fix Should not mess with lessUpdate Delete lastWeightPatch after successful patch * Check to make sure the daily summaries have not been partially updated. * Revised to reflect latest wee_database incantation * Revised wee_database and interval weighting paras, added windSpeed recalc para * Removed vacuum * Updated what has been done and committed * Better diagnostics with partial update of the daily summaries. * MySQL now uses transaction isolation level READ COMMITTED * Add .config (#204) * Changed semantics of "$last_xxx" tags. * Template test updates including fixing issue #201 (#205) * Add .config * Standardise test skin for index.html.tml to remove "%x %X" locale dependent formatting. Note: Used 24 hour time as AM/PM can also be locale dependent. Also include fixes for line formatting in some other test files and expected results. * sysctlbyname is not available on every platform, so catch AttributeError too, otherwise cheetah fails with a name error * Got rid of tabs * update usb mode info for acurite models * Removed intervalgenRoundTS and archiveDaysAgoSpan. * Simplified tags. Got template test working again. * Somehow, style "indent" got lost * Documented new "$ago" tags. * Documented $ago options, as well as .start and .end * Update changes doc * Stupid typo * use markdown instead of html for README * indenting seems to confuse markdown's handling of urls * Left over $last_day in $spans example (#206) * Add .config * Standardise test skin for index.html.tml to remove "%x %X" locale dependent formatting. Note: Used 24 hour time as AM/PM can also be locale dependent. Also include fixes for line formatting in some other test files and expected results. * Left over $last_day in $spans example. * a user-specified sensor_map will update, not replace, the default sensor_map * Updated the upgrade guide. * ensure that weewx-multi works on deb and rpm as well as setup.py * document the sensor mapping changes * Added comprehensive example to customizing guide. * Updated TODO * Got the weighting update to work on MySQL * added crude caching of pressure in wmr300 * Fixed typo * Fixed little errors. Consistently use "tag", instead of "dot code." * no altimeter, just pressure and barometer * Fixed little HTML problems. * Reformatted, in anticipation of a refactoring of some sections. * Fixed location of 'Version' So it works with automatic replacement of 'Version' * initial experiment with different fonts * override jquery-ui hijacking of toc font family * bring header highlighting to h2 to correlate with toc. reduce post-header margins. * fix some table header cruft. prevent indent sections from overflowing right margin. let the table cells breath. * fix broken tty formatting * Checkpoint Work in progress. * Refined examples of creating new units and groups * Polishing. Or, maybe fiddling? * minor css fixes. rearrange troubleshooting sections about pressure. * minor cleanup to readme * Fixed test suites Add MySQL back to template tests. * Reworked the iteration examples. * Corrected and clarified the units used in the "electricity" example. * remove write_timeout since naming is inconsistent between pyserial versions and there is no backward compatibility * remove write_timeout from ultimeter * Minor changes to users guide * increase body size to 100%. background for code. weeWX in titles. true bold for monospace. prep for direct font comparisons. * Clarified the role of encoding * update utilities guide with suggested wee_database descriptions and args * more compaction * increase margins on html examples. use droid serif for html examples. * Clarified a few things * decode weewx into weeWX * include transaction limit defaults * eliminate transaction-limit * weeWX fixes in install pages. more fixes to utilities. * missed a few code spans * fix version label alignment * use only major.minor for docs * Fix error where import fields that are None can cause Source.mapRawData() to crash in some cases * Updated TODO and NEW_FEATURES * There will be no daykeys if the daily summaries have been dropped. * Restructure usage string, hard code transaction days * dry-run goes with fix-strings, not check-strings * Log daily summary version * No need to check for weewx.debug * include examples in manifest * added examples and extensions to data_files * do not install sample extensions * Can now specify date field separator for Cumulus imports, weewx -> weeWX * Clarified option strings * Rudimentary test of selective daily summary rebuild * No longer allows selective rebuild of the daily summaries if the summaries are not complete * Hardwired UTF-8 encoding, but with a warning comment. * Hardwired UTF-8 encoding, but with a warning comment. * Documented Cumulus import separator config option * Added comments about Tools. * Changed to execute in user environment * Ported to PyMySQL as an alternative to python-mysqldb See https://github.com/PyMySQL/PyMySQL for a description of PyMySQL * Recognize additional MySQL "Can't connect" error * Fixed error in test suites Subsequent tests depended on ordering of a dictionary. * pymysql seems to have problems connecting via file socket unless it is told explicitly about it. * Workaround for pypy compiler * Defaults now support MySQLdb over pymysql * Fixed bug in record augmentation. The augmentation was happening without giving StdConvert a chance to do its thing. * More clear msg when encountering an ImportError * Clarified the relationship between archive period and report_timing option, aded note regarding primacy of the report cycle * Reverted back to MySQLdb only version. * Changed config option names but never changed the code! * One transaction for updating daily high/lows and archive record Formerly, these were done in two transactions. * v3.7.0a2 * Adjustable value for how long to wait after a bad uploader login. Option retry_login. Fixes issue #212. * Fixed Cumulus import rain field issue wee_import will try to use field 26(AA) - midnight reset daily rainfall but if not available due it will revert to field 9(J) or 11(L) * Switched back to __str__ when extracting string out of template. .respond() doesn't seem to encode Unicode characters properly * Fix errors in wee_import WU step-by-step, remove Cumulus version caveat on rain * Now uses dedicated test users 'weewx1' and 'weewx2' * Formal check of the various MySQLdb exceptions. * Added sqlite3 exceptions. * Reworked check_strings() output (#213) Reworked check_strings() screen and syslog output: - now gives progress ala --rebuild-daily - syslog is silent for --check-strings and --fix-strings with --dry-run - left 'Preparing' (rather than 'Starting') but added 'this may take a while' as there is a significant delay in dbmanager.genBatchRows() initialising at line 619 (well there was for 400k records) * reduce debug log spewage in wmr300 driver * Finished formal test of errors * First cut at V3.7 exception hierarchy * Ported the weedb sqlite driver to the new exception hierarchy. * Ported MySQLdb to the new database exception hierarchy. * windSpeed fix now gracefully handles no windSpeed summary table, tweaked --update output/logging * Now picks up absence of windSpeed daily summary * The weedb Connection object can now be used in a "with" clause. * The weedb Cursor object can now be used in a "with" clause. * V3.7.0a3 * more code removal * code formatting only * use apt instructions for debian installs * Reworked --wee-database section of Utilities Guide to reflect current wee-database operation - revised usage - reword --rebuild-daily - reword --check - reword --update * More details on upgrading * Moved start time to just before applying the patch * Accumulator is now initialized with override values from weewx.conf * Added sentences about wee_import/interval and weight patching multiple dbs * remove extensions from rpm and deb packages * v3.7.0b1 * Cleaned up some HTML warnings. * must do a try loop at the read level so we can skip the no data 'errors' and return empty buffer so that a subsequent write will get the station to talk again * Slightly more robust mechanism for decoding last time a file was FTP'd. * adjust wording of weighting description * added examples * simplify * fixed typo * fixed typo * read /etc/default before bailing out * make init script work properly with /etc/default/weewx * make output consistent * more simplification * keep PEP happy * avoid resource consumption from slow reports by extending the StdReport.max_wait. provide log messages when it happens. * new features have move to the roadmap * no more todo items left * do not emit default sensor_map to the config * aborted attempt to get additional battery status * bump to 3.7.0b2 * do recipe using wget instead of curl * forgot the O option to wget * do not warn when calibrations are ignored - the implementation resulted in too many log entries * get rid of tabs * added notes about wmr300 rain counter and logger * added norwegian 'no data available' as 'Ingen data er tilgjengelige' * simplify. eliminate more passive voice and gerunds. * avoid run-on * provide better feedback for operational errors. make manager logging more consistent. * fix typo in wmr300 ConfEditor * decode heatindex and windchill from wmr300 sensor outputs * fixed bad extract_signed invocation * fixed wmr* partial packets note * added mysqldb install instructions to userguide * minor html fixes * added link to wee_extension * clarify acurite sensor transmission periods. * added battery status for all wmr100 remote t/h sensors * added battery status for all wmr100 remote t sensors * document changes to wmr100, wmr200, wmr9x8 drivers. fix 'Calculatios' typo. * fixed inverted wmr200 battery status * rename fault_out to out_fault to match pattern of other faults * make battery status labels consistent across all wmr drivers * wmr300 driver moves from rc to 0.18 * bump to 3.7.0b3 * css fixes: neutralize the glaring yellow; brighten the note green to more closely match the tone of warning red; @media tweaks to match font changes. * fix column title * distinguish selection color from code color * fixed shift bug in weewx-multi * Fixed (I think) issue #219 * update logwatch script to properly handle revised generator log messages refer commit03c3e4ef57 (diff-3cefdd7265f340e9683b0a2d0417b70f)* normalize the quick-start * Merge branch 'development', remote branch 'origin' * fix layout table width on installation pages * v3.7.0b4 * use released_versions instead of previous_versions * Merge branch 'development', remote branch 'origin' * parameterize release rule. make release rule idempotent. * replace cheetahtemplate.org with pythonhosted.org * wee_database --help output was slightly different to reality * Removed BOM at beginning of customizing.htm * wee_database --help output was slightly different to reality * Reworded comment on whether to --update daily summaries. * cater for change in manager log output * remove misleading windGustDir info * bump to 3.7.0 * Added date to change log * escape the dollars in release target * fixed log syntax * adjust log level for wmr100 bad usb report * emit rapidfire cache info only when debug >= 3 * enable post_interval overrides for WOW uploader * Fixes issue #230, exception when using Rapidfire with metric units * Added StdRESTbase back in. It seems that some uploaders still depend on it. * Fixed problem that prevented a MySQL port from being specified. * Added antialias GIF to list of files to be installed. * Make sure GIF files get uploaded * distribute examples in a single directory * distinguish docs/examples vs examples * Fixed bug that prevented a port from being specified for MySQL installations. * Removed redundant change log entry * Add MySQL Error 2003 to exceptions (#234) * Added PR #234 to change log * Documented change in location of the examples * update examples paths in remaining guides. explicitly list all path changes for examples. * By default, autocommit is now enabled for the MySQL driver. Fixes issue #237. Included regression test.
170 lines
7.3 KiB
Plaintext
170 lines
7.3 KiB
Plaintext
# EXAMPLE CONFIGURATION FILE FOR IMPORTING FROM CSV FILES
|
|
#
|
|
# Copyright (c) 2009-2016 Tom Keffer <tkeffer@gmail.com> and Gary Roderick.
|
|
# See the file LICENSE.txt for your rights.
|
|
|
|
##############################################################################
|
|
|
|
# Specify the source. Available options are:
|
|
# CSV - import obs from a single CSV format file
|
|
# WU - import obs from a Weather Underground PWS history
|
|
# Cumulus - import obs from a one or more Cumulus monthly log files
|
|
# Format is:
|
|
# source = (CSV | WU | Cumulus)
|
|
source = CSV
|
|
|
|
##############################################################################
|
|
|
|
[CSV]
|
|
# Parameters used when importing from a CSV file
|
|
|
|
# Path and name of our CSV source file. Format is:
|
|
# file = full path and filename
|
|
file = /var/tmp/data.csv
|
|
|
|
# If there is no mapped interval field how will the interval field be
|
|
# determined for the imported records. Available options are:
|
|
# derive - Derive the interval field from the timestamp of successive
|
|
# records. This setting is best used when the imported records
|
|
# are equally spaced in time and there are no missing records.
|
|
# conf - Use the interval setting from weewx.conf. This setting is
|
|
# best used if the records to be imported have been produced by
|
|
# weeWX using the same archive interval as set in weewx.conf on
|
|
# this machine.
|
|
# x - Use a fixed interval of x minutes for every record. This
|
|
# setting is best used if the records to be imported are
|
|
# equally based in time but there are some missing records.
|
|
#
|
|
# Note: If there is a mapped interval field then this setting will be
|
|
# ignored.
|
|
# Format is:
|
|
# interval = (derive | conf | x)
|
|
interval = derive
|
|
|
|
# Should the [StdQC] max/min limits in weewx.conf be applied to the
|
|
# imported data. This may be useful if the source has extreme values that
|
|
# are clearly incorrect for some observations. Available options are:
|
|
# True - weewx.conf [StdQC] max/min limits are applied.
|
|
# False - weewx.conf [StdQC] max/min limits are not applied.
|
|
# Format is:
|
|
# qc = (True | False)
|
|
qc = True
|
|
|
|
# Should any missing derived observations be calculated from the imported
|
|
# data if possible. Available options are:
|
|
# True - Any missing derived observations are calculated.
|
|
# False - Any missing derived observations are not calculated.
|
|
# Format is:
|
|
# calc_missing = (True | False)
|
|
calc_missing = True
|
|
|
|
# Imported records are written to archive in transactions of tranche
|
|
# records at a time. Increase for faster throughput, decrease to reduce
|
|
# memory requirements. Format is:
|
|
# tranche = x
|
|
# where x is an integer
|
|
tranche = 250
|
|
|
|
# Specify whether a UV sensor was used to produce any UV observations.
|
|
# Available options are:
|
|
# True - UV sensor was used and UV data will be imported.
|
|
# False - UV sensor was not used and any UV data will not be imported.
|
|
# UV fields will be set to None/NULL.
|
|
# For a CSV import UV_sensor should be set to False if a UV sensor was
|
|
# NOT present when the import data was created. Otherwise it may be set to
|
|
# True or omitted. Format is:
|
|
# UV_sensor = (True | False)
|
|
UV_sensor = True
|
|
|
|
# Specify whether a solar radiation sensor was used to produce any solar
|
|
# radiation observations. Available options are:
|
|
# True - Solar radiation sensor was used and solar radiation data will
|
|
# be imported.
|
|
# False - Solar radiation sensor was not used and any solar radiation
|
|
# data will not be imported. radiation fields will be set to
|
|
# None/NULL.
|
|
# For a CSV import solar_sensor should be set to False if a solar radiation
|
|
# sensor was NOT present when the import data was created. Otherwise it may
|
|
# be set to True or omitted. Format is:
|
|
# solar_sensor = (True | False)
|
|
solar_sensor = True
|
|
|
|
# Date-time format of CSV field from which the weeWX archive record
|
|
# dateTime field is to be extracted. wee_import first attempts to interpret
|
|
# date/time info in this format, if this fails it then attempts to
|
|
# interpret it as a timestamp and if this fails it then raises an error.
|
|
# Uses Python strptime() format codes.
|
|
# raw_datetime_format = Python strptime() format string
|
|
raw_datetime_format = %Y-%m-%d %H:%M:%S
|
|
|
|
# Does the imported rain field represent the total rainfall since the last
|
|
# record or a cumulative value. Available options are:
|
|
# discrete - rain field represents total rainfall since last record
|
|
# cumulative - rain field represents a cumulative rainfall reset at
|
|
# midnight
|
|
# rain = (discrete | cumulative)
|
|
rain = cumulative
|
|
|
|
# Lower and upper bounds for imported wind direction. It is possible,
|
|
# particularly for a calculated direction, to have a value (eg -45) outside
|
|
# of the weeWX limits (0 to 360 inclusive). Format is:
|
|
#
|
|
# wind_direction = lower,upper
|
|
#
|
|
# where :
|
|
# lower is the lower limit of acceptable wind direction in degrees
|
|
# (may be negative)
|
|
# upper is the upper limit of acceptable wind direction in degrees
|
|
#
|
|
# Imported values from lower to upper will be normalised to the range 0 to
|
|
# 360. Values outside of the parameter range will be stored as None.
|
|
# Default is -360,360.
|
|
wind_direction = -360,360
|
|
|
|
# Map CSV record fields to weeWX archive fields. Format is:
|
|
#
|
|
# weewx_archive_field_name = csv_field_name, weewx_unit_name
|
|
#
|
|
# where:
|
|
# weewx_archive_field_name - An observation name in the weeWX database
|
|
# schema.
|
|
# csv_field_name - The name of a field from the CSV file.
|
|
# weewx_unit_name - The name of the units, as defined in weeWX,
|
|
# used by csv_field_name. wee_import will do
|
|
# the necessary conversions to the unit system
|
|
# used by the weeWX archive.
|
|
# For example,
|
|
# outTemp = Temp, degree_C
|
|
# would map the CSV field Temp, in degrees C, to the archive field outTemp.
|
|
#
|
|
# If a field mapping exists for the weeWX usUnits archive field then the
|
|
# units option may be omitted for each mapped field.
|
|
#
|
|
# weeWX archive fields that do not exist in the CSV data may be omitted.
|
|
# Any omitted fields that are derived (eg dewpoint) may be calculated
|
|
# during import using the equivalent of the weeWX StdWXCalculate service
|
|
# through setting the calc-missing parameter above.
|
|
[[FieldMap]]
|
|
dateTime = timestamp, unix_epoch
|
|
usUnits =
|
|
interval =
|
|
barometer = barometer, inHg
|
|
pressure =
|
|
altimeter =
|
|
inTemp =
|
|
outTemp = Temp, degree_F
|
|
inHumidity =
|
|
outHumidity = humidity, percent
|
|
windSpeed = windspeed, mile_per_hour
|
|
windDir = wind, degree_compass
|
|
windGust = gust, mile_per_hour
|
|
windGustDir = gustDir, degree_compass
|
|
rainRate = rate, inch_per_hour
|
|
rain = dayrain, inch
|
|
dewpoint =
|
|
windchill =
|
|
heatindex =
|
|
ET =
|
|
radiation =
|
|
UV =
|