diff --git a/CHANGES.txt b/CHANGES.txt index 16225f22..692a1f41 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -4,8 +4,10 @@ CHANGE HISTORY 1.4.0 02/21/10 Changed the architecture of stats.py to one that uses very late binding. The -SQL statements are not run until template evaluation. This greatly reduces -the amount of memory required, as well as greatly simplifying the code. +SQL statements are not run until template evaluation. This reduces +the amount of memory required (by about 10MB), reduces memory fragmentation, +as well as greatly simplifying the code (file stats.py shed 100 lines of code). +Execution time is approximately the same. Added option week_start. Default is 6 (Sunday). diff --git a/docs/readme.htm b/docs/readme.htm index 5d87d789..18c04daa 100644 --- a/docs/readme.htm +++ b/docs/readme.htm @@ -1,12 +1,12 @@ - - - - - - - - -The Weewx weather system + + + + + + + + +The Weewx weather system - - - - -

The weewx weather system
-Version 1.4

-

Table of Contents

-
    -
  1. -

    Copyright

    -
  2. -
  3. -

    About weewx

    -
  4. -
  5. -

    Downloading - weewx

    -
  6. -
  7. -

    Prerequisites

    -
  8. -
  9. -

    Installing - weewx

    -
  10. -
  11. -

    Configuring - weewx

    -
  12. -
  13. -

    Running weewx

    -
  14. -
  15. -

    Compatibility with - wview

    -
  16. -
  17. -

    HTML Generation

    -
  18. -
  19. -

    Monitoring - weewx

    -
  20. -
  21. -

    Architectural Notes

    -
  22. -
  23. -

    Customizing - weewx

    -
  24. -
-

1. Copyright

-

(c) 2009 by Tom Keffer <tkeffer@gmail.com>

-

This program is free software: you can redistribute it and/or modify it under -the terms of the GNU General Public License as published by the Free Software Foundation, -either version 3 of the License, or (at your option) any later version.

-

This program is distributed in the hope that it will be useful, but WITHOUT ANY -WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A -PARTICULAR PURPOSE. See the GNU General Public License for more details.

-

You should have received a copy of the GNU General Public License along with -this program. If not, see -http://www.gnu.org/licenses.

-

2. About weewx

-

I wrote weewx over the winter of 2008-2009 for two -reasons: it was a wet and miserable winter here in Oregon with not much else to -do, so there was no good reason not to, and because I wanted a simple, easy-to-understand -server to run my Davis VantagePro2 weather station on a Linux box. I had been using -wview, which is a very high-performance, -stable, and feature rich system authored by Mark Teel with lots of users. Written -in C, it's an efficient system that can run on underpowered boxes. -In exchange, it's huge (45,000+ lines of code), tightly integrated in with its companion -library, radlib (another 14,000+ lines), and complex. But, if you're looking to -be in good company and you want to run on inexpensive, featherweight machines such -as the Linksys NSLU2, you can't -beat it.

-

Having made a career in C++ and Java, I was also interested in some more modern -languages, so I thought I'd try either Python or Ruby (although, truth be told, -the roots of Python are nearly as old as C++!). I picked Python because its libraries -are more mature and there are many more choices for third party libraries.

-

Weewx weighs in at under 7,000 lines of code, of which 2,500 are -comments. However, to be fair, it is missing features such as support for other weather -stations and for metric units. On the other hand, it offers very -powerful configuration, templating, and customization options, making it very easy to customize or -to add these things. It is also architecturally very simple and easy to understand.

-

3. Downloading weewx

-

weewx can be downloaded from its -SourceForge page: -https://sourceforge.net/projects/weewx. -

-

4. Prerequisites

-

4.1 Python

-

Python V2.5 or V2.6 is required. The newer V3.0 distribution will not work.

-

4.2 Required packages

-

The following external packages are required to use weewx.

-
    -
  1. sqlite3 (Version 3.5 or greater) A - SQL database written in C, which weewx uses to store - data pulled from the weather station. Comes with Debian and many other Linux - distributions.
  2. -
  3. pysqlite (Version 2.5 - or greater) The Python interface to sqlite3.
  4. -
  5. configobj (Version - 4.5 or greater) Manages the configuration file weewx.conf.
  6. -
  7. pyserial (Version - 1.35 or greater) Manages the serial connection to the weather station.
  8. -
  9. Cheetah (Version 2.0 or greater) - The HTML templating engine.
  10. -
  11. Python Imaging Library (Version - 1.1.6 or greater) Also known as PIL, this is included in many Python distributions.
  12. -
-

There are two general strategies for installing these prerequisites:

-
    -
  1. Use operating system tools, such as apt-get (or - its graphical equivalent Synaptic Package Manager) for Debian/Ubuntu or - yast for SuSE; or
  2. -
  3. Use the Python tool - easy_install.
  4. -
-

Option #1 is easier, but if your Linux distribution does not come with such tools, -you may have to use easy_install. Brief instructions for -both approaches are given below.

-

Installation on Debian distributions (including Ubuntu) using -apt-get

-

The instructions that follow are for using the Debian tool -apt_get, but the same package names would be used should you chose to use -a graphical interface such as the Synaptic Package Manager.

-

sqlite3

-

My Ubuntu 8.10 system came with V3.5.9, which works just fine. However, if you -need to install:

-

sudo apt-get install sqlite3

-

pysqlite

-

Easily installed:

-

sudo apt-get install python-pysqlite2

-

configobj

-

Easily installed:

-

sudo apt-get install python-configobj

-

pyserial

-

Easily installed:

-

sudo apt-get install python-serial

-

Cheetah

-

Easily installed:

-

sudo apt-get install python-cheetah

-

Python Imaging Library (PIL)

-

My version of Python came with V1.1.6, which works great. Nothing needs to be -done.

-

Installation on SuSE using yast

-

My SuSE 11.1 system came with some of the prerequisites installed, some available -through yast, and three that required -easy_install. To start, you will have to install the gcc -compiler:

-

sudo yast -i gcc

-

Then install easy_install:

-

sudo yast -i python-setuptools

-

On my system, some scripts wanted to install themselves into -/usr/local/lib/python2.6/site-packages, which didn't exist. -If this is the case, you may have to create these directories before running -easy_install:

-

sudo mkdir /usr/local/lib/python2.6

-

sudo mkdir /usr/local/lib/python2.6/site-packages

-

sqlite3

-

My SuSE 11.1 system came with V3.6.4, which works just fine. However, if you -need to install:

-

sudo yast -i sqlite3

-

pysqlite

-

Install using easy_install. See -comments below about installing pysqlite -using easy_install. On my SuSE 11.1 system, I had to install -the gcc compiler and the sqlite3 development environment first:

-

sudo yast -i gcc

-

sudo yast -i sqlite-devel

-

Then I was able to install pysqlite using easy_install. -However, because the hosting site for pysqlite had changed recently, I had to give -the URL explicitly:

-

sudo easy_install -http://pysqlite.googlecode.com/files/pysqlite-2.5.5.tar.gz

-

configobj

-

Install using easy_install:

-

sudo easy_install configobj

-

pyserial

-

Install using yast:

-

sudo yast-i python-serial

-

Cheetah

-

Install using easy_install (My system emitted a bunch -of, apparently, benign warnings):

-

sudo easy_install Cheetah

-

Python Imaging Library (PIL)

-

Install using yast

-

sudo yast -i python-imaging

-

Installation using easy_install

-

An alternative approach to installing the required packages is by using the Python -setup tool "easy_install", part of the -python-setuptools package. -Refer to their instructions on how to install this tool.

-

Once installed, installing the rest of the packages is very easy.

-

sqlite3

-

My Ubuntu 8.10 system came with V3.5.9, which works just fine. If you do not -have sqlite3, refer to the sqlite webpage for -installation instructions.

-

pysqlite

-

While Version 2.3.X of pysqlite -is included with many versions of Python, the more recent 2.5.X or greater is required -in order to take advantage of transaction contexts. Hence, you may have to install -or upgrade. Because pysqlite builds a C library, you may have to install the Python -development environment first, if you have not already done so. Generally, this -means installing the gcc compiler. You may also have to install the sqlite3 development -environment as well.

-

With the development environment in place, you can easily build and install pysqlite:

-

easy_install pysqlite

-

If your system already has a version of pysqlite installed, but it is not a high -enough version (easy_install will tell you the version -you have), then you may have to force an upgrade:

-

easy_install --upgrade pysqlite

-

Note that at the time of this writing (24 Oct 2009), the hosting site for pysqlite -has changed to one on googlecode, and -easy_install could not find it. You may have to find and -give the URL explicitly to easy_install (adjust version -numbers as necessary):

-

easy_install http://pysqlite.googlecode.com/files/pysqlite-2.5.5.tar.gz

-

configobj

-

Easily installed:

-

easy_install configobj

-

pyserial

-

easy_install pyserial

-

Cheetah

-

easy_install Cheetah

-

Python Imaging Library (PIL)

-

My version of Python came with V1.1.6, which works great.

-

4.3 System requirements

-

I run weewx on a 500MHz system with an AMD Geode processor -and 512 MB of memory.  Configured this way, it consumes about 5% of the CPU -and about 75MB of total memory.

-

4.4 Weather station hardware requirements

-

At this point, only the -Davis VantagePro2 -is supported, and even then, only the "Revision B" version (firmware dated on or -after 22 April 2002). It would be very easy to port to a "Revision A" station or -even the original VantagePro, but I don't have access to the hardware to test it.

-

5. Installing weewx

-

Because weewx is "Pure Python", that is it is 100% -Python with no "C" modules to compile, installing it is very easy. Furthermore, -it uses the standard Python -distutils install method, which is very easy and flexible. Detailed instructions -follow.

-

5.1 Unpacking

-

Start by unpack the tar ball (substitute your version for X.Y.Z) into any convenient -directory where you have write permission

-

tar xvf weewx-X.Y.Z

-

Then change directory into it:

-

cd weewx-X.Y.Z

-

5.1 Choosing the WEEWX_ROOT directory

-

Next step is figuring out where you want to install weewx. -If WEEWX_ROOT symbolizes the root location of -the weewx directory hierarchy, then

- -

By default, the location for WEEWX_ROOT is -/home/weewx. However, it can be changed by editing the -file setup.cfg. If you wish to install someplace else, -open up setup.cfg and change the line

-

home = /home/weewx

-

to reflect your decision.

-

5.2 Build and install

-

Build the distribution

-

./setup.py build

-

(Because weewx is pure Python this doesn't actually -build anything, but it does arrange files for the final installation)

-

Then install it. If you have write permission in the directory where -weewx will go (i.e., $WEEWX_ROOT), -then type

-

./setup.py install

-

Otherwise, if you do not have write permission, you will have to use sudo:

-

sudo ./setup.py install

-

Upgrading

-

Before upgrading from a previous version of weewx, -check the upgrade notes, found in file upgrading.htm, to see if there any -specific actions you need to do.

-

You then generally follow the procedure above.

-

In particular, before starting, be sure to set -home in the file -setup.cfg.

-

The build and install process will do the following for you.

- -

5.3 Permissions

-

Strictly speaking it is not necessary to install or run weewx -with root privileges. You only need read/write access to the serial port for your -hardware. For example, if your hardware has a USB interface, on Ubuntu and SuSE:

-

sudo chmod a+rw /dev/ttyUSB0

-

Once done, if you edit setup.cfg to install into a -directory where you have write permissions, you can install and run -weewx without any root privileges at all.

-

5.4 Final note on installation

-

Because weewx is pure Python, it actually does not -have to be "built" and "installed" at all! You can just simply run it out of whatever -directory you unpack it into (after, of course, editing weewx.conf -to reflect your local environment). I do this all the time when testing. However, -the setup.py script does include special provisions for -updating your configuration file weewx.conf, which can -be handy when upgrading to a later version.

-

6. Configuring weewx

-

This section covers configuring your archive and statistical database (if necessary; -this step is required only if you are moving from -wview to weewx), configuring your weather -station, and configuring the configuration file weewx.conf.

-

In the following, $WEEWX_ROOT refers to the -weewx root directory, generally /home/weewx. -

-

6.1 Configuring the databases

-

NOTE: This section is necessary only if you are moving from -wview to weewx -and wish to transfer your old data over. If you are starting afresh, you do not -need to follow this section — the two main databases are created and populated automatically -by weewx.

-

If you wish to transfer your wview data, note that two databases are maintained by weewx:

- -

Because wview and weewx use identical schema for the first of these (the archive -database), it can be just copied over. However, the second (the statistical databases) -is different — the weewx statistical database must be built manually and backfilled. -This is done using the configuration script configure.py. -

-

Here's a summary of how to transfer your wview data to weewx.

-

mkdir $WEEWX_ROOT/archive

-

cp /usr/local/var/wview/archive/wview-archive.sdb $WEEWX_ROOT/archive/weewx.sdb

-

$WEEWX_ROOT/bin/configure.py --create-stats $WEEWX_ROOT/weewx.conf

-

$WEEWX_ROOT/bin/configure.py --backfill-stats $WEEWX_ROOT/weewx.conf

-

If your existing database is large, backfilling could take some time. On my modest -500 MHz fit-PC -with 512 MB of memory it took a little over 4 minutes for a year and a half (25 -MB) of data (while wview was running in the background).

-

6.2 Configuring your weather station

-

The only two variables weewx tries to manage on the -VantagePro are the time and the archive interval.

-

Time

-

The time on the VP is automatically synchronized with the -weewx server nominally every four hours (changeable by the user). You should run a -NTP daemon on your -server to insure that it is synchronized with the correct time. Doing so will greatly -reduce errors, especially if you send data to services such as the Weather Underground.

-

Archive interval

-

The archive interval is set in the main configuration file -$WEEWX_ROOT/weewx.conf. Look for the entry archive_interval -in the VantagePro section. Set it to the number of seconds. -Valid entries are 60, 300, 600, 900, 1800, 3600, and 7200. However, if you are ftp'ing -lots of files to a server, setting it to 60 seconds may not give enough time to -have them all uploaded before the next archive record is due. If this is the case, -you should pick an archive interval of at least 300 seconds, or trim the number -of files you are using.

-

After setting to the desired interval, run the configure.py -script to set it on the VantagePro. If it differs from the old archive interval, -the main memory log of the VantagePro will be cleared.

-

$WEEWX_ROOT/bin/configure.py --configure-VantagePro $WEEWX_ROOT/weewx.conf

-

6.3 Editing the configuration file weewx.conf

-

Virtually every conceivable configuration option is in the configuration file -$WEEWX_ROOT/weewx.conf. Most of the important -ones are up near the top of the file. They are all documented in this section, although -you can safely ignore most of them. The truly important ones, the ones you are likely -to have to customize for your station, are shown in -bold face and in blue.

-

Default values are provided for many of them, meaning that if they are not listed -in the configuration file at all, weewx will -pick sensible values. When the documentation below gives a "default value" this -is what it means. However, all options have been given values in the configuration -file that ships with weewx, so you can see what they look -like. The value given in this shipped configuration file is not necessarily the -same as the "default value".

-

What follows is organized by the different sections of the configuration file.

-

General

-

The options declared at the top are not actually part of any section. There are -two:

-

debug

-

Set to 1 to have the program perform extra debug checks, as well as emit extra -information on the log file. Otherwise, set to 0. Default is 0 (no debug).

-

socket_timeout

-

Set to how long to wait before declaring a socket time out. This is used when -FTP'ing data to a web server or sending data to the Weather Underground. Twenty -(20) seconds is reasonable. Default is 20.

-

[Station]

-

This section covers options relating to the entire weather station setup.

-

WEEWX_ROOT

-

Set to the root directory of the weewx file hierarchy -for this station, nominally '/home/weewx'. This value -will be set automatically by the setup script setup.py -to reflect the choice you made in the configuration file setup.cfg. -Required. No default.

-

location

-

The station location should be a string that describes the geography of where -you weather station is located, such as 'Hood River, Oregon'. Required. No default.

-

latitude
-longitude

-

The lat/lon should be set in decimal degrees, negative for southern and eastern -hemispheres, respectively. Required. No default.

-

altitude

-

Should be set to the altitude of the station. In this version, the only unit -accepted is feet. Required. No default.

-

rain_year_start

-

If your area uses a rain year that starts on something other than the first of -January, you may want to set this variable. For example, set to 10 if your rain -year starts in October (as mine does). Default is 1.

-

radar_url

-

This variable is available in the HTML templates. Set it to an appropriate URL -to display a radar image for your area. No default.

+.style1 { + background-color: #FFFFCC; +} + + + + + +

The weewx weather system
+Version 1.4

+

Table of Contents

+
    +
  1. +

    Copyright

    +
  2. +
  3. +

    About weewx

    +
  4. +
  5. +

    Downloading + weewx

    +
  6. +
  7. +

    Prerequisites

    +
  8. +
  9. +

    Installing + weewx

    +
  10. +
  11. +

    Configuring + weewx

    +
  12. +
  13. +

    Running weewx

    +
  14. +
  15. +

    Compatibility with + wview

    +
  16. +
  17. +

    HTML Generation

    +
  18. +
  19. +

    Monitoring + weewx

    +
  20. +
  21. +

    Architectural Notes

    +
  22. +
  23. +

    Customizing + weewx

    +
  24. +
+

1. Copyright

+

(c) 2009 by Tom Keffer <tkeffer@gmail.com>

+

This program is free software: you can redistribute it and/or modify it under +the terms of the GNU General Public License as published by the Free Software Foundation, +either version 3 of the License, or (at your option) any later version.

+

This program is distributed in the hope that it will be useful, but WITHOUT ANY +WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A +PARTICULAR PURPOSE. See the GNU General Public License for more details.

+

You should have received a copy of the GNU General Public License along with +this program. If not, see +http://www.gnu.org/licenses.

+

2. About weewx

+

I wrote weewx over the winter of 2008-2009 for two +reasons: it was a wet and miserable winter here in Oregon with not much else to +do, so there was no good reason not to, and because I wanted a simple, easy-to-understand +server to run my Davis VantagePro2 weather station on a Linux box. I had been using +wview, which is a very high-performance, +stable, and feature rich system authored by Mark Teel with lots of users. Written +in C, it's an efficient system that can run on underpowered boxes. +In exchange, it's huge (45,000+ lines of code), tightly integrated in with its companion +library, radlib (another 14,000+ lines), and complex. But, if you're looking to +be in good company and you want to run on inexpensive, featherweight machines such +as the Linksys NSLU2, you can't +beat it.

+

Having made a career in C++ and Java, I was also interested in some more modern +languages, so I thought I'd try either Python or Ruby (although, truth be told, +the roots of Python are nearly as old as C++!). I picked Python because its libraries +are more mature and there are many more choices for third party libraries.

+

Weewx weighs in at under 7,000 lines of code, of which 2,500 are +comments. However, to be fair, it is missing features such as support for other weather +stations and for metric units. On the other hand, it offers very +powerful configuration, templating, and customization options, making it very easy to customize or +to add these things. It is also architecturally very simple and easy to understand.

+

3. Downloading weewx

+

weewx can be downloaded from its +SourceForge page: +https://sourceforge.net/projects/weewx. +

+

4. Prerequisites

+

4.1 Python

+

Python V2.5 or V2.6 is required. The newer V3.0 distribution will not work.

+

4.2 Required packages

+

The following external packages are required to use weewx.

+
    +
  1. sqlite3 (Version 3.5 or greater) A + SQL database written in C, which weewx uses to store + data pulled from the weather station. Comes with Debian and many other Linux + distributions.
  2. +
  3. pysqlite (Version 2.5 + or greater) The Python interface to sqlite3.
  4. +
  5. configobj (Version + 4.5 or greater) Manages the configuration file weewx.conf.
  6. +
  7. pyserial (Version + 1.35 or greater) Manages the serial connection to the weather station.
  8. +
  9. Cheetah (Version 2.0 or greater) + The HTML templating engine.
  10. +
  11. Python Imaging Library (Version + 1.1.6 or greater) Also known as PIL, this is included in many Python distributions.
  12. +
+

There are two general strategies for installing these prerequisites:

+
    +
  1. Use operating system tools, such as apt-get (or + its graphical equivalent Synaptic Package Manager) for Debian/Ubuntu or + yast for SuSE; or
  2. +
  3. Use the Python tool + easy_install.
  4. +
+

Option #1 is easier, but if your Linux distribution does not come with such tools, +you may have to use easy_install. Brief instructions for +both approaches are given below.

+

Installation on Debian distributions (including Ubuntu) using +apt-get

+

The instructions that follow are for using the Debian tool +apt_get, but the same package names would be used should you chose to use +a graphical interface such as the Synaptic Package Manager.

+

sqlite3

+

My Ubuntu 8.10 system came with V3.5.9, which works just fine. However, if you +need to install:

+

sudo apt-get install sqlite3

+

pysqlite

+

Easily installed:

+

sudo apt-get install python-pysqlite2

+

configobj

+

Easily installed:

+

sudo apt-get install python-configobj

+

pyserial

+

Easily installed:

+

sudo apt-get install python-serial

+

Cheetah

+

Easily installed:

+

sudo apt-get install python-cheetah

+

Python Imaging Library (PIL)

+

My version of Python came with V1.1.6, which works great. Nothing needs to be +done.

+

Installation on SuSE using yast

+

My SuSE 11.1 system came with some of the prerequisites installed, some available +through yast, and three that required +easy_install. To start, you will have to install the gcc +compiler:

+

sudo yast -i gcc

+

Then install easy_install:

+

sudo yast -i python-setuptools

+

On my system, some scripts wanted to install themselves into +/usr/local/lib/python2.6/site-packages, which didn't exist. +If this is the case, you may have to create these directories before running +easy_install:

+

sudo mkdir /usr/local/lib/python2.6

+

sudo mkdir /usr/local/lib/python2.6/site-packages

+

sqlite3

+

My SuSE 11.1 system came with V3.6.4, which works just fine. However, if you +need to install:

+

sudo yast -i sqlite3

+

pysqlite

+

Install using easy_install. See +comments below about installing pysqlite +using easy_install. On my SuSE 11.1 system, I had to install +the gcc compiler and the sqlite3 development environment first:

+

sudo yast -i gcc

+

sudo yast -i sqlite-devel

+

Then I was able to install pysqlite using easy_install. +However, because the hosting site for pysqlite had changed recently, I had to give +the URL explicitly:

+

sudo easy_install +http://pysqlite.googlecode.com/files/pysqlite-2.5.5.tar.gz

+

configobj

+

Install using easy_install:

+

sudo easy_install configobj

+

pyserial

+

Install using yast:

+

sudo yast-i python-serial

+

Cheetah

+

Install using easy_install (My system emitted a bunch +of, apparently, benign warnings):

+

sudo easy_install Cheetah

+

Python Imaging Library (PIL)

+

Install using yast

+

sudo yast -i python-imaging

+

Installation using easy_install

+

An alternative approach to installing the required packages is by using the Python +setup tool "easy_install", part of the +python-setuptools package. +Refer to their instructions on how to install this tool.

+

Once installed, installing the rest of the packages is very easy.

+

sqlite3

+

My Ubuntu 8.10 system came with V3.5.9, which works just fine. If you do not +have sqlite3, refer to the sqlite webpage for +installation instructions.

+

pysqlite

+

While Version 2.3.X of pysqlite +is included with many versions of Python, the more recent 2.5.X or greater is required +in order to take advantage of transaction contexts. Hence, you may have to install +or upgrade. Because pysqlite builds a C library, you may have to install the Python +development environment first, if you have not already done so. Generally, this +means installing the gcc compiler. You may also have to install the sqlite3 development +environment as well.

+

With the development environment in place, you can easily build and install pysqlite:

+

easy_install pysqlite

+

If your system already has a version of pysqlite installed, but it is not a high +enough version (easy_install will tell you the version +you have), then you may have to force an upgrade:

+

easy_install --upgrade pysqlite

+

Note that at the time of this writing (24 Oct 2009), the hosting site for pysqlite +has changed to one on googlecode, and +easy_install could not find it. You may have to find and +give the URL explicitly to easy_install (adjust version +numbers as necessary):

+

easy_install http://pysqlite.googlecode.com/files/pysqlite-2.5.5.tar.gz

+

configobj

+

Easily installed:

+

easy_install configobj

+

pyserial

+

easy_install pyserial

+

Cheetah

+

easy_install Cheetah

+

Python Imaging Library (PIL)

+

My version of Python came with V1.1.6, which works great.

+

4.3 System requirements

+

I run weewx on a 500MHz system with an AMD Geode processor +and 512 MB of memory.  Configured this way, it consumes about 5% of the CPU +and about 75MB of total memory.

+

4.4 Weather station hardware requirements

+

At this point, only the +Davis VantagePro2 +is supported, and even then, only the "Revision B" version (firmware dated on or +after 22 April 2002). It would be very easy to port to a "Revision A" station or +even the original VantagePro, but I don't have access to the hardware to test it.

+

5. Installing weewx

+

Because weewx is "Pure Python", that is it is 100% +Python with no "C" modules to compile, installing it is very easy. Furthermore, +it uses the standard Python +distutils install method, which is very easy and flexible. Detailed instructions +follow.

+

5.1 Unpacking

+

Start by unpack the tar ball (substitute your version for X.Y.Z) into any convenient +directory where you have write permission

+

tar xvf weewx-X.Y.Z

+

Then change directory into it:

+

cd weewx-X.Y.Z

+

5.1 Choosing the WEEWX_ROOT directory

+

Next step is figuring out where you want to install weewx. +If WEEWX_ROOT symbolizes the root location of +the weewx directory hierarchy, then

+ +

By default, the location for WEEWX_ROOT is +/home/weewx. However, it can be changed by editing the +file setup.cfg. If you wish to install someplace else, +open up setup.cfg and change the line

+

home = /home/weewx

+

to reflect your decision.

+

5.2 Build and install

+

Build the distribution

+

./setup.py build

+

(Because weewx is pure Python this doesn't actually +build anything, but it does arrange files for the final installation)

+

Then install it. If you have write permission in the directory where +weewx will go (i.e., $WEEWX_ROOT), +then type

+

./setup.py install

+

Otherwise, if you do not have write permission, you will have to use sudo:

+

sudo ./setup.py install

+

Upgrading

+

Before upgrading from a previous version of weewx, +check the upgrade notes, found in file upgrading.htm, to see if there any +specific actions you need to do.

+

You then generally follow the procedure above.

+

In particular, before starting, be sure to set +home in the file +setup.cfg.

+

The build and install process will do the following for you.

+ +

5.3 Permissions

+

Strictly speaking it is not necessary to install or run weewx +with root privileges. You only need read/write access to the serial port for your +hardware. For example, if your hardware has a USB interface, on Ubuntu and SuSE:

+

sudo chmod a+rw /dev/ttyUSB0

+

Once done, if you edit setup.cfg to install into a +directory where you have write permissions, you can install and run +weewx without any root privileges at all.

+

5.4 Final note on installation

+

Because weewx is pure Python, it actually does not +have to be "built" and "installed" at all! You can just simply run it out of whatever +directory you unpack it into (after, of course, editing weewx.conf +to reflect your local environment). I do this all the time when testing. However, +the setup.py script does include special provisions for +updating your configuration file weewx.conf, which can +be handy when upgrading to a later version.

+

6. Configuring weewx

+

This section covers configuring your archive and statistical database (if necessary; +this step is required only if you are moving from +wview to weewx), configuring your weather +station, and configuring the configuration file weewx.conf.

+

In the following, $WEEWX_ROOT refers to the +weewx root directory, generally /home/weewx. +

+

6.1 Configuring the databases

+

NOTE: This section is necessary only if you are moving from +wview to weewx +and wish to transfer your old data over. If you are starting afresh, you do not +need to follow this section — the two main databases are created and populated automatically +by weewx.

+

If you wish to transfer your wview data, note that two databases are maintained by weewx:

+ +

Because wview and weewx use identical schema for the first of these (the archive +database), it can be just copied over. However, the second (the statistical databases) +is different — the weewx statistical database must be built manually and backfilled. +This is done using the configuration script configure.py. +

+

Here's a summary of how to transfer your wview data to weewx.

+

mkdir $WEEWX_ROOT/archive

+

cp /usr/local/var/wview/archive/wview-archive.sdb $WEEWX_ROOT/archive/weewx.sdb

+

$WEEWX_ROOT/bin/configure.py --create-stats $WEEWX_ROOT/weewx.conf

+

$WEEWX_ROOT/bin/configure.py --backfill-stats $WEEWX_ROOT/weewx.conf

+

If your existing database is large, backfilling could take some time. On my modest +500 MHz fit-PC +with 512 MB of memory it took a little over 4 minutes for a year and a half (25 +MB) of data (while wview was running in the background).

+

6.2 Configuring your weather station

+

The only two variables weewx tries to manage on the +VantagePro are the time and the archive interval.

+

Time

+

The time on the VP is automatically synchronized with the +weewx server nominally every four hours (changeable by the user). You should run a +NTP daemon on your +server to insure that it is synchronized with the correct time. Doing so will greatly +reduce errors, especially if you send data to services such as the Weather Underground.

+

Archive interval

+

The archive interval is set in the main configuration file +$WEEWX_ROOT/weewx.conf. Look for the entry archive_interval +in the VantagePro section. Set it to the number of seconds. +Valid entries are 60, 300, 600, 900, 1800, 3600, and 7200. However, if you are ftp'ing +lots of files to a server, setting it to 60 seconds may not give enough time to +have them all uploaded before the next archive record is due. If this is the case, +you should pick an archive interval of at least 300 seconds, or trim the number +of files you are using.

+

After setting to the desired interval, run the configure.py +script to set it on the VantagePro. If it differs from the old archive interval, +the main memory log of the VantagePro will be cleared.

+

$WEEWX_ROOT/bin/configure.py --configure-VantagePro $WEEWX_ROOT/weewx.conf

+

6.3 Editing the configuration file weewx.conf

+

Virtually every conceivable configuration option is in the configuration file +$WEEWX_ROOT/weewx.conf. Most of the important +ones are up near the top of the file. They are all documented in this section, although +you can safely ignore most of them. The truly important ones, the ones you are likely +to have to customize for your station, are shown in +bold face and in blue.

+

Default values are provided for many of them, meaning that if they are not listed +in the configuration file at all, weewx will +pick sensible values. When the documentation below gives a "default value" this +is what it means. However, all options have been given values in the configuration +file that ships with weewx, so you can see what they look +like. The value given in this shipped configuration file is not necessarily the +same as the "default value".

+

What follows is organized by the different sections of the configuration file.

+

General

+

The options declared at the top are not actually part of any section. There are +two:

+

debug

+

Set to 1 to have the program perform extra debug checks, as well as emit extra +information on the log file. Otherwise, set to 0. Default is 0 (no debug).

+

socket_timeout

+

Set to how long to wait before declaring a socket time out. This is used when +FTP'ing data to a web server or sending data to the Weather Underground. Twenty +(20) seconds is reasonable. Default is 20.

+

[Station]

+

This section covers options relating to the entire weather station setup.

+

WEEWX_ROOT

+

Set to the root directory of the weewx file hierarchy +for this station, nominally '/home/weewx'. This value +will be set automatically by the setup script setup.py +to reflect the choice you made in the configuration file setup.cfg. +Required. No default.

+

location

+

The station location should be a string that describes the geography of where +you weather station is located, such as 'Hood River, Oregon'. Required. No default.

+

latitude
+longitude

+

The lat/lon should be set in decimal degrees, negative for southern and eastern +hemispheres, respectively. Required. No default.

+

altitude

+

Should be set to the altitude of the station. In this version, the only unit +accepted is feet. Required. No default.

+

rain_year_start

+

If your area uses a rain year that starts on something other than the first of +January, you may want to set this variable. For example, set to 10 if your rain +year starts in October (as mine does). Default is 1.

+

radar_url

+

This variable is available in the HTML templates. Set it to an appropriate URL +to display a radar image for your area. No default.

week_start

Start of the week. 0=Monday, 1= Tuesday, ... , 6 = Sunday. Default is 6 (Sunday)

-

heating_base
-cooling_base

-

Set to the base temperature for calculating heating and cooling degree-days, -respectively. The default is 65.0 for both.

-

hemispheres

-

Set to suitable abbreviations for the four hemispheres. Default is "N", "S", -"E", "W"

-

clock_check

-

How often to check the station's onboard clock for drift, in seconds. Default -is 14400 (every 4 hours)

-

cache_loop_data

-

Set to 1 (one) to cache LOOP data, otherwise, set to zero. Most users will -not want to change this unless you have a specialized application. Default is 1 -(i.e., cache LOOP data).

-

station_type

-

Set to the type of hardware you are using. For this version, only 'VantagePro' -is accepted. Required.

-

[VantagePro]

-

This section is for options relating to the VantagePro hardware.

-

port

-

Set to the port name used by your station. Example, /dev/ttyUSB0 -is a common location for USB ports under Debian, /dev/ttyS0 -for serial ports. Required. No default.

-

baudrate

-

Set to the baudrate of your station. The default is 19200.

-

archive_interval

-

Set to the desired archive interval of your station, in seconds. This variable -is only used when setting up your station. Otherwise, this value is read directly -from the station. Required if you configure your station. -No default.

-

iss_id

-

Set to the ID number of your Integrated Sensor Suite (ISS). This is used in the -formula to calculate reception quality for wireless stations. The default is 1.

-

archive_delay

-

How long to wait in seconds after the top of an archiving interval before fetching -new data off the station. For example, if your archive interval is 5 minutes and -archive_delay is set to 15, then the data will be fetched at 00:00:15, 00:05:15, -00:10:15, etc. This delay is to give the station a few seconds to archive the data -internally, and in case your server has any other tasks to do at the top of the -minute. Default is 15 seconds.

-

timeout

-

How many seconds  to wait for a response from the station before giving -up. Default is 5 seconds.

-

wait_before_retry

-

How many seconds to wait before retrying again. Unless you have a good reason -to change it, this value should be left at the default, as it is long enough for -the station to offer new data, but not so long as to go into a new loop packet (which -arrive every 2 seconds). Default is 1.2 seconds.

-

max_retries

-

How many times to try again before giving up. Default is 4.

-

max_drift

-

The maximum amount of drift to tolerate, in seconds, in the VantagePro's onboard -clock, before resetting the clock. Default is 5.

-

unit_system

-

What unit system is in use on your weather station hardware. Possible values -are '1' (Imperial) or '2' (Metric). As far as I know, all Davis instruments support -only Imperial. In any case, Imperial is the only system supported by -weewx. Default is 1.

-

[FTP]

-

If you FTP your images and HTML files to an external web server, -weewx can FTP them for you. It does an incremental update, -that is, it only FTPs any files that have changed, saving the outgoing bandwidth -of your Internet connection.

-

If you do not use such a server, comment out the first four options below.

-

user

-

Set to the username you use for your FTP connection to your web server. Required. -No default.

-

password

-

Set to the password you use for your FTP connection to your web server. Required. -No default.

-

server

-

Set to the name of your web server (e.g., -www.threefools.org, in my case). Required. No default

-

path

-

Set to the path where the weather data will be stored on your webserver (e.g., -'/weather'). NB: some FTP servers require a -leading slash ('/'), some don't. Required. No default.

-

passive

-

Set to 1 if you wish to use the more modern, FTP passive mode, 0 if you wish -to use active mode. Passive mode generally works better through firewalls, but not -all FTP servers do a good job of supporting it. See -Active FTP vs. Passive FTP, a Definitive -Explanation for a good explanation of the difference. Default is 1 (passive -mode).

-

max_retries

-

Weewx will try up to this many times to FTP a file -up to your server before giving up. Default is 3.

-

[Wunderground]

-

Weewx can send your current data to the Weather -Underground. If you do not wish to do this, comment out the two options below.

-

station

-

Set to your Weather Underground station ID (e.g., KORHOODR3). -Required.

-

password

-

Set to your Weather Underground password. Required.

-

[Archive]

-

This section is for configuring the sqlite3 database on which the station archive -data is stored.

-

archive_file

-

The path, relative to the WEEWX_ROOT directory, to -the database. Required

-

unit_system

-

What unit system to use inside the database. Required. The only one supported -right now is '1', the Imperial (U.S.) system

-

[Stats]

-

This section is for configuring the sqlite3 database on which the station statistics -are stored.

-

stats_file

-

The path, relative to the WEEWX_ROOT directory to the -statistical database. Required.

-

stats_types

-

The list of types for which statistics will be kept. Types not listed will not -be available for generating HTML pages. Optional. The default is all types, resulting -in a possibly much bigger than necessary stats database (do you really have four -different soil moisture sensors?) The list that ships with the configuration file -will work for most stations and probably will not have to be modified.

-

[Images]

-

This section, which controls which images (plots) get generated and with which -options, is by far the most complicated. However, it is extremely flexible and powerful.

-

Time periods

-

The section consists of one or more sub-sections, one for each time period (day, week, -month, and year). These sub-sections define the nature of aggregation and plot types -for the time period. For example, here's a typical set of options for sub-section -[[month_images]], controlling how images that cover a -month period are generated:

-

[[month_images]]

-

  x_label_format = %d

-

  bottom_label_format = %m/%d/%y %H:%M

-

  time_length = 2592000 # == 30 days

-

  aggregate_type = avg

-

  aggregate_interval = 10800 # == 3 hours

-

The option x_label_format gives a -strftime() -type format for the x-axis. In this example, it will only show days (format option -"%d"). The bottom_label_format -is the format used to time stamp the image at the bottom. In this example, it will -show the time as 10/25/09 15:35. A plot will cover a nominal -30 days, and all items included in it will use an aggregate type of averaging over -3 hours.

-

Image files

-

Within each sub-section is another nesting, one for each image to be generated. -The title of each sub-sub-section is the filename to be used for the image. -Finally, at one additional nesting level (!) are the logical names of all the -line types to be drawn in the image.  Values specified in the -level above can be overridden. For example, here's a typical set of options for -sub-sub-section [[[monthrain]]]:

-

[[[monthrain]]]

-

  plot_type = bar

-

  yscale = None, None, 0.02

-

  [[[[rain]]]]

-

    aggregate_type = sum

-

    aggregate_interval = 86400

-

    label = Rain (daily avg)

-

This will generate an image file with name monthrain.png. -It will be a bar plot. Option yscale controls the y-axis -scaling — if left out, the scale will automatically be chosen. However, in this -example we are choosing to exercise some degree of control by specifying values -explicitly. It is a 3-way tuple (ylow, -yhigh, min_interval), where -ymin and ymax are the minimum -and maximum y-axis values, respectively, and min_interval -is the minimum tick interval. If set to 'None', the corresponding -value will be automatically chosen. So, in this example, we are letting -weewx pick sensible y minimum and maximum values, but -we are requiring that the tick increment (min_interval) -be at least 0.02.

-

Continuing on with the example above, there will be only one plot "line" (it -will actually be a series of bars) and it will -have logical name 'rain'. Because we haven't said -otherwise, the SQL data type to be used for this line will be the same as its -logical name, that is, rain, but this can be -overridden (see below). The aggregation type will be summing (overriding -the averaging specified in sub-section [[month_images]]), -so you get the total rain over the aggregate period (rather than the average) over -an aggregation interval of 86,400 seconds (one day). The plot line will be titled -with the indicated label ('Rain (daily avg)')

-

Including more than one SQL type in a plot

-

More than one SQL type can be included in a plot. For example, here's how to -generate a plot with the week's outside temperature as well as dewpoint:

-

[[[monthtempdew]]]

-

  [[[[outTemp]]]]

-

  [[[[dewpoint]]]]

-

This would create an image in file monthtempdew.png -that includes a line plot of both outside temperature and dewpoint.

-

Including the same SQL type more than once in a plot

-

Another example. Say you want a plot of the day's temperature, overlaid with -hourly averages. Here, you are using the same data type ('outTemp') -for both plot lines, the first with averages, the second without. If you do the -obvious it won't work:

-

## WRONG ##

-

[[[daytemp_with_avg]]]

-

  [[[[outTemp]]]]

-

    aggregate_type = avg

-

    aggregate_interval = 3600

-

  [[[[outTemp]]]]  # OOPS! The same section name appears more than -once!

-

The option parser does not allow the same section name ('outTemp' -in this case) to appear more than once at a given level in the configuration -file, so an error will be declared (technical reason: formally, the sections are -an unordered dictionary). If you wish for the same SQL type to appear more than -once in a plot then there is a trick you must know: use option -data_type. This will override the default action that -the logical line name is used for the SQL type. So, our example would look like -this:

-

[[[daytemp_with_avg]]]

-

    [[[[a_logical_name]]]]

-

      data_type = outTemp

-

      aggregate_type = avg

-

      aggregate_interval = 3600

-

    [[[[outTemp]]]]

-

Here, the first logical line has been given the name "a_logical_name" -to distinguish it from the second line "outTemp". We -have specified that the first line will use data type outTemp -and that it will use averaging over a one hour period. The second also uses -outTemp, but will not use averaging.

-

The result is a nice plot of the day's temperature, overlaid with a 3-hour -smoothed average.

-

Progressive vector plots

-

Weewx can produce progressive vector plots as well as the more conventional -x-y plots. To produce these, use plot type 'vector'. -You need a vector type to produce this kind of plot. There are two: 'windvec', -and 'windgustvec'. While they don't actually appear in -the SQL database, weewx understands that they represent special vector-types. -The first, 'windvec', represents the average wind in -an archive period, the second, 'windgustvec' the max -wind in an archive period. Here's how to produce a progressive vector plot of -the year's biggest daily wind gusts, along with daily averages:

-

[[[yeargustoverlay]]]

-

  aggregate_interval = 86400

-

  [[[[windvec]]]]

-

    plot_type = vector

-

    aggregate_type = avg

-

  [[[[windgustvec]]]]

-

    plot_type = vector

-

    aggregate_type = max

-

This will produce an image file with name -yeargustoverlay.png. It will consist of two progressive vector plots, -both using daily aggregation (86,400 seconds). For the first set of vectors, the daily -average will be used. In the second, the max of the gusts will be used.

-

By default, the sticks in the progressive wind plots point towards the wind -source. That is, the stick for a wind from the west will point left. If you have -a chronic wind direction (as I do), you may want to rotate the default direction -so that all the vectors don't line up over the x-axis, overlaying each other. Do -this by using option vector_rotate. For example, with -my chronic westerlies, I set vector_rotate to 90.0, so -winds out of the west point straight up.

-

If you use this kind of plot (the out-of-the-box version of -weewx includes daily, weekly, monthly, and yearly -progressive wind plots), a small compass rose will be put in the lower-left -corner of the image to show the orientation of North.

-

Summary

-

Studying this section in the shipped version of weewx.conf -will give you ideas about the many different image plot configurations that are -possible without hacking the code.

-

[Labels]

-

This section controls how images are labeled. It consists of three sub-sections:

-

[[Generic]]

-

This sub-sections specifies default labels to be used for each SQL type. For -example, options

-

inTemp = Inside Temperature

-

outTemp = Outside Temperature

-

would cause the given labels to be used for plots involving SQL types -inTemp and outTemp..

-

[[ImperialFormats]]

-

This sub-section is used to specify what format to be used for y-axis labels -in image plots that use Imperial (U.S.) units. It is also used for unit labels in -HTML file generation. For example, the options

-

outTemp = %.1f

-

rain =    %.2f

-

would cause the given formats to be used when formatting outside temperature -and rain axes, respectively. The - -formatting codes are those used by Python, and are very similar to C's sprintf() -codes.

-

[[ImperialUnits]]

-

This sub-section specifies what unit labels to be used for the y-axis in image -plots that use Imperial (U.S.) units. For example, the options

-

outTemp = \xb0F

-

rain = ' in'

-

would cause outside temperature to have unit labels '°F' -and rain to have labels ' in'. (NB: the code -\xb0 is the hexadecimal value b0, -which in many encodings encodes the degree sign.)

-

[HTML]

-

Section [HTML] has two options and two sub-sections. For additional information -on HTML generation see the section below.

-

template_root

-

This option specifies the directory, relative to WEEWX_ROOT, -where the HTML templates can be found. Required. No default.

-

html_root

-

This option specifies the directory, relative to WEEWX_ROOT, -where the generated HTML files should be put. Required. No default.

-

[[ImperialUnits]]

-

This subsection is similar to its eponymous counterpart in section -[Labels] above, except it is used for HTML generation. -It is useful to have a separate section because HTML uses special 'entity' codes -to code special characters, such as the degree sign. For example, the options

-

outTemp = &deg;F

-

rain =    ' in'

-

would cause outside temperature and rain to have unit labels  '°F' -and ' in', respectively.

-

[[Time]]

-

This subsection is used for time labels in HTML generation. It uses -strftime() -formats. For example

-

week  = %H:%M on %A

-

month = %d-%b-%Y %H:%M

-

would specify that week data should use a format such as "15:20 -on Sunday", while month data should look like "06-Oct-2009 -15:20"

-

7. Running weewx

-

Weewx can be run either from the command line (useful -for diagnostic purposes because it will print out a summary of every LOOP data), -or as a daemon. When first trying weewx, it's probably -best to run it from the command line because you will be able to see command line -diagnostics, as well as log messages.

-

7.1 Running from the command line

-

Weewx can easily be run from the command line. Start -by making sure you have appropriate permissions to the serial port your weather -station uses. For example, if you are using a plain old serial port:

-

sudo chmod 666 /dev/ttyS0

-

Then run the main loop program, weewxd.py, giving the -configuration file as its only parameter:

-

$WEEWX_ROOT/bin/weewxd.py $WEEWX_ROOT/weewx.conf

-

It should start by downloading any archive data from your weather station into -the database $WEEWX_ROOT/archive/weewx.sdb. As -the Davis VantagePro can store a couple thousand archive records internally, this -could take a minute or two. I've found this process particularly slow on SuSE for -some reason.

-

Weewx will then start monitoring LOOP data, printing -a short version of the received data on standard output, about once every two seconds.

-

7.2 Running as a daemon

-

For unattended operations it is best to have weewx -run as a daemon, started automatically when the server is rebooted. Start by selecting the appropriate run script. They can be found under -$WEEWX_ROOT/start_script.

- - - - - - - - - -
SuSE:$WEEWX_ROOT/start_script/SuSE/weewx
Debian/Ubuntu:$WEEWX_ROOT/start_script/Debian/weewx
-

Check the chosen script to make sure the variable WEEWX_ROOT -inside has been set to the proper root directory for your weewx -installation (it should have been set to the correct value automatically by the -install process, but it's worth checking).

-

Copy it to the proper location for your system:

- - - - - - - - - -
SuSE:cp $WEEWX_ROOT/start_script/SuSE/weewx /etc/init.d
Debian/Ubuntu:cp $WEEWX_ROOT/start_script/Debian/weewx /etc/init.d
-

Make sure the script is executable

- - - - - - - - - -
SuSE:chmod +x /etc/init.d/weewx
Debian/Ubuntu:chmod +x /etc/init.d/weewx
-

Create symbolic links in the run level directories:

- - - - - - - - - -
SuSE:/usr/lib/lsb/install_initd /etc/init.d/weewx
Debian/Ubuntu:update-rc.d weewx defaults 98
-

Weewx will now start automatically whenever your system -is booted. You can also manually start, stop, and restart the -weewx daemon:

-

/etc/init.d/weewx start

-

/etc/init.d/weewx stop

-

/etc/init.d/weewx restart

-

By default, the scripts are designed to have weewx -run at run levels 2, 3, 4 and 5. Incidentally, a nice tool for setting run levels -with Debian (Ubuntu) systems is sysv-rc-conf. -It uses a curses interface to allow you to change easily which run level any of -your daemons runs at. There is a similar tool on SuSE. From the start menu run the -YAST Control Center, then look for Systesm Services (Runlevel). Pick "Expert" mode -to see the run levels.

-

8. Compatibility with -wview

-

The sqlite3 archive database used by weewx (nominally, -weewx.sdb) is completely compatible with the database -used by wview (usually called -wview-archive.sdb), at least as of Version 5.2.X. The -schema and its semantics is identical. However, the statistical file -stats.sdb is different, and must be rebuilt

-

9. HTML Generation

-

HTML generation is done using the Cheetah -templating engine. This is a very powerful engine, which essentially lets you have -the full semantics of Python available in your templates. As this would make the -templates incomprehensible to anyone but a Python programmer, -weewx adopts a very small subset of its power.

-

Generally, any value is specified by using a 'dot' code. For example:

-

$month.outTemp.max

-

$month.outTemp.maxtime

-

$current.outTemp

-

would code the max outside temperature for the month, the time it occurred, and -the current outside temperature, respectively. So, an HTML file that looks like

-

<html>

-

  <head>

-

    <title>Current conditions</title>

-

  </head>

-

  <body>

-

  <p>Current temperature = $current.outTemp</p>

-

  <p>Max for the month is $month.outTemp.max, which occurred at

-

$month.outTemp.maxtime</p>

-

  </body>

-

</html>

-

would be all you need for a very simple HTML page that would display the text: -

-

Current temperature = 51.0°F
-Max for the month is 68.8°F, which occurred at 07-Oct-2009 15:15

-

The format that was used to format the temperature (51.0) -is specified in section [Labels][[ImperialFormats]]. -The unit label °F is from section -[HTML][[ImperialUnits]], while the time -format is from [HTML][[Time]].

-

The "dot" code has up to three parts.

-
    -
  1. The first part is the time period, and can be one of - current, day, week, - month, year, or - rainyear.
  2. -
  3. The second part is the "SQL" type. This is something like 'outTemp', - 'rain', 'wind', etc. It - is called the "SQL" type because the identifier is identical to that used in - the schema of the SQL databases with three exceptions. First, type 'wind' - is a special hybrid type and does not appear in the SQL database. It brings - together the several different SQL types 'windSpeed', - 'windDir', windGust', and - 'windGustDir' under one roof (all are still available, - should you wish to use them for a specialized application). Exceptions number - two and three are 'heatdeg' and 'cooldeg', - heating and cooling degree-days, respectively, which are synthesized from average - outside temperature and do not appear directly in the database.
  4. -
  5. The last position is the aggregation type, available for any time period - except for 'current'. The table below shows what aggregation - types are available for which types.
  6. -
-

9.1 Types

-

The following types are available to be used in your template (assuming your -station supports them and you have specified that it be stored in your stats database. -See section stats_types in the weewx.conf -configuration file).

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Typeminmintimemaxmaxtimeavgsumrmsvecavgvecdir
barometerXXXXX    
inTempXXXXX    
outTempXXXXX    
inHumidityXXXXX    
outHumidityXXXXX    
windXXXXX XXX
rainXXXXXX   
dewpointXXXXX    
windchillXXXXX    
heatindexXXXXX    
heatdeg     X   
cooldeg     X   
ETXXXXX    
radiationXXXXX    
UVXXXXX    
extraTemp1
- extraTemp2
- extraTemp3
XXXXX    
soilTemp1
- soilTemp2
- soilTemp3
XXXXX    
leafTemp1
- leafTemp2
XXXXX    
extraHumid1
- extraHumid2
XXXXX    
soilMoist1
- soilMoist2
- soilMoist3
- soilMoist4
XXXXX    
leafWet1
- leafWet2
XXXXX    
rxCheckPercentXXXXX    
-

10. Monitoring weewx

-

Weewx logs many events to the system log. On Debian -systems, this is /var/log/syslog, on SuSE, -/var/log/messages. Your system may use yet another place. -When troubleshooting the system, be sure to check it!

-

Setting the option debug in weewx.conf -to 1 (one) will generate many more checks and output and -can be useful for debugging.

-

11. Architectural notes

-

11.1 Goals

-

The primary goals of weewx are:

- -

11.2 Strategies

-

To meet these goals, the following strategies were used:

- -

While weewx is nowhere near as fast at generating images -and HTML as its predecessor, wview (this is partially -because it uses fancier fonts and a way more powerful templating engine), it is -'fast enough' for all platforms but the slowest. I run it regularly on a 500 MHz -machine where generating the 9 images used in the "Current Conditions" page takes -just under 2 seconds. Compare this with wview's 0.4 seconds.

-

Unfortunately, the architectural goal of one code base is likely to be broken -with the arrival of Python V3.X. It has so many changes that are not backwards compatible -with V2.X, that a separate code base will most likely be needed. My intention is -to stick with the V2.5 and V2.6 versions until V3.X is so widespread it cannot be -ignored, then make a permanent switch. I doubt this will affect the average -weewx user.

-

11.3 Run time internals

-

At a high-level, weewx consists of an engine class called StdEngine. It is -responsible for loading any "services" that are to be run and arranging for them -to be called when key events occur, such as the arrival of LOOP data. One of the -standard services is a "processing service," which arranges to do things such as -generate HTML files and images, and FTP'ing of data to a web server. Additional -tasks can be added to the list of things the processing service will do. Another -standard service creates a queue into which the timestamp of new data is put in. -It then watches this queue, and when new data arrives, it posts it to the -Weather Underground

-

It is easy to add new services. The source distribution includes an example -new service called "MyAlarm," which sends an email -when an arbitrary expression evaluates True.

-

All writes to the databases are protected by transactions. You can kill the program -at any time (either Control-C if run from the command line or "/etc/init.d/weewx -stop" if a daemon) without fear of corrupting the databases.

-

The code makes ample use of exceptions to insure graceful recovery from problems -such as network outages. It also monitors socket and console timeouts, restarting -whatever it was working on several times before giving up. In the case of an unrecoverable -console error (such as the console not responding at all), the program waits 60 -seconds then restarts the program from the top.

-

Any "hard" exceptions, that is those that do not involve network and console -timeouts and are most likely due to a logic error, are logged, reraised, and ultimately -cause thread termination. If this happens in the main thread (not likely and hasn't -happened to me yet), then this causes program termination.

-

11.4 Terminology

-

This is a glossary of terminology used throughout the code.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
packetSomething obtained off the weather station. Frequently uses a complex - internal encoding, so it requires some processing to be useful.
recordSomething obtained off the SQL database.
archive packetA packet obtained off the store on the weather station. For example, - with a Davis VantagePro, it's obtained using their DMPAFT - command.
loop packetA packet with the current observations. For example, with a Davis VantagePro, - it's obtained using their LOOP command.
archive recordA record obtained off the SQL database
tuple-timeAn instance of the Python object time.struct_time. - This is a 9-wise tuple that represent a time. It could be in either local - time or UTC, though usually the former. See module time - for more information. They are useful because they are a little closer in - format to what the Davis VantagePro uses, although they still require a - bit of processing. Variables carrying tuple time usually have a suffix '_tt'.
epoch timeSometimes referred to as "unix time," or "unix epoch time." The number - of seconds since the epoch, which is 1 Jan 1970 00:00:00 UTC. Hence, it - always represents UTC (well.... after adding a few leap seconds. But, close - enough). This is the time used on the sqlite archive and appears as type - 'dateTime' in the SQL schema, perhaps an unfortunate - name because of the similarity to the Python type 'datetime'. Very easy - to manipulate, but it's an opaque big number.
time stampA variable in unix epoch time. Always in UTC. Variables carrying a time - stamp usually have a suffix '_ts'.
datetimeAn instance of the Python object datetime.datetime. - Variables of type datetime usually have a suffix '_dt'.
-

11.5 Units

-

This package is imperial (U.S.) units only. However, it has been set up to make -it easy to extend to metric.

-

In general, there are three different areas where the unit system makes a difference.: -

-
    -
  1. On the weather station. As far as I know, the Davis VantagePro series supports - only imperial units internally. So, any translation to metric must be done in - one of the other two areas:
  2. -
  3. In the database. The unit system of any individual record is indicated by - the "usUnits" field. The numerical value 1 indicates imperial. Other numbers - haven't been assigned (yet). Right now, only imperial units is supported. -
  4. -
  5. In the presentation (i.e., html and image files). Right now, whatever unit - system is in use in the database is carried through.
  6. -
-

The transition from 1 to 2, i.e., from data in the VantagePro to the database, -is handled by a translation function. Right now, only one is supplied, -weewx.VantagePro.translateArchiveToImperial. Others could -easily be introduced.

-

11.6 Value "None"

-

The Python special value 'None' is used throughout -to signal a missing data point. All functions expect it.

-

However, the time value must never be 'None'. This -is because it is used as the primary key in the SQL database.

-

11.7 Time

-

Weewx stores all data in UTC (roughly, "Greenwich" -or "Zulu") time. However, usually one is interested in weather events in local time -and want image and HTML generation to reflect that. Furthermore, most weather stations -are configured in local time. This requires that many data times be converted back -and forth between UTC and local time. To avoid tripping up over time zones and daylight -savings time, weeewx generally uses Python routines to -do this conversion. Nowhere in the code base is there any explicit recognition of -DST. Instead, its presence is implicit in the conversions. At times, this can cause -the code to be relatively inefficient.

-

For example, if one wanted to plot something every 3 hours in UTC time, it would -be very simple: to get the next plot point, just add 10,800 to the epoch time:

-

next_ts = last_ts + 10800

-

But, if one wanted to plot something for every 3 hours in local time -(that is, at 0000, 0300, 0600, etc.), despite a possible DST change in the middle, -one could modify the above to recognize whether a DST transition occurs sometime -between last_ts and the next three hours and, if so, make -the necessary adjustments. This is generally what wview -does. Weewx takes a different approach and converts from -UTC to local, does the arithmetic, then converts back. This is inefficient, but -bulletproof against changes in DST algorithms, etc:

-

time_dt = datetime.datetime.fromtimestamp(last_ts)

-

delta = datetime.timedelta(seconds=10800)

-

next_dt = time_dt + delta

-

next_ts = int(time.mktime(next_dt.timetuple()))

-

Other time conversion problems are handled in a similar manner.

-

12. Customizing weewx

-

See file customizing.htm for details on how to -customize weewx.

- - - - +

heating_base
+cooling_base

+

Set to the base temperature for calculating heating and cooling degree-days, +respectively. The default is 65.0 for both.

+

hemispheres

+

Set to suitable abbreviations for the four hemispheres. Default is "N", "S", +"E", "W"

+

clock_check

+

How often to check the station's onboard clock for drift, in seconds. Default +is 14400 (every 4 hours)

+

cache_loop_data

+

Set to 1 (one) to cache LOOP data, otherwise, set to zero. Most users will +not want to change this unless you have a specialized application. Default is 1 +(i.e., cache LOOP data).

+

station_type

+

Set to the type of hardware you are using. For this version, only 'VantagePro' +is accepted. Required.

+

[VantagePro]

+

This section is for options relating to the VantagePro hardware.

+

port

+

Set to the port name used by your station. Example, /dev/ttyUSB0 +is a common location for USB ports under Debian, /dev/ttyS0 +for serial ports. Required. No default.

+

baudrate

+

Set to the baudrate of your station. The default is 19200.

+

archive_interval

+

Set to the desired archive interval of your station, in seconds. This variable +is only used when setting up your station. Otherwise, this value is read directly +from the station. Required if you configure your station. +No default.

+

iss_id

+

Set to the ID number of your Integrated Sensor Suite (ISS). This is used in the +formula to calculate reception quality for wireless stations. The default is 1.

+

archive_delay

+

How long to wait in seconds after the top of an archiving interval before fetching +new data off the station. For example, if your archive interval is 5 minutes and +archive_delay is set to 15, then the data will be fetched at 00:00:15, 00:05:15, +00:10:15, etc. This delay is to give the station a few seconds to archive the data +internally, and in case your server has any other tasks to do at the top of the +minute. Default is 15 seconds.

+

timeout

+

How many seconds  to wait for a response from the station before giving +up. Default is 5 seconds.

+

wait_before_retry

+

How many seconds to wait before retrying again. Unless you have a good reason +to change it, this value should be left at the default, as it is long enough for +the station to offer new data, but not so long as to go into a new loop packet (which +arrive every 2 seconds). Default is 1.2 seconds.

+

max_retries

+

How many times to try again before giving up. Default is 4.

+

max_drift

+

The maximum amount of drift to tolerate, in seconds, in the VantagePro's onboard +clock, before resetting the clock. Default is 5.

+

unit_system

+

What unit system is in use on your weather station hardware. Possible values +are '1' (Imperial) or '2' (Metric). As far as I know, all Davis instruments support +only Imperial. In any case, Imperial is the only system supported by +weewx. Default is 1.

+

[FTP]

+

If you FTP your images and HTML files to an external web server, +weewx can FTP them for you. It does an incremental update, +that is, it only FTPs any files that have changed, saving the outgoing bandwidth +of your Internet connection.

+

If you do not use such a server, comment out the first four options below.

+

user

+

Set to the username you use for your FTP connection to your web server. Required. +No default.

+

password

+

Set to the password you use for your FTP connection to your web server. Required. +No default.

+

server

+

Set to the name of your web server (e.g., +www.threefools.org, in my case). Required. No default

+

path

+

Set to the path where the weather data will be stored on your webserver (e.g., +'/weather'). NB: some FTP servers require a +leading slash ('/'), some don't. Required. No default.

+

passive

+

Set to 1 if you wish to use the more modern, FTP passive mode, 0 if you wish +to use active mode. Passive mode generally works better through firewalls, but not +all FTP servers do a good job of supporting it. See +Active FTP vs. Passive FTP, a Definitive +Explanation for a good explanation of the difference. Default is 1 (passive +mode).

+

max_retries

+

Weewx will try up to this many times to FTP a file +up to your server before giving up. Default is 3.

+

[Wunderground]

+

Weewx can send your current data to the Weather +Underground. If you do not wish to do this, comment out the two options below.

+

station

+

Set to your Weather Underground station ID (e.g., KORHOODR3). +Required.

+

password

+

Set to your Weather Underground password. Required.

+

[Archive]

+

This section is for configuring the sqlite3 database on which the station archive +data is stored.

+

archive_file

+

The path, relative to the WEEWX_ROOT directory, to +the database. Required

+

unit_system

+

What unit system to use inside the database. Required. The only one supported +right now is '1', the Imperial (U.S.) system

+

[Stats]

+

This section is for configuring the sqlite3 database on which the station statistics +are stored.

+

stats_file

+

The path, relative to the WEEWX_ROOT directory to the +statistical database. Required.

+

stats_types

+

The list of types for which statistics will be kept. Types not listed will not +be available for generating HTML pages. Optional. The default is all types, resulting +in a possibly much bigger than necessary stats database (do you really have four +different soil moisture sensors?) The list that ships with the configuration file +will work for most stations and probably will not have to be modified.

+

[Images]

+

This section, which controls which images (plots) get generated and with which +options, is by far the most complicated. However, it is extremely flexible and powerful.

+

Time periods

+

The section consists of one or more sub-sections, one for each time period (day, week, +month, and year). These sub-sections define the nature of aggregation and plot types +for the time period. For example, here's a typical set of options for sub-section +[[month_images]], controlling how images that cover a +month period are generated:

+

[[month_images]]

+

  x_label_format = %d

+

  bottom_label_format = %m/%d/%y %H:%M

+

  time_length = 2592000 # == 30 days

+

  aggregate_type = avg

+

  aggregate_interval = 10800 # == 3 hours

+

The option x_label_format gives a +strftime() +type format for the x-axis. In this example, it will only show days (format option +"%d"). The bottom_label_format +is the format used to time stamp the image at the bottom. In this example, it will +show the time as 10/25/09 15:35. A plot will cover a nominal +30 days, and all items included in it will use an aggregate type of averaging over +3 hours.

+

Image files

+

Within each sub-section is another nesting, one for each image to be generated. +The title of each sub-sub-section is the filename to be used for the image. +Finally, at one additional nesting level (!) are the logical names of all the +line types to be drawn in the image.  Values specified in the +level above can be overridden. For example, here's a typical set of options for +sub-sub-section [[[monthrain]]]:

+

[[[monthrain]]]

+

  plot_type = bar

+

  yscale = None, None, 0.02

+

  [[[[rain]]]]

+

    aggregate_type = sum

+

    aggregate_interval = 86400

+

    label = Rain (daily avg)

+

This will generate an image file with name monthrain.png. +It will be a bar plot. Option yscale controls the y-axis +scaling — if left out, the scale will automatically be chosen. However, in this +example we are choosing to exercise some degree of control by specifying values +explicitly. It is a 3-way tuple (ylow, +yhigh, min_interval), where +ymin and ymax are the minimum +and maximum y-axis values, respectively, and min_interval +is the minimum tick interval. If set to 'None', the corresponding +value will be automatically chosen. So, in this example, we are letting +weewx pick sensible y minimum and maximum values, but +we are requiring that the tick increment (min_interval) +be at least 0.02.

+

Continuing on with the example above, there will be only one plot "line" (it +will actually be a series of bars) and it will +have logical name 'rain'. Because we haven't said +otherwise, the SQL data type to be used for this line will be the same as its +logical name, that is, rain, but this can be +overridden (see below). The aggregation type will be summing (overriding +the averaging specified in sub-section [[month_images]]), +so you get the total rain over the aggregate period (rather than the average) over +an aggregation interval of 86,400 seconds (one day). The plot line will be titled +with the indicated label ('Rain (daily avg)')

+

Including more than one SQL type in a plot

+

More than one SQL type can be included in a plot. For example, here's how to +generate a plot with the week's outside temperature as well as dewpoint:

+

[[[monthtempdew]]]

+

  [[[[outTemp]]]]

+

  [[[[dewpoint]]]]

+

This would create an image in file monthtempdew.png +that includes a line plot of both outside temperature and dewpoint.

+

Including the same SQL type more than once in a plot

+

Another example. Say you want a plot of the day's temperature, overlaid with +hourly averages. Here, you are using the same data type ('outTemp') +for both plot lines, the first with averages, the second without. If you do the +obvious it won't work:

+

## WRONG ##

+

[[[daytemp_with_avg]]]

+

  [[[[outTemp]]]]

+

    aggregate_type = avg

+

    aggregate_interval = 3600

+

  [[[[outTemp]]]]  # OOPS! The same section name appears more than +once!

+

The option parser does not allow the same section name ('outTemp' +in this case) to appear more than once at a given level in the configuration +file, so an error will be declared (technical reason: formally, the sections are +an unordered dictionary). If you wish for the same SQL type to appear more than +once in a plot then there is a trick you must know: use option +data_type. This will override the default action that +the logical line name is used for the SQL type. So, our example would look like +this:

+

[[[daytemp_with_avg]]]

+

    [[[[a_logical_name]]]]

+

      data_type = outTemp

+

      aggregate_type = avg

+

      aggregate_interval = 3600

+

    [[[[outTemp]]]]

+

Here, the first logical line has been given the name "a_logical_name" +to distinguish it from the second line "outTemp". We +have specified that the first line will use data type outTemp +and that it will use averaging over a one hour period. The second also uses +outTemp, but will not use averaging.

+

The result is a nice plot of the day's temperature, overlaid with a 3-hour +smoothed average.

+

Progressive vector plots

+

Weewx can produce progressive vector plots as well as the more conventional +x-y plots. To produce these, use plot type 'vector'. +You need a vector type to produce this kind of plot. There are two: 'windvec', +and 'windgustvec'. While they don't actually appear in +the SQL database, weewx understands that they represent special vector-types. +The first, 'windvec', represents the average wind in +an archive period, the second, 'windgustvec' the max +wind in an archive period. Here's how to produce a progressive vector plot of +the year's biggest daily wind gusts, along with daily averages:

+

[[[yeargustoverlay]]]

+

  aggregate_interval = 86400

+

  [[[[windvec]]]]

+

    plot_type = vector

+

    aggregate_type = avg

+

  [[[[windgustvec]]]]

+

    plot_type = vector

+

    aggregate_type = max

+

This will produce an image file with name +yeargustoverlay.png. It will consist of two progressive vector plots, +both using daily aggregation (86,400 seconds). For the first set of vectors, the daily +average will be used. In the second, the max of the gusts will be used.

+

By default, the sticks in the progressive wind plots point towards the wind +source. That is, the stick for a wind from the west will point left. If you have +a chronic wind direction (as I do), you may want to rotate the default direction +so that all the vectors don't line up over the x-axis, overlaying each other. Do +this by using option vector_rotate. For example, with +my chronic westerlies, I set vector_rotate to 90.0, so +winds out of the west point straight up.

+

If you use this kind of plot (the out-of-the-box version of +weewx includes daily, weekly, monthly, and yearly +progressive wind plots), a small compass rose will be put in the lower-left +corner of the image to show the orientation of North.

+

Summary

+

Studying this section in the shipped version of weewx.conf +will give you ideas about the many different image plot configurations that are +possible without hacking the code.

+

[Labels]

+

This section controls how images are labeled. It consists of three sub-sections:

+

[[Generic]]

+

This sub-sections specifies default labels to be used for each SQL type. For +example, options

+

inTemp = Inside Temperature

+

outTemp = Outside Temperature

+

would cause the given labels to be used for plots involving SQL types +inTemp and outTemp..

+

[[ImperialFormats]]

+

This sub-section is used to specify what format to be used for y-axis labels +in image plots that use Imperial (U.S.) units. It is also used for unit labels in +HTML file generation. For example, the options

+

outTemp = %.1f

+

rain =    %.2f

+

would cause the given formats to be used when formatting outside temperature +and rain axes, respectively. The + +formatting codes are those used by Python, and are very similar to C's sprintf() +codes.

+

[[ImperialUnits]]

+

This sub-section specifies what unit labels to be used for the y-axis in image +plots that use Imperial (U.S.) units. For example, the options

+

outTemp = \xb0F

+

rain = ' in'

+

would cause outside temperature to have unit labels '°F' +and rain to have labels ' in'. (NB: the code +\xb0 is the hexadecimal value b0, +which in many encodings encodes the degree sign.)

+

[HTML]

+

Section [HTML] has two options and two sub-sections. For additional information +on HTML generation see the section below.

+

template_root

+

This option specifies the directory, relative to WEEWX_ROOT, +where the HTML templates can be found. Required. No default.

+

html_root

+

This option specifies the directory, relative to WEEWX_ROOT, +where the generated HTML files should be put. Required. No default.

+

[[ImperialUnits]]

+

This subsection is similar to its eponymous counterpart in section +[Labels] above, except it is used for HTML generation. +It is useful to have a separate section because HTML uses special 'entity' codes +to code special characters, such as the degree sign. For example, the options

+

outTemp = &deg;F

+

rain =    ' in'

+

would cause outside temperature and rain to have unit labels  '°F' +and ' in', respectively.

+

[[Time]]

+

This subsection is used for time labels in HTML generation. It uses +strftime() +formats. For example

+

week  = %H:%M on %A

+

month = %d-%b-%Y %H:%M

+

would specify that week data should use a format such as "15:20 +on Sunday", while month data should look like "06-Oct-2009 +15:20"

+

7. Running weewx

+

Weewx can be run either from the command line (useful +for diagnostic purposes because it will print out a summary of every LOOP data), +or as a daemon. When first trying weewx, it's probably +best to run it from the command line because you will be able to see command line +diagnostics, as well as log messages.

+

7.1 Running from the command line

+

Weewx can easily be run from the command line. Start +by making sure you have appropriate permissions to the serial port your weather +station uses. For example, if you are using a plain old serial port:

+

sudo chmod 666 /dev/ttyS0

+

Then run the main loop program, weewxd.py, giving the +configuration file as its only parameter:

+

$WEEWX_ROOT/bin/weewxd.py $WEEWX_ROOT/weewx.conf

+

It should start by downloading any archive data from your weather station into +the database $WEEWX_ROOT/archive/weewx.sdb. As +the Davis VantagePro can store a couple thousand archive records internally, this +could take a minute or two. I've found this process particularly slow on SuSE for +some reason.

+

Weewx will then start monitoring LOOP data, printing +a short version of the received data on standard output, about once every two seconds.

+

7.2 Running as a daemon

+

For unattended operations it is best to have weewx +run as a daemon, started automatically when the server is rebooted. Start by selecting the appropriate run script. They can be found under +$WEEWX_ROOT/start_script.

+ + + + + + + + + +
SuSE:$WEEWX_ROOT/start_script/SuSE/weewx
Debian/Ubuntu:$WEEWX_ROOT/start_script/Debian/weewx
+

Check the chosen script to make sure the variable WEEWX_ROOT +inside has been set to the proper root directory for your weewx +installation (it should have been set to the correct value automatically by the +install process, but it's worth checking).

+

Copy it to the proper location for your system:

+ + + + + + + + + +
SuSE:cp $WEEWX_ROOT/start_script/SuSE/weewx /etc/init.d
Debian/Ubuntu:cp $WEEWX_ROOT/start_script/Debian/weewx /etc/init.d
+

Make sure the script is executable

+ + + + + + + + + +
SuSE:chmod +x /etc/init.d/weewx
Debian/Ubuntu:chmod +x /etc/init.d/weewx
+

Create symbolic links in the run level directories:

+ + + + + + + + + +
SuSE:/usr/lib/lsb/install_initd /etc/init.d/weewx
Debian/Ubuntu:update-rc.d weewx defaults 98
+

Weewx will now start automatically whenever your system +is booted. You can also manually start, stop, and restart the +weewx daemon:

+

/etc/init.d/weewx start

+

/etc/init.d/weewx stop

+

/etc/init.d/weewx restart

+

By default, the scripts are designed to have weewx +run at run levels 2, 3, 4 and 5. Incidentally, a nice tool for setting run levels +with Debian (Ubuntu) systems is sysv-rc-conf. +It uses a curses interface to allow you to change easily which run level any of +your daemons runs at. There is a similar tool on SuSE. From the start menu run the +YAST Control Center, then look for Systesm Services (Runlevel). Pick "Expert" mode +to see the run levels.

+

8. Compatibility with +wview

+

The sqlite3 archive database used by weewx (nominally, +weewx.sdb) is completely compatible with the database +used by wview (usually called +wview-archive.sdb), at least as of Version 5.2.X. The +schema and its semantics is identical. However, the statistical file +stats.sdb is different, and must be rebuilt

+

9. HTML Generation

+

HTML generation is done using the Cheetah +templating engine. This is a very powerful engine, which essentially lets you have +the full semantics of Python available in your templates. As this would make the +templates incomprehensible to anyone but a Python programmer, +weewx adopts a very small subset of its power.

+

Generally, any value is specified by using a 'dot' code. For example:

+

$month.outTemp.max

+

$month.outTemp.maxtime

+

$current.outTemp

+

would code the max outside temperature for the month, the time it occurred, and +the current outside temperature, respectively. So, an HTML file that looks like

+

<html>

+

  <head>

+

    <title>Current conditions</title>

+

  </head>

+

  <body>

+

  <p>Current temperature = $current.outTemp</p>

+

  <p>Max for the month is $month.outTemp.max, which occurred at

+

$month.outTemp.maxtime</p>

+

  </body>

+

</html>

+

would be all you need for a very simple HTML page that would display the text: +

+

Current temperature = 51.0°F
+Max for the month is 68.8°F, which occurred at 07-Oct-2009 15:15

+

The format that was used to format the temperature (51.0) +is specified in section [Labels][[ImperialFormats]]. +The unit label °F is from section +[HTML][[ImperialUnits]], while the time +format is from [HTML][[Time]].

+

The "dot" code has up to three parts.

+
    +
  1. The first part is the time period, and can be one of + current, day, week, + month, year, or + rainyear.
  2. +
  3. The second part is the "SQL" type. This is something like 'outTemp', + 'rain', 'wind', etc. It + is called the "SQL" type because the identifier is identical to that used in + the schema of the SQL databases with three exceptions. First, type 'wind' + is a special hybrid type and does not appear in the SQL database. It brings + together the several different SQL types 'windSpeed', + 'windDir', windGust', and + 'windGustDir' under one roof (all are still available, + should you wish to use them for a specialized application). Exceptions number + two and three are 'heatdeg' and 'cooldeg', + heating and cooling degree-days, respectively, which are synthesized from average + outside temperature and do not appear directly in the database.
  4. +
  5. The last position is the aggregation type, available for any time period + except for 'current'. The table below shows what aggregation + types are available for which types.
  6. +
+

In addition, if the first part of the dot code represents a time period +longer than a day (e.g., a week, month, year, or rainyear), then it can be +iterated over. This example uses a Cheetah 'for' loop to iterate over all months +in a year, printing out each month's min and max temperature.

+

<html>

+

  <head>

+

    <title>Year stats by month</title>

+

  </head>

+

  <body>

+

  <p>Min, max temperatures +by month:</p>

+

  # for $month in $year.months

+

    $month.outTemp.min $month.outTemp.max

+

  # end for 

+

  </body>

+

</html>

+

See the NOAA template files NOAA_month.tmpl and +NOAA_year.tmpl for examples using iteration, as well +as explicit formatting.

+

9.1 Types

+

The following types are available to be used in your template (assuming your +station supports them and you have specified that it be stored in your stats database. +See section stats_types in the weewx.conf +configuration file).

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Typeminmintimemaxmaxtimeavgsumrmsvecavgvecdir
barometerXXXXX    
inTempXXXXX    
outTempXXXXX    
inHumidityXXXXX    
outHumidityXXXXX    
windXXXXX XXX
rainXXXXXX   
dewpointXXXXX    
windchillXXXXX    
heatindexXXXXX    
heatdeg    XX   
cooldeg    XX   
ETXXXXX    
radiationXXXXX    
UVXXXXX    
extraTemp1
+ extraTemp2
+ extraTemp3
XXXXX    
soilTemp1
+ soilTemp2
+ soilTemp3
XXXXX    
leafTemp1
+ leafTemp2
XXXXX    
extraHumid1
+ extraHumid2
XXXXX    
soilMoist1
+ soilMoist2
+ soilMoist3
+ soilMoist4
XXXXX    
leafWet1
+ leafWet2
XXXXX    
rxCheckPercentXXXXX    
+

10. Monitoring weewx

+

Weewx logs many events to the system log. On Debian +systems, this is /var/log/syslog, on SuSE, +/var/log/messages. Your system may use yet another place. +When troubleshooting the system, be sure to check it!

+

Setting the option debug in weewx.conf +to 1 (one) will generate many more checks and output and +can be useful for debugging.

+

11. Architectural notes

+

11.1 Goals

+

The primary goals of weewx are:

+ +

11.2 Strategies

+

To meet these goals, the following strategies were used:

+ +

While weewx is nowhere near as fast at generating images +and HTML as its predecessor, wview (this is partially +because it uses fancier fonts and a way more powerful templating engine), it is +'fast enough' for all platforms but the slowest. I run it regularly on a 500 MHz +machine where generating the 9 images used in the "Current Conditions" page takes +just under 2 seconds. Compare this with wview's 0.4 seconds.

+

Unfortunately, the architectural goal of one code base is likely to be broken +with the arrival of Python V3.X. It has so many changes that are not backwards compatible +with V2.X, that a separate code base will most likely be needed. My intention is +to stick with the V2.5 and V2.6 versions until V3.X is so widespread it cannot be +ignored, then make a permanent switch. I doubt this will affect the average +weewx user.

+

11.3 Run time internals

+

At a high-level, weewx consists of an engine class called StdEngine. It is +responsible for loading any "services" that are to be run and arranging for them +to be called when key events occur, such as the arrival of LOOP data. One of the +standard services is a "processing service," which arranges to do things such as +generate HTML files and images, and FTP'ing of data to a web server. Additional +tasks can be added to the list of things the processing service will do. Another +standard service creates a queue into which the timestamp of new data is put in. +It then watches this queue, and when new data arrives, it posts it to the +Weather Underground

+

It is easy to add new services. The source distribution includes an example +new service called "MyAlarm," which sends an email +when an arbitrary expression evaluates True.

+

All writes to the databases are protected by transactions. You can kill the program +at any time (either Control-C if run from the command line or "/etc/init.d/weewx +stop" if a daemon) without fear of corrupting the databases.

+

The code makes ample use of exceptions to insure graceful recovery from problems +such as network outages. It also monitors socket and console timeouts, restarting +whatever it was working on several times before giving up. In the case of an unrecoverable +console error (such as the console not responding at all), the program waits 60 +seconds then restarts the program from the top.

+

Any "hard" exceptions, that is those that do not involve network and console +timeouts and are most likely due to a logic error, are logged, reraised, and ultimately +cause thread termination. If this happens in the main thread (not likely and hasn't +happened to me yet), then this causes program termination.

+

11.4 Terminology

+

This is a glossary of terminology used throughout the code.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
packetSomething obtained off the weather station. Frequently uses a complex + internal encoding, so it requires some processing to be useful.
recordSomething obtained off the SQL database.
archive packetA packet obtained off the store on the weather station. For example, + with a Davis VantagePro, it's obtained using their DMPAFT + command.
loop packetA packet with the current observations. For example, with a Davis VantagePro, + it's obtained using their LOOP command.
archive recordA record obtained off the SQL database
tuple-timeAn instance of the Python object time.struct_time. + This is a 9-wise tuple that represent a time. It could be in either local + time or UTC, though usually the former. See module time + for more information. They are useful because they are a little closer in + format to what the Davis VantagePro uses, although they still require a + bit of processing. Variables carrying tuple time usually have a suffix '_tt'.
epoch timeSometimes referred to as "unix time," or "unix epoch time." The number + of seconds since the epoch, which is 1 Jan 1970 00:00:00 UTC. Hence, it + always represents UTC (well.... after adding a few leap seconds. But, close + enough). This is the time used on the sqlite archive and appears as type + 'dateTime' in the SQL schema, perhaps an unfortunate + name because of the similarity to the Python type 'datetime'. Very easy + to manipulate, but it's an opaque big number.
time stampA variable in unix epoch time. Always in UTC. Variables carrying a time + stamp usually have a suffix '_ts'.
datetimeAn instance of the Python object datetime.datetime. + Variables of type datetime usually have a suffix '_dt'.
+

11.5 Units

+

This package is imperial (U.S.) units only. However, it has been set up to make +it easy to extend to metric.

+

In general, there are three different areas where the unit system makes a difference.: +

+
    +
  1. On the weather station. As far as I know, the Davis VantagePro series supports + only imperial units internally. So, any translation to metric must be done in + one of the other two areas:
  2. +
  3. In the database. The unit system of any individual record is indicated by + the "usUnits" field. The numerical value 1 indicates imperial. Other numbers + haven't been assigned (yet). Right now, only imperial units is supported. +
  4. +
  5. In the presentation (i.e., html and image files). Right now, whatever unit + system is in use in the database is carried through.
  6. +
+

The transition from 1 to 2, i.e., from data in the VantagePro to the database, +is handled by a translation function. Right now, only one is supplied, +weewx.VantagePro.translateArchiveToImperial. Others could +easily be introduced.

+

11.6 Value "None"

+

The Python special value 'None' is used throughout +to signal a missing data point. All functions expect it.

+

However, the time value must never be 'None'. This +is because it is used as the primary key in the SQL database.

+

11.7 Time

+

Weewx stores all data in UTC (roughly, "Greenwich" +or "Zulu") time. However, usually one is interested in weather events in local time +and want image and HTML generation to reflect that. Furthermore, most weather stations +are configured in local time. This requires that many data times be converted back +and forth between UTC and local time. To avoid tripping up over time zones and daylight +savings time, weeewx generally uses Python routines to +do this conversion. Nowhere in the code base is there any explicit recognition of +DST. Instead, its presence is implicit in the conversions. At times, this can cause +the code to be relatively inefficient.

+

For example, if one wanted to plot something every 3 hours in UTC time, it would +be very simple: to get the next plot point, just add 10,800 to the epoch time:

+

next_ts = last_ts + 10800

+

But, if one wanted to plot something for every 3 hours in local time +(that is, at 0000, 0300, 0600, etc.), despite a possible DST change in the middle, +one could modify the above to recognize whether a DST transition occurs sometime +between last_ts and the next three hours and, if so, make +the necessary adjustments. This is generally what wview +does. Weewx takes a different approach and converts from +UTC to local, does the arithmetic, then converts back. This is inefficient, but +bulletproof against changes in DST algorithms, etc:

+

time_dt = datetime.datetime.fromtimestamp(last_ts)

+

delta = datetime.timedelta(seconds=10800)

+

next_dt = time_dt + delta

+

next_ts = int(time.mktime(next_dt.timetuple()))

+

Other time conversion problems are handled in a similar manner.

+

12. Customizing weewx

+

See file customizing.htm for details on how to +customize weewx.

+ + + + diff --git a/weeutil/weeutil.py b/weeutil/weeutil.py index 884d8cb1..b5444b43 100644 --- a/weeutil/weeutil.py +++ b/weeutil/weeutil.py @@ -312,7 +312,6 @@ def archiveWeekSpan(time_ts, startOfWeek = 6, grace = 30): start at midnight of the day considered the start of the week, and be one week long. """ - print "start of week = ", startOfWeek time_ts -= grace _day_date = datetime.date.fromtimestamp(time_ts) _day_of_week = _day_date.weekday() @@ -493,16 +492,6 @@ def startOfArchiveDay(time_ts, grace = 30): """ return startOfDay(time_ts - grace) -# _time_dt = datetime.datetime.fromtimestamp(time_ts) -# if (_time_dt.hour, _time_dt.minute, _time_dt.second) == (0, 0, 0): -# _time_dt -= datetime.timedelta(days=1) -# _time_tt = _time_dt.timetuple() -# _bod_ts = time.mktime((_time_tt.tm_year, -# _time_tt.tm_mon, -# _time_tt.tm_mday, -# 0, 0, 0, 0, 0, -1)) -# return int(_bod_ts) - def secs_to_string(secs): """ diff --git a/weewx.conf b/weewx.conf index 2d859137..3bf10e10 100644 --- a/weewx.conf +++ b/weewx.conf @@ -726,6 +726,5 @@ socket_timeout = 20 [[ReportEngine]] # The list of reports the report engine should run: -# report_list = weewx.reportengine.FileGen, weewx.reportengine.ImageGen, weewx.reportengine.Ftp - report_list = weewx.reportengine.FileGen, + report_list = weewx.reportengine.FileGen, weewx.reportengine.ImageGen, weewx.reportengine.Ftp \ No newline at end of file diff --git a/weewx/reportengine.py b/weewx/reportengine.py index 82b1fc50..793eda56 100644 --- a/weewx/reportengine.py +++ b/weewx/reportengine.py @@ -52,8 +52,11 @@ class StdReportEngine(threading.Thread): Runs through the list of report classes, instantiating each one, then calling its "start()" method.""" - syslog.syslog(syslog.LOG_DEBUG, "reportengine: Running reports for time %s" % - weeutil.weeutil.timestamp_to_string(self.gen_ts)) + if self.gen_ts: + syslog.syslog(syslog.LOG_DEBUG, "reportengine: Running reports for time %s" % + weeutil.weeutil.timestamp_to_string(self.gen_ts)) + else: + syslog.syslog(syslog.LOG_DEBUG, "reportengine: Running reports for latest time in the database.") # Put the whole thing in a try block so we can log any exceptions that # might bubble up