diff --git a/README.md b/README.md index 0baa46e1..4d9d0741 100644 --- a/README.md +++ b/README.md @@ -103,7 +103,10 @@ The code lives in a git repository: [https://github.com/weewx/weewx](https://github.com/weewx/weewx) -The `master` branch is stable, the `development` branch contains changes that will appear in the next release. See the file [CONTRIBUTING.txt](CONTRIBUTING.txt) for details about branching, and the file [PACKAGING.txt](PACKAGING.txt) for details about the process of creating a release. +See [Notes for WeeWX developers](https://weewx.com/docs/latest/devnotes/) to +learn about how to contribute to WeeWX. See the file +[PACKAGING.txt](PACKAGING.txt) for details about the process of creating a +release. ## Documentation and Support diff --git a/docs_src/devnotes.md b/docs_src/devnotes.md index 801bfc50..aaece7a3 100644 --- a/docs_src/devnotes.md +++ b/docs_src/devnotes.md @@ -60,16 +60,16 @@ To meet these goals, the following strategies were used: [ConfigObj](https://configobj.readthedocs.io) module, by Michael Foord and Nicola Larosa, was chosen to read the configuration file. This allows many options that might otherwise have to go in the - code, to be in a configuration file. + code to be in a configuration file. - A powerful templating engine. The [Cheetah](https://cheetahtemplate.org/) module was chosen for - generating html and other types of files from templates. Cheetah + generating HTML and other types of files from templates. Cheetah allows *search list extensions* to be defined, making it easy to extend WeeWX with new template tags. - Pure Python. The code base is 100% Python — no underlying C - libraries need be built to install WeeWX. This also means no + libraries need to be built in order to install WeeWX. This also means no Makefiles are needed. While WeeWX is nowhere near as fast at generating images and HTML as its @@ -80,10 +80,10 @@ MHz machine where generating the 9 images used in the *Current Conditions* page takes just under 2 seconds (compared with 0.4 seconds for wview). -All writes to the databases are protected by transactions. You can kill +Transactions protect all writes to the databases. You can kill the program at any time without fear of corrupting the databases. -The code makes ample use of exceptions to insure graceful recovery from +The code makes ample use of exceptions to ensure graceful recovery from problems such as network outages. It also monitors socket and console timeouts, restarting whatever it was working on several times before giving up. In the case of an unrecoverable console error (such as the @@ -112,7 +112,7 @@ a difference: 2. In the database. Either US or Metric can be used. -3. In the presentation (i.e., html and image files). +3. In the presentation (i.e., HTML and image files). The general strategy is that measurements are converted by service `StdConvert` as they come off the weather station into a target @@ -142,34 +142,32 @@ is used as the primary key in the SQL database. ## Time -WeeWX stores all data in UTC (roughly, *Greenwich* or *Zulu*) time. -However, usually one is interested in weather events in local time and -want image and HTML generation to reflect that. Furthermore, most -weather stations are configured in local time. This requires that many -data times be converted back and forth between UTC and local time. To -avoid tripping up over time zones and daylight savings time, WeeWX -generally uses Python routines to do this conversion. Nowhere in the -code base is there any explicit recognition of DST. Instead, its -presence is implicit in the conversions. At times, this can cause the -code to be relatively inefficient. +WeeWX stores all data in Unix epoch time (roughly, *UTC*, *Greenwich* or *Zulu*) +time. However, usually one is interested in weather events in local time and +wants image and HTML generation to reflect that. Furthermore, most weather +stations are configured in local time. This requires that many data times be +converted back and forth between Unix time and local time. To avoid tripping up +over time zones and daylight savings time, WeeWX generally uses Python routines +to do this conversion. Nowhere in the code base is there any explicit +recognition of DST. Instead, its presence is implicit in the conversions. At +times, this can cause the code to be relatively inefficient. -For example, if one wanted to plot something every 3 hours in UTC time, -it would be very simple: to get the next plot point, just add 10,800 to +For example, if one wanted to plot something every 3 hours in Unix time, +it would be very simple: to get the next plot point, add 10,800 to the epoch time: ``` next_ts = last_ts + 10800 ``` -But, if one wanted to plot something for every 3 hours *in local time* -(that is, at 0000, 0300, 0600, etc.), despite a possible DST change in -the middle, then things get a bit more complicated. One could modify the -above to recognize whether a DST transition occurs sometime between -`last_ts` and the next three hours and, if so, make the necessary -adjustments. This is generally what `wview` does. WeeWX takes a -different approach and converts from UTC to local, does the arithmetic, -then converts back. This is inefficient, but bulletproof against changes -in DST algorithms, etc.: +But if one wanted to plot something for every 3 hours *in local time* (that is, +at 0000, 0300, 0600, etc.), despite a possible DST change in the middle, then +things get a bit more complicated. One could modify the above to recognize +whether a DST transition occurs sometime between `last_ts` and the next three +hours and, if so, make the necessary adjustments. This is generally what `wview` +does. WeeWX takes a different approach and converts from UTC to local, does the +arithmetic, then converts back. This is inefficient, but bulletproof against +changes in DST algorithms, etc.: ``` time_dt = datetime.datetime.fromtimestamp(last_ts) @@ -188,9 +186,9 @@ times will probably be incorrect. ## Archive records An archive record's timestamp, whether in software or in the database, -represents the *end time* of the record. For example, a record -timestamped 05-Feb-2016 09:35, includes data from an instant after -09:30, through 09:35. Another way to think of it is that it is exclusive +represents the *end time* of the record. For example, with a 5-minute archive +time, a record timestamped 05-Feb-2016 09:35 includes data from an instant +after 09:30, through 09:35. Another way to think of it is that it is exclusive on the left, inclusive on the right. Schematically: ``` @@ -198,7 +196,7 @@ on the left, inclusive on the right. Schematically: ``` Database queries should reflect this. For example, to find the maximum -temperature for the hour between timestamps 1454691600 and 1454695200, +temperature for the hour between timestamps `1454691600` and `1454695200`, the query would be: ``` @@ -206,12 +204,12 @@ SELECT MAX(outTemp) FROM archive WHERE dateTime > 1454691600 and dateTime <= 1454695200; ``` -This ensures that the record at the beginning of the hour (1454691600) +This ensures that the record at the beginning of the hour (`1454691600`) does not get included (it belongs to the previous hour), while the -record at the end of the hour (1454695200) does. +record at the end of the hour (`1454695200`) does. -One must be constantly be aware of this convention when working with -timestamped data records. +One must be constantly aware of this convention when working with timestamped +data records. Better yet, if you need this kind of information, use an [xtypes](https://github.com/weewx/weewx/wiki/WeeWX-V4-user-defined-types) @@ -224,7 +222,7 @@ max_temp = weewx.xtypes.get_aggregate('outTemp', db_manager) ``` -It will not only make sure the limits of the query are correct, but will +It will not only make sure the limits of the query are correct but will also decide whether the daily summary optimization can be used ([details below](#daily-summaries)). If not, it will use the regular archive table. @@ -314,7 +312,7 @@ This is what the table columns mean: dateTime - The time of the start of day in unix epoch time. This is the primary key in the database. It must be unique, and it cannot be null. + The time of the start of day in Unix epoch time. This is the primary key in the database. It must be unique, and it cannot be null. min @@ -322,7 +320,7 @@ This is what the table columns mean: mintime - The time in unix epoch time of the minimum temperature. + The time in Unix epoch time of the minimum temperature. max @@ -330,7 +328,7 @@ This is what the table columns mean: maxtime - The time in unix epoch time of the maximum temperature. + The time in Unix epoch time of the maximum temperature. sum @@ -363,7 +361,7 @@ Now consider an extensive variable such as `rain`. The total rainfall for the day will be given by the field `sum`. So, calculating the total rainfall for the year can be done by scanning and summing only 365 records, instead of potentially tens, or even hundreds, -of thousands of records. This results in a dramatic speed up for report +of thousands of records. This results in a dramatic speed-up for report generation, particularly on slower machines such as the Raspberry Pi, working off an SD card. @@ -486,7 +484,7 @@ they are not used consistently. _ts first_ts - Variable is a timestamp in unix epoch time. + Variable is a timestamp in Unix epoch time. _dt @@ -581,7 +579,7 @@ To clean up after running tests: make test-clean ``` -To set up mysql server with user and permissions for testing: +To set up a MySQL server with the correct users and permissions for testing: ``` make test-setup ``` @@ -600,7 +598,7 @@ extensive and helpful. We generally follow Vincent Driessen's [branching model](http://nvie.com/posts/a-successful-git-branching-model/). Ignore -the complicated diagram at the beginning of the article, and just focus +the complicated diagram at the beginning of the article and focus on the text. In this model, there are two key branches: - 'master'. Fixes go into this branch. We tend to use fewer @@ -636,16 +634,16 @@ on this topic. ### Python -[JetBrain's PyCharm](http://www.jetbrains.com/pycharm/) is exellent, +[JetBrain's PyCharm](http://www.jetbrains.com/pycharm/) is excellent, and now there's a free Community Edition. It has many advanced features, yet is structured that you need not be exposed to them until you need them. Highly recommended. ### HTML and Javascript -For Javascript, [JetBrain's +For JavaScript, [JetBrain's WebStorm](http://www.jetbrains.com/webstorm/) is excellent, particularly -if you will be using a framework such as Node.js or Express.js. +if you are using a framework such as Node.js or Express.js. ## Glossary @@ -716,7 +714,7 @@ SQLite would be epoch time -Sometimes referred to as "unix time," or "unix epoch time." +Sometimes referred to as "Unix time," or "Unix epoch time." The number of seconds since the epoch, which is 1 Jan 1970 00:00:00 UTC. Hence, it always represents UTC (well... after adding a few leap seconds... but, close enough). This is the time used in the databases and appears as type @@ -770,7 +768,7 @@ A complete set of units used together. Either US, time stamp -A variable in unix epoch time. Always in UTC. Variables carrying a time stamp +A variable in Unix epoch time. Always in UTC. Variables carrying a time stamp usually have a suffix _ts.