weewx/DEV_NOTES.txt

checklist for doing a release:

1. Check weewx.conf for local changes.  In particular, check to make sure:
     1. WEEWX_ROOT points to /home/weewx
     2. debug=0 in weewx.conf
2. Make sure the version is correct
     1. weewx.__init__.py
     2. make version
3. Make sure all changes have been logged
     1. docs/changes.txt
     2. make deb-changelog
     3. make redhat-changelog
     4. make suse-changelog
     5. docs/upgrading.htm
4. Create the packages
     1. make src-package
     2. make debian-packages
     3. make redhat-packages
     4. make suse-packages
5. Sanity check on production machine: unpack the tarball, install, run
6. Tag it
     1. git tag -a v1.3.1 -m "Version 1.3.1 release"
7. Upload to weewx.com
     1. make upload-src
     2. make upload-debian
     3. make upload-redhat
     4. make upload-suse
     6. make upload-docs
8. Update the deb repository
     1. make pull-apt-repo
     2. make update-apt-repo
     3. make push-apt-repo
     4. ssh weewx.com rsync -arv /var/www/html/aptly-test/ /var/www/html/aptly
9. Update the redhat repository
     1. make pull-yum-repo
     2. make update-yum-repo
     3. make push-yum-repo
     4. ssh weewx.com rsync -arv /var/www/html/yum-test/ /var/www/html/yum
10. Update the suse repository
     1. make pull-suse-repo
     2. make update-suse-repo
     3. make push-suse-repo
     4. ssh weewx.com rsync -arv /var/www/html/suse-test/ /var/www/html/suse
11. Announce the release to the weewx user's group.



release process by command ----------------------------------------------------

# on any host, do a fresh checkout, set the version and update changelogs
git clone https://github.com/weewx/weewx.git
vi bin/weewx/__init__.py
make version
make deb-changelog
make redhat-changelog
make suse-changelog
vi docs/changes.txt        # as needed
vi docs/upgrading.htm      # as needed
git commit -a -m "release x.y.z"
git push

# build src, upload to development_versions
rm -rf build dist
make src-package upload-src

# on debian host, build debian packages, upload to development_versions
rm -rf build dist
make debian-packages upload-debian

# on redhat host, build the redhat packages, upload to development_versions
rm -rf build dist
make redhat-packages upload-redhat

# on suse host, build the suse packages, upload to development_versions
rm -rf build dist
make suse-packages upload-suse

# tag the release
git tag -a vX.Y.Z -m "tag release X.Y.Z"

# upload the docs
git clone https://github.com/weewx/website.git
make upload-docs

# move the new files into position
make release

# update the apt repository
make pull-apt-repo
make update-apt-repo (requires local installation of aptly)
make push-apt-repo

# update the yum repository
make pull-yum-repo
make update-yum-repo (requires local installation of createrepo)
make push-yum-repo

# update the suse repository
make pull-suse-repo
make update-suse-repo (requires local installation of createrepo)
make push-suse-repo


pre-requisites ----------------------------------------------------------------

to build debian package install the following (use 'apt-get install'):
  git
  rsync
  gpg
  debhelper
  python3-distutils

to build redhat package install the following (use 'yum install'):
  git
  rsync
  rpm-build
  rpm-sign

to build suse package install the following (use 'zypper install'):
  git
  rsync
  rpm-build


howto -------------------------------------------------------------------------

how to update the version number:
  perl -pi -e 's/__version__=".*"/__version__="x.y.z"/' bin/weewx/__init__.py
  make version                  # this propagates the version number everywhere
  git commit -a -m "bump to version x.y.z"

how to build source package:
  make src-package

how to build debian packages:
  make deb-changelog
  emacs pkg/debian/changelog    # add any package-specific changes, if any
  git commit -m "update deb changelog" pkg/debian/changelog
  make debian-packages

how to build redhat packages:
  make redhat-changelog
  emacs pkg/changelog.el       # add any package-specific changes, if any
  git commit -m "update redhat changelog" pkg/changelog.el
  make redhat-packages

how to build redhat packages:
  make suse-changelog
  emacs pkg/changelog.suse     # add any package-specific changes, if any
  git commit -m "update suse changelog" pkg/changelog.suse
  make suse-packages

to display debconf variables:
  sudo debconf-show weewx

to manually purge debconf variables:
  echo PURGE | sudo debconf-communicate weewx

to sign rpm packages you need .rpmmacros in your home directory:
~/.rpmmacros
  %_gpg_name  YOUR_NAME_HERE

to generate gpg key used for signing packages:
  gpg --gen-key
  gpg --list-keys

to export the text version of a public key:
  gpg --export -a "YOUR_NAME_HERE" > username.gpg.key

source install:
  python ./setup.py install
  python ./setup.py install home=/opt/weewx-x.y.z     (this does not work!)

debian install/remove:
  apt-get install weewx         # install with apt
  apt-get remove weewx          # remove with apt
  apt-get purge weewx           # purge removes /etc/weewx and debconf settings
  dpkg -i weewx_x.y.z-r.deb     # install
  (apt-get install weewx)       # finish install if dependencies failed
  dpkg -r weewx                 # remove
  dpkg -P weewx                 # purge

redhat install/remove:
  yum install weewx-x.y.z-r.rpm [--nogpgcheck]    # install with yum
  yum remove weewx                                # remove with yum
  rpm -i weewx-x.y.z-r.rpm                        # install with rpm directly
  rpm -e weewx                                    # remove with rpm

suse install/remove:
  zypper install weewx-x.y.z-r.rpm [--nogpgcheck] # install with zypper
  zypper remove weewx                             # remove with zypper
  rpm -i weewx-x.y.z-r.rpm                        # install with rpm directly
  rpm -e weewx                                    # remove with rpm


howto: deb repository ---------------------------------------------------------

aptly has two different mechanisms for doing a 'publish': switch or update.
we use snapshots, and publish using 'publish switch', rather than publishing
using a simple 'publish update'.

there are two apt repositories: python2 and python3

to do apt repo updates you must first install aptly:
  https://www.aptly.info/download/
for example, on debian:
  echo "deb http://repo.aptly.info/ squeeze main" | sudo tee /etc/apt/sources.list.d/aptly.list
  wget -qO - https://www.aptly.info/pubkey.txt | sudo apt-key add -
  sudo apt-get update
  sudo apt-get install aptly

create local debian repo using aptly:
  aptly repo create -distribution=squeeze -component=main -architectures=all python2-weewx
  aptly repo create -distribution=buster -component=main -architectures=all python3-weewx

put a bunch of deb files into an empty apt repo:
  for f in `ls distdir`; do aptly repo add python2-weewx distdir/$f; done

create a snapshot:
  aptly snapshot create python-weewx-x.y.z-n from repo python2-weewx
  aptly snapshot create python3-weewx-x.y.z-n from repo python3-weewx

publish using snapshot:
  aptly publish -architectures=all snapshot python-weewx-x.y.z-n python2
  aptly publish -architectures=all snapshot python3-weewx-x.y.z-n python3

update using 'publish switch':
  aptly repo add python2-weewx dist/python-weewx_x.y.z-n_all.deb
  aptly snapshot create python-weewx-x.y.z-n from repo python2-weewx
  aptly publish switch squeeze python2 python-weewx-x.y.z-n

  aptly repo add python3-weewx dist/python3-weewx_x.y.z-n_all.deb
  aptly snapshot create python3-weewx-x.y.z-n from repo python3-weewx
  aptly publish switch buster python3 python3-weewx-x.y.z-n

update using 'publish update':
  aptly publish repo -architectures=all python2-weewx squeeze
  aptly repo add python2-weewx dist/squeeze/python-weewx_x.y.z-n_all.deb
  aptly publish update squeeze python2

  aptly publish repo -architectures=all python3-weewx buster
  aptly repo add python3-weewx dist/buster/python3-weewx_x.y.z-n_all.deb
  aptly publish update buster python3

clone the published apt repo to local space:
  mkdir -p ~/.aptly
  rsync -arv USER@weewx.com:/var/www/html/aptly-test/ ~/.aptly

synchronize local aptly changes with the published apt repo:
  rsync -arv ~/.aptly/ USER@weewx.com:/var/www/html/aptly-test

switch from testing to production (this is done at weewx.com):
  rsync -arv /var/www/html/aptly-test/ /var/www/html/aptly

for clients to use an apt repo at weewx.com:
  curl -s http://weewx.com/keys.html | sudo apt-key add -
  echo "deb [arch=all] http://weewx.com/apt/ squeeze main" | sudo tee /etc/apt/sources.list.d/weewx.list
  echo "deb [arch=all] http://weewx.com/apt/ buster main" | sudo tee /etc/apt/sources.list.d/python3-weewx.list


howto: yum repository ---------------------------------------------------------

create yum repo:
  mkdir -p ~/.yum/weewx/{el7,el8}/RPMS

update local yum repo with latest rpm:
  cp *.el7.rpm ~/.yum/weewx/el7/RPMS
  createrepo -o ~/.yum/weewx/el7 ~/.yum/weewx/el7
  cp *.el8.rpm ~/yum/weewx/el8/RPMS
  createrepo -o ~/.yum/weewx/el8 ~/.yum/weewx/el8

clone the published yum repo to local space:
  mkdir -p ~/.yum
  rsync -arv USER@weewx.com:/var/www/html/yum-test/ ~/.yum

synchronize local yum changes with published yum repo:
  rsync -arv ~/.yum/ USER@weewx.com:/var/www/html/yum-test

switch from testing to production (this is done at weewx.com):
  rsync -arv /var/www/html/yum-test/ /var/www/html/yum


notes -------------------------------------------------------------------------

there are multiple changelogs:
  docs/changes.txt - definitive changelog for the application
  pkg/debian/changelog - changes to the debian packaging
  pkg/changelog.el - changes to the redhat packaging
  pkg/changelog.suse - changes to the suse packaging

when signing, gpg info must match the name and email in the latest package
changelog entry.

the debian changelog *must* have a version number that matches the app version.
the redhat package will build if the version numbers do not match.  use the
rpm-changelog and deb-changelog targets to ensure that changelog versions match
the application version for a release.

there are many ways to build a debian package.  first tried dpkg (uses DEBIAN
dir and is fairly low-level) but that does not create changes and source diffs.
then dried dpkg-buildpackage (uses debian dir and is higher level) but misses
the config and templates.  ended up using dpkg-buildpackage with some manual
(scripted) file manipulation.

both debian and redhat deprecate the use of '/usr/bin/env python' as the
shebang - they want a very tight binding to the operating system python as the
shebang.  in fact, since late 2019 rehat build tools see any shebang other than
a tight coupling to the operating system's python as an error and refuse to
accept it.  however, the bsd platforms prefer this as the shebang, and the
other approach fails on bsd.  macos uses something else.

weewx supports python 2.7 and any python 3.5+ so it should not be tightly
coupled to any specific python installation.

so use the source code should use the '/usr/bin/env python' shebang.  on
platforms that refuse to accept this, replace the package builder will replace
this with whatever that platform will accept.  the /etc/default/weewx plus
shell stubs in /usr/bin/wee* is used in platform installations to provide
python flexibility so that users do not have to install weewx multiple times
to experiment with different versions of python.  users who run weewx directly
from a git clone get all of the 'env python' goodness of being able to specify
whatever python interpreter they want.


unit tests --------------------------------------------------------------------

prerequisites:

python 2.7
python-usb
python-mock 0.6
pyephem

to set up mysql server with user and permissions for testing:

make test-setup

to run all unit tests:

make test
(note: do not run this as root)

to clean up after running tests:

make test-clean

guidelines:

unit tests should put transient files in /var/tmp/weewx_test


testing -----------------------------------------------------------------------

what to test when creating debian and redhat packages:
  install, upgrade, remove, purge
  install, modify files, remove
  install, modify files, upgrade

any linux:
- new install using setup.py to /home/weewx
  setup.py install

- upgrade using setup.py to /home/weewx
  setup.py install
  modify /home/weewx/weewx.conf
  setup.py install

- new install using setup.py to /opt/weewx
  setup.py install home=/opt/weewx

- upgrade using setup.py to /opt/weewx
  setup.py install home=/opt/weewx
  modify /opt/weewx/weewx.conf
  setup.py install home=/opt/weewx

on centos and suse:

- new install using rpm
  rpm -i weewx_x.y.z.rpm

- upgrade using rpm
  rpm -i weewx_r.s.t.rpm
  rpm -U weewx_x.y.z.rpm

- upgrade using rpm with extensions installed
  rpm -i weewx_r.s.t.rpm
  wee_extension --install cmon
  rpm -U weewx_x.y.z.rpm

debian:

- new install usinb dpkg
  dpkg -i weewx_x.y.z.deb

- upgrade using dpkg take maintainer's version of weewx.conf
  dpkg -i weewx_r.s.t.deb
  modify /etc/weewx/weewx.conf
  dpkg -i weewx_x.y.z.deb

- upgrade using dpkg use old version of weewx.conf
  dpkg -i weewx_r.s.t.deb
  modify /etc/weewx/weewx.conf
  dpkg -i weewx_x.y.z.deb

- reconfigure using dpkg
  dpkg-reconfigure weewx

all platforms:

- installation and removal of extensions
  wee_extension --install cmon
  wee_extension --install pmon
  wee_extension --remove cmon

- reconfigure using wee_config
  wee_config --reconfigure
  wee_config --reconfigure --driver=weewx.drivers.vantage

- list drivers
  wee_config --list-drivers