Files
home-information/docs/dev/shared/environment-variables.md
Tony C 52999d9e99 Make docker compose the ongoing management surface (#382) (#394)
* Add docker compose templates and env-var drift check (#382)

Phase 1 of issue #382 (docker compose support):

Repo-root templates for users integrating HI into their own compose stack:
 - docker-compose.example.yml — published image, container_name: hi, $HOME/.hi
   volume defaults; no healthcheck stanza (Dockerfile's HEALTHCHECK applies)
 - local.env.example — generated from env-generate.py --example; preamble
   explicitly warns install.sh users not to drop it at ~/.hi/env/local.env

Drift prevention across the three env-var sources:
 - install.sh gains --list-env-vars, extracting names from its own heredoc via
   a unique terminator (INSTALL_ENV_FILE_EOF) so the listing cannot drift from
   what the script actually writes
 - env-generate.py gains a SETTING_SECTIONS canonical declaration that seeds
   self._settings_map and drives --example output; validate_settings() runs
   before _write_file() and fails the run on undeclared keys or unset values
 - deploy/env-drift-check.sh compares the three name sets and prints a clean
   labeled diff on mismatch
 - Wired into make env-drift-check, make check, and a CI step in
   django-tests.yml

install.sh SECRET_KEY charset narrowed to exclude characters that can confuse
docker compose's env_file parser (\", ', \\, \$, #, =, \`).

* Add docker compose path to install.sh and update.sh (#382)

Phase 2 of issue #382 (docker compose support):

install.sh:
 - check_docker_compose probes `docker compose version` (no install offer,
   no platform branching)
 - create_compose_file writes a fully-resolved compose file to
   ~/.hi/docker-compose.yml with container_name: hi so legacy
   `docker logs/stop/start hi` work identically across both code paths
 - Existing compose file is backed up to .BAK.<timestamp> before overwriting
   (protects hand-edits for reverse-proxy labels, custom networks, etc.)
 - start_container branches on HAS_COMPOSE: compose up -d when available,
   original docker run when not
 - show_success adds docker restart hi and the update.sh canonical update path

update.sh:
 - Same check_docker_compose pattern
 - update_via_compose runs `docker compose pull` then `up -d`
 - Branches when both compose is available AND ~/.hi/docker-compose.yml
   exists (pre-Phase-2 installs stay on the legacy recreation flow)

Container is named `hi` on both paths so post-install management commands
documented in `docs/Installation.md` work uniformly regardless of which
code path created the install.

* Split Installation.md into a simple-user doc and a Deployment.md (#382)

Phase 3 of issue #382 (docker compose support):

docs/Installation.md (290 → 123 lines):
 - Quick Installation → Next Steps (moved from the bottom) → Managing your
   installation → Updates → Environment Variable Changes → Removing your
   installation → Troubleshooting
 - "Managing your installation" fills the post-install management gap and
   uses only legacy `docker logs/stop/start/restart hi` commands, which work
   for both install-time code paths (the compose file is generated either
   way and container_name: hi is set on both paths)
 - Manual Installation section removed entirely; docs/dev/Setup.md already
   covers `make docker-build/run` for the from-source/developer audience
 - More Help points users at Deployment.md for advanced topics

docs/Deployment.md (new, 87 lines):
 - Network Access Configuration
 - Auto-Start on Reboot
 - User Management
 - Using docker compose directly (compose verbs as an equivalent alternative
   to legacy docker commands)
 - Integrating into your own compose stack (with the env-file format gotcha
   spelled out: no export, no shell quoting, no ${VAR} interpolation)
 - Pointer to the Integrations Guide

README.md:
 - "Need more control?" updated to point at both Installation.md and
   Deployment.md
 - Resources → Users list adds Deployment Options

* Code-review polish and add env-var ritual doc (#382)

deploy/env-generate.py:
 - validate_settings() moved from generate_env_file() into _write_file()
   so any future code path that writes the env file is guarded
 - Spacing fixes for project convention (inner spaces in update( { ... } ),
   sorted( extra ) / sorted( missing ))
 - "extra" error wording hints at typo in __init__ overlay as a possible
   cause, not just "add to SETTING_SECTIONS"
 - Drive-by colon spacing on the pre-existing HI_SUPPRESS_AUTHENTICATION
   overlay key

docs/dev/shared/environment-variables.md (new):
 - Documents the 4-place ritual when adding an env var dependency:
   EnvironmentSettings (server.py), SETTING_SECTIONS + value assignment
   (env-generate.py), install.sh heredoc, regenerated local.env.example
 - Explains what make env-drift-check covers (three sources) and
   what it deliberately does not (server.py — field names diverge by
   design; other os.environ.get callers — caught only by review)

src/hi/environment/server.py:
 - Pointer comment on EnvironmentSettings dataclass referencing the new doc

* Move dev init scripts into dev/ and make them self-locating

Two top-level scripts (init-env-dev.sh, init-claude.sh) carried hardcoded
personal paths that made them effectively unusable for other contributors,
and their visibility at the repo root suggested otherwise. Moved into
dev/ and rewritten to be portable:

dev/init-env-dev.sh:
 - Computes PROJ_ROOT via BASH_SOURCE so absolute paths to
   venv/bin/activate and .private/env/development.sh resolve regardless
   of the caller's working directory
 - Header comment now states the script must be sourced (the `return 1`
   failure paths only behave correctly when sourced)

dev/init-claude.sh:
 - Self-locates PROJ_ROOT the same way; the previous `cd ~/proj/hi`
   personal hardcode is gone
 - `export PATH="~/.local/bin:..."` → `export PATH="$HOME/.local/bin:..."`
   (tilde inside double quotes does not expand)
 - `gh auth status --hostname "$host"` → `--hostname github.com` (the
   prior `$host` was undefined)

Unrelated to issue #382; bundled on this branch for convenience.
2026-06-01 21:36:12 -05:00

3.4 KiB

Environment Variables

Adding (or removing, or renaming) an environment variable that the app reads touches four files. The cross-source drift check covers three of them automatically; the fourth — the app-side dataclass — relies on code review.

When to read this

Any time you add a new env-var dependency in app code: new integration credentials, a new feature flag, a new config knob. If the app reads it from os.environ (directly or via EnvironmentSettings), the ritual below applies.

Files to update

In recommended order:

  1. src/hi/environment/server.pyEnvironmentSettings dataclass. Add a field with type and default. Use = None for required vars (absence raises ImproperlyConfigured); use an empty/zero/false default for optional.

    Field names are intentionally not required to match env-var names — EnvironmentSettings.SECRET_KEY reads DJANGO_SECRET_KEY, EnvironmentSettings.REDIS_HOST reads HI_REDIS_HOST, etc. The mapping (prefix-stripping and any explicit rename) is handled inside EnvironmentSettings. Pick a field name that reads naturally in app code; the env-var name follows the project's HI_ / DJANGO_ convention.

  2. deploy/env-generate.py — two edits in the same file:

    • Add an entry to SETTING_SECTIONS under the appropriate section (or add a new section). Include a placeholder value and inline guidance on required vs. optional.
    • Assign a real value for the var, either in the __init__ overlay (self._settings_map.update(...)) for vars with a fixed default, or in generate_env_file() for vars filled in by interactive prompts.

    validate_settings() fails the run if a var is declared in SETTING_SECTIONS but never assigned (or vice versa), so this step is self-checking the moment you run make env-build.

  3. install.sh — the heredoc between << INSTALL_ENV_FILE_EOF and INSTALL_ENV_FILE_EOF (the unique terminator lets install.sh --list-env-vars extract names unambiguously). Add a KEY=value line. Use a literal default for fixed-value vars or ${SHELL_VAR} interpolation for generated values (e.g. DJANGO_SECRET_KEY=${DJANGO_SECRET_KEY}).

  4. local.env.example at the repo root — do not hand-edit. Regenerate from env-generate.py:

    python3 deploy/env-generate.py --example > local.env.example
    

Verification

After the four edits, run the drift check:

make env-drift-check

It compares the variable-name sets across install.sh, deploy/env-generate.py, and local.env.example and fails with a labeled diff on mismatch. Also wired into make check and the Check Env-Var Drift step in .github/workflows/django-tests.yml.

What drift-check does not cover

  • EnvironmentSettings (server.py) — field names diverge from env-var names by design, so a structural cross-check would either be lossy or require a rename map per field. Verifying the dataclass change matches the env-var change is a code-review responsibility.
  • Other code paths that read os.environ directly — app code should route env access through EnvironmentSettings, but nothing enforces it. If a contributor adds an os.environ.get('HI_SOMETHING_NEW') call elsewhere, only review catches it.

If a var reaches production without install.sh and the example file knowing about it, users won't have it set. The drift check is the first line; code review is the second.