Operations » History » Revision 5
Revision 4 (Redmine Admin, 10/31/2026 07:01 PM) → Revision 5/6 (Redmine Admin, 10/31/2026 07:25 PM)
*Chapter 5 of the dmz-web documentation · headings are numbered like the cheat sheets · index: [[Wiki]]*
{{>toc}}
## 5.1 Services
```bash
systemctl --user daemon-reload # after editing any Quadlet file
systemctl --user list-units 'db-*' 'site-*' 'net-*' 'nginx-proxy-manager*'
systemctl --user status db-<name>.service site-<name>.service
systemctl --user restart db-<name>.service site-<name>.service
journalctl --user -u site-<name>.service -e
/usr/libexec/podman/quadlet -dryrun -user # validate Quadlet syntax
```
## 5.2 Listing / inspecting
```bash
podman ps -a
podman network ls ; podman volume ls ; podman secret ls
podman inspect <ctr> --format '{{.ImageName}}'
podman inspect <ctr> --format '{{range .Mounts}}{{.Name}} -> {{.Destination}}{{"\n"}}{{end}}'
podman inspect <ctr> | jq '.[0].Config.Env'
podman logs -f <ctr>
podman exec -it <ctr> bash # root shell inside, no password
```
## 5.3 Environment variables inside a container
```bash
podman exec <ctr> printenv
podman exec <ctr> printenv | grep -iE 'ROOT|PASSWORD|KEY'
podman exec <ctr> printenv MYSQL_ROOT_PASSWORD | od -c | tail -3 # reveals a stray trailing \n
```
## 5.4 Secrets
Injected as env at container start (`Secret=<secret>,type=env,target=VAR`).
MariaDB/PostgreSQL only read their `*_PASSWORD` env on **first**
initialisation of an empty data dir; afterwards the env var is inert and the
real password is whatever is in the DB.
```bash
podman secret inspect --showsecret <secret>
podman secret inspect --showsecret --format '{{.SecretData}}' <secret>
# always create WITHOUT a trailing newline
printf '%s' 'VALUE' | podman secret create <secret> -
printf '%s' "$(openssl rand -base64 32)" | podman secret create <secret> -
# replace: rm + create, then restart the consumers
podman secret rm <secret> && printf '%s' 'VALUE' | podman secret create <secret> -
```
## 5.5 Fixing a secret that has a trailing `\n` (legacy WordPress sites)
```bash
cat > ~/tmp/fixpw.sh <<'EOF'
ROOTPW="$(printf '%s' "$MYSQL_ROOT_PASSWORD")" # $(...) strips the trailing \n
WPPW="$(printf '%s' "$MYSQL_PASSWORD")"
mariadb -uroot -p"$MYSQL_ROOT_PASSWORD" <<SQL
ALTER USER 'root'@'localhost' IDENTIFIED BY '$ROOTPW';
ALTER USER 'root'@'%' IDENTIFIED BY '$ROOTPW';
ALTER USER 'wordpress'@'%' IDENTIFIED BY '$WPPW';
FLUSH PRIVILEGES;
SQL
EOF
podman exec -i db-<name> sh < ~/tmp/fixpw.sh
for s in <name>-db-root-password <name>-db-password; do
v="$(podman secret inspect --showsecret --format '{{.SecretData}}' "$s")"
podman secret rm "$s"; printf '%s' "$v" | podman secret create "$s" -
done
systemctl --user restart db-<name>.service site-<name>.service # run back to back with the SQL half
```
## 5.6 Database engines and version upgrades
Every stack has its own DB container; the engine and version are in the DB unit (`Image=`). Rule
for all of them: **dump first, change one `Image=` line in the repo copy, commit, copy the unit,
daemon-reload, restart, check** (§3.1.7 and §3.1.6, same procedure as the Snipe-IT image upgrade in §4.3). Which
of the two cases applies depends on the kind of jump.
| Engine | Patch / minor (same major) | Major jump |
|--------|----------------------------|------------|
| MariaDB (WordPress, Snipe-IT) | Change the tag or digest, restart. The data directory is kept; schema upgrades are applied by the server. Dump beforehand. | Never straight on the live volume. 1. Dump all databases. 2. Start the new version as a **scratch** container with an empty volume and restore the dump there. 3. Check the application against it (staging first). 4. Only then switch the real unit. Do not rely on `MARIADB_AUTO_UPGRADE` as the only safety net: it upgrades the system tables on start, but a failed or half-done upgrade cannot be undone without the dump. If you do keep the volume, run `mariadb-upgrade` inside the container once after the start and read its output. |
| PostgreSQL (Redmine) | Change the minor tag (e.g. `16.4` → `16.6`), restart. | The data directory format changes: **dump and restore** (`pg_dumpall` or `pg_dump -Fc` per database into a new empty volume under the new major version). Do not just change the tag; the container would refuse to start or, worse, you would be tempted to delete the volume. |
| TimescaleDB on PG 16 (PF) | The `timescaledb:<x.y.z>-pg16` tag can move to a newer extension version; after the restart run `ALTER EXTENSION timescaledb UPDATE;` in the PF database (in a fresh session) and check `\dx`. Dump first. | A PostgreSQL major (`-pg16` → `-pg17`) is the dump-and-restore case above, with the Timescale restore steps from `pf-mgmt-cheatsheet.md` (`timescaledb_pre_restore()` / `timescaledb_post_restore()`). |
A dump that was never restored is not a backup: restore it once into a scratch container and look
at the data before you depend on it. Keep the old image tag and the old volume until the new
version has run for a few days; then remove them.
## 5.7 One root shell for everything — `as-web` and `pf-upgrade.sh` (root)
Until now a job that needs both identities (root for `deploy-commit.sh`, websvcmaster for `podman` and `systemctl --user`) meant switching users back and forth. Two scripts keep you in the root shell and switch for you. Both are in the repo (`root-bin/`), installed to `/usr/local/sbin/` by `deploy-publish.sh` (root:root 0750) and both **ask before they change anything** they cannot undo by themselves.
**`as-web`** runs a command as websvcmaster with the rootless environment a console login would have (`HOME`, `XDG_RUNTIME_DIR=/run/user/<uid>`, the user bus), starting in websvcmaster's home:
```bash
as-web # interactive shell as websvcmaster; "exit" returns to root
as-web podman ps
as-web systemctl --user status site-pf.service
as-web bash -c 'cd ~ && podman exec db-pf pg_dump -U pf_user -Fc pf_db > ~/tmp/x.dump' # redirections need bash -c
```
It refuses when websvcmaster has no user manager running (`loginctl enable-linger websvcmaster`, §0).
**`pf-upgrade.sh <version>`** is the procedure of the PF sheet §8.3.1 in one go: pre-flight (tarball readable and a PF source, sha256 shown or checked with `-s`, version newer than the unit's, old tree name free) → backup → tarball into `~/deploy` → `apps/pf` kept as `apps/pf-<old>`, unpack, `podman build`, `Image=` in the unit → `deploy-commit.sh` → copy the unit, `daemon-reload`, restart **only** `site-pf`, wait for *healthy* → log tail and `check-sites.sh`. A failed build puts the old tree back and leaves the unit untouched.
```bash
pf-upgrade.sh -n 1.1.6 # dry run: all checks, every command printed, nothing changed
pf-upgrade.sh -s <sha256> 1.1.6 # tarball /root/pf-app-1.1.6.tar.gz (SFTP), stop unless the sha256 matches, ask before each step
pf-upgrade.sh -y -B 1.1.6 # no questions, no backup (only when the release says "no schema change")
```
Options: `-t <file>` other tarball path, `-f` same or older version (rebuild / rollback), `-h` help. Rollback: `pf-upgrade.sh -f -t <old tarball> <old version>`, or the manual steps of §8.3.1; when the release notes say the new version stored data the old one cannot read (splits, §8.5.5), restore the dump instead.
---
_Source: `dmz-web-podman-cheatsheet.md` v1.13 v1.8 in the `dmz-web-deploy` repo ([[Releases]]). Edit the wiki and the repo copy together._