IT Support » DMZ-WEB Infrastructure

*Chapter 7 of the dmz-web documentation · headings are numbered like the cheat sheets · index: [[Wiki]]*

{{>toc}}

## 7.1 What is up right now?

```bash

~/deploy/check-sites.sh                          # every site-* container: status + HTTP answer through NPM

podman ps --format 'table {{.Names}}\t{{.Status}}\t{{.Networks}}'

systemctl --user list-units 'net-*' 'db-*' 'site-*' 'nginx-proxy-manager*' --no-pager

systemctl --user --failed                        # anything that died

```

## 7.2 What is defined (even if not running)?

```bash

ls ~/.config/containers/systemd/                 # the live units = the list of sites

systemctl --user list-unit-files 'net-*' 'db-*' 'site-*' 'nginx-proxy-manager*'

podman ps -a                                     # containers incl. stopped/exited

podman volume ls ; podman network ls ; podman secret ls

/usr/libexec/podman/quadlet -dryrun -user 2>&1 | grep -iE 'error|warn'   # unit syntax problems

```

A site = `net-<name>.network` + `db-<name>.container` + `site-<name>.container`

(+ `.volume` files). If the files are there and `podman ps` doesn't show the

containers, the services are stopped or failed — see next.

## 7.3 Bringing a site back

Order matters (network → db → site); `Requires=`/`After=` do it for you when

you start the site unit, but starting them one by one shows where it fails:

```bash

systemctl --user start net-<name>-network.service

systemctl --user start db-<name>.service   && sleep 5 && podman logs --tail 20 db-<name>

systemctl --user start site-<name>.service && sleep 5 && podman logs --tail 20 site-<name>

systemctl --user status site-<name>.service --no-pager

journalctl --user -u site-<name>.service -e --no-pager | tail -30

```

Typical causes: image pull failed (no egress / registry down) → `podman pull <image>`;

secret missing → `podman secret ls` (a container whose `Secret=` is missing

won't start); volume removed → the DB initialises **empty** on next start, stop

immediately and restore from backup; `Restart=always` loop → `podman logs`.

## 7.4 NPM says 502 / site unreachable but container is up

```bash

podman inspect nginx-proxy-manager | jq '.[0].NetworkSettings.Networks | keys'   # must list net-<name>

podman network inspect net-<name> | jq '.[0].containers | map(.name)'            # db, site, nginx-proxy-manager

podman exec nginx-proxy-manager sh -c 'getent hosts site-<name>; curl -sI http://site-<name>/ | head -1'

```

NPM not in the network → the live `nginx-proxy-manager.container` lost the

`Network=` line (see [[NPM]]) → fix the unit, `cp`, `daemon-reload`,

`restart nginx-proxy-manager.service`. Name doesn't resolve inside NPM → NPM

joined the network after it started → same restart. Restarting NPM interrupts

**all** sites for ~10–20 s.

## 7.5 Snipe-IT shows HTTP 500 — how do I find out why?

The long `#67 /var/www/html/vendor/...` list is only the request path. The cause is

the `production.ERROR:` line above it:

```bash

podman exec site-snipeit sh -c 'grep -nE "production\.(ERROR|CRITICAL)" /var/www/html/storage/logs/laravel.log | cut -c1-400 | tail -8'

```

* `Email "${MAIL_ENV_FROM_ADDR}" does not comply with addr-spec of RFC 2822` → From/Reply-To

  were never set; the image's placeholder is not an address. Set `MAIL_FROM_ADDR` and

  `MAIL_REPLYTO_ADDR` in the unit (Snipe-IT chapter, "Mail"). It only breaks the email step;

  the record itself (e.g. the new user) is saved.

* `Table 'snipeit.password_resets' doesn't exist` from `auth:clear-resets`, repeating every

  15 minutes → the scheduler is running against a database whose tables were not created yet

  (pre-flight "Create database tables" not clicked). In this install the errors stopped once

  the tables existed. Check: `podman exec db-snipeit sh -c 'mariadb -uroot -p"$MYSQL_ROOT_PASSWORD" -e "show tables from snipeit like \"password%\""'`.

## 7.6 Snipe-IT mail: `451 4.4.4 ... ATTR5` from Exchange

Seen in the tinker test above when the recipient was external. It is **not** an authentication

or TLS problem and not a Snipe-IT setting: Exchange accepted the connection, then refused a

message to a recipient outside the tenant. First thing to establish is whether internal works:

send the test to `sysadmin@mdsdefense.com` (passes here). If internal passes and external

fails, look at the connector / relay permission in Exchange, not at the unit. If **internal

also** fails, compare the From address with Redmine's (Administration → Settings → Email

notifications → "Emission email address") and the host's egress IP with the connector's IP list

(`curl -s https://api.ipify.org` on the host).

## 7.7 Does it come back after a reboot?

Yes, by design, if three things hold:

1. **Lingering** for websvcmaster (the user manager starts at boot without a login):

   `loginctl show-user websvcmaster | grep Linger` → `Linger=yes` (root: `loginctl enable-linger websvcmaster`).

2. Every unit has `[Install] WantedBy=default.target` — Quadlet units are

   *enabled by that section*, not by `systemctl enable` (which doesn't work on

   generated units). Check: `grep -L 'WantedBy=default.target' ~/.config/containers/systemd/*.container`

   → must print nothing.

3. `Restart=always` in `[Service]` so a crash is retried.

Dry test without rebooting: `systemctl --user stop site-<name>.service db-<name>.service && systemctl --user start default.target`

→ both come back. Real test: reboot the host, then `check-sites.sh` after ~2 min

(MariaDB + Puma + Apache need a moment).

## 7.8 After a host reboot — checklist

```bash

loginctl show-user websvcmaster | grep Linger     # yes

systemctl --user --failed                         # empty

~/deploy/check-sites.sh                           # all 200/30x

firewall-cmd --list-all                           # (root) 81/444 + 8181 rich rules still there

```

## 7.9 Data safety reminders

* Secrets live only in `~/.local/share/containers/storage/secrets/` — back up

  that directory (or re-create the secrets from your password store); without

  `snipeit-app-key` the Snipe-IT DB is unreadable, without the DB secrets the

  containers won't start.

* Volumes: `podman volume export <vol> > file.tar` / `podman volume import <vol> file.tar`,

  plus DB dumps ([[WordPress]], [[Redmine]], [[Snipe-IT]]).

* `podman system reset` or removing `~/.local/share/containers` deletes

  **everything** above; never run it on this host.

---

_Source: `dmz-web-podman-cheatsheet.md` v1.2 in the `dmz-web-deploy` repo ([[Releases]]). Edit the wiki and the repo copy together._