fail2ban-webui/docs/operations.md

# Operations

All commands are run in the directory with `docker-compose.yaml` (e.g. `/opt/fail2ban-webui`).

## Status and logs

```bash
docker compose ps                              # Up … (healthy)
docker compose logs --tail 100 fail2ban-webui
docker compose logs -f fail2ban-webui | grep -E 'AUDIT|LOGIN|ERROR|WARNING'
docker compose run --rm fail2ban-webui check   # every agent reachable?
```

Log lines of interest:

| Line | Meaning |
|---|---|
| `fail2ban-webui 1.0.0 – hosts: … – auth: … – loki: … – geoip: …` | start summary |
| `AUDIT {…}` | ban/unban with user, client IP, hosts and result |
| `LOGIN_OK` / `LOGIN_FAILED` / `LOGIN_LOCKED` / `LOGOUT` | sign-in events |
| `OIDC_FAILED` / `OIDC_DENIED` | SSO errors / account not allowed |

The audit log is also kept in `data/audit.log` (one JSON object per line):

```bash
tail -n 20 data/audit.log | python3 -m json.tool --json-lines
```

On the hosts every agent action is in syslog:

```bash
journalctl -t f2b-webctl --since today
```

## Update

```bash
cp -a .env docker-compose.yaml config /backups/fail2ban-webui_$(date +%F)/   # or your backup location
docker compose pull
docker compose up -d
docker compose ps
docker compose run --rm fail2ban-webui check
```

Read the [changelog](../CHANGELOG.md) first – if it mentions a new agent version, update
the agents as well (re-run `install.sh` with the same options, see [agent.md](agent.md#update)).
The `1` tag only moves within the 1.x series; breaking changes get a new major version.

Rebuild/pull regularly even without new features: the image is rebuilt with current
Debian security updates.

## Backup

| What | Why |
|---|---|
| `.env` | settings, password hash, OIDC secret |
| `config/hosts.yaml` | host list |
| `data/ssh/` | private key and pinned host keys – without them every agent needs the new key |
| `data/audit.log` | audit trail |
| `data/secret_key` | only keeps sessions valid; losing it signs everybody out |

`data/geoip/` is downloaded again automatically.

```bash
tar -cpzf /backups/fail2ban-webui_$(date +%F_%H-%M).tar.gz .env docker-compose.yaml config data
sha256sum /backups/fail2ban-webui_*.tar.gz | tail -n1
```

The archive contains the private SSH key – store it accordingly.

## Moving to another host

1. `docker compose stop` on the old host, create the archive as above.
2. Copy it, unpack with `tar -xpzf` (keeps owner 10050 and permissions).
3. On **every agent** add the new address to `from=` – easiest by re-running
   `install.sh … --from <new address>`.
4. `docker compose up -d`, then `check`.

## Rotating the SSH key

```bash
docker compose run --rm fail2ban-webui keygen --force
# on every host:
bash install.sh --key-file new.pub --from … (same options as before)
```

The installer replaces only entries with the same key; remove the old one with
`uninstall.sh` first or delete its line from `authorized_keys` (it contains
`command="…f2b-webctl"`).

## Protecting the login with fail2ban

The UI locks a client after 5 failed logins itself. To ban persistent attackers on the
firewall, let fail2ban on the **proxy host** read the container log.

`/etc/fail2ban/filter.d/fail2ban-webui.conf`

```ini
[Definition]
# Docker json-file log: {"log":"… WARNING webui: LOGIN_FAILED user=x ip=203.0.113.7\n",…}
# user names never contain spaces (they are sanitised), so only the real " ip=" matches
failregex = LOGIN_FAILED user=\S* ip=<HOST>
            LOGIN_LOCKED ip=<HOST>
ignoreregex =
```

`/etc/fail2ban/jail.d/fail2ban-webui.local`

```ini
[fail2ban-webui]
enabled  = true
filter   = fail2ban-webui
backend  = polling
logpath  = /var/lib/docker/containers/*/*-json.log
maxretry = 5
findtime = 1h
bantime  = 1d
# the proxy runs in Docker? Then ban in the DOCKER-USER chain:
# banaction = iptables-allports[chain=DOCKER-USER]
```

```bash
fail2ban-client reload
fail2ban-regex /var/lib/docker/containers/*/*-json.log /etc/fail2ban/filter.d/fail2ban-webui.conf
```

This only works when the container sees the real client address (`TRUSTED_PROXY_HOPS`
set correctly). User names are logged with spaces and control characters replaced, so a
crafted user name cannot inject a second log line.

## Troubleshooting

| Symptom | Cause / fix |
|---|---|
| Container exits: `… is not writable for UID 10050` | `chown -R 10050:10050 data` |
| Container exits: `AUTH_MODE=local needs AUTH_PASSWORD_HASH` | run `hash-password`, paste the line with single quotes |
| Container exits: `No hosts configured` | `config/hosts.yaml` missing or empty; on Windows/Docker Desktop: the drive cannot be bind-mounted |
| Login always fails after pasting the hash | single quotes missing in `.env` – Compose ate the `$` parts |
| Everybody locked out together / audit shows the proxy IP | `TRUSTED_PROXY_HOPS` not set |
| `Host key verification failed` | host not in `known_hosts` or its key changed → `keyscan`, compare fingerprints |
| `Permission denied (publickey)` | key not installed for that user, wrong `user:` in `hosts.yaml`, wrong `from=` (the address the host sees!), or sshd does not read that `authorized_keys` file |
| `sudo: a password is required` | sudo rule missing → re-run `install.sh` |
| `The agent does not run as root` | forced command without `sudo -n` for a non-root user → re-run `install.sh` |
| `Timeout talking to …` | firewall/port, or the host is slow – raise `SSH_TIMEOUT` |
| `The jail manual-block does not exist on this host` | installed with `--no-jail` or with another `--manual-jail`; check `MANUAL_JAIL` in `/etc/f2b-webctl.conf` |
| Bans have no start/end time | fail2ban older than 0.11 (no `--with-time`) |
| Banned IP still reaches a Docker service | banaction blocks only `INPUT` – see [Docker hosts](agent.md#docker-hosts) |
| OIDC: `redirect_uri` mismatch | provider must allow exactly `<PUBLIC_URL>/auth/callback` |
| OIDC: `mismatching_state` | session cookie lost between redirect and callback: `PUBLIC_URL` host differs from the one in the browser, or `https` without `Secure`/proxy headers |
| OIDC: "account … is not allowed" | user not in `OIDC_ALLOWED_GROUPS`; check that the provider sends the `groups` claim |