getent hosts portal.bytek.ca`
Expected internal result:
- `portal.bytek.ca` resolves to `192.168.2.182`.
If the command returns no output, container DNS is failing.
Do not change OIDC client IDs or secrets before DNS is proven healthy.
---
## Browser DNS Over HTTPS
Firefox-based browsers, including Zen, can bypass Pi-hole through DNS over HTTPS.
This previously prevented access to:
- `pve.bytek.ca`
Pi-hole correctly resolved the private hostname, but Zen attempted to use protected public DNS instead.
Recommended setting:
- **DNS over HTTPS:** Off or Default Protection.
If a private hostname works by IP but not by name:
1. Check the hostname from the operating system.
2. Query Pi-hole directly.
3. Disable forced browser DNS over HTTPS.
4. Clear the operating-system DNS cache.
5. Clear the browser DNS cache.
6. Close idle browser sockets.
7. Restart the browser.
Useful Windows commands include:
- `Resolve-DnsName pve.bytek.ca`
- `nslookup pve.bytek.ca 192.168.2.65`
- `Clear-DnsClientCache`
- `ipconfig /flushdns`
---
## DNS Validation
### Test an Internal Application Record
Use:
`nslookup portal.bytek.ca 192.168.2.65`
Expected result:
- `portal.bytek.ca` resolves to `192.168.2.182`.
### Test Proxmox Resolution
Use:
`nslookup pve.bytek.ca 192.168.2.65`
Expected result:
- `pve.bytek.ca` resolves to `192.168.2.254`.
### Test Public Resolution
Use:
`nslookup google.com 192.168.2.65`
Expected result:
- One or more public addresses are returned.
### Test DNS Over TCP
Use:
`nc -zv -w 3 192.168.2.65 53`
### Test DNS Over UDP
Use:
`nc -zvu -w 3 192.168.2.65 53`
Both TCP and UDP DNS should be available to approved LAN clients.
---
## Service Health
Confirm that the Pi-hole LXC is running from Proxmox.
Confirm the guest address using:
`ip -br -4 addr`
Confirm DNS listeners using:
`ss -lntup | grep ':53'`
Confirm the Pi-hole DNS service using the appropriate service command for the installed Pi-hole version.
Review recent service logs if DNS queries fail.
---
## Management Access
Pi-hole management should remain private.
Approved access paths include:
- Home LAN.
- Trusted remote-access VPN.
- Proxmox console.
- SSH from an approved administrative source.
Do not route Pi-hole administration through:
- The public VPS.
- Traefik public HTTPS.
- Authentik ForwardAuth.
- A public WHC hostname.
- Direct Bell router port forwarding.
---
## Why Pi-hole Is Not Behind Authentik
Pi-hole is part of the dependency chain for internal Authentik and application resolution.
If Pi-hole management depended on Authentik, the following circular failure could occur:
1. Pi-hole DNS fails.
2. Authentik’s hostname stops resolving internally.
3. Authentik login fails.
4. Pi-hole administration is protected by Authentik.
5. The administrator cannot reach Pi-hole to repair DNS.
Pi-hole therefore uses local credentials and private network access.
---
## Upstream DNS
Pi-hole forwards non-local queries to configured upstream resolvers.
The chosen upstream resolver configuration should be documented after verification.
When changing upstream DNS:
1. Record the current configuration.
2. Change only one resolver setting at a time.
3. Test public DNS.
4. Test internal local records.
5. Test DNSSEC if enabled.
6. Test IPv4 and IPv6 behavior separately.
7. Confirm application containers still resolve external services.
Do not assume that an upstream failure means Pi-hole itself is unavailable.
---
## IPv6 Considerations
The Bytek homelab mainly uses IPv4 addressing.
If IPv6 is not intentionally configured end to end:
- Avoid publishing unintended AAAA records.
- Verify clients are not receiving an unusable IPv6 DNS path.
- Check browser connection behavior.
- Check whether applications prefer IPv6 before IPv4.
- Document any intentional IPv6 disabling.
An incorrect public AAAA record can cause clients to attempt an unavailable path before falling back to IPv4.
---
## Firewall Policy
Pi-hole must accept DNS from approved clients.
Typical requirements include:
- LAN clients to TCP 53.
- LAN clients to UDP 53.
- Approved Docker hosts to TCP and UDP 53.
- Future VPN clients to TCP and UDP 53.
- Administrative SSH from approved LAN sources.
- Private management access from the LAN.
Avoid exposing DNS publicly.
A publicly accessible recursive DNS resolver can be abused.
The Bell router must not forward public TCP or UDP port 53 to Pi-hole.
---
## Monitoring
Recommended Uptime Kuma monitors include:
| Monitor | Target |
|---|
| Pi-hole guest | Ping `192.168.2.65` |
| Pi-hole DNS UDP | `192.168.2.65:53` |
| Pi-hole DNS TCP | `192.168.2.65:53` |
| Internal Authentik record | `portal.bytek.ca` |
| Internal Proxmox record | `pve.bytek.ca` |
| Public DNS resolution | Selected external hostname |
- Guest unavailable.
- DNS service unavailable.
- Internal records unavailable.
- Public forwarding unavailable.
- Upstream DNS unavailable.
---
## Common Failure Scenarios
### Pi-hole Guest Is Down
Symptoms:
- Internal hostnames fail.
- Containers using Pi-hole cannot resolve names.
- Uptime Kuma reports several services down.
- Browser applications may fail internally.
Check:
- Proxmox guest status.
- LXC startup.
- Guest network address.
- Pi-hole service.
- Backup availability.
### Public DNS Fails but Local Records Work
Possible causes:
- Upstream resolver unavailable.
- Internet outage.
- DNSSEC issue.
- Firewall blocking outbound DNS.
- Incorrect upstream configuration.
### Local Records Fail but Public DNS Works
Possible causes:
- Local record deleted.
- Typographical error.
- Different Pi-hole instance answering.
- Client using another DNS server.
- Browser DNS over HTTPS.
- Cached stale record.
### VM Host Resolves but Container Does Not
Possible causes:
- Docker embedded DNS issue.
- Container started before Pi-hole.
- Explicit Docker DNS is missing.
- Docker network is unhealthy.
- Guest firewall blocks DNS.
Recovery may require recreating the container after Pi-hole is working.
### One Host Uses the Wrong Address
Possible causes:
- Client DNS cache.
- Browser DNS cache.
- Forced DNS over HTTPS.
- Public resolver used instead of Pi-hole.
- Hosts-file override.
- Multiple active network adapters.
- VPN DNS override.
---
## Power-Failure Recovery
After an electrical outage, Pi-hole should start before applications that depend on split DNS.
Recommended validation order:
1. Confirm Proxmox is running.
2. Confirm the Pi-hole LXC started.
3. Confirm Pi-hole owns `192.168.2.65`.
4. Confirm TCP and UDP port 53.
5. Confirm one local record.
6. Confirm one public hostname.
7. Start or validate Traefik.
8. Validate Authentik.
9. Restart affected containers whose DNS remained broken.
10. Confirm Uptime Kuma monitors recover.
Application containers may need to be recreated or restarted if Docker cached a broken DNS path during startup.
---
## Backup
The Pi-hole LXC is included in the Proxmox scheduled backup job.
The backup should preserve:
- Pi-hole configuration.
- Local DNS records.
- DHCP reservations.
- Adlists.
- Allow and deny lists.
- Custom DNS settings.
- Network configuration.
- Administrative configuration.
A Pi-hole configuration export may be maintained in addition to the Proxmox backup.
The export should be stored securely because it may contain internal network details.
---
## Recovery Procedure
If Pi-hole is unavailable:
1. Open Proxmox using `192.168.2.254` if private DNS is unavailable.
2. Confirm the Pi-hole LXC is running.
3. Open the LXC console.
4. Confirm the interface owns `192.168.2.65`.
5. Confirm the default route.
6. Confirm TCP and UDP port 53.
7. Confirm the Pi-hole DNS service.
8. Test a local record from inside the LXC.
9. Test a public record.
10. Review logs.
11. Restore from a Proxmox backup if the configuration is damaged.
If Pi-hole is restored under a temporary CT ID:
- Keep the restored container disconnected from the live LAN initially.
- Verify the configuration.
- Do not run two Pi-hole instances with the same IP.
- Shut down the failed instance before activating the restored one.
---
## Address-Change Procedure
Changing Pi-hole’s address would affect most of the environment.
Before changing `192.168.2.65`:
1. Document the current state.
2. Update the Bell router or DHCP DNS settings.
3. Update Docker Compose files with explicit DNS.
4. Update Uptime Kuma.
5. Update future VPN client DNS.
6. Update firewall rules.
7. Update service documentation.
8. Flush client DNS caches.
9. Recreate containers that use explicit Docker DNS.
10. Test all split-DNS applications.
Avoid changing Pi-hole’s address unless required.
---
## Security Checklist
- Pi-hole management is LAN or VPN only.
- Public TCP and UDP port 53 are not forwarded.
- Local administrator credentials are stored securely.
- SSH uses key-based authentication.
- DNS access is limited to approved networks.
- Local DNS records contain only intended hostnames.
- Proxmox does not depend on Pi-hole.
- Pi-hole is not behind Authentik.
- Backups include the Pi-hole LXC.
- A configuration export exists or is planned.
- Browser DNS over HTTPS behavior is documented.
- Upstream DNS settings are documented.
- Recovery by private IP has been tested.
---
## Validation Checklist
- Pi-hole LXC is running.
- Pi-hole owns `192.168.2.65`.
- TCP port 53 responds.
- UDP port 53 responds.
- Public names resolve.
- Internal hostnames resolve to Traefik.
- `pve.bytek.ca` resolves directly to Proxmox.
- No public WHC record exists for `pve.bytek.ca`.
- Traefik containers can use Pi-hole.
- Uptime Kuma can use Pi-hole.
- Nextcloud can resolve Authentik.
- BookStack can resolve Authentik.
- Vikunja can resolve Authentik.
- DNS over HTTPS is not bypassing Pi-hole on administrative browsers.
- Proxmox backup includes Pi-hole.
---
## Document Control
- **Owner:** Bryan Gagne-Plante
- **Private address:** `192.168.2.65`
- **Primary role:** LAN DNS and split DNS
- **Secondary role:** DHCP reservations
- **Guest type:** LXC
- **Guest ID:** Confirm current CT ID
- **Management scope:** LAN or trusted VPN
- **Upstream resolvers:** Confirm current configuration
- **Last verified:** YYYY-MM-DD
- **Last DNS recovery test:** YYYY-MM-DD
- **Last backup test:** YYYY-MM-DD
- **Last configuration export:** YYYY-MM-DD
- **Known limitations:** Internal application resolution depends on one Pi-hole instance
# Traefik
## Purpose
Traefik is the central HTTPS reverse proxy for the Bytek homelab.
Traefik receives HTTPS requests, selects the correct application according to the requested hostname, and forwards the request to the appropriate private backend.
Traefik provides:
- HTTPS termination.
- Hostname-based routing.
- Let’s Encrypt certificate issuance and renewal.
- ACME DNS-01 validation through the WHC cPanel API.
- Routing to private application backends.
- Authentik ForwardAuth for selected administrative services.
- A dashboard for routers, services, middlewares, and certificates.
- Consistent internal and external application URLs.
- Protection of private backend addresses from direct public exposure.
---
## Service Information
| Setting | Value |
|---|
| Service name | Traefik |
| Private IP address | `192.168.2.182` |
| HTTPS entry point | TCP 443 |
| Public dashboard hostname | `proxy.bytek.ca` |
| Deployment directory | `/opt/traefik` |
| Container DNS | `192.168.2.65` |
| Certificate authority | Let’s Encrypt |
| ACME challenge type | DNS-01 |
| DNS automation | WHC cPanel API |
| Normal dashboard authentication | Authentik ForwardAuth |
| Secondary dashboard authentication | Basic Auth while retained |
| Backup coverage | Proxmox scheduled backup |
| Proxmox VM ID | Confirm current VM ID |
---
## Architecture Role
Traefik is the common application-routing layer for both external and internal clients.
### External Request Path
1. WHC public DNS resolves the application hostname to the VPS at `38.29.213.101`.
2. The VPS receives HTTPS traffic.
3. The VPS forwards the traffic through WireGuard.
4. The home WireGuard gateway sends the request to Traefik at `192.168.2.182`.
5. Traefik examines the requested hostname.
6. Traefik forwards the request to the matching private backend.
### Internal Request Path
1. A LAN client queries Pi-hole.
2. Pi-hole resolves the application hostname to `192.168.2.182`.
3. The client connects directly to Traefik.
4. Traefik forwards the request to the matching private backend.
The same application hostname is used internally and externally.
---
## Application Routing Reference
| Hostname | Backend Destination | Authentication |
|---|
| `portal.bytek.ca` | `192.168.2.162:9000` | Authentik native authentication |
| `cloud.bytek.ca` | `192.168.2.100:11000` | Nextcloud native OIDC |
| `projects.bytek.ca` | `192.168.2.121:3456` | Vikunja native OIDC |
| `docs.bytek.ca` | BookStack private IP on TCP 6875 | BookStack native OIDC |
| `status.bytek.ca` | Uptime Kuma or Authentik embedded outpost | Authentik Proxy Provider |
| `proxy.bytek.ca` | Traefik internal dashboard service | Authentik ForwardAuth |
---
## Configuration Locations
| Purpose | Location |
|---|
| Docker Compose file | `/opt/traefik/compose.yml` |
| Static Traefik configuration | `/opt/traefik/traefik.yml` |
| Environment variables | `/opt/traefik/.env` |
| Dynamic configuration | `/opt/traefik/dynamic/` |
| Certificate state | `/opt/traefik/data/acme.json` |
| Traefik log | `/opt/traefik/logs/traefik.log` |
| Access log | Confirm current path |
| Additional persistent data | `/opt/traefik/data/` |
---
## Dynamic Configuration
Each application normally has its own dynamic configuration file.
Known files include:
| File | Purpose |
|---|
| `authentik.yml` | Authentik router and service |
| `authentik-forwardauth.yml` | Shared Authentik middleware and outpost service |
| `bookstack.yml` | BookStack router and service |
| `dashboard.yml` | Traefik dashboard router |
| `nextcloud.yml` | Nextcloud router and service |
| `uptime-kuma.yml` | Uptime Kuma router and service |
| `vikunja.yml` | Vikunja router and service |
1. Create a dated or descriptive backup copy.
2. Change only the required section.
3. Validate the YAML indentation.
4. Check the Traefik log.
5. Test the route locally.
6. Test the route from the LAN.
7. Test the route externally if the service is public.
8. Remove obsolete backup files after the change is documented.
---
## Static Configuration
The static Traefik configuration defines platform-level settings such as:
- Entry points.
- File-provider location.
- Dashboard availability.
- Logging.
- Access logging.
- Certificate resolvers.
- ACME settings.
- Trusted forwarded headers.
- Provider behavior.
Static configuration changes normally require a Traefik container restart or recreation.
Dynamic routing changes are usually detected automatically by the file provider when file watching is enabled.
---
## Docker DNS
The Traefik container explicitly uses Pi-hole at `192.168.2.65`.
This configuration was added after Docker’s embedded resolver failed to resolve the Let’s Encrypt ACME API following a power interruption.
The observed error reported that the Docker resolver at `127.0.0.11` was misbehaving.
Explicit Pi-hole DNS allows Traefik to resolve:
- Let’s Encrypt endpoints.
- WHC API endpoints.
- Internal Bytek hostnames.
- External Docker and operating-system dependencies.
Verify the configured Docker DNS using:
`sudo docker inspect --format='DNS={{json .HostConfig.Dns}}' traefik`
Expected result:
- `192.168.2.65` appears in the DNS list.
The container’s internal resolver file may still show Docker’s embedded resolver at `127.0.0.11`.
That is normal when Docker forwards requests to the explicitly configured upstream DNS server.
---
## Container Management
Move to the deployment directory before running Compose commands:
`cd /opt/traefik`
Check the stack:
`sudo docker compose ps`
View recent logs:
`sudo docker compose logs --tail 100 traefik`
Follow logs live:
`sudo docker compose logs -f traefik`
Validate the Compose file:
`sudo docker compose config --quiet`
Apply normal Compose changes:
`sudo docker compose up -d`
Recreate Traefik after changing container-level settings such as DNS:
`sudo docker compose up -d --force-recreate traefik`
Restart Traefik after a static configuration change:
`sudo docker compose restart traefik`
A normal restart does not apply changes to Docker-level settings such as container DNS, network mode, volume mounts, or environment variables. Those changes require container recreation.
---
## Naming and Routing Standard
Every Traefik application should have:
- A descriptive router name.
- A descriptive service name.
- A hostname rule.
- The `websecure` entry point.
- TLS enabled.
- The correct certificate resolver.
- A private backend destination.
- Host-header forwarding where required.
- Authentication middleware only when appropriate.
A route should not contain Authentik ForwardAuth if the application already uses native OIDC and requires direct API or client access.
---
## Applications That Must Not Use ForwardAuth
Do not attach Authentik ForwardAuth to:
- Nextcloud.
- Vikunja.
- BookStack.
- Authentik itself.
- Proxmox VE.
- Nextcloud AIO management.
- Plex.
- Game-server traffic.
### Nextcloud
Nextcloud mobile clients, desktop clients, WebDAV, CalDAV, CardDAV, and application authentication must reach Nextcloud directly.
Nextcloud uses native OIDC.
### Vikunja
Vikunja uses native Authentik OIDC.
### BookStack
BookStack uses native Authentik OIDC.
### Authentik
Authentik must never depend on its own ForwardAuth middleware.
### Proxmox VE
Proxmox uses native OIDC and remains outside Traefik.
### Plex
Plex clients use Plex authentication and Plex-specific APIs.
---
## Applications That Use Authentik Proxy Integration
### Traefik Dashboard
The Traefik dashboard uses single-application ForwardAuth.
The dashboard request is validated by the Authentik embedded outpost before access is granted.
### Uptime Kuma
Uptime Kuma uses an Authentik Proxy Provider because Uptime Kuma does not provide the same native OIDC integration as the other applications.
Selected status-page and monitoring endpoints may bypass authentication when public access is intentionally required.
---
## Traefik Dashboard Authentication
The dashboard is available at:
[Open the Traefik Dashboard](https://proxy.bytek.ca/)
Access is restricted by:
- Authentik group `bytek-admin`.
- Authentik policy `Allow bytek.ca users`.
- Policy engine mode `ALL`.
During the initial ForwardAuth validation, Traefik Basic Auth remains enabled as a second authentication layer.
The expected initial flow is:
1. The administrator opens `proxy.bytek.ca`.
2. Traefik invokes Authentik ForwardAuth.
3. Authentik validates the user and MFA.
4. Authentik checks the application bindings.
5. Traefik receives the authenticated request.
6. Traefik presents the existing Basic Auth challenge.
7. The administrator enters the Traefik Basic Auth credentials.
8. The dashboard opens.
Basic Auth may be removed only after ForwardAuth and recovery procedures are fully tested.
---
## Authentik ForwardAuth Middleware
The ForwardAuth middleware sends authentication requests to the Authentik embedded outpost.
The outpost backend is hosted by Authentik at `192.168.2.162` on TCP port 9000.
The middleware should trust the forwarded request headers and return Authentik identity headers to the protected application.
Typical identity headers include:
- Authentik username.
- Authentik group list.
- Authentik entitlements.
- Email address.
- Display name.
- User identifier.
- Provider metadata.
- Application metadata.
- Outpost metadata.
Do not log full authentication headers in a location accessible to untrusted users.
---
## Authentik Outpost Callback Route
The path under `/outpost.goauthentik.io/` must remain reachable without the ForwardAuth middleware.
If the callback path is protected by the same middleware, an authentication loop can occur.
The callback router should:
- Match `proxy.bytek.ca`.
- Match the `/outpost.goauthentik.io/` path prefix.
- Use the Authentik outpost service.
- Have a higher routing priority than the dashboard router.
- Use HTTPS.
- Avoid the dashboard ForwardAuth middleware.
Test the outpost with:
`curl -k -I https://proxy.bytek.ca/outpost.goauthentik.io/ping`
Expected result:
- HTTP 204.
A normal browser link is not useful for this endpoint because an HTTP 204 response intentionally contains no page content.
---
## Certificate Management
Traefik obtains HTTPS certificates from Let’s Encrypt.
The current design uses:
- ACME.
- DNS-01 validation.
- WHC cPanel API.
- Certificate resolver named `letsencrypt`.
Each public router should reference the correct certificate resolver.
A router without the certificate resolver may use the Traefik default certificate instead of requesting the intended hostname certificate.
---
## ACME DNS-01 Flow
Certificate issuance follows this sequence:
1. A Traefik router requests a certificate.
2. Traefik contacts the Let’s Encrypt ACME API.
3. Traefik creates the required DNS challenge through the WHC API.
4. Let’s Encrypt validates the challenge.
5. Traefik receives the certificate.
6. Traefik stores the certificate state in `acme.json`.
7. Traefik serves the certificate for the matching hostname.
Traefik must have working DNS before this process can succeed.
---
## Certificate Storage
Certificate state is stored in:
`/opt/traefik/data/acme.json`
The file must:
- Remain persistent across container recreation.
- Have restrictive permissions.
- Be included in backups.
- Never be committed to a public repository.
- Never be pasted into BookStack.
- Never be deleted during routine troubleshooting.
Deleting `acme.json` can remove all stored certificates and account state.
That may trigger unnecessary reissuance and certificate-rate-limit risk.
---
## Certificate Validation
Inspect a certificate served by Traefik using:
`openssl s_client -connect 127.0.0.1:443 -servername projects.bytek.ca /dev/null | openssl x509 -noout -subject -issuer -dates -ext subjectAltName`
Verify:
- The subject or Subject Alternative Name contains the requested hostname.
- The issuer is Let’s Encrypt.
- The certificate is currently valid.
- The expiration date is reasonable.
- Traefik is not serving the default certificate.
Repeat with the relevant hostname when validating another service.
---
## Certificate Troubleshooting
### Certificate Missing From the Dashboard
Check:
1. The dynamic router exists.
2. The hostname rule is correct.
3. TLS is enabled.
4. The certificate resolver is named correctly.
5. Traefik DNS works.
6. The WHC API credentials work.
7. The ACME log contains the hostname.
8. The DNS challenge was validated.
### ACME DNS Failure
A previously observed error reported:
- Failure to contact the Let’s Encrypt API.
- DNS lookup through `127.0.0.11`.
- Docker DNS server misbehaving.
The correction was to configure the Traefik container to use Pi-hole explicitly and recreate the container.
### Wrong Certificate Served
Possible causes include:
- Router not loaded.
- Hostname typo.
- Wrong certificate resolver.
- Duplicate router rule.
- Default certificate served.
- Client or browser cache.
- DNS points to the wrong server.
---
## Application Onboarding
When adding a new public application:
1. Confirm the application backend works directly.
2. Reserve the VM or LXC IP in Pi-hole.
3. Add the public WHC A record pointing to `38.29.213.101`.
4. Add the Pi-hole local DNS record pointing to `192.168.2.182`.
5. Create a dynamic Traefik configuration file.
6. Add the hostname router.
7. Add the private backend service.
8. Enable TLS with the correct resolver.
9. Validate YAML formatting.
10. Confirm Traefik loaded the router.
11. Confirm certificate issuance.
12. Test from the Traefik VM.
13. Test from the home LAN.
14. Test from an external network.
15. Add Uptime Kuma monitors.
16. Update BookStack documentation.
---
## Backend Validation
Before troubleshooting an application through Traefik, test the direct backend from the Traefik VM.
Examples:
### Nextcloud
`nc -zv -w 5 192.168.2.100 11000`
### Vikunja
`curl -i http://192.168.2.121:3456/health`
### Authentik
`curl -I http://192.168.2.162:9000/-/health/ready/`
### BookStack
Use the confirmed BookStack IP on TCP port 6875.
A direct backend failure must be resolved before changing the Traefik router.
---
## Local Router Validation
Traefik can be tested locally without relying on public DNS.
Example for Vikunja:
`curl -k --resolve projects.bytek.ca:443:127.0.0.1 https://projects.bytek.ca/health`
This forces the requested hostname to the local Traefik listener.
A successful response confirms:
- The router loaded.
- TLS negotiation works.
- Traefik selected the expected service.
- The backend is reachable.
- The application responded.
---
## Logging
Review the main Traefik log using:
`sudo tail -n 100 /opt/traefik/logs/traefik.log`
Filter common issues using:
`sudo grep -iE 'error|acme|certificate|middleware|service|router|dns' /opt/traefik/logs/traefik.log | tail -n 100`
Look for:
- YAML parsing errors.
- Unknown fields.
- Missing services.
- Missing middlewares.
- Incorrect certificate resolver.
- ACME validation failures.
- DNS resolution failures.
- Backend connection failures.
- Router conflicts.
If access logging is enabled, use it to distinguish:
- Requests reaching Traefik.
- Router selection.
- Backend status.
- HTTP response code.
- Client source.
- Authentication redirects.
Do not copy session cookies, bearer tokens, or authorization headers into BookStack.
---
## YAML Validation
YAML indentation must use spaces, not tabs.
Check dynamic configuration files using:
`sudo grep -RnP '\t' /opt/traefik/dynamic/`
Expected result:
- No output.
Review a specific file using:
`sudo cat /opt/traefik/dynamic/.yml`
Common YAML problems include:
- Incorrect indentation.
- Tabs.
- Misspelled field names.
- Router placed under the wrong section.
- Service placed under the wrong section.
- Incorrect middleware name.
- Incorrect backend URL.
- Old IP address.
- Duplicate router name.
---
## IP Address Changes
If an application IP changes:
1. Confirm the new reservation in Pi-hole.
2. Confirm the VM owns the new address.
3. Update the Traefik backend.
4. Search `/opt/traefik` for the old address.
5. Update Proxmox firewall rules.
6. Update Authentik internal hosts if applicable.
7. Update Uptime Kuma monitors.
8. Test the direct backend.
9. Test the local Traefik route.
10. Test the public route.
11. Update BookStack documentation.
A stale backend address causes gateway errors even when the application itself is healthy.
---
## Firewall Requirements
The Traefik VM must accept:
- TCP 443 from the home LAN.
- TCP 443 from the WireGuard gateway.
- Administrative SSH from approved LAN sources.
- Monitoring traffic where required.
Application VM firewalls must permit Traefik to reach their backend ports.
Known requirements include:
| Destination | Source | Port |
|---|
| Nextcloud | `192.168.2.182` | TCP 11000 |
| Vikunja | `192.168.2.182` | TCP 3456 |
| Authentik | `192.168.2.182` | TCP 9000 |
| BookStack | `192.168.2.182` | TCP 6875 |
| Uptime Kuma direct route, if retained | `192.168.2.182` | TCP 3001 |
---
## Monitoring
Recommended Uptime Kuma monitors include:
| Monitor | Target |
|---|
| Traefik VM | Ping `192.168.2.182` |
| Traefik HTTPS | TCP 443 |
| Authentik route | `portal.bytek.ca` readiness endpoint |
| Nextcloud route | Nextcloud status endpoint |
| Vikunja route | Vikunja health endpoint |
| BookStack route | BookStack login endpoint |
| Certificate expiry | Each important public hostname |
| Public VPS to Traefik path | External application monitor |
- Traefik VM unavailable.
- TCP 443 unavailable.
- Router unavailable.
- Certificate failure.
- Backend unavailable.
- Authentik middleware failure.
- VPS or WireGuard path failure.
---
## Power-Failure Recovery
After a power interruption:
1. Confirm Pi-hole is running.
2. Confirm the WireGuard gateway is running.
3. Confirm Authentik is ready.
4. Confirm the Traefik VM owns `192.168.2.182`.
5. Confirm Docker is active.
6. Confirm the Traefik container is running.
7. Confirm the container uses Pi-hole DNS.
8. Review Traefik logs for DNS errors.
9. Confirm routers loaded.
10. Confirm certificates are present.
11. Test Authentik locally.
12. Test an application locally.
13. Test a public application externally.
14. Restart or recreate Traefik only if required.
If Traefik started before Pi-hole and DNS remains broken, recreate the Traefik container after Pi-hole is healthy.
---
## Backup
The Traefik VM is included in the Proxmox scheduled backup job.
Protect the following paths:
- `/opt/traefik/compose.yml`
- `/opt/traefik/traefik.yml`
- `/opt/traefik/.env`
- `/opt/traefik/dynamic/`
- `/opt/traefik/data/acme.json`
- Relevant logs.
- Any authentication files.
- Any custom certificate files.
- Any dashboard Basic Auth configuration.
Secrets must remain in a password manager or encrypted backup.
BookStack should document the secret location, not the secret itself.
---
## Recovery Procedure
If Traefik configuration is broken:
1. SSH into the Traefik VM.
2. Confirm Docker is active.
3. Confirm the Traefik container state.
4. Review the most recent log entries.
5. Identify the most recently changed dynamic file.
6. Restore the known-good backup copy.
7. Confirm the file provider reloads the configuration.
8. Test the affected router locally.
9. Restart Traefik only if the static configuration changed or reload failed.
10. Confirm certificates remain present.
11. Test from the LAN.
12. Test externally.
If the VM must be restored:
1. Restore the most recent Proxmox backup under a temporary VM ID.
2. Disconnect the restored VM network adapter.
3. Verify `/opt/traefik`.
4. Verify `acme.json`.
5. Verify environment and dynamic configuration.
6. Shut down the failed production VM.
7. Assign the expected network identity.
8. Start the restored VM.
9. Confirm `192.168.2.182`.
10. Confirm Docker and Traefik.
11. Test all routers.
12. Reconnect public traffic only after validation.
---
## Security Checklist
- Traefik TCP 443 is the only intended public application entry point.
- Private backend ports are not publicly exposed.
- The Traefik dashboard requires `bytek-admin`.
- The dashboard callback path bypasses ForwardAuth.
- Authentik is not protected by its own ForwardAuth.
- Nextcloud uses native OIDC.
- Vikunja uses native OIDC.
- BookStack uses native OIDC.
- Certificate API credentials are protected.
- `acme.json` has restrictive permissions.
- Container DNS points to Pi-hole.
- The Docker socket is mounted read-only where supported.
- The Docker API is not publicly exposed.
- SSH uses keys.
- Dynamic configuration is backed up.
- Basic Auth recovery credentials are stored securely.
- Proxmox backup includes the Traefik VM.
---
## Validation Checklist
- Traefik VM owns `192.168.2.182`.
- Docker is active.
- Traefik container is running.
- TCP 443 is listening.
- Pi-hole DNS works inside the container.
- Dynamic YAML contains no tabs.
- Authentik router works.
- Nextcloud router works.
- Vikunja router works.
- BookStack router works.
- Uptime Kuma router works.
- Dashboard authentication works.
- Outpost ping returns HTTP 204.
- Certificates match every public hostname.
- Certificate renewal works.
- Public ingress works through the VPS and WireGuard.
- Internal split-DNS access works directly.
- `acme.json` is included in backup.
- A known-good dynamic configuration backup exists.
---
## Document Control
- **Owner:** Bryan Gagne-Plante
- **Private address:** `192.168.2.182`
- **Public dashboard:** `proxy.bytek.ca`
- **Primary entry point:** TCP 443
- **Deployment directory:** `/opt/traefik`
- **Certificate authority:** Let’s Encrypt
- **ACME challenge:** DNS-01 through WHC cPanel API
- **Container DNS:** `192.168.2.65`
- **Dashboard authentication:** Authentik ForwardAuth
- **Secondary authentication:** Basic Auth while retained
- **Proxmox VM ID:** Confirm current VM ID
- **Last verified:** YYYY-MM-DD
- **Last certificate-renewal test:** YYYY-MM-DD
- **Last ForwardAuth test:** YYYY-MM-DD
- **Last restore test:** YYYY-MM-DD
- **Known limitation:** Public applications depend on one Traefik VM and one certificate store
# Authentik
## Purpose
Authentik provides centralized identity and access management for the Bytek homelab.
Authentik allows users to access multiple applications with one account while applying consistent authentication, multi-factor authentication, group membership, and access policies.
Authentik provides:
- Central user accounts.
- Multi-factor authentication.
- OpenID Connect.
- OAuth2 providers.
- Proxy Providers.
- ForwardAuth.
- Group-based application access.
- Email-domain policies.
- Application entitlements.
- An embedded proxy outpost.
- Central application launch links.
- Administrative auditing and event records.
---
## Service Information
| Setting | Value |
|---|
| Service name | Authentik |
| Private IP address | `192.168.2.162` |
| Backend port | TCP 9000 |
| Public hostname | `portal.bytek.ca` |
| Public access | Traefik |
| Deployment directory | `/opt/authentik` |
| Database | PostgreSQL |
| Supporting service | Redis |
| Proxy component | Embedded outpost |
| Normal user group | `bytek-users` |
| Administrator group | `bytek-admin` |
| Backup coverage | Proxmox scheduled backup |
| Proxmox VM ID | Confirm current VM ID |
---
## Architecture Role
Authentik is the trusted identity provider for the Bytek environment.
Applications rely on Authentik to answer these questions:
- Who is the user?
- Has the user completed authentication?
- Has the user completed MFA?
- Is the user a member of the required group?
- Does the user satisfy the application policy?
- Should the user be allowed to continue to the application?
Authentik does not normally replace application-specific permissions.
For example:
- Authentik authenticates a Proxmox user, but Proxmox controls the user’s ACL.
- Authentik authenticates a BookStack user, but BookStack controls the user’s roles.
- Authentik authenticates a Nextcloud user, but Nextcloud controls shares and application permissions.
- Authentik authenticates a Vikunja user, but Vikunja controls projects and task permissions.
---
## Request Flow
A normal OIDC login follows this process:
1. The user opens an application.
2. The application redirects the browser to `portal.bytek.ca`.
3. Traefik forwards the request to Authentik at `192.168.2.162:9000`.
4. Authentik identifies the requested application.
5. Authentik requests credentials if no valid session exists.
6. Authentik requests MFA according to the authentication flow.
7. Authentik evaluates the application’s group and policy bindings.
8. Authentik issues a signed authorization response.
9. Authentik redirects the browser to the application callback.
10. The application validates the response and creates or matches the local account.
11. The application applies its own permissions.
---
## Core Components
### Authentik Server
The Authentik server provides:
- The web interface.
- Login and authorization flows.
- OAuth2 and OIDC endpoints.
- Provider endpoints.
- Administrative interfaces.
- API services.
- User-facing application portal.
### Authentik Worker
The worker performs background tasks such as:
- Scheduled processing.
- Event handling.
- Email-related tasks.
- Outpost-related processing.
- Provider maintenance.
- Background synchronization.
### PostgreSQL
PostgreSQL stores Authentik configuration and identity data.
PostgreSQL contains critical information including:
- Users.
- Groups.
- Policies.
- Applications.
- Providers.
- Flows.
- Stages.
- Events.
- Outpost configuration.
- Authentication settings.
PostgreSQL is the most important Authentik data component to protect.
### Redis
Redis supports Authentik’s runtime and task-processing requirements.
Redis is required for normal stack operation but PostgreSQL remains the authoritative persistent data source.
### Embedded Outpost
The embedded outpost provides:
- ForwardAuth.
- Proxy Provider routing.
- Authentication callbacks.
- Identity-header forwarding.
- Access enforcement for applications without native OIDC.
---
## Application Portal
The Authentik user portal displays applications available to the signed-in user.
The portal should show only applications that the user is allowed to access.
Normal users may see:
- Nextcloud.
- Vikunja.
- BookStack.
Administrators may additionally see:
- Proxmox VE.
- Traefik dashboard.
- Uptime Kuma.
Application visibility is controlled by:
- Group bindings.
- Policy bindings.
- Policy engine mode.
- Application launch URL.
- User authentication state.
---
## Group Model
### bytek-users
The `bytek-users` group grants access to normal user-facing applications.
Applications using this group include:
- Nextcloud.
- Vikunja.
- BookStack.
Users should be added to `bytek-users` only when access to normal Bytek applications is required.
### bytek-admin
The `bytek-admin` group grants access to administrative applications.
Applications using this group include:
- Proxmox VE.
- Traefik dashboard.
- Uptime Kuma dashboard.
Membership in `bytek-admin` must be limited to trusted infrastructure administrators.
### Administrator Membership
An administrator may belong to both groups:
- `bytek-users`
- `bytek-admin`
This allows access to both normal applications and administrative interfaces.
---
## Shared Email-Domain Policy
Applications also use the policy:
- `Allow bytek.ca users`
The policy validates the user according to the configured Bytek email-domain rules.
The application policy engine is normally set to `ALL`.
For a normal application, access requires:
- Membership in `bytek-users`.
- Successful evaluation of `Allow bytek.ca users`.
For an administrative application, access requires:
- Membership in `bytek-admin`.
- Successful evaluation of `Allow bytek.ca users`.
---
## Policy Engine Mode
Authentik applications normally use policy engine mode `ALL`.
With `ALL`, every enabled binding must pass.
A normal application should therefore have:
| Binding Type | Binding |
|---|
| Group | `bytek-users` |
| Policy | `Allow bytek.ca users` |
| Binding Type | Binding |
|---|
| Group | `bytek-admin` |
| Policy | `Allow bytek.ca users` |
---
## Application Integration Matrix
| Application | Integration Type | Required Group |
|---|
| Nextcloud | Native OIDC | `bytek-users` |
| Vikunja | Native OIDC | `bytek-users` |
| BookStack | Native OIDC | `bytek-users` |
| Proxmox VE | Native OIDC | `bytek-admin` |
| Traefik dashboard | Single-application ForwardAuth | `bytek-admin` |
| Uptime Kuma | Proxy Provider | `bytek-admin` |
---
| Setting | Value |
|---|
| Application slug | `nextcloud` |
| Integration | OpenID Connect user backend |
| Callback | `https://cloud.bytek.ca/apps/user_oidc/code` |
| Subject mode | User UUID |
| Scopes | `openid profile email` |
| Required group | `bytek-users` |
| Setting | Value |
|---|
| Application slug | `vikunja` |
| Integration | Native OIDC |
| Callback | `https://projects.bytek.ca/auth/openid/authentik` |
| Subject mode | User UUID |
| Scopes | `openid profile email` |
| Required group | `bytek-users` |
| Setting | Value |
|---|
| Application slug | `bookstack` |
| Integration | Native OIDC |
| Callback | `https://docs.bytek.ca/oidc/callback` |
| Subject mode | User UUID |
| Required group | `bytek-users` |
| Signing | RSA signing key |
| Token encryption | Disabled |
| Setting | Value |
|---|
| Application slug | `proxmox` |
| Integration | Native OIDC |
| Callback | `https://pve.bytek.ca:8006` |
| Subject mode | User UUID |
| Required group | `bytek-admin` |
| Selected scopes | `openid`, `email`, and `profile` |
---
## OIDC Provider Standard
New OIDC providers should normally use:
| Provider Setting | Standard Value |
|---|
| Client type | Confidential |
| Subject mode | Based on the user’s UUID |
| Authorization flow | Default implicit-consent provider flow |
| Signing key | Selected |
| Encryption key | None |
| Redirect type | Strict Authorization |
| Policy engine | ALL |
---
## OIDC Scope Mappings
Providers must include the scopes required by the application.
Common scopes include:
- `openid`
- `email`
- `profile`
The mapping names may appear as Authentik default OAuth scope mappings.
Missing scope mappings can allow the initial browser authorization to complete while causing the application to fail later when requesting profile or userinfo data.
### Proxmox Scope Issue
Proxmox initially returned an authentication failure because the Authentik provider did not have the expected scopes selected.
The detailed Proxmox log reported a failure while contacting the userinfo endpoint.
Selecting the required mappings resolved the issue.
### Scope Review Procedure
When an OIDC application fails:
1. Open the Authentik provider.
2. Review selected scope mappings.
3. Confirm `openid`.
4. Confirm `email`.
5. Confirm `profile`.
6. Review any application-specific mappings.
7. Save the provider.
8. Retry in a completely new private browser session.
9. Review the application log.
---
## Redirect URI Requirements
Redirect URIs must match the application exactly.
A mismatch can cause:
- Invalid redirect errors.
- Authorization rejection.
- Login loops.
- Callback failures.
- Token validation failures.
### Known Redirect URIs
| Application | Strict Authorization URI |
|---|
| Nextcloud | `https://cloud.bytek.ca/apps/user_oidc/code` |
| Vikunja | `https://projects.bytek.ca/auth/openid/authentik` |
| BookStack | `https://docs.bytek.ca/oidc/callback` |
| Proxmox VE | `https://pve.bytek.ca:8006` |
---
## Post-Logout Redirects
Where supported, providers should include approved post-logout destinations.
BookStack uses destinations such as:
- BookStack home.
- BookStack login page.
- BookStack login page with automatic OIDC initiation prevented.
Post-logout redirects must use the actual production hostname.
Do not use obsolete provisional hostnames such as `wiki.bytek.ca`.
---
## ForwardAuth Applications
### Traefik Dashboard
The Traefik dashboard uses a Proxy Provider configured for single-application ForwardAuth.
| Setting | Value |
|---|
| Application slug | `traefik-dashboard` |
| External host | `https://proxy.bytek.ca` |
| Required group | `bytek-admin` |
| Shared policy | `Allow bytek.ca users` |
| Outpost | Embedded outpost |
| Setting | Value |
|---|
| Application slug | `uptime-kuma` |
| External host | `https://status.bytek.ca` |
| Internal host | `http://192.168.2.115:3001` |
| Required group | `bytek-admin` |
| Shared policy | `Allow bytek.ca users` |
| Outpost | Embedded outpost |
---
## Embedded Outpost
The embedded outpost handles Authentik proxy integrations.
Applications assigned to the embedded outpost include:
- Traefik dashboard.
- Uptime Kuma.
The outpost must be able to reach each protected application backend.
For Uptime Kuma, Authentik at `192.168.2.162` must be permitted to reach `192.168.2.115` on TCP port 3001.
The outpost callback path must remain accessible through Traefik.
---
## Outpost Health
The outpost health endpoint should return HTTP 204.
For the Traefik dashboard, test:
`curl -k -I https://proxy.bytek.ca/outpost.goauthentik.io/ping`
Expected result:
- HTTP 204.
A 204 response contains no page body and is expected.
If the endpoint redirects to login or loops repeatedly, review:
- Callback router priority.
- Middleware assignment.
- Outpost application assignment.
- External host.
- Traefik service destination.
- Host header forwarding.
---
## Reverse-Proxy Requirements
Traefik must preserve the original request information for Authentik.
Important forwarded values include:
- Original host.
- Original HTTPS scheme.
- Client forwarding information.
- Forwarded protocol.
- WebSocket compatibility.
Incorrect forwarded headers can cause:
- CSRF cookie errors.
- Origin mismatch.
- Redirect loops.
- Wrong callback URLs.
- Session-cookie problems.
An Authentik CSRF error previously reported that the CSRF cookie was not set.
When diagnosing a CSRF error, review:
- Browser cookies.
- Browser privacy extensions.
- Request Origin header.
- Request Host header.
- Traefik host-header forwarding.
- Authentik system information.
- Existing stale browser sessions.
---
## DNS Requirements
Authentik must be reachable from:
- User browsers.
- Traefik.
- Nextcloud containers.
- Vikunja.
- BookStack.
- Proxmox.
- Uptime Kuma where required.
- The Authentik embedded outpost.
### Internal Resolution
LAN clients using Pi-hole resolve:
- `portal.bytek.ca` to `192.168.2.182`.
### Proxmox Resolution
Proxmox uses Quad9 and normally resolves:
- `portal.bytek.ca` to `38.29.213.101`.
This sends Proxmox’s server-to-server OIDC requests through:
1. VPS.
2. WireGuard.
3. Traefik.
4. Authentik.
### Container Resolution
Application containers that require private split DNS should use Pi-hole at `192.168.2.65`.
If an OIDC discovery endpoint becomes unreachable after a power failure, verify DNS from inside the application container before changing provider credentials.
---
## Health Endpoints
### Direct Readiness
Direct backend:
`http://192.168.2.162:9000/-/health/ready/`
Expected result:
- HTTP 200.
### Routed Readiness
Public hostname:
[Open Authentik Readiness](https://portal.bytek.ca/)
Expected result:
- HTTP 200.
### Embedded Outpost
The outpost ping endpoint should return:
- HTTP 204.
---
## Container Management
Move to the Authentik deployment directory:
`cd /opt/authentik`
Check container state:
`sudo docker compose ps`
List Compose service names:
`sudo docker compose config --services`
View recent logs:
`sudo docker compose logs --tail 100`
View server logs:
`sudo docker compose logs --tail 100 server`
View worker logs:
`sudo docker compose logs --tail 100 worker`
The exact service names should be confirmed using the Compose service listing before running service-specific commands.
---
## Expected Container State
The Authentik stack should have healthy or running equivalents of:
- Authentik server.
- Authentik worker.
- PostgreSQL.
- Redis.
The server should not be considered ready until PostgreSQL and Redis are available.
A running Docker container does not necessarily mean the application is ready.
Use the readiness endpoint for application-level validation.
---
## User Administration
When creating a user:
1. Create or confirm the user’s email address.
2. Add the user to the required groups.
3. Confirm MFA enrollment.
4. Confirm the user appears in the application portal.
5. Test one user-facing application.
6. Test application logout.
7. Confirm no duplicate account is created.
8. Review application-specific permissions.
Do not add normal users to `bytek-admin`.
---
## Administrator Administration
An Authentik administrator should:
- Belong to `bytek-admin`.
- Belong to `bytek-users` if normal applications are required.
- Have MFA enabled.
- Use a unique password.
- Have recovery methods stored securely.
- Avoid using the local break-glass accounts for daily work.
Administrative group membership should be reviewed periodically.
---
## MFA
Authentik provides MFA for applications using OIDC or proxy authentication.
MFA should be required for:
- Proxmox.
- Traefik dashboard.
- Uptime Kuma.
- Nextcloud.
- Vikunja.
- BookStack.
- Authentik administration.
Preserve recovery codes or backup authentication devices.
Do not rely on one physical authenticator without a documented recovery method.
---
## Local Application Recovery Accounts
Authentik must not be the only recovery path.
### Proxmox
Use:
- `root@pam`
### Nextcloud
Use:
- Local Nextcloud administrator.
- Direct local login path.
### BookStack
Switch:
- `AUTH_METHOD` from `oidc` to `standard`.
Then sign in using the dedicated local administrator.
### Uptime Kuma
Restore direct private access and re-enable local authentication.
### Traefik
Restore a known-good dashboard file or use retained Basic Auth.
---
## Common Failure Scenarios
### Application Shows Permission Denied
Possible causes:
- User is not in the required group.
- An extra group binding is attached.
- Policy engine is `ALL`.
- One binding returns false.
- Email-domain policy fails.
- Application is not assigned to the user.
Review the Authentik denial explanation.
A binding shown as `None` in the explanation may correspond to a direct group-membership check.
### Discovery Endpoint Is Unreachable
Possible causes:
- Application container DNS failure.
- Pi-hole unavailable.
- Traefik unavailable.
- Authentik unavailable.
- Certificate issue.
- Local-address protection inside the application.
- Wrong application slug.
Test discovery from the application host and container.
### CSRF Cookie Not Set
Possible causes:
- Stale browser session.
- Browser cookie blocking.
- Wrong Origin header.
- Wrong Host header.
- Incorrect reverse-proxy forwarding.
- Proxy or VPN interference.
Test using a new private browser window.
### Token Payload Cannot Be Parsed
Possible cause:
- Authentik provider encryption key is selected.
Correction:
- Keep the signing key.
- Clear the encryption key.
- Start a fresh authorization flow.
### User’s Email Is Already Taken
The application may not automatically link an OIDC account to a local account based only on email.
For BookStack:
- Use a separate local break-glass administrator email.
- Allow Authentik to create the normal OIDC account.
- Assign BookStack roles separately.
### Authentication Works but Application Is Empty
Authentication succeeded, but application authorization is missing.
For Proxmox:
- Assign an ACL at `/`.
- Select the required role.
- Enable propagation.
---
## OIDC Troubleshooting Order
When an OIDC application fails:
1. Keep the local administrator session open.
2. Confirm Authentik readiness.
3. Confirm Traefik routing.
4. Confirm the application provider.
5. Confirm the application slug.
6. Confirm client ID and client secret.
7. Confirm redirect URI.
8. Confirm selected scope mappings.
9. Confirm signing key.
10. Confirm encryption key is empty.
11. Confirm user group membership.
12. Confirm policy results.
13. Test the discovery endpoint.
14. Test DNS from the application container.
15. Review the application log.
16. Review Authentik events.
17. Retry from a completely new private browser session.
Do not recreate the provider until the detailed failure is understood.
---
## Monitoring
Recommended Uptime Kuma monitors include:
| Monitor | Target |
|---|
| Authentik VM | Ping `192.168.2.162` |
| Direct readiness | `http://192.168.2.162:9000/-/health/ready/` |
| Routed readiness | `https://portal.bytek.ca/-/health/ready/` |
| Authentik certificate | `portal.bytek.ca` |
| PostgreSQL availability | Internal service-level check |
| Embedded outpost | Outpost ping endpoint |
- VM unavailable.
- Container unavailable.
- Database unavailable.
- Direct readiness failure.
- Traefik failure.
- Certificate failure.
- Public VPS or WireGuard failure.
- Outpost failure.
# Uptime Kuma
## Purpose
Uptime Kuma provides availability monitoring for the Bytek homelab.
Uptime Kuma monitors services at multiple layers so an outage can be isolated quickly.
Monitoring can determine whether a failure is caused by:
- A stopped VM or LXC.
- A network problem.
- A Proxmox firewall rule.
- A stopped Docker container.
- An unhealthy application.
- A database problem.
- Pi-hole DNS.
- Traefik routing.
- An expired or invalid certificate.
- Authentik proxy authentication.
- The WireGuard tunnel.
- The public VPS.
---
## Service Information
| Setting | Value |
|---|
| Service name | Uptime Kuma |
| Private IP address | `192.168.2.115` |
| Backend port | TCP 3001 |
| Public hostname | `status.bytek.ca` |
| Deployment directory | `/opt/uptime-kuma` |
| Container name | `uptime-kuma` |
| Persistent data | `/app/data` inside the container |
| Database | Embedded MariaDB |
| Container DNS | `192.168.2.65` |
| Public routing | Traefik |
| Target authentication | Authentik Proxy Provider |
| Required Authentik group | `bytek-admin` |
| Backup coverage | Proxmox scheduled backup |
| Proxmox VM ID | Confirm current VM ID |
---
## Architecture Role
Uptime Kuma monitors both private backends and routed application hostnames.
A routed monitor follows this path:
1. Uptime Kuma resolves the application hostname through Pi-hole.
2. Pi-hole returns Traefik’s private IP address, `192.168.2.182`.
3. Uptime Kuma connects to Traefik.
4. Traefik selects the application router.
5. Traefik forwards the request to the private backend.
6. Uptime Kuma records the response status and latency.
A direct backend monitor bypasses Traefik and connects directly to the application VM.
Using both monitor types helps distinguish infrastructure failures from reverse-proxy failures.
---
## Monitoring Layers
Each important service should have up to three monitoring layers.
### VM or LXC Monitor
A ping monitor verifies that the guest is reachable on the network.
A successful ping proves:
- The guest is running.
- The virtual network interface is active.
- The network path permits ICMP.
A successful ping does not prove that the application is healthy.
### Direct Backend Monitor
A direct backend monitor connects to the private application address and port.
A successful backend monitor proves:
- The guest is reachable.
- The application port is open.
- The application or web service responds.
A direct backend monitor bypasses:
- Pi-hole hostname resolution.
- Traefik routing.
- Public certificates.
- The VPS.
- WireGuard public ingress.
### Routed Application Monitor
A routed monitor uses the normal HTTPS application hostname.
A successful routed monitor proves:
- Pi-hole resolves the hostname.
- Traefik is reachable.
- The router is loaded.
- The certificate is valid.
- The backend is reachable.
- The application responds.
---
## Failure Interpretation
| VM Monitor | Backend Monitor | Routed Monitor | Likely Failure |
|---|
| Down | Down | Down | VM, LXC, Proxmox, or network failure |
| Up | Down | Down | Docker, application, database, or backend firewall failure |
| Up | Up | Down | Pi-hole, Traefik, TLS, router, or Authentik failure |
| Up | Up | Up | Service is operating normally |
| Up | Down | Up | Monitor target or direct-backend firewall configuration may be incorrect |
| Down | Up | Up | ICMP is blocked, but the application is available |
---
## Docker Deployment
The Uptime Kuma deployment is managed using Docker Compose.
Deployment directory:
`/opt/uptime-kuma`
Before running Compose commands, move into the deployment directory using:
`cd /opt/uptime-kuma`
Check container state using:
`sudo docker compose ps`
View recent logs using:
`sudo docker compose logs --tail 100 uptime-kuma`
Follow logs live using:
`sudo docker compose logs -f uptime-kuma`
Validate the Compose configuration using:
`sudo docker compose config --quiet`
Apply normal configuration changes using:
`sudo docker compose up -d`
Recreate the container after changing Docker-level settings such as DNS, volumes, environment variables, or networking using:
`sudo docker compose up -d --force-recreate uptime-kuma`
---
## Persistent Data
Uptime Kuma stores its application data under `/app/data` inside the container.
The persistent host mapping should be confirmed from the current Compose file.
The persistent data includes information such as:
- Monitors.
- Monitor history.
- Notifications.
- Status pages.
- Maintenance windows.
- Application settings.
- User configuration.
- Embedded database files.
- Uploaded status-page assets.
The persistent-data directory must remain mounted across container recreation.
Never remove the persistent volume while troubleshooting a container-startup problem.
---
## Database
The current Uptime Kuma deployment uses embedded MariaDB.
Database health is reported by the container health check.
The container should not be considered fully healthy until the database is available.
Check container health using:
`sudo docker inspect --format='{{.State.Health.Status}}' uptime-kuma`
Expected result:
- `healthy`
If the result is `starting`, allow the health check to continue while reviewing logs.
If the result is `unhealthy`, inspect the Uptime Kuma logs before restarting or recreating the container.
---
## Docker DNS
The Uptime Kuma container explicitly uses Pi-hole at `192.168.2.65`.
This allows Uptime Kuma to resolve internal split-DNS records.
Verify container DNS using:
`sudo docker inspect --format='DNS={{json .HostConfig.Dns}}' uptime-kuma`
Expected result:
- `192.168.2.65` appears in the DNS list.
Test an internal hostname from inside the container using:
`sudo docker exec uptime-kuma getent hosts portal.bytek.ca`
Expected result:
- `portal.bytek.ca` resolves to `192.168.2.182`.
Test another internal hostname using:
`sudo docker exec uptime-kuma getent hosts projects.bytek.ca`
Expected result:
- `projects.bytek.ca` resolves to `192.168.2.182`.
---
## DNS Failure After a Power Interruption
Docker DNS previously failed after an electrical outage.
Symptoms included:
- Monitors failing after services had restarted.
- Internal hostnames not resolving inside containers.
- Application hosts resolving correctly while container resolution failed.
- OIDC or certificate operations reporting unreachable endpoints.
Recommended recovery sequence:
1. Confirm Pi-hole is running.
2. Confirm Pi-hole owns `192.168.2.65`.
3. Confirm the Uptime Kuma VM resolves internal hostnames.
4. Confirm the Uptime Kuma container resolves internal hostnames.
5. Verify the container’s explicit DNS configuration.
6. Restart the Uptime Kuma container.
7. Recreate the container if Docker-level DNS configuration changed.
8. Confirm the monitors recover.
Do not rebuild Uptime Kuma or delete persistent data because of a DNS-only failure.
---
## Public Dashboard Routing
The Uptime Kuma dashboard is available at:
[Open Uptime Kuma](https://status.bytek.ca/)
Internal clients resolve `status.bytek.ca` to Traefik at `192.168.2.182`.
External clients resolve `status.bytek.ca` to the public VPS at `38.29.213.101`.
The public path includes:
1. WHC public DNS.
2. Public VPS.
3. WireGuard.
4. Home WireGuard gateway.
5. Traefik.
6. Authentik embedded outpost when enabled.
7. Uptime Kuma at `192.168.2.115:3001`.
---
## Authentication Design
Uptime Kuma does not use the same native OIDC design as Nextcloud, Vikunja, BookStack, and Proxmox.
The target design uses an Authentik Proxy Provider.
The expected authentication flow is:
1. The administrator opens `status.bytek.ca`.
2. Traefik forwards the request to the Authentik embedded outpost.
3. Authentik checks the existing authentication session.
4. Authentik requests credentials and MFA if required.
5. Authentik evaluates the application bindings.
6. Authentik proxies the request to Uptime Kuma.
7. Uptime Kuma displays the dashboard.
---
## Authentik Application
| Setting | Value |
|---|
| Application name | Uptime Kuma |
| Application slug | `uptime-kuma` |
| Provider type | Proxy Provider |
| Provider mode | Proxy |
| External host | `https://status.bytek.ca` |
| Internal host | `http://192.168.2.115:3001` |
| Required group | `bytek-admin` |
| Required policy | `Allow bytek.ca users` |
| Policy engine | `ALL` |
| Outpost | Authentik Embedded Outpost |
---
## Embedded Outpost Connectivity
The embedded outpost runs through the Authentik service at `192.168.2.162`.
Authentik must be able to reach Uptime Kuma at:
- `192.168.2.115`
- TCP port 3001
The Uptime Kuma VM firewall must allow:
- Source `192.168.2.162`
- Destination TCP port 3001
Test from the Authentik VM using:
`curl -I http://192.168.2.115:3001/`
A healthy Uptime Kuma response may redirect to `/dashboard`.
---
## Authentication Migration Safety
Do not disable Uptime Kuma’s local authentication before the Authentik proxy works.
Use this migration order:
1. Create the Authentik application and Proxy Provider.
2. Assign the application to the embedded outpost.
3. Allow Authentik to reach TCP port 3001.
4. Route `status.bytek.ca` through the outpost.
5. Test Authentik authentication.
6. Confirm the Uptime Kuma local login still appears behind Authentik.
7. Test with the `bytek-admin` account.
8. Test denial with a non-administrator account.
9. Confirm public status pages still work if required.
10. Disable Uptime Kuma local authentication.
11. Enable Trust Proxy.
12. Restrict direct TCP 3001 access.
13. Test recovery.
During the intermediate state, both Authentik and Uptime Kuma authentication may appear.
That is intentional until the Authentik path is validated.
---
## Disabling Local Authentication
Only disable local Uptime Kuma authentication after Authentik is proven reliable.
Before disabling local authentication:
- Record the Uptime Kuma administrator password.
- Confirm Proxmox backup coverage.
- Create a Proxmox snapshot if desired.
- Confirm direct private access.
- Confirm Authentik access.
- Confirm a non-admin user is denied.
- Confirm public status pages.
- Document the recovery process.
After disabling local authentication:
- Enable Trust Proxy.
- Restrict direct TCP 3001 access to the Authentik VM.
- Remove unnecessary direct Traefik-to-Kuma access if Traefik now routes to the outpost.
- Retain the original administrator password for recovery.
---
## Direct Backend Security
When Uptime Kuma local authentication is disabled, direct access to `192.168.2.115:3001` must be restricted.
Otherwise, a LAN client could bypass Authentik and reach an unauthenticated dashboard.
The final firewall should permit TCP port 3001 only from approved sources such as:
| Source | Purpose |
|---|
| `192.168.2.162` | Authentik embedded outpost |
| Administrative workstation during testing | Temporary recovery access |
| Approved monitoring source | Only if required |
---
## Public Status Pages
Some Uptime Kuma status pages may be intended for unauthenticated public viewing.
Authentik Proxy Provider exclusions may be required for:
- Published status pages.
- Status-page API requests.
- Static assets.
- Uploaded status-page files.
- Badge endpoints.
- Push-monitor endpoints.
- Public icons.
Unauthenticated-path exclusions must be as narrow as practical.
Do not exclude the entire Uptime Kuma API.
After configuring exclusions:
1. Open a public status page in a private browser.
2. Confirm no Authentik login is required.
3. Open the dashboard.
4. Confirm Authentik login is required.
5. Test a push monitor.
6. Test a status badge if used.
7. Review Authentik and Traefik logs.
---
## Push Monitors
Push monitors rely on a unique URL that an external process calls to report success.
If Uptime Kuma sits behind Authentik, the push path must remain reachable without an interactive login.
Protect push-monitor URLs as secrets.
Do not publish push URLs in BookStack.
If a push monitor stops updating after enabling Authentik:
1. Confirm the push path is excluded.
2. Confirm Traefik routes the request correctly.
3. Confirm the unique token is unchanged.
4. Review the Uptime Kuma monitor history.
5. Review Traefik access logs.
6. Review Authentik outpost logs.
---
## Monitor Naming Standard
Use consistent names so the dashboard is easy to scan.
Recommended patterns include:
- `Service VM`
- `Service Backend Direct`
- `Service Through Traefik`
- `Service External`
- `Service Certificate`
- `Service DNS`
Examples:
- `Nextcloud VM`
- `Nextcloud AIO Apache`
- `Nextcloud Through Traefik`
- `Vikunja VM`
- `Vikunja Backend Direct`
- `Vikunja Through Traefik`
- `BookStack VM`
- `BookStack Backend Direct`
- `BookStack Through Traefik`
---
## Core Infrastructure Monitors
Recommended core monitors include:
| Monitor | Suggested Type | Target |
|---|
| Proxmox VE | HTTPS or TCP | `192.168.2.254:8006` |
| Pi-hole VM or LXC | Ping | `192.168.2.65` |
| Pi-hole DNS TCP | TCP | `192.168.2.65:53` |
| Pi-hole DNS UDP | UDP or DNS | `192.168.2.65:53` |
| WireGuard gateway | Ping | `192.168.2.64` |
| Public VPS | Ping or TCP | `38.29.213.101` |
| Public VPS HTTPS | TCP | `38.29.213.101:443` |
| Traefik VM | Ping | `192.168.2.182` |
| Traefik HTTPS | TCP | `192.168.2.182:443` |
| Authentik direct | HTTP | Direct readiness endpoint |
| Authentik routed | HTTPS | Routed readiness endpoint |
---
| Monitor | Type | Target |
|---|
| Nextcloud VM | Ping | `192.168.2.100` |
| Nextcloud AIO Apache | TCP | `192.168.2.100:11000` |
| Nextcloud Through Traefik | HTTPS | `https://cloud.bytek.ca/status.php` |
- HTTP 200.
---
## Vikunja Monitors
| Monitor | Type | Target |
|---|
| Vikunja VM | Ping | `192.168.2.121` |
| Vikunja Backend Direct | HTTP | `http://192.168.2.121:3456/health` |
| Vikunja Through Traefik | HTTPS | `https://projects.bytek.ca/health` |
- HTTP 200.
The Vikunja VM firewall must allow Uptime Kuma at `192.168.2.115` to reach TCP port 3456.
---
## BookStack Monitors
| Monitor | Type | Target |
|---|
| BookStack VM | Ping | Confirm BookStack IP |
| BookStack Backend Direct | HTTP | BookStack IP on port 6875 |
| BookStack Through Traefik | HTTPS | `https://docs.bytek.ca/login` |
---
## Authentik Monitors
| Monitor | Type | Target |
|---|
| Authentik VM | Ping | `192.168.2.162` |
| Authentik Direct Readiness | HTTP | `http://192.168.2.162:9000/-/health/ready/` |
| Authentik Routed Readiness | HTTPS | `https://portal.bytek.ca/-/health/ready/` |
| Authentik Certificate | Certificate | `portal.bytek.ca` |
| Authentik Outpost | HTTP | Outpost ping endpoint |
- Direct readiness returns HTTP 200.
- Routed readiness returns HTTP 200.
- Outpost ping returns HTTP 204.
---
## Traefik Monitors
| Monitor | Type | Target |
|---|
| Traefik VM | Ping | `192.168.2.182` |
| Traefik HTTPS | TCP | `192.168.2.182:443` |
| Traefik Dashboard | HTTPS | `https://proxy.bytek.ca/` |
| Traefik Certificate | Certificate | `proxy.bytek.ca` |
---
## Proxmox Monitor Considerations
Proxmox may return a response that is valid but not HTTP 200 for some request methods.
Avoid relying on an unsupported HTTP HEAD request.
A TCP monitor on port 8006 can verify listener availability.
An HTTPS monitor can verify the login page if configured with an appropriate method and accepted status range.
Proxmox remains private and should be monitored using its private address.
---
## Certificate Monitoring
Important public hostnames should have certificate-expiry monitoring.
Recommended hostnames include:
- `portal.bytek.ca`
- `cloud.bytek.ca`
- `projects.bytek.ca`
- `docs.bytek.ca`
- `status.bytek.ca`
- `proxy.bytek.ca`
Certificate monitoring should alert before expiration.
A valid application response does not guarantee that certificate renewal automation remains healthy.
---
## Notification Channels
Uptime Kuma can notify administrators when a monitor changes state.
The configured notification methods should be recorded after verification.
Potential channels may include:
- Email.
- Microsoft Teams webhook.
- Discord webhook.
- Matrix.
- Telegram.
- Other supported notification services.
Notification credentials must remain outside BookStack.
BookStack should document:
- Notification channel name.
- Intended recipients.
- Severity policy.
- Test date.
- Secret-storage location.
---
## Notification Policy
Recommended notification behavior:
### Critical Infrastructure
Immediate notification for:
- Proxmox.
- Pi-hole DNS.
- WireGuard.
- Traefik.
- Authentik.
- Public VPS.
### Applications
Notify after enough failed checks to avoid alerts from brief application restarts.
### Maintenance
Create maintenance windows before:
- Proxmox reboots.
- Application upgrades.
- Docker stack recreation.
- Firewall maintenance.
- Planned internet outages.
- Storage maintenance.
This prevents expected maintenance from generating unnecessary alerts.
---
## Monitor Timing
Monitor intervals should balance timely detection with unnecessary load.
Suggested principles:
- Critical service checks may run more frequently.
- Application backends can use moderate intervals.
- Certificate checks need less frequent execution.
- Avoid aggressively monitoring database internals without a clear operational need.
- Use retries before marking services down.
- Configure heartbeat intervals based on expected update frequency.
These values should be tuned after observing false positives and actual recovery behavior.
---
## Status Pages
Status pages provide a consolidated view of selected monitors.
A public status page should include only services appropriate for public disclosure.
Avoid publishing:
- Private IP addresses.
- Proxmox details.
- Pi-hole details.
- Authentik backend details.
- Internal database status.
- Hypervisor storage status.
- Internal firewall information.
A private administrative status page may include more detail behind Authentik.
---
## Maintenance Windows
Use maintenance windows for planned outages.
Common maintenance events include:
- Proxmox host reboot.
- Electrical maintenance.
- Docker upgrades.
- Authentik upgrades.
- Nextcloud AIO upgrades.
- Traefik configuration changes.
- Storage work.
- Firewall work.
- Backup restoration testing.
Record:
- Reason for maintenance.
- Affected monitors.
- Date and time.
- Expected service impact.
- Actual result.
---
## Firewall Requirements
Uptime Kuma requires outbound access to monitored services.
Application firewalls may require narrow inbound rules from:
- `192.168.2.115`
Known direct-monitor requirements include:
| Destination | Port |
|---|
| Pi-hole | TCP and UDP 53 |
| Proxmox VE | TCP 8006 |
| Traefik | TCP 443 |
| Authentik | TCP 9000 |
| Nextcloud | TCP 11000 |
| Vikunja | TCP 3456 |
| BookStack | TCP 6875 |
---
## Health Validation
Check the VM address using:
`ip -br -4 addr`
Check the container state using:
`sudo docker compose ps`
Check container health using:
`sudo docker inspect --format='{{.State.Health.Status}}' uptime-kuma`
Confirm direct access using:
`curl -I http://192.168.2.115:3001/`
Confirm DNS inside the container using:
`sudo docker exec uptime-kuma getent hosts portal.bytek.ca`
Confirm the public route using:
`curl -I https://status.bytek.ca/`
Interpret redirects according to the currently enabled authentication layers.
---
## Logging
Review Uptime Kuma logs using:
`sudo docker compose logs --tail 100 uptime-kuma`
Follow logs live using:
`sudo docker compose logs -f uptime-kuma`
Review logs for:
- Database startup failures.
- Monitor connection failures.
- DNS errors.
- Certificate errors.
- Notification failures.
- Proxy-header errors.
- Authentication errors.
- File-permission problems.
- Persistent-volume errors.
Do not paste access tokens, notification secrets, cookies, or push-monitor URLs into BookStack.
---
## Common Failure Scenarios
### Many Hostname Monitors Fail Simultaneously
Likely causes:
- Pi-hole unavailable.
- Container DNS failure.
- Traefik unavailable.
- WireGuard outage.
- VPS outage.
Check direct IP monitors before assuming every application failed.
### Direct Backends Work but Routed Monitors Fail
Likely causes:
- Pi-hole split DNS.
- Traefik router.
- Certificate.
- Authentik proxy.
- Hostname mismatch.
### VM Ping Fails but HTTP Works
Likely cause:
- ICMP blocked by Proxmox or guest firewall.
The VM is likely available.
### Kuma Dashboard Works by IP but Not by Hostname
Check:
- Pi-hole.
- Traefik.
- Certificate.
- Authentik provider.
- Browser DNS over HTTPS.
### Authentik Login Works but Kuma Is Unreachable
Check:
- Internal host in the Authentik Proxy Provider.
- Authentik VM firewall access to TCP 3001.
- Uptime Kuma container.
- Embedded outpost assignment.
- Host-header forwarding.
### Public Status Page Requires Login
Check:
- Authentik unauthenticated-path patterns.
- Status-page slug.
- Static asset exclusions.
- Status API exclusions.
- Traefik routing.
### Push Monitor Stops Working
Check:
- Push path exclusion.
- Push token.
- Traefik access logs.
- Authentik outpost logs.
- Monitor heartbeat interval.
---
## Power-Failure Recovery
After a power interruption:
1. Confirm Proxmox.
2. Confirm Pi-hole.
3. Confirm the WireGuard gateway.
4. Confirm Authentik.
5. Confirm Traefik.
6. Confirm the Uptime Kuma VM owns `192.168.2.115`.
7. Confirm Docker.
8. Confirm the container.
9. Confirm container DNS.
10. Confirm embedded MariaDB health.
11. Confirm direct dashboard access.
12. Confirm the routed dashboard.
13. Confirm monitors recover.
14. Restart or recreate the container only if DNS remains broken.
Uptime Kuma should start after the core DNS, identity, and routing services when practical.
---
## Backup
The Uptime Kuma VM is included in the Proxmox scheduled backup job.
Protect:
- Persistent `/app/data`.
- Docker Compose file.
- Environment file, if used.
- Any custom certificates.
- Notification configuration.
- Status-page assets.
- Monitor history.
- Embedded database.
A stopped Proxmox backup provides the strongest whole-VM consistency for a baseline.
Snapshot-mode backup may be used for routine operation where downtime is undesirable.
---
## Recovery Procedure
If Uptime Kuma is corrupted or unavailable:
1. Confirm the VM is running.
2. Confirm `192.168.2.115`.
3. Confirm Docker.
4. Confirm the persistent-data mount.
5. Confirm the container.
6. Review logs.
7. Confirm embedded database health.
8. Test direct TCP port 3001.
9. Test Pi-hole DNS.
10. Test the Traefik route.
11. Test the Authentik proxy.
If the VM must be restored:
1. Restore the latest backup under a temporary VM ID.
2. Disconnect the restored VM network adapter.
3. Verify Docker and persistent data.
4. Verify monitor configuration.
5. Shut down the failed production VM.
6. Assign the expected network identity.
7. Start the restored VM.
8. Confirm `192.168.2.115`.
9. Confirm the dashboard.
10. Confirm notifications and monitors.
11. Reconnect the Authentik route.
---
## Authentik Recovery
If the Authentik proxy prevents access:
1. Restore the known-good direct Traefik route.
2. Temporarily permit the administrative workstation to reach TCP 3001.
3. Access Uptime Kuma using the private IP.
4. Re-enable Uptime Kuma local authentication.
5. Repair the Authentik Proxy Provider.
6. Test Authentik with a private browser.
7. Confirm `bytek-admin` access.
8. Confirm a normal user is denied.
9. Disable local authentication again only after validation.
10. Restrict TCP 3001 again.
Record the original administrator password securely even after local authentication is disabled.
---
## Security Checklist
- Uptime Kuma is not directly exposed publicly on TCP 3001.
- Dashboard access requires `bytek-admin`.
- Authentik policy engine uses `ALL`.
- Direct backend access is restricted after local authentication is disabled.
- Trust Proxy is enabled when required.
- Public status-page exclusions are narrow.
- Push URLs are treated as secrets.
- Notification credentials are protected.
- Container DNS is `192.168.2.65`.
- Persistent data is backed up.
- Local recovery credentials are stored securely.
- Proxmox backup includes the Uptime Kuma VM.
- Public status pages do not reveal sensitive infrastructure details.
---
## Validation Checklist
- Uptime Kuma VM owns `192.168.2.115`.
- Docker is active.
- Uptime Kuma container is running.
- Container health is `healthy`.
- Embedded MariaDB is available.
- Persistent data is mounted.
- Container DNS uses Pi-hole.
- Internal Bytek names resolve.
- Direct TCP 3001 responds.
- Routed dashboard responds.
- Authentik Proxy Provider works.
- `bytek-admin` is allowed.
- Normal users are denied.
- Public status pages work as intended.
- Push monitors work.
- Notifications are tested.
- Certificate monitoring works.
- Proxmox backup includes the VM.
- Recovery access has been tested.
---
## Document Control
- **Owner:** Bryan Gagne-Plante
- **Private address:** `192.168.2.115`
- **Backend port:** TCP 3001
- **Public hostname:** `status.bytek.ca`
- **Deployment directory:** `/opt/uptime-kuma`
- **Database:** Embedded MariaDB
- **Container DNS:** `192.168.2.65`
- **Authentication:** Authentik Proxy Provider
- **Required group:** `bytek-admin`
- **Proxmox VM ID:** Confirm current VM ID
- **Last verified:** YYYY-MM-DD
- **Last notification test:** YYYY-MM-DD
- **Last Authentik proxy test:** YYYY-MM-DD
- **Last public status-page test:** YYYY-MM-DD
- **Last restore test:** YYYY-MM-DD
- **Known limitation:** Monitoring, status history, and alerting depend on one Uptime Kuma VM