daybook/deploy/cloud_agent_deploy.md

# Cloud Deployment Runbook For A Dedicated Deploy Agent

This document is written for an autonomous AI agent responsible for deploying
and operating Daybook on a cloud VM, without a human walking through each
step. Follow it in order. Where a decision depends on information you don't
have (VM IP, domain, whether TLS is required), ask the operator once, then
proceed autonomously for routine redeploys.

For background on the app itself and the general (human-facing) version of
this workflow, see `docs/deployment.md`. This file is the operational SOP;
that one is the reference.

## Source Of Truth

The canonical source is a private Forgejo repository:

```text
https://git.cygnus.systems/peter/daybook.git
```

Do not deploy from a copied/rsynced tree. Always deploy from a git clone of
this repository so `git pull` is available for fast redeploys.

## 0. Get Repo Access

You need read access to a private repo. Prefer a **repo-scoped SSH deploy
key** over a personal access token — it can only read this one repo, unlike
a token tied to a full user account.

1. Generate an SSH key pair on the VM (or ask the operator for one to install):
   ```bash
   ssh-keygen -t ed25519 -C "daybook-deploy-agent" -f ~/.ssh/daybook_deploy -N ""
   ```
2. Ask the operator to add the **public** key (`~/.ssh/daybook_deploy.pub`) as
   a Deploy Key on the repo in Forgejo (repo → Settings → Deploy Keys →
   read-only is sufficient), or hand it to them directly — never ask them to
   give you their personal account credentials.
3. Configure SSH to use it for this host. **Note:** this Forgejo's git-SSH
   listens on port `2222`, not the default `22` (chosen so it doesn't clash
   with the box's own system `sshd`) — connecting on `22` hits system SSH
   instead and will fail or hang, not just warn. Don't assume `22` even if
   this changes again later; confirm with the operator if unsure.
   ```bash
   cat >> ~/.ssh/config <<'EOF'
   Host git.cygnus.systems
     Port 2222
     IdentityFile ~/.ssh/daybook_deploy
     IdentitiesOnly yes
   EOF
   ```
4. Verify:
   ```bash
   ssh -T git@git.cygnus.systems
   ```
   If you'd rather not rely on `~/.ssh/config`, clone with the port explicit
   instead: `ssh://git@git.cygnus.systems:2222/peter/daybook.git`.

If SSH access can't be arranged, fall back to an HTTPS access token the
operator generates (Forgejo → user Settings → Applications → Generate New
Token, read-only `repo` scope) and clone with:

```bash
git clone https://<username>:<token>@git.cygnus.systems/peter/daybook.git /opt/daybook
```

## 1. Prerequisites On The VM

Confirm before proceeding; install if missing:

```bash
docker --version || curl -fsSL https://get.docker.com | sh
docker compose version
git --version
```

Confirm outbound network access to pull the Python base image and packages,
and confirm you know the VM's public IP or DNS name — you'll need it for
`DJANGO_ALLOWED_HOSTS` and for reporting the final URL back.

## 2. Clone

```bash
sudo mkdir -p /opt/daybook
sudo chown "$USER:$USER" /opt/daybook
git clone git@git.cygnus.systems:peter/daybook.git /opt/daybook
cd /opt/daybook
```

## 3. Configure Environment

```bash
cp deploy/env.example deploy/.env
```

Generate a real secret key — do not deploy with the placeholder:

```bash
python3 -c "import secrets; print('DJANGO_SECRET_KEY=' + secrets.token_urlsafe(50))"
```

Edit `deploy/.env`:

```text
DJANGO_DEBUG=0
DJANGO_SECRET_KEY=<generated-secret>
DJANGO_ALLOWED_HOSTS=<vm-public-ip>,<domain-if-any>,localhost,127.0.0.1
DAYBOOK_DATA_DIR=/data
DATABASE_PATH=/data/db.sqlite3
ARTIFACTS_ROOT=/data/artifacts_store
```

There is no shared API token to set — Daybook issues a personal token per
user from Settings → Account once a user exists (step 5). Never commit
`deploy/.env`; it is already gitignored.

`deploy/.env` will be read and written by whatever agent runs this runbook.
Treat it as low-stakes: it should never hold anything more sensitive than
this app's own `DJANGO_SECRET_KEY`. If the target host has its own
documented convention for secrets (e.g. composed by reference at execution
time rather than written to a file), follow that instead — ask the operator
if unsure which applies here.

## 4. Network Access

**If this VM is reachable from the public internet, a reverse proxy with TLS
in front of Daybook is not optional — it's the only path.** Bearer tokens are
this app's entire write-auth model, and they'd otherwise go over plaintext
HTTP straight to a port on the public IP. Direct port exposure (below) is
only for private/VPN-only networks with no public exposure.

- **Internet-facing (the common case):** set `DAYBOOK_BIND_ADDR=127.0.0.1` in
  `deploy/.env` so the container port is not reachable from outside the VM at
  all, then open `80`/`443` in the VM's firewall/security group for the
  reverse proxy (see §8). Do not open `8000` externally.
- **Private/VPN-only network:** open TCP `8000` to the addresses that should
  reach it (never `0.0.0.0/0`), and leave `DAYBOOK_BIND_ADDR` unset.

Before opening any port, check whether one is already open and in use —
`ss -tlnp | grep -E ':80|:443|:8000'` — rather than assuming a clean slate.
Firewall/security-group configuration is provider-specific (AWS security
group, GCP firewall rule, DigitalOcean cloud firewall, or the VM's own
`ufw`/`iptables`). Ask the operator which provider/console to use if you
don't have API access to it yourself.

## 5. Build, Start, Create A User

If the target has a separate, larger data volume than its main disk (common
on space-constrained VMs), point Daybook's persistent data at it before
starting: set `DAYBOOK_HOST_DATA_DIR=/path/on/that/volume` in `deploy/.env`.
Ask the operator whether such a volume exists and where before defaulting to
the main disk — filling the main disk on a host running other services is a
bigger problem than it would be on a dedicated box. Create that same path
(the one you just set in `deploy/.env`, or `deploy/data` if you left it
unset) before starting the stack:

```bash
mkdir -p deploy/data   # or the DAYBOOK_HOST_DATA_DIR path if you set one above
docker compose -f deploy/docker-compose.yml up -d --build
docker compose -f deploy/docker-compose.yml exec daybook python manage.py createsuperuser
```

The entrypoint runs migrations and collects static files automatically on
every start.

## 6. Verify

```bash
docker compose -f deploy/docker-compose.yml ps
curl -fsS http://127.0.0.1:8000/api/capabilities
```

A `403` with `"Login or a valid API token is required"` is the **expected**
response when unauthenticated — it confirms the app is up. This `curl` runs
against `127.0.0.1` from on the VM itself, so it works regardless of whether
`DAYBOOK_BIND_ADDR` is set. Log in from a browser at whichever address is
actually externally reachable at this point in the runbook — `http://<vm-ip>:8000/`
if you're on the private/VPN-only path from §4, or `https://<domain>/` through
the reverse proxy once §8 is done if you're on the internet-facing path — with
the superuser from step 5, grab a personal token from Settings → Account, then
smoke-test a post:

```bash
curl -fsS -X POST http://127.0.0.1:8000/api/artifacts \
  -H "Authorization: Bearer <token-from-settings>" \
  -H "Content-Type: application/json" \
  -d '{"source": "deploy-agent", "kind": "markdown", "title": "Deployment Smoke Test", "content": "# OK\n\nDeployed successfully."}'
```

Confirm the response includes `id`, `url`, and `api_url`, and that opening
`url` in a browser renders the artifact.

## 7. Redeploy Loop (routine updates)

This is the command you run every time you're asked to ship a new version:

```bash
cd /opt/daybook
PREV_SHA=$(git rev-parse HEAD)
git pull
docker compose -f deploy/docker-compose.yml up -d --build
```

Then repeat the verification in step 6 (capabilities check at minimum). If it
fails:

```bash
git checkout "$PREV_SHA"
docker compose -f deploy/docker-compose.yml up -d --build
```

to roll back, then report the failure (build log / migration error / failed
health check) back to the operator rather than retrying blindly.

`deploy/.env` and the data directory (wherever `DAYBOOK_HOST_DATA_DIR` points,
or `deploy/data` by default) are untracked, so `git pull` never touches them —
no need to back them up before a routine redeploy.

## 8. Hardening For Internet-Facing Deployments

If this VM is reachable from the public internet (not just a private
network/VPN), do this before calling the deployment done:

- Put a reverse proxy in front with TLS. Bearer tokens go over plaintext HTTP
  otherwise.
- Add the proxy's public hostname to `DJANGO_ALLOWED_HOSTS`.
- Set `DJANGO_CSRF_TRUSTED_ORIGINS=https://<public-hostname>` in `deploy/.env`,
  **and** make sure the proxy forwards `X-Forwarded-Proto: https` (examples
  below). Without both, login fails with "CSRF verification failed" — Django
  can't tell the original request was HTTPS once a proxy terminates TLS and
  forwards plain HTTP to the container.
- Keep `DJANGO_DEBUG=0` (already set above).
- Restrict SSH to key-based auth only; disable password login if you have
  access to do so.
- Confirm the VM's firewall only allows the ports actually needed (80/443,
  and SSH from wherever the operator connects from).

**Check for an existing reverse proxy before introducing a new one** —
`ss -tlnp | grep -E ':80|:443'`. A VM already running other services almost
always already has one (Apache, Nginx, Traefik) terminating TLS for
everything on it; a second process trying to bind 80/443 will just fail to
start, and stopping the existing proxy to "free the port" would take every
other service on the box down with it. If one exists, add a vhost to it
instead. Apache example:

```apache
<VirtualHost *:443>
    ServerName daybook.example.com
    ProxyPreserveHost On
    RequestHeader set X-Forwarded-Proto "https"
    ProxyPass / http://127.0.0.1:8000/
    ProxyPassReverse / http://127.0.0.1:8000/
    SSLEngine on
    SSLCertificateFile /etc/letsencrypt/live/daybook.example.com/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/daybook.example.com/privkey.pem
</VirtualHost>
```

`RequestHeader set X-Forwarded-Proto "https"` requires `a2enmod headers`; it's
not optional (see the CSRF note above). (`certbot --apache` will provision and
wire up the certificate for you.)

Only reach for a standalone proxy like Caddy if the target genuinely has
nothing on 80/443 yet — it still needs the same header:

```text
daybook.example.com {
    reverse_proxy 127.0.0.1:8000 {
        header_up X-Forwarded-Proto https
    }
}
```

## 9. Backup

If you set `DAYBOOK_HOST_DATA_DIR` in step 5, the real data lives there, not
at `deploy/data` — back up whichever path is actually configured, read
straight from `deploy/.env` rather than assumed, so this keeps working
whether or not that override is set:

```bash
cd /opt/daybook
DATA_DIR="$(grep -oP '(?<=^DAYBOOK_HOST_DATA_DIR=).*' deploy/.env || echo deploy/data)"
tar -czf "/tmp/daybook-data-$(date +%Y%m%d-%H%M%S).tar.gz" -C "$(dirname "$DATA_DIR")" "$(basename "$DATA_DIR")"
```

Copy the resulting archive off the VM (to object storage, the operator's
machine, etc.) — a backup that only exists on the VM it's protecting against
doesn't help if the VM is lost.

## Deployment Checklist

Before reporting the deployment complete, confirm all of these:

- [ ] Cloned via git (not copied), with a working `git pull`.
- [ ] `deploy/.env` has a real, non-placeholder `DJANGO_SECRET_KEY`.
- [ ] `DJANGO_ALLOWED_HOSTS` includes the actual hostname/IP being used.
- [ ] If behind a reverse proxy: `DJANGO_CSRF_TRUSTED_ORIGINS` is set to the
      public HTTPS origin, and the proxy forwards `X-Forwarded-Proto: https`.
      Verify by actually logging in through the proxy, not just curling
      `/api/capabilities` — CSRF only bites on POST (login, artifact writes).
- [ ] `docker compose -f deploy/docker-compose.yml ps` shows `daybook` healthy.
- [ ] `/api/capabilities` returns `403` unauthenticated, `200` with a token.
- [ ] At least one superuser exists with a working personal API token.
- [ ] A smoke-test artifact posts and renders successfully.
- [ ] If internet-facing: TLS is in front of the app, not raw HTTP on `8000`.
- [ ] A backup of the data directory (`deploy/data`, or `DAYBOOK_HOST_DATA_DIR`
      if you set it) has been taken and copied off the VM.
- [ ] If the target host has its own service-inventory, monitoring, or
      changelog conventions for new services, they've been followed — ask
      the operator if unsure what those are.

Report back to the operator: the app URL, confirmation of the checklist
above, and the deploy SSH key fingerprint or token identity used (not the
key/token itself) for their records.