daybook/deploy/cloud_agent_deploy.md

Cloud Deployment Runbook For A Dedicated Deploy Agent

This is written for a capable (Opus-class) agent deploying and operating Daybook on a cloud VM. Treat it as a guide and a source of Daybook-specific facts, not a rigid script — you're expected to use judgment, adapt to the actual VM, and read the authoritative sources rather than follow steps blindly. The steps below are a sensible default path; deviate where the host warrants it.

Ground truth, in order of authority (prefer these over this doc if they ever disagree):

• The repo — https://git.cygnus.systems/peter/daybook.git (private Forgejo). Everything ships from here; deploy from a clone so git pull works. Note: this Forgejo's git-SSH is on port 2222, not 22.
• deploy/ — Dockerfile, docker-compose.yml, entrypoint.sh, env.example. These are the deployment: the image installs all Python deps from requirements.txt (Django, DRF, Markdown, Pygments, nh3, openpyxl, gunicorn, whitenoise), and entrypoint.sh runs migrate + collectstatic on every start, so schema and static assets are always current. Read them; don't assume.
• docs/deployment.md — the general (host-agnostic) deployment reference and rationale.

Where a decision needs info you don't have (VM IP, domain, whether it's internet-facing, whether a reverse proxy/secondary volume exists), ask the operator once, then proceed autonomously for routine redeploys. The rest of this file is the recommended path plus the non-obvious, Daybook-specific gotchas (SSH port, CSRF-behind-proxy, bind address, data volume, healthcheck) that are easy to get wrong.

0. Get Repo Access

You need read access to a private repo. Prefer a repo-scoped SSH deploy key over a personal access token — it can only read this one repo, unlike a token tied to a full user account.

1. Generate an SSH key pair on the VM (or ask the operator for one to install):

   ssh-keygen -t ed25519 -C "daybook-deploy-agent" -f ~/.ssh/daybook_deploy -N ""
   

2. Ask the operator to add the public key (~/.ssh/daybook_deploy.pub) as a Deploy Key on the repo in Forgejo (repo → Settings → Deploy Keys → read-only is sufficient), or hand it to them directly — never ask them to give you their personal account credentials.
3. Configure SSH to use it for this host. Note: this Forgejo's git-SSH listens on port 2222, not the default 22 (chosen so it doesn't clash with the box's own system sshd) — connecting on 22 hits system SSH instead and will fail or hang, not just warn. Don't assume 22 even if this changes again later; confirm with the operator if unsure.

   cat >> ~/.ssh/config <<'EOF'
   Host git.cygnus.systems
     Port 2222
     IdentityFile ~/.ssh/daybook_deploy
     IdentitiesOnly yes
   EOF
   

4. Verify:

   ssh -T git@git.cygnus.systems
   

   If you'd rather not rely on ~/.ssh/config, clone with the port explicit instead: ssh://git@git.cygnus.systems:2222/peter/daybook.git.

If SSH access can't be arranged, fall back to an HTTPS access token the operator generates (Forgejo → user Settings → Applications → Generate New Token, read-only repo scope) and clone with:

git clone https://<username>:<token>@git.cygnus.systems/peter/daybook.git /opt/daybook

1. Prerequisites On The VM

Confirm before proceeding; install if missing:

docker --version || curl -fsSL https://get.docker.com | sh
docker compose version
git --version

Confirm outbound network access to pull the Python base image and packages, and confirm you know the VM's public IP or DNS name — you'll need it for DJANGO_ALLOWED_HOSTS and for reporting the final URL back.

2. Clone

sudo mkdir -p /opt/daybook
sudo chown "$USER:$USER" /opt/daybook
git clone git@git.cygnus.systems:peter/daybook.git /opt/daybook
cd /opt/daybook

3. Configure Environment

cp deploy/env.example deploy/.env

Generate a real secret key — do not deploy with the placeholder:

python3 -c "import secrets; print('DJANGO_SECRET_KEY=' + secrets.token_urlsafe(50))"

Edit deploy/.env:

DJANGO_DEBUG=0
DJANGO_SECRET_KEY=<generated-secret>
DJANGO_ALLOWED_HOSTS=<vm-public-ip>,<domain-if-any>,localhost,127.0.0.1
DAYBOOK_DATA_DIR=/data
DATABASE_PATH=/data/db.sqlite3
ARTIFACTS_ROOT=/data/artifacts_store

There is no shared API token to set — Daybook issues a personal token per user from Settings → Account once a user exists (step 5). Never commit deploy/.env; it is already gitignored.

deploy/.env will be read and written by whatever agent runs this runbook. Treat it as low-stakes: it should never hold anything more sensitive than this app's own DJANGO_SECRET_KEY. If the target host has its own documented convention for secrets (e.g. composed by reference at execution time rather than written to a file), follow that instead — ask the operator if unsure which applies here.

4. Network Access

If this VM is reachable from the public internet, a reverse proxy with TLS in front of Daybook is not optional — it's the only path. Bearer tokens are this app's entire write-auth model, and they'd otherwise go over plaintext HTTP straight to a port on the public IP. Direct port exposure (below) is only for private/VPN-only networks with no public exposure.

• Internet-facing (the common case): set DAYBOOK_BIND_ADDR=127.0.0.1 in deploy/.env so the container port is not reachable from outside the VM at all, then open 80/443 in the VM's firewall/security group for the reverse proxy (see §8). Do not open 8000 externally.
• Private/VPN-only network: open TCP 8000 to the addresses that should reach it (never 0.0.0.0/0), and leave DAYBOOK_BIND_ADDR unset.

Before opening any port, check whether one is already open and in use — ss -tlnp | grep -E ':80|:443|:8000' — rather than assuming a clean slate. Firewall/security-group configuration is provider-specific (AWS security group, GCP firewall rule, DigitalOcean cloud firewall, or the VM's own ufw/iptables). Ask the operator which provider/console to use if you don't have API access to it yourself.

5. Build, Start, Create A User

If the target has a separate, larger data volume than its main disk (common on space-constrained VMs), point Daybook's persistent data at it before starting: set DAYBOOK_HOST_DATA_DIR=/path/on/that/volume in deploy/.env. Ask the operator whether such a volume exists and where before defaulting to the main disk — filling the main disk on a host running other services is a bigger problem than it would be on a dedicated box. Create that same path (the one you just set in deploy/.env, or deploy/data if you left it unset) before starting the stack:

mkdir -p deploy/data   # or the DAYBOOK_HOST_DATA_DIR path if you set one above
docker compose -f deploy/docker-compose.yml up -d --build
docker compose -f deploy/docker-compose.yml exec daybook python manage.py createsuperuser

The entrypoint runs migrations and collects static files automatically on every start.

6. Verify

docker compose -f deploy/docker-compose.yml ps
curl -s -o /dev/null -w "%{http_code}\n" http://127.0.0.1:8000/api/capabilities   # expect 403

A 403 with "Login or a valid API token is required" is the expected response when unauthenticated — it confirms the app is up. (Don't use curl -f here: --fail treats the 403 as an error and hides it. The container's own healthcheck hits the login page instead, which is 200 without auth.) This curl runs against 127.0.0.1 from on the VM itself, so it works regardless of whether DAYBOOK_BIND_ADDR is set. Log in from a browser at whichever address is actually externally reachable at this point in the runbook — http://<vm-ip>:8000/ if you're on the private/VPN-only path from §4, or https://<domain>/ through the reverse proxy once §8 is done if you're on the internet-facing path — with the superuser from step 5, grab a personal token from Settings → Account, then smoke-test a post:

curl -fsS -X POST http://127.0.0.1:8000/api/artifacts \
  -H "Authorization: Bearer <token-from-settings>" \
  -H "Content-Type: application/json" \
  -d '{"source": "deploy-agent", "kind": "markdown", "title": "Deployment Smoke Test", "content": "# OK\n\nDeployed successfully."}'

Confirm the response includes id, url, and api_url, and that opening url in a browser renders the artifact.

7. Redeploy Loop (routine updates)

Releases are tagged vX.Y.Z (SemVer). Deploy a specific tag rather than a moving branch, so what's live is always a known, named version — and note the currently-deployed one before switching, for a clean rollback target:

cd /opt/daybook
PREV_TAG=$(git describe --tags --exact-match 2>/dev/null || git rev-parse HEAD)  # what's live now
git fetch --tags
git checkout v0.1.0        # the tag you were told to deploy (latest: git tag -l | sort -V | tail -1)
docker compose -f deploy/docker-compose.yml up -d --build

Then repeat the verification in step 6 (and confirm /api/capabilities reports the version you expect). If it fails, roll back to the previous version and report the failure (build log / migration error / failed health check) rather than retrying blindly:

git checkout "$PREV_TAG"
docker compose -f deploy/docker-compose.yml up -d --build

Caveat: rollback reverts code, not data — a release that already applied a DB migration may not cleanly downgrade against the migrated db.sqlite3. For a routine forward release that's fine; if a rollback is needed after a migration, restore the pre-deploy deploy/data backup (see §9) alongside the code checkout.

deploy/.env and the data directory (wherever DAYBOOK_HOST_DATA_DIR points, or deploy/data by default) are untracked, so git pull never touches them — no need to back them up before a routine redeploy.

8. Hardening For Internet-Facing Deployments

If this VM is reachable from the public internet (not just a private network/VPN), do this before calling the deployment done:

• Put a reverse proxy in front with TLS. Bearer tokens go over plaintext HTTP otherwise.
• Add the proxy's public hostname to DJANGO_ALLOWED_HOSTS.
• Set DJANGO_CSRF_TRUSTED_ORIGINS=https://<public-hostname> in deploy/.env, and make sure the proxy forwards X-Forwarded-Proto: https (examples below). Without both, login fails with "CSRF verification failed" — Django can't tell the original request was HTTPS once a proxy terminates TLS and forwards plain HTTP to the container.
• Keep DJANGO_DEBUG=0 (already set above).
• Restrict SSH to key-based auth only; disable password login if you have access to do so.
• Confirm the VM's firewall only allows the ports actually needed (80/443, and SSH from wherever the operator connects from).

Check for an existing reverse proxy before introducing a new one — ss -tlnp | grep -E ':80|:443'. A VM already running other services almost always already has one (Apache, Nginx, Traefik) terminating TLS for everything on it; a second process trying to bind 80/443 will just fail to start, and stopping the existing proxy to "free the port" would take every other service on the box down with it. If one exists, add a vhost to it instead. Apache example:

<VirtualHost *:443>
    ServerName daybook.example.com
    ProxyPreserveHost On
    RequestHeader set X-Forwarded-Proto "https"
    ProxyPass / http://127.0.0.1:8000/
    ProxyPassReverse / http://127.0.0.1:8000/
    SSLEngine on
    SSLCertificateFile /etc/letsencrypt/live/daybook.example.com/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/daybook.example.com/privkey.pem
</VirtualHost>

RequestHeader set X-Forwarded-Proto "https" requires a2enmod headers; it's not optional (see the CSRF note above). (certbot --apache will provision and wire up the certificate for you.)

Only reach for a standalone proxy like Caddy if the target genuinely has nothing on 80/443 yet — it still needs the same header:

daybook.example.com {
    reverse_proxy 127.0.0.1:8000 {
        header_up X-Forwarded-Proto https
    }
}

9. Backup

If you set DAYBOOK_HOST_DATA_DIR in step 5, the real data lives there, not at deploy/data — back up whichever path is actually configured, read straight from deploy/.env rather than assumed, so this keeps working whether or not that override is set:

cd /opt/daybook
DATA_DIR="$(grep -oP '(?<=^DAYBOOK_HOST_DATA_DIR=).*' deploy/.env || echo deploy/data)"
tar -czf "/tmp/daybook-data-$(date +%Y%m%d-%H%M%S).tar.gz" -C "$(dirname "$DATA_DIR")" "$(basename "$DATA_DIR")"

Copy the resulting archive off the VM (to object storage, the operator's machine, etc.) — a backup that only exists on the VM it's protecting against doesn't help if the VM is lost.

Deployment Checklist

Before reporting the deployment complete, confirm all of these:

• Cloned via git (not copied), with a working git pull.
• deploy/.env has a real, non-placeholder DJANGO_SECRET_KEY.
• DJANGO_ALLOWED_HOSTS includes the actual hostname/IP being used.
• If behind a reverse proxy: DJANGO_CSRF_TRUSTED_ORIGINS is set to the public HTTPS origin, and the proxy forwards X-Forwarded-Proto: https. Verify by actually logging in through the proxy, not just curling /api/capabilities — CSRF only bites on POST (login, artifact writes).
• docker compose -f deploy/docker-compose.yml ps shows daybook healthy.
• /api/capabilities returns 403 unauthenticated, 200 with a token.
• At least one superuser exists with a working personal API token.
• A smoke-test artifact posts and renders successfully.
• If internet-facing: TLS is in front of the app, not raw HTTP on 8000.
• A backup of the data directory (deploy/data, or DAYBOOK_HOST_DATA_DIR if you set it) has been taken and copied off the VM.
• If the target host has its own service-inventory, monitoring, or changelog conventions for new services, they've been followed — ask the operator if unsure what those are.

Report back to the operator: the app URL, confirmation of the checklist above, and the deploy SSH key fingerprint or token identity used (not the key/token itself) for their records.