daybook/planning/daybook_planning.md

# Daybook — Project Planning

## Purpose

**Daybook** is a self-hosted web application where AI agents publish their
outputs (task lists, research reports, slide decks, images, etc.) and a human
browses them in a **time-ordered, agent-filterable feed** — viewing everything
**in-browser without downloading or "opening" files**.

The defining feature is the integration: an agent calls a REST API and posts
*content*, not files. For HTML it can send just a body fragment and Daybook
supplies the CSS. The agent never deals with styling, file naming, or storage.

## Core principles

- **View, don't download.** Everything renders inline in the browser.
- **Agent-agnostic.** Any agent/persona can publish; the human filters by persona.
- **Time-ordered.** The primary axis is the day; flipping back to yesterday is trivial.
- **Minimal agent burden.** Agents send content + a few metadata fields. The server
  owns styling, storage layout, IDs, and dates.
- **Self-hosted.** A single deployable unit running on the user's own machine/server.
- **Dumb viewer, smart ingest.** Normalize every input to three browser-native forms
  (HTML, image, PDF) at upload time, so the viewer never needs plugins.

## Actors

- **Agent (publisher):** an authenticated machine client (e.g. Claude Code, a
  persona script). POSTs artifacts via REST.
- **Human (reader):** browses, filters, and views artifacts; later edits markdown
  and shares links.

## Scope

### MVP
- REST ingest endpoint accepting: HTML fragment, full HTML, markdown, image, PDF, binary.
- Server-side **named CSS themes**; automatic wrapping of HTML fragments.
- Storage in a `persona/day/id` layout plus a metadata table.
- Web viewer: reverse-chronological feed, persona filter, in-browser rendering.

### Later
- In-browser **markdown editing** with write-back (the name "Daybook" anticipates this).
- Slide decks via a **reveal.js** theme; `pptx`/`docx`/`xlsx` → PDF fallback.
- **Share links** (password + expiry) for individual artifacts.
- Tags, search, and full-text indexing.

## Architecture

```
Agent ──POST /api/artifacts──▶  Daybook (Django + DRF)
                                  │
                                  ├─ normalize-on-ingest
                                  │    fragment → wrap with theme CSS → .html
                                  │    markdown → render → .html
                                  │    deck     → reveal.js scaffold → .html
                                  │    office   → LibreOffice headless → .pdf
                                  │    image/pdf → store as-is
                                  │
                                  ├─ write file to  artifacts/{persona}/{day}/{id}.{ext}
                                  └─ insert Artifact row (metadata index)

Human ──GET / (feed)──▶  list Artifact rows (reverse-chron, persona filter)
       ──GET /a/{id}──▶  render via <iframe> (html/pdf) or <img> (image)
```

The viewer only ever handles **HTML, image, or PDF**, because everything is
collapsed into those three at ingest. No client-side office/pptx rendering.

## Data model

`Artifact`

| Field | Type | Notes |
|---|---|---|
| `id` | str (8-char) | short unique id, used in URLs (`/a/{id}`) |
| `persona` | str (indexed) | the agent that published it; primary filter |
| `title` | str | display title |
| `kind` | str | `html_fragment` \| `html_full` \| `markdown` \| `deck` \| `image` \| `pdf` \| `binary` |
| `theme` | str | named CSS theme used at render (nullable) |
| `day` | date (indexed) | publication day; feed grouping key |
| `created_at` | datetime | exact timestamp |
| `path` | str | relative path to stored, normalized file |
| `original_kind` | str | what the agent sent, pre-normalization (audit) |
| `tags` | json/list | later: filtering/search |
| `source_prompt` | text | optional: the prompt that produced it |

Share fields (`share_token`, `share_expires`, `share_password_hash`) added in the
share-links milestone, not MVP.

## REST API (MVP contract)

All ingest endpoints require an API token (per-persona; see Open Questions).

### `POST /api/artifacts`  (JSON — text content)
```json
{
  "persona": "hermes",
  "kind": "html_fragment",        // or html_full | markdown | deck
  "theme": "report",              // optional; defaults per kind
  "title": "Daily Tasks 2026-06-10",
  "content": "<h1>Today</h1><ul><li>...</li></ul>",
  "tags": ["tasks"],              // optional
  "source_prompt": "..."          // optional
}
```
Response: `{ "id": "a1b2c3d4", "url": "/a/a1b2c3d4" }`

### `POST /api/artifacts`  (multipart — binary)
Fields: `persona`, `title`, `tags`; file part `file` (image / pdf / office).
Office/deck files are converted to PDF on ingest.

### `GET /api/artifacts?persona=&day=&tag=`
Returns metadata list (reverse-chronological). Drives the feed.

### `GET /a/{id}`
Serves the rendered artifact (HTML/PDF/image) for inline viewing.

## Theming (server-side CSS)

The server owns styling so agents can send markup only. Themes live as static
files and are selected by name in the POST body:

```
static/themes/report.css
static/themes/tasks.css
static/themes/deck.css        # reveal.js theme
static/themes/base.css        # default
```

Fragment wrapping (conceptual):

```python
WRAP = """<!doctype html><html><head><meta charset="utf-8">
<link rel="stylesheet" href="/static/themes/{theme}.css">
<title>{title}</title></head>
<body class="artifact">{body}</body></html>"""
```

- `html_fragment` → wrapped with the chosen theme.
- `markdown` → rendered to HTML, then wrapped.
- `deck` → wrapped in reveal.js scaffold (agent sends `<section>` blocks).
- `html_full` → stored verbatim (agent owns everything).

## Ingest normalization pipeline

| Agent sends | Step | Stored as | Viewer |
|---|---|---|---|
| HTML fragment | wrap + theme | `.html` | iframe |
| Full HTML | verbatim | `.html` | iframe |
| Markdown | render + wrap | `.html` | iframe |
| Deck (sections) | reveal.js wrap | `.html` | iframe |
| Image | verbatim | image | `<img>` |
| PDF | verbatim | `.pdf` | native / pdf.js |
| pptx/docx/xlsx | LibreOffice → PDF | `.pdf` | native / pdf.js |

## Viewer

- **Feed view (`/`):** reverse-chronological list grouped by day, with a persona
  filter (dropdown or sidebar) and, later, tag/search. Each entry shows title,
  persona, time, and kind.
- **Artifact view (`/a/{id}`):** renders inline — `<iframe>` for HTML/PDF,
  `<img>` for images. No download step required (download remains available).
- Markdown edit (later): the stored source markdown is editable in-browser and
  re-rendered on save; keep the original `.md` alongside the rendered `.html`.

## Storage layout

```
artifacts/
  {persona}/
    {YYYY-MM-DD}/
      {id}.html | {id}.pdf | {id}.png ...
      {id}.md            # kept when source is markdown (for editing)
```

Files on disk stay greppable, portable, and trivially backed up; the DB is an
index over them, not the source of truth for content.

## Tech stack

- **Django + Django REST Framework** — API, metadata, viewer.
- **SQLite** (dev) → **PostgreSQL** (later) for the metadata index.
- **Files on disk** for artifact content.
- **LibreOffice headless** in the container for office/deck → PDF.
- **markdown** (python-markdown) for md rendering; **reveal.js** for decks.
- **pdf.js** optional, only if the native PDF viewer is unsatisfactory.
- **Docker** — single image, self-hosted.

## Milestones

1. **M1 — Core ingest + storage.** `Artifact` model, JSON POST for
   `html_fragment`/`html_full`/`markdown`, theme wrapping, disk write, `/a/{id}` view.
2. **M2 — Viewer.** Feed (`/`) with reverse-chron listing and persona filter;
   inline iframe/img rendering.
3. **M3 — Binary + conversion.** Multipart upload; image/PDF passthrough;
   LibreOffice headless for office/deck → PDF.
4. **M4 — Decks.** reveal.js theme + `deck` kind.
5. **M5 — Editing.** In-browser markdown edit + re-render.
6. **M6 — Sharing.** Share tokens with password + expiry.
7. **M7 — Search/tags.** Tag filter and full-text search.

## Future features

- Per-persona dashboards / stats.
- Webhooks or "latest artifact" endpoints for chaining agents.
- Retention/archival policies per persona.
- Multi-user reader auth with per-persona visibility.
- **Interactive questions artifact type:** agents send JSON with multiple-choice questions; viewer renders an interactive form; user selections are sent back to the agent via message queue, email, or API callback. Use cases: agent feedback loops, preference gathering, approval workflows.

## Open questions / decisions

- **Output standardization:** push agents toward HTML/markdown/reveal.js
  (cheap, robust) vs. accept native office files (LibreOffice conversion is the
  heaviest, flakiest path). Recommendation: prefer the former, keep conversion
  as a fallback.
- **Agent auth:** one API token per persona, or a shared token + persona field?
- **Reader auth:** single-user (simplest) vs. multi-user from the start?
- **ID scheme:** random short id vs. date-prefixed sortable id.