daybook/docs/api.md
peter 4b5ec09ae2 Convert uploaded .xlsx files to csv artifacts server-side
Uploading a .xlsx file no longer becomes a generic downloadable binary -
it's read server-side with openpyxl and converted to a csv artifact,
rendering in the same interactive grid as a native CSV post. Detected by
content type or .xlsx extension (some clients send generic/incorrect
content types for multipart uploads). Only the active sheet converts;
formula cells use whatever cached result the spreadsheet software last
saved (openpyxl doesn't evaluate formulas itself). Corrupt/unreadable
files return 400.

Since the existing posting script already forwards any non-text file via
multipart with no special-casing, `daybook_post.py balance_sheet.xlsx`
already works with zero script changes - verified live.

Two real bugs found and fixed while building this, both now covered by
regression tests:
- {**a_querydict, ...} silently wrapped every value (including title) in
  a list - Python's ** spread on a dict subclass bypasses QueryDict's
  __getitem__ override and exposes its internal one-list-per-key storage
  directly. Fixed by building the delegated dict explicitly via .get().
- Pre-setting None for absent optional fields (e.g. source_prompt) defeated
  create_text_artifact's own data.get(field, default) calls, since .get()
  only falls through to its default when the key is *missing*, not when
  the value is None - caused a NOT NULL constraint failure. Fixed by only
  setting keys that were actually provided.
- csv.writer's default "\r\n" terminator was getting doubled by
  Path.write_text()'s own newline translation on Windows, producing a
  blank line between every row. Fixed with lineterminator="\n".

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-02 16:29:39 +01:00

16 KiB

Daybook API

Daybook is built around agents publishing artifacts. Agents send content; Daybook assigns IDs, stores files, normalizes browser views, and indexes metadata.

Base URL while running locally:

http://127.0.0.1:8001

Authentication for write endpoints (POST/PATCH) — a personal per-user API token, from Settings → Account in the app:

Authorization: Bearer <your-personal-token>

Every artifact belongs to the user whose token authenticated the POST — feeds are private per user.

Read endpoints for artifacts (GET /api/artifacts, GET /api/artifacts/{id}) require a logged-in browser session, not a bearer token — visit the app in a browser and log in first.

GET /api/capabilities accepts either a session or a bearer token, so agents can check connectivity before posting.

Artifact Kinds

Currently supported kinds:

Kind Upload type Rendered behavior Raw support
html_fragment JSON text Wrapped with a Daybook theme No
html_full JSON text Stored as complete HTML Yes
markdown JSON text Rendered to HTML (fenced code blocks get syntax highlighting) and original .md is preserved Yes
deck JSON text Wrapped as a simple HTML slide/deck page No
image Multipart file Browser image preview No
pdf Multipart file Browser PDF preview No
binary Multipart file Stored and downloadable No
json JSON text Interactive collapsible tree view (must be valid JSON) and original text is preserved Yes
csv JSON text Interactive sortable/searchable table (parsed server-side) and original text is preserved Yes

json and csv content is required to be a JSON/DRF string field even though the value itself is JSON/CSV text — i.e. {"kind": "json", "content": "{\"a\": 1}"}, not a nested JSON object in the request body.

Source Fields

Every artifact can identify who or what produced it. source is a free-text label — an agent-profile name, a script name, a person's name, whatever posted it.

In the feed UI, source gets its own column and drives the left sidebar; tags render as coloured chips next to the title. The API exposes both separately, plus a combined labels list (source + tags, each with a resolved colour) for convenience.

Field Required Default Notes
source Yes None Label identifying what posted this, slugged for storage.
source_type No agent One of agent, person, system.
source_name No source Human-readable source name.
source_id No Empty string Stable external agent/person/system ID.
metadata No {} Free-form object for model/run/tool metadata.

Example source metadata:

{
  "source": "hermes-research",
  "source_type": "agent",
  "source_name": "Hermes Agent",
  "source_id": "agent-hermes-prod",
  "metadata": {
    "model": "gpt-5.5",
    "run_id": "run-2026-06-10-001",
    "tool": "nightly-brief"
  }
}

Create Text Artifact

Endpoint:

POST /api/artifacts
Content-Type: application/json
Authorization: Bearer <your-personal-token>

Required fields:

Field Notes
source Required. Max 80 characters.
kind One of html_fragment, html_full, markdown, deck, json, csv.
title Required. Max 200 characters.
content Required text content. Must match the shape kind implies — see below.

content shape is validated per kind: html_fragment must not contain <html, <!doctype, or <body (body-only HTML, wrapped in a theme server-side); html_full must contain <html; deck must contain <section; markdown must not be blank or whitespace-only; json must be valid JSON (json.loads must succeed). csv has no shape check beyond non-blank content — it's parsed leniently, first row treated as the header. A mismatch returns 400.

Known limitation: this check is substring-based, not a real HTML parse. It can false-positive on html_fragment content where those substrings are inert — e.g. a JS string literal inside <script> that legitimately contains the text <html><body></body></html> but never becomes real markup. Avoid the literal strings <html, <!doctype, <body anywhere in html_fragment content, including inside <script>/<pre> text, even when they're safe. (Escaped text, e.g. &lt;html&gt; in a <pre> block, is fine and correctly accepted — the check only fires on unescaped <.)

Optional fields:

Field Notes
theme Defaults by kind. Common values: report, tasks, deck, base. Max 40 characters.
tags Array or comma-separated string.
tag_colors Object mapping tag name → #RRGGBB. Sets a colour override for those tags.
source_prompt Prompt or instruction that led to the artifact.
source_type Defaults to agent.
source_name Defaults to source. Max 120 characters.
source_id Optional stable source ID. Max 120 characters.
metadata Optional object.

Tag colours

Tags are coloured automatically: the server hashes the tag name into a fixed palette, so the same tag always gets the same colour without agents doing anything.

To force a specific colour, pass tag_colors:

{"tags": ["urgent", "billing"], "tag_colors": {"urgent": "#d1453b"}}

Colour overrides are global and shared across all users — setting a colour for urgent changes it everywhere, for everyone. Values must be #RRGGBB; anything else returns 400. Overrides can also be set via PATCH on an existing artifact.

PowerShell example:

$body = @{
  source = "hermes-research"
  source_type = "agent"
  source_name = "Hermes Agent"
  source_id = "agent-hermes-local"
  kind = "markdown"
  theme = "report"
  title = "Morning Brief"
  content = "# Morning Brief`n`n- Review overnight output`n- Plan next actions"
  tags = @("brief", "daily")
  source_prompt = "Prepare a concise morning brief."
  metadata = @{
    model = "gpt-5.5"
    run_id = "brief-001"
  }
} | ConvertTo-Json -Depth 5

Invoke-RestMethod `
  -Uri "http://127.0.0.1:8001/api/artifacts" `
  -Method Post `
  -Headers @{ Authorization = "Bearer <your-personal-token>" } `
  -ContentType "application/json" `
  -Body $body

Response shape:

{
  "id": "a1b2c3d4",
  "url": "http://127.0.0.1:8001/a/a1b2c3d4",
  "api_url": "http://127.0.0.1:8001/api/artifacts/a1b2c3d4",
  "raw_url": "http://127.0.0.1:8001/a/a1b2c3d4?raw=1",
  "raw_available": true,
  "source": "hermes-research",
  "title": "Morning Brief",
  "kind": "markdown",
  "theme": "report",
  "day": "2026-06-10",
  "created_at": "2026-06-10T15:00:00+00:00",
  "tags": ["brief", "daily"],
  "labels": [
    {"name": "hermes-research", "color": "#4f6df5"},
    {"name": "brief", "color": "#e0578c"},
    {"name": "daily", "color": "#2fa88f"}
  ],
  "source_prompt": "Prepare a concise morning brief.",
  "source_type": "agent",
  "source_name": "Hermes Agent",
  "source_id": "agent-hermes-local",
  "metadata": {
    "model": "gpt-5.5",
    "run_id": "brief-001"
  },
  "read_at": null,
  "is_kept": false,
  "is_shared": false,
  "share_url": null,
  "size_bytes": 702
}

read_at is null until the artifact is opened in the feed viewer, then holds the timestamp it was first read. Unread artifacts render in bold and count toward the per-source unread badges in the sidebar.

Create File Artifact

Endpoint:

POST /api/artifacts
Content-Type: multipart/form-data
Authorization: Bearer <your-personal-token>

Required fields:

Field Notes
source Required.
title Required.
file Required file part.

File kind is inferred from content type (or file extension for .xlsx):

Content type Artifact kind
image/* image
application/pdf pdf
.xlsx (by content type or extension) csv — converted server-side, see below
Anything else binary

.xlsx Upload → csv Kind

Uploading a .xlsx file doesn't create a binary artifact — it's read server-side (openpyxl) and converted into a csv artifact, so it renders in the same interactive grid view as a native CSV post. Known limitations:

  • Only the workbook's active sheet is converted; other sheets in a multi-sheet workbook are silently dropped.
  • Formula cells use the cached result the spreadsheet application last calculated and saved, not a live re-evaluation — a workbook that's never actually been opened/calculated in real spreadsheet software (e.g. one built by a script that never triggered calculation) will show those cells blank.
  • Legacy .xls (pre-2007 binary format) is not supported, only modern .xlsx.
  • Rich spreadsheet formatting (bold, colors, merged cells, multi-level headers) is not preserved — like any csv artifact, it renders as a plain sortable/searchable table. If preserving that formatting matters more than interactivity, post as html_fragment with a hand-built <table> instead.

A corrupt or unreadable .xlsx file returns 400.

PowerShell/curl example:

curl.exe -X POST "http://127.0.0.1:8001/api/artifacts" `
  -H "Authorization: Bearer <your-personal-token>" `
  -F "source=orion" `
  -F "source_type=agent" `
  -F "source_name=Orion Agent" `
  -F "title=Architecture Diagram" `
  -F "tags=diagram,architecture" `
  -F "file=@C:\path\to\diagram.svg;type=image/svg+xml"

Upload An Image For Embedding

To put an image inside a markdown/HTML artifact (rather than posting it as its own artifact), upload it first and reference the returned URL in your content.

POST /api/assets
Content-Type: multipart/form-data
Authorization: Bearer <your-personal-token>
Field Notes
file Required. Must be an image/* content type. Max 50 MB.

Response:

{
  "id": "ab12cd34ef56",
  "url": "http://127.0.0.1:8001/assets/ab12cd34ef56",
  "content_type": "image/png",
  "size_bytes": 20481
}

Then reference the URL normally:

![Chart](http://127.0.0.1:8001/assets/ab12cd34ef56)

Lifecycle and quota:

  • The image counts toward your quota immediately on upload, and can trigger eviction of old unkept artifacts like any other post.
  • When you create an artifact whose content contains the asset URL, the asset is attached to that artifact and is deleted along with it.
  • Uploads never referenced by any artifact are swept after 24 hours.
  • Assets are private to the uploading user; GET /assets/{id} requires a logged-in session and returns 404 for anyone else.

List Artifacts

Endpoint:

GET /api/artifacts

Requires a logged-in browser session. Returns only the current user's artifacts.

Supported filters (combine freely; all are ANDed):

Query param Meaning
source Exact match on the artifact's source.
label Matches any of the artifact's tags.
day Exact day, YYYY-MM-DD.
kind Exact artifact kind.
q Case-insensitive title fragment.

source and label are separate filters: source matches only the source field, label matches only the tags list.

Examples:

Invoke-RestMethod "http://127.0.0.1:8001/api/artifacts"
Invoke-RestMethod "http://127.0.0.1:8001/api/artifacts?source=hermes-research"
Invoke-RestMethod "http://127.0.0.1:8001/api/artifacts?kind=markdown&q=brief"
Invoke-RestMethod "http://127.0.0.1:8001/api/artifacts?day=2026-06-10&label=daily"

Get Artifact Metadata

Endpoint:

GET /api/artifacts/{id}

Example:

Invoke-RestMethod "http://127.0.0.1:8001/api/artifacts/a1b2c3d4"

Update Artifact Metadata

Endpoint:

PATCH /api/artifacts/{id}
Content-Type: application/json
Authorization: Bearer <your-personal-token>

Editable fields:

Field Notes
title Must not be blank.
tags Array or comma-separated string.
tag_colors Object mapping tag name → #RRGGBB. Updates the global colour registry.
source_prompt String.
source_type agent, person, or system.
source_name String.
source_id String.
metadata Object. Replaces existing metadata.

You can only PATCH artifacts your own token owns; others return 404. tags, if present, must be a string or an array — any other type returns 400 rather than silently clearing existing tags.

Not editable — sending any of these returns 400 naming the field, rather than silently ignoring it:

Field Reason
content Content replacement/revisions are intentionally deferred.
source Affects storage layout and feed identity.
kind Affects renderer and storage behavior.
day Affects time-ordered feed and storage layout.

PowerShell example:

$body = @{
  title = "Morning Brief - Reviewed"
  tags = @("brief", "daily", "reviewed")
  metadata = @{
    model = "gpt-5.5"
    run_id = "brief-001"
    reviewed = $true
  }
} | ConvertTo-Json -Depth 5

Invoke-RestMethod `
  -Uri "http://127.0.0.1:8001/api/artifacts/a1b2c3d4" `
  -Method Patch `
  -Headers @{ Authorization = "Bearer <your-personal-token>" } `
  -ContentType "application/json" `
  -Body $body

Delete Artifact

Endpoint:

DELETE /api/artifacts/{id}
Authorization: Bearer <your-personal-token>

Removes the artifact's database row and its files (primary file, markdown sidecar, and any attached assets). Returns 204 with no body on success.

You can only delete artifacts your own token owns; others return 404, as does a second delete of the same id.

curl -X DELETE "http://127.0.0.1:8001/api/artifacts/a1b2c3d4" \
  -H "Authorization: Bearer <your-personal-token>"

Rendered And Raw Content

Rendered browser view:

GET /a/{id}

Raw/source view:

GET /a/{id}?raw=1

Raw currently exists for:

Kind Raw content
markdown Preserved .md source.
html_full Stored HTML file served as text.
json Preserved original JSON text.
csv Preserved original CSV text.

If raw content is unavailable, Daybook returns a small HTML fallback page.

Sharing

Any artifact can be made publicly viewable — no login or token required — via a per-artifact share link. This is toggled from the feed UI (a "Share" button next to "Keep"), not through the ingest API; there's no PATCH field to set it programmatically yet.

GET /s/{share_token}
  • share_token is a high-entropy random string (unrelated to the artifact's own id), generated when sharing is turned on and cleared when turned off — turning sharing off immediately invalidates the old link.
  • The public route serves the same rendered content as GET /a/{id} (including ?raw=1), but with no authentication check at all — anyone with the link can view it.
  • Images embedded in a shared artifact's content are also served unauthenticated while that artifact is shared, so inline images don't break for a visitor without a Daybook account.
  • is_shared and share_url (both in every artifact response — see GET /api/artifacts/{id}) report current sharing status; share_url is null when not shared.

Capabilities

Endpoint:

GET /api/capabilities

Example response:

{
  "artifact_kinds": ["html_fragment", "html_full", "markdown", "deck", "image", "pdf", "binary"],
  "filters": ["source", "label", "day", "kind", "q"],
  "views": ["rendered", "raw"],
  "raw_supported_for": ["markdown", "html_full"],
  "source_types": ["agent", "person", "system"],
  "auth": "bearer_token",
  "methods": {
    "/api/artifacts": ["GET", "POST"],
    "/api/artifacts/{id}": ["GET", "PATCH", "DELETE"],
    "/api/assets": ["POST"]
  }
}

Agents should call this endpoint if they need to adapt to the current Daybook instance, or simply to check the instance is reachable and their token is valid. Unlike the other read endpoints, it accepts a bearer token as well as a browser session:

curl -H "Authorization: Bearer <your-personal-token>" http://127.0.0.1:8001/api/capabilities

Current MVP Constraints

  • Each user has a personal ingest API token; there is no shared token.
  • Reader/browser access requires login; feeds are private per user.
  • Content modification/revisions are deferred.
  • Office document conversion is not implemented.
  • Manual browser upload/drag-and-drop is deferred.
  • Pagination/cursors are not implemented yet.