Commit graph

15 commits

Author SHA1 Message Date
fb69261218 Add GET /api/skill endpoint serving the posting skill
Serves skills/daybook-posting/SKILL.md so users/agents can fetch the
posting instructions from the running instance instead of hunting for the
file. The server rewrites the DAYBOOK_URL config line to the request's own
origin, so the text handed to an agent already points at the right server;
the recipient only supplies their token.

- Same auth as /api/capabilities (session or bearer token); no user data.
- Raw text/markdown by default, ?format=json returns {version, url, skill}.
- Advertised via skill_url + a /api/skill methods entry in capabilities.
- Ships in the Docker image (COPY . /app, skills/ not dockerignored).
- Tests for auth, origin rewrite, JSON mode, session auth, capabilities ad.
- docs/api.md + CHANGELOG updated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-03 15:50:04 +01:00
ca8e364010 Fix healthcheck + verify commands; reframe deploy runbook for a capable agent
Real bug: the container healthcheck (and the docs' verify step) hit
/api/capabilities, which returns 403 unauthenticated; curl --fail treats
403 as an error, so a healthy container would report "unhealthy" and the
verify step would look like a failure. Healthcheck now hits /accounts/login/
(200, no auth). Verify commands now print the status code and note not to
use curl -f on the 403.

Also reframed cloud_agent_deploy.md's intro for an Opus-class deploy agent:
it's a guide + Daybook-specific gotchas, not a rigid script — points at the
authoritative sources (repo incl. the git-SSH :2222 note, deploy/ files,
requirements.txt deps auto-installed, entrypoint runs migrate+collectstatic,
docs/deployment.md) and trusts the agent's judgment. Dropped the now-
redundant Source Of Truth section.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-03 14:32:39 +01:00
5c53643349 Document that html_full is intentionally not validated
Records the decision to keep html_full fully flexible: no sanitization, no
CSP, and no HTML/CSS validation — malformed markup is kept as-is. The
themed kinds (markdown/html_fragment/deck) remain sanitized, which also
normalizes them to well-formed HTML.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-03 10:50:06 +01:00
e3c5773e8f Fix #014: sanitize themed text kinds + script-blocking CSP (stored XSS)
markdown/deck/html_fragment rendered author-influenced content verbatim,
so <script>/<img onerror>/etc. executed on the Daybook origin — self-XSS
against the owner's session, and (post-sharing) unauthenticated via /s/.

Two layers, both in artifacts/views.py:
- Create-time allowlist sanitization via nh3 (maintained ammonia/Rust
  sanitizer, not deprecated bleach). Keeps formatting/tables/links/images
  (incl. relative /assets URLs) and codehilite markup; drops script, on*
  handlers, style, unknown tags, and javascript:/data: URLs. html_full is
  deliberately NOT sanitized (documented verbatim escape hatch).
- Serve-time CSP (script-src 'none') for the sanitized kinds, on both
  /a/{id} and the unauthenticated /s/{token} route. Being a response
  header, it also neutralizes scripts in artifacts stored before this fix.
  Not applied to html_full or json/csv (json/csv data is already escaped;
  their viewers need their own JS).

Verified live across all kinds; 7 regression tests added. 102 tests pass.

Not addressed (noted in the issue): CSV formula injection,
CSRF_COOKIE_HTTPONLY (moot with XSS closed), nonce CSP for json/csv/full.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-03 10:44:49 +01:00
c9edac62cf Review docs/api.md and docs/llm_posting_helper.md for accuracy and completeness
Same pass as the SKILL.md review. Fixed factually-wrong content and filled
gaps against the current code.

docs/api.md:
- Capabilities example response was stale: artifact_kinds missing json/csv,
  raw_supported_for missing json/csv. Corrected to match live output, with
  a note that the endpoint is the authoritative source.
- "Office document conversion is not implemented" is now false - replaced
  with the accurate scope (.xlsx -> csv only; .docx/.pptx/.xls not).
- Source Fields overview table omitted source_prompt entirely; added it,
  plus a note that source_name/source_id/source_prompt must be strings
  (400, not coerced) and metadata must be an object.
- Kinds table: noted tables now render styled in markdown/html_fragment,
  and made explicit that html_full is served verbatim with no styling.
- MVP constraints: noted the public-share-link exception to "feeds private
  per user" and clarified PATCH edits metadata only.

docs/llm_posting_helper.md:
- Added .xlsx upload, --upload-image image embedding, a Source & Theme
  Flags table (all six source-*/theme flags were undocumented here), and a
  note that --meta/--meta-json merge and are distinct from tags.
- Added a scope note that the helper only creates - update/delete need the
  API, sharing is feed-UI-only.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-02 16:49:19 +01:00
4b5ec09ae2 Convert uploaded .xlsx files to csv artifacts server-side
Uploading a .xlsx file no longer becomes a generic downloadable binary -
it's read server-side with openpyxl and converted to a csv artifact,
rendering in the same interactive grid as a native CSV post. Detected by
content type or .xlsx extension (some clients send generic/incorrect
content types for multipart uploads). Only the active sheet converts;
formula cells use whatever cached result the spreadsheet software last
saved (openpyxl doesn't evaluate formulas itself). Corrupt/unreadable
files return 400.

Since the existing posting script already forwards any non-text file via
multipart with no special-casing, `daybook_post.py balance_sheet.xlsx`
already works with zero script changes - verified live.

Two real bugs found and fixed while building this, both now covered by
regression tests:
- {**a_querydict, ...} silently wrapped every value (including title) in
  a list - Python's ** spread on a dict subclass bypasses QueryDict's
  __getitem__ override and exposes its internal one-list-per-key storage
  directly. Fixed by building the delegated dict explicitly via .get().
- Pre-setting None for absent optional fields (e.g. source_prompt) defeated
  create_text_artifact's own data.get(field, default) calls, since .get()
  only falls through to its default when the key is *missing*, not when
  the value is None - caused a NOT NULL constraint failure. Fixed by only
  setting keys that were actually provided.
- csv.writer's default "\r\n" terminator was getting doubled by
  Path.write_text()'s own newline translation on Windows, producing a
  blank line between every row. Fixed with lineterminator="\n".

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-02 16:29:39 +01:00
91ea19d26b Add per-artifact public sharing
Any artifact can now be made publicly viewable via a share link, no login
required - opt-in per artifact, toggled from the feed UI next to Keep.

- Artifact.share_token: nullable, unique, high-entropy (secrets.token_urlsafe(32)).
  NULL = not shared; SQL treats each NULL as distinct so uniqueness still
  holds for the many not-shared rows.
- New public route GET /s/<token> (shared_artifact_view), completely
  unauthenticated, serving the same rendered/raw content as the owner
  route via a shared serve_artifact_file() helper.
- toggle_artifact_share: generates/clears the token: mirrors
  toggle_artifact_keep's pattern.
- Fixed a real gap this surfaced: embedded images in a shared artifact's
  content would otherwise break for an anonymous visitor, since asset_view
  was owner-only. It now also allows access when the asset's parent
  artifact is currently shared - unattached (orphaned) assets stay private
  regardless.
- serialize_artifact now reports is_shared/share_url so the API reflects
  sharing status (read-only for now; toggling stays UI-only).
- Feed UI: Share button (mirrors Keep's toggle interaction), a copyable
  share-link box that appears when shared, and a "shared" badge on the
  artifact row - distinct blue accent from Keep's green so both can be
  visible on the same row without confusion.

Verified live end-to-end (toggle on/off, public access, fake-token 404,
old-link-dies-on-unshare, embedded-asset access before/after sharing)
before writing the 12 automated tests in tests/views/test_sharing.py.
Also fixed two unrelated stale spots noticed in docs/api.md while adding
the Sharing section: the example response was missing is_kept/size_bytes,
and the raw-content table didn't list json/csv (added earlier this
session but never documented there).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-02 16:00:39 +01:00
e137725cab Add json/csv support to the posting script (issue #012)
TEXT_KINDS and TEXT_SUFFIXES now include json/csv, with matching
sniff_text_kind branches so both explicit --kind and positional auto-mode
work. Added validate_payload's JSON shape check mirroring the server.
Updated SKILL.md/docs/llm_posting_helper.md with real usage examples,
replacing the stale --as-markdown workaround guidance (kept only for
genuinely unstructured code, which still has no dedicated kind). Also
fixed an unrelated stale doc line found in the same table: html_full's
validation was documented as requiring <!doctype AND <html, but only
<html is actually required.

Verified via dry-run (explicit --kind, auto-mode, invalid-JSON rejection)
and one real end-to-end post through the script against the live dev
instance.

Added tests/scripts/test_daybook_post.py - 17 tests, plain unittest, zero
new dependencies, loaded via importlib since the script isn't a package.
Closes the test-coverage question the issue raised.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-02 15:38:47 +01:00
532b4bd4eb Document #009's substring-check limitation; fix #011's CDN docs and theme typo
#009: html_fragment's content-shape check is substring-based, so it can
false-positive on inert text inside <script>/<pre> (e.g. a JS string
literal that happens to contain "<html>"). Documented as a known
limitation in docs/api.md rather than implementing real HTML parsing - a
proper fix is a much bigger change than the narrow trigger justifies, and
substring matching is the same approach the client helper already uses.

#011: added a curated CDN Snippets section to SKILL.md for html_full
(Tailwind, Chart.js, with an offline/air-gapped caveat), scoped explicitly
to that kind only. Fixed daybook_post.py's --theme help text, which named
a non-existent "bootstrap" theme (written ahead of a vendored-Bootstrap
feature that was planned but never built, not a typo).

Also updated SKILL.md's kind cheat sheet, which never mentioned the
json/csv kinds and still pointed at the old --as-markdown workaround for
them - and filed issue #012 for a real gap that surfaced doing so: the
posting script itself doesn't support --kind json/csv yet, only the raw API
does.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-02 15:32:22 +01:00
e5d2f192df Add json and csv artifact kinds with interactive viewers
Gap 1 from planning/2026-09-02-artifact_kind_gaps_for_frequent_agent_output.md.
Chose new first-class kinds over riding on html_fragment, resolving that
doc's open design question in favor of filtering support, raw-source
preservation (like markdown's .md sidecar), and server-owned validation.

- json: validated with json.loads at post time (400 on invalid JSON),
  rendered with renderjson (collapsible tree), vendored locally.
- csv: parsed server-side with the stdlib csv module into columns/rows and
  handed to Grid.js as plain data - no client-side CSV-parsing plugin
  needed, keeping normalization server-owned. Vendored locally (MIT).
- Both preserve their original text as a sidecar file, generalizing what
  was previously markdown-only handling (storage.py's SIDECAR_SUFFIXES) so
  disk-size accounting, eviction cleanup, and ?raw=1 all cover json/csv too.
- Raw JSON is embedded in a <script type="application/json"> block with
  `</script` occurrences escaped (safe because JSON's \/ escape round-trips
  correctly) - verified with a live </script>-injection payload against the
  actual running server, not just unit tests.

New migration (0012) for the KIND_CHOICES change - SQLite doesn't enforce
choices at the schema level, but Django's migration state tracks it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-02 14:56:20 +01:00
af9ecf28a9 Add syntax highlighting for fenced code blocks in markdown artifacts
Enables codehilite on the markdown renderer (Pygments, 'friendly' style,
vendored as static/themes/pygments.css) so fenced code blocks get real
syntax highlighting with zero client-side JS - matching the "themes are
CSS-only" stance in framework_artifact_rendering.md. Verified live against
the running dev instance: a fenced python block renders with highlighted
keyword/string/function spans.

This was Gap 2 from planning/2026-09-02-artifact_kind_gaps_for_frequent_agent_output.md,
picked as the first fix since it needed no new kind or design decision -
just enabling an existing markdown extension.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-02 14:39:02 +01:00
07328a3412 Fix issues #002-#006 found by the LLM test harness
- #006 (high, data loss): make_room_for() no longer deletes unkept
  artifacts on a doomed request - checks whether eviction can possibly
  succeed before deleting anything. Also fixed a related latent bug found
  while testing: the function returned evicted Artifact objects after
  calling .delete() on them, so their .id was always None (Django clears
  pk on delete); now returns ids captured beforehand.
- #003: added server-side field-length validation (source/title/
  source_name/source_id/theme) on POST and PATCH, matching the model's
  own max_length limits, which were previously unenforced on the ingest
  path entirely.
- #004: ported daybook_post.py's content-shape validation
  (html_fragment/html_full/deck/markdown) to the server, closing the gap
  for any client that doesn't go through the helper script.
- #005: PATCH now rejects wrong-typed `tags` (previously silently wiped
  existing tags) and rejects immutable fields (content/source/kind/day)
  with 400 instead of silently no-op'ing them.
- #002: deleted the misleading skills/daybook-posting/.env.example (a
  zero-dependency script that never reads .env files) and clarified
  SKILL.md/docs/llm_posting_helper.md that these must be real
  process/shell env vars.

Adds tests/unit/ for storage-layer tests. 46 tests passing (was 25).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-02 13:38:59 +01:00
f2f2a9c96b Close out issue 001: document CSRF/reverse-proxy requirement
The app-side fix (CSRF_TRUSTED_ORIGINS, SECURE_PROXY_SSL_HEADER in
settings.py) was already committed as d458cd3 after the first Cloud-VM-1
deployment hit "CSRF verification failed" on login behind Apache. This
finishes the rest of that issue's checklist: documents
DJANGO_CSRF_TRUSTED_ORIGINS in deploy/env.example, adds the required
X-Forwarded-Proto header to the Apache/Caddy examples in
cloud_agent_deploy.md (plus a checklist item that verifies via an actual
login, not just an unauthenticated capabilities curl), and adds the same
proxy-agnostic guidance to docs/deployment.md.

Also introduces issues/, a tracked one-file-per-issue log for the project
going forward.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-02 12:28:41 +01:00
4a5ea8944b Add generic deployment docs and fix stale references
Adds docs/deployment.md, a platform-agnostic guide (any Linux host with
Docker) built around a git-pull based fast-redeploy workflow, replacing the
Proxmox-specific deploy/proxmox_llm_deploy.md which was also stale (old
`persona` field, single-shared-token model predating multi-user auth).

Also: drops the unused DAYBOOK_API_TOKEN from deploy/env.example, documents
the new DELETE endpoint and capabilities `methods` field in docs/api.md and
docs/agent_quickstart.md, and fixes CLAUDE.md's OpenCode note now that
opencode.json is untracked (machine-specific MCP config).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-01 22:14:34 +01:00
940a703cd7 Initial public commit
Daybook: a self-hosted, multi-user Django + DRF app where agents, scripts,
or people publish artifacts and each logged-in user browses their own
time-ordered, label-filterable feed.

This squashes prior local development history (kept privately on a separate
branch) into a single clean starting point for the public repository.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-01 21:55:02 +01:00