peter/daybook
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
19 commits 1 branch 17 tags 2.4 MiB
Commit graph

19 commits

This branch
This branch
All branches
Author SHA1 Message Date
peter
91ea19d26b Add per-artifact public sharing 
Any artifact can now be made publicly viewable via a share link, no login
required - opt-in per artifact, toggled from the feed UI next to Keep.

- Artifact.share_token: nullable, unique, high-entropy (secrets.token_urlsafe(32)).
  NULL = not shared; SQL treats each NULL as distinct so uniqueness still
  holds for the many not-shared rows.
- New public route GET /s/<token> (shared_artifact_view), completely
  unauthenticated, serving the same rendered/raw content as the owner
  route via a shared serve_artifact_file() helper.
- toggle_artifact_share: generates/clears the token: mirrors
  toggle_artifact_keep's pattern.
- Fixed a real gap this surfaced: embedded images in a shared artifact's
  content would otherwise break for an anonymous visitor, since asset_view
  was owner-only. It now also allows access when the asset's parent
  artifact is currently shared - unattached (orphaned) assets stay private
  regardless.
- serialize_artifact now reports is_shared/share_url so the API reflects
  sharing status (read-only for now; toggling stays UI-only).
- Feed UI: Share button (mirrors Keep's toggle interaction), a copyable
  share-link box that appears when shared, and a "shared" badge on the
  artifact row - distinct blue accent from Keep's green so both can be
  visible on the same row without confusion.

Verified live end-to-end (toggle on/off, public access, fake-token 404,
old-link-dies-on-unshare, embedded-asset access before/after sharing)
before writing the 12 automated tests in tests/views/test_sharing.py.
Also fixed two unrelated stale spots noticed in docs/api.md while adding
the Sharing section: the example response was missing is_kept/size_bytes,
and the raw-content table didn't list json/csv (added earlier this
session but never documented there).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-02 16:00:39 +01:00
peter
e137725cab Add json/csv support to the posting script (issue #012) 
TEXT_KINDS and TEXT_SUFFIXES now include json/csv, with matching
sniff_text_kind branches so both explicit --kind and positional auto-mode
work. Added validate_payload's JSON shape check mirroring the server.
Updated SKILL.md/docs/llm_posting_helper.md with real usage examples,
replacing the stale --as-markdown workaround guidance (kept only for
genuinely unstructured code, which still has no dedicated kind). Also
fixed an unrelated stale doc line found in the same table: html_full's
validation was documented as requiring <!doctype AND <html, but only
<html is actually required.

Verified via dry-run (explicit --kind, auto-mode, invalid-JSON rejection)
and one real end-to-end post through the script against the live dev
instance.

Added tests/scripts/test_daybook_post.py - 17 tests, plain unittest, zero
new dependencies, loaded via importlib since the script isn't a package.
Closes the test-coverage question the issue raised.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-02 15:38:47 +01:00
peter
532b4bd4eb Document #009's substring-check limitation; fix #011's CDN docs and theme typo 
#009: html_fragment's content-shape check is substring-based, so it can
false-positive on inert text inside <script>/<pre> (e.g. a JS string
literal that happens to contain "<html>"). Documented as a known
limitation in docs/api.md rather than implementing real HTML parsing - a
proper fix is a much bigger change than the narrow trigger justifies, and
substring matching is the same approach the client helper already uses.

#011: added a curated CDN Snippets section to SKILL.md for html_full
(Tailwind, Chart.js, with an offline/air-gapped caveat), scoped explicitly
to that kind only. Fixed daybook_post.py's --theme help text, which named
a non-existent "bootstrap" theme (written ahead of a vendored-Bootstrap
feature that was planned but never built, not a typo).

Also updated SKILL.md's kind cheat sheet, which never mentioned the
json/csv kinds and still pointed at the old --as-markdown workaround for
them - and filed issue #012 for a real gap that surfaced doing so: the
posting script itself doesn't support --kind json/csv yet, only the raw API
does.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-02 15:32:22 +01:00
peter
09ea0a8b4f Reject non-string source_name/source_id/source_prompt (issue #008) 
These accepted any JSON type on both POST and PATCH, silently mangled
into Python repr() strings at save time (a dict became the literal text
"{'a': 1}", unparseable as JSON) - and the immediate response echoed the
raw pre-save value, while a later GET returned the mangled one, an
internally inconsistent API.

Added validate_string_fields(), rejecting non-string values with 400
before they reach save(), matching #005's approach for tags. Verified live
and with new tests.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-02 15:27:07 +01:00
peter
d1b6a19eb3 Fix concurrent posts racing past quota (issue #010) 
make_room_for() read usage before any write lock was held, so concurrent
requests could each read stale data and both conclude "this fits" -
confirmed reproducible (measured 616 bytes over a 1551-byte quota with 5
truly concurrent posts).

Fix: force SQLite to acquire its write lock immediately via a no-op UPDATE
on the user's UserProfile row, before reading `used`, wrapped in
transaction.atomic() through the subsequent Artifact.objects.create() so
the lock is held across the whole eviction+insert sequence. Added a 20s
DB timeout as headroom for legitimate contention.

Deliberately no retry/backoff logic - this project's actual load (a
low-volume self-hosted tool) doesn't need it, and SQLite has no row-level
locking regardless, so quota-affecting writes serialize globally either
way. Did fix a real bug found while verifying this: exception handling
around artifact creation only caught QuotaExceeded, so a genuine lock
timeout left the just-written file orphaned on disk - both call sites now
clean up on any exception.

Verified live against the real dev instance (twice, with 5 truly
simultaneous posts against a tight quota) rather than via an automated
test - Django's in-memory shared-cache test database uses a different,
non-retryable SQLite locking model that fails for reasons unrelated to
this fix's correctness.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-02 15:22:37 +01:00
peter
696cfe7d65 Style tables in markdown/html_fragment content 
Markdown pipe-tables and hand-written html_fragment <table> markup already
rendered - the only existing rule was 'max-width: 100%', so they came out
as bare, unbordered browser-default tables. Adds borders, header
background, cell padding, and zebra striping, plus a display:block +
overflow-x:auto pattern so wide tables scroll horizontally instead of
squeezing columns or breaking page layout.

Lives in base.css, which report.css and tasks.css both @import, so it
applies to every existing and future markdown/html_fragment artifact
immediately - no agent-side change needed, since this is rendered live
from CSS rather than baked into stored HTML.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-02 15:01:26 +01:00
peter
e5d2f192df Add json and csv artifact kinds with interactive viewers 
Gap 1 from planning/2026-09-02-artifact_kind_gaps_for_frequent_agent_output.md.
Chose new first-class kinds over riding on html_fragment, resolving that
doc's open design question in favor of filtering support, raw-source
preservation (like markdown's .md sidecar), and server-owned validation.

- json: validated with json.loads at post time (400 on invalid JSON),
  rendered with renderjson (collapsible tree), vendored locally.
- csv: parsed server-side with the stdlib csv module into columns/rows and
  handed to Grid.js as plain data - no client-side CSV-parsing plugin
  needed, keeping normalization server-owned. Vendored locally (MIT).
- Both preserve their original text as a sidecar file, generalizing what
  was previously markdown-only handling (storage.py's SIDECAR_SUFFIXES) so
  disk-size accounting, eviction cleanup, and ?raw=1 all cover json/csv too.
- Raw JSON is embedded in a <script type="application/json"> block with
  `</script` occurrences escaped (safe because JSON's \/ escape round-trips
  correctly) - verified with a live </script>-injection payload against the
  actual running server, not just unit tests.

New migration (0012) for the KIND_CHOICES change - SQLite doesn't enforce
choices at the schema level, but Django's migration state tracks it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-02 14:56:20 +01:00
peter
af9ecf28a9 Add syntax highlighting for fenced code blocks in markdown artifacts 
Enables codehilite on the markdown renderer (Pygments, 'friendly' style,
vendored as static/themes/pygments.css) so fenced code blocks get real
syntax highlighting with zero client-side JS - matching the "themes are
CSS-only" stance in framework_artifact_rendering.md. Verified live against
the running dev instance: a fenced python block renders with highlighted
keyword/string/function spans.

This was Gap 2 from planning/2026-09-02-artifact_kind_gaps_for_frequent_agent_output.md,
picked as the first fix since it needed no new kind or design decision -
just enabling an existing markdown extension.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-02 14:39:02 +01:00
peter
d9fc28aec0 Add handover doc summarizing issues #002-#006 fixes for the LLM test project 
Confirms all five are live on the local dev instance without a restart
(autoreload picked them up), and suggests where to probe next - boundary
cases on the new length/content-shape validation, other wrong-typed PATCH
fields beyond tags, and eviction race conditions #006 doesn't cover.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-02 13:56:43 +01:00
peter
07328a3412 Fix issues #002-#006 found by the LLM test harness 
- #006 (high, data loss): make_room_for() no longer deletes unkept
  artifacts on a doomed request - checks whether eviction can possibly
  succeed before deleting anything. Also fixed a related latent bug found
  while testing: the function returned evicted Artifact objects after
  calling .delete() on them, so their .id was always None (Django clears
  pk on delete); now returns ids captured beforehand.
- #003: added server-side field-length validation (source/title/
  source_name/source_id/theme) on POST and PATCH, matching the model's
  own max_length limits, which were previously unenforced on the ingest
  path entirely.
- #004: ported daybook_post.py's content-shape validation
  (html_fragment/html_full/deck/markdown) to the server, closing the gap
  for any client that doesn't go through the helper script.
- #005: PATCH now rejects wrong-typed `tags` (previously silently wiped
  existing tags) and rejects immutable fields (content/source/kind/day)
  with 400 instead of silently no-op'ing them.
- #002: deleted the misleading skills/daybook-posting/.env.example (a
  zero-dependency script that never reads .env files) and clarified
  SKILL.md/docs/llm_posting_helper.md that these must be real
  process/shell env vars.

Adds tests/unit/ for storage-layer tests. 46 tests passing (was 25).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-02 13:38:59 +01:00
peter
c1ea5c7990 Note the permanent test/testpwd fixture user for LLM API testing 
AGENT_DAYBOOK__LLM_TEST (a sibling project) always uses this account to
test the API. Documenting it here so a future reset of the local dev
database gets it recreated instead of silently breaking that project.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-02 12:59:14 +01:00
peter
edacb86800 Add admin-only Settings -> Users page for creating accounts 
Peter needed a second user for throwaway testing without going through
Django admin each time. Superusers now get a "Users" tab in Settings
(hidden for regular users) with a create-user form and a list of existing
accounts. New users are regular (non-staff, non-superuser) by default, with
an optional "Grant admin access" checkbox; creating one immediately issues
their API token and displays it once, since that's the credential an agent
account needs and a human account can otherwise regenerate for themselves
later from their own Settings -> Account.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-02 12:33:38 +01:00
peter
f2f2a9c96b Close out issue 001: document CSRF/reverse-proxy requirement 
The app-side fix (CSRF_TRUSTED_ORIGINS, SECURE_PROXY_SSL_HEADER in
settings.py) was already committed as d458cd3 after the first Cloud-VM-1
deployment hit "CSRF verification failed" on login behind Apache. This
finishes the rest of that issue's checklist: documents
DJANGO_CSRF_TRUSTED_ORIGINS in deploy/env.example, adds the required
X-Forwarded-Proto header to the Apache/Caddy examples in
cloud_agent_deploy.md (plus a checklist item that verifies via an actual
login, not just an unauthenticated capabilities curl), and adds the same
proxy-agnostic guidance to docs/deployment.md.

Also introduces issues/, a tracked one-file-per-issue log for the project
going forward.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-02 12:28:41 +01:00
peter
d458cd3673 Fix CSRF failure behind a TLS-terminating reverse proxy 
Django had no CSRF_TRUSTED_ORIGINS and no SECURE_PROXY_SSL_HEADER, so
it always treated requests as plain HTTP. Login over HTTPS behind
Apache/Nginx failed CSRF verification as a result.
 2026-09-02 12:17:16 +01:00
peter
f7dd98f6e5 Fix backup step to follow the configurable data path (F1) 
A follow-up review of the previous fix caught that M1's DAYBOOK_HOST_DATA_DIR
override wasn't carried through to the backup step or checklist - both still
hardcoded deploy/data. Exactly when the override matters (a host with a
separate/larger volume), the backup would silently tar the wrong directory
while the checklist still read as complete.

Backup and checklist now derive the path from deploy/.env instead of
restating it. Also documents in docker-compose.yml why the .env
interpolation only resolves correctly when invoked as
`docker compose -f deploy/docker-compose.yml`, per a non-blocking note from
the same review.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-01 22:45:18 +01:00
peter
441d000d33 Fix deploy runbook against a real, already-crowded target host 
Addresses a review of deploy/cloud_agent_deploy.md against Cloud-VM-1 (a
shared Hetzner VPS already running ~10 other services, including the
Forgejo instance this repo is hosted on):

- Blocking: git.cygnus.systems' git-SSH listens on port 2222, not 22 -
  fixed the SSH config example and clone fallback, with a note not to
  assume 22 if it changes again.
- High: container port now binds via configurable DAYBOOK_BIND_ADDR
  (defaults to all interfaces, set to 127.0.0.1 when a reverse proxy fronts
  it) instead of always publishing to 0.0.0.0; reverse-proxy+TLS is now the
  only path presented for internet-facing hosts, not a coequal option with
  direct exposure.
- High: replaced the Caddy-only hardening example with an Apache vhost +
  certbot path (matching how this specific host already terminates TLS for
  everything else on it), plus a check-before-assuming-80/443-are-free step.
- High: added mem_limit/mem_reservation to docker-compose.yml so this
  doesn't risk OOM-killing other services on a memory-tight box.
- Medium: added configurable DAYBOOK_HOST_DATA_DIR so persistent data can
  land on a separate/larger volume instead of a space-constrained main disk.
- Medium/Low: checklist now prompts checking the target's own
  service-inventory/monitoring/changelog conventions, and .env guidance
  notes it should stay low-stakes since a deploy agent reads/writes it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-01 22:38:03 +01:00
peter
d84753c839 Add cloud deployment runbook for the dedicated deploy agent 
Self-contained operational SOP for an autonomous agent deploying and
operating Daybook on a cloud VM: bootstrapping its own scoped SSH deploy
key, cloud-specific network/firewall setup, the git-pull redeploy loop with
rollback, TLS hardening for internet-facing hosts, and a completion
checklist.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-01 22:23:55 +01:00
peter
4a5ea8944b Add generic deployment docs and fix stale references 
Adds docs/deployment.md, a platform-agnostic guide (any Linux host with
Docker) built around a git-pull based fast-redeploy workflow, replacing the
Proxmox-specific deploy/proxmox_llm_deploy.md which was also stale (old
`persona` field, single-shared-token model predating multi-user auth).

Also: drops the unused DAYBOOK_API_TOKEN from deploy/env.example, documents
the new DELETE endpoint and capabilities `methods` field in docs/api.md and
docs/agent_quickstart.md, and fixes CLAUDE.md's OpenCode note now that
opencode.json is untracked (machine-specific MCP config).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-01 22:14:34 +01:00
peter
940a703cd7 Initial public commit 
Daybook: a self-hosted, multi-user Django + DRF app where agents, scripts,
or people publish artifacts and each logged-in user browses their own
time-ordered, label-filterable feed.

This squashes prior local development history (kept privately on a separate
branch) into a single clean starting point for the public repository.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-01 21:55:02 +01:00