Moves the 11 Fixed issues (#001-006, #008-012) into issues/_archive_/,
which is gitignored — they stay on local disk and in git history but drop
out of the committed tree, keeping the issues/ dir focused on what's open.
The sibling test project reads this dir by local path, so it still sees
the archived files. Adds the three new open issues found in round-3
testing: #014 (stored XSS in markdown/deck/html_fragment — High), #015
(RecursionError -> 500 on deeply nested JSON), #016 (ragged CSV rows lose
data in the rendered table). Documents the archive convention in the README.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Covers everything shipped since the #007 handoff: the json/csv kinds,
.xlsx conversion, public sharing, syntax highlighting/table styling, and
the #008-#012 fixes. Prioritizes round-3 probe suggestions by risk, with
sharing (the new unauthenticated surface) and the #010 quota race (no
automated coverage) called out as the highest-value targets.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The UI stylesheet already centralized its palette in a :root block, so this
mostly meant (1) migrating the ~30 colors still hardcoded in rules into
variables so they flip in one place, and (2) adding a [data-theme="dark"]
override palette. Artifact rendering is untouched — those use the separate
static/themes/*.css and the preview backdrop stays white by design, since
artifacts (charts, reports) are authored light.
- Toggle lives in the gear-menu dropdown (Dark mode / Light mode), persists
to localStorage.
- A tiny inline script in each page <head> (feed, settings, login) applies
the stored theme before first paint, so dark-mode users never see a white
flash on load.
- color-scheme: dark on the dark palette makes native form controls
(inputs, buttons) render dark too.
- Semantic accents (kept=green, shared=blue, today, danger) got
dark-appropriate variants; status/data-viz colors (usage bar, tag chips)
and the white artifact backdrop are intentionally left as-is.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Same pass as the SKILL.md review. Fixed factually-wrong content and filled
gaps against the current code.
docs/api.md:
- Capabilities example response was stale: artifact_kinds missing json/csv,
raw_supported_for missing json/csv. Corrected to match live output, with
a note that the endpoint is the authoritative source.
- "Office document conversion is not implemented" is now false - replaced
with the accurate scope (.xlsx -> csv only; .docx/.pptx/.xls not).
- Source Fields overview table omitted source_prompt entirely; added it,
plus a note that source_name/source_id/source_prompt must be strings
(400, not coerced) and metadata must be an object.
- Kinds table: noted tables now render styled in markdown/html_fragment,
and made explicit that html_full is served verbatim with no styling.
- MVP constraints: noted the public-share-link exception to "feeds private
per user" and clarified PATCH edits metadata only.
docs/llm_posting_helper.md:
- Added .xlsx upload, --upload-image image embedding, a Source & Theme
Flags table (all six source-*/theme flags were undocumented here), and a
note that --meta/--meta-json merge and are distinct from tags.
- Added a scope note that the helper only creates - update/delete need the
API, sharing is feed-UI-only.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Found six real flags with zero mention anywhere in the doc:
--source-type, --source-name, --source-id, --source-prompt, --meta-json,
--dry-run. Added a new "Source & Provenance Fields" section for the
source-* flags and covered the rest inline/in Rules.
Fixed:
- html_full's cheat-sheet entry overstated the requirement as needing
doctype/html/head/body; the actual check only requires <html>.
- The "Images Inside Content" section had grown to cover four unrelated
topics (images, CSV/JSON, xlsx, code-as-markdown) under a header
describing only the first one. Split into properly-titled sections:
Images Inside Content, CSV/JSON/Spreadsheets, Other Code.
- An unclosed parenthesis in the xlsx limitations paragraph, rewritten as
a bulleted list for clarity while fixing it.
- Nothing anywhere mentioned that this script only creates artifacts -
update/delete need the raw API, sharing is feed-UI-only with no API
path at all. Added an explicit scope note.
- --source-name's actual default (the resolved source, which can come
from DAYBOOK_SOURCE, not just --source) was previously undocumented
entirely, so "same as --source" would have been imprecise once added.
Added: available --theme values (report/tasks/base) and which kinds
ignore it, and a note that markdown/html_fragment tables now render
styled automatically (from this session's earlier CSS pass).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Uploading a .xlsx file no longer becomes a generic downloadable binary -
it's read server-side with openpyxl and converted to a csv artifact,
rendering in the same interactive grid as a native CSV post. Detected by
content type or .xlsx extension (some clients send generic/incorrect
content types for multipart uploads). Only the active sheet converts;
formula cells use whatever cached result the spreadsheet software last
saved (openpyxl doesn't evaluate formulas itself). Corrupt/unreadable
files return 400.
Since the existing posting script already forwards any non-text file via
multipart with no special-casing, `daybook_post.py balance_sheet.xlsx`
already works with zero script changes - verified live.
Two real bugs found and fixed while building this, both now covered by
regression tests:
- {**a_querydict, ...} silently wrapped every value (including title) in
a list - Python's ** spread on a dict subclass bypasses QueryDict's
__getitem__ override and exposes its internal one-list-per-key storage
directly. Fixed by building the delegated dict explicitly via .get().
- Pre-setting None for absent optional fields (e.g. source_prompt) defeated
create_text_artifact's own data.get(field, default) calls, since .get()
only falls through to its default when the key is *missing*, not when
the value is None - caused a NOT NULL constraint failure. Fixed by only
setting keys that were actually provided.
- csv.writer's default "\r\n" terminator was getting doubled by
Path.write_text()'s own newline translation on Windows, producing a
blank line between every row. Fixed with lineterminator="\n".
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Grid.js and renderjson (vendored for the csv/json kinds) are reachable as
plain static files from any artifact, including html_full - a better
default than an external CDN for table/JSON rendering specifically, since
it works offline and has no third-party dependency.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Any artifact can now be made publicly viewable via a share link, no login
required - opt-in per artifact, toggled from the feed UI next to Keep.
- Artifact.share_token: nullable, unique, high-entropy (secrets.token_urlsafe(32)).
NULL = not shared; SQL treats each NULL as distinct so uniqueness still
holds for the many not-shared rows.
- New public route GET /s/<token> (shared_artifact_view), completely
unauthenticated, serving the same rendered/raw content as the owner
route via a shared serve_artifact_file() helper.
- toggle_artifact_share: generates/clears the token: mirrors
toggle_artifact_keep's pattern.
- Fixed a real gap this surfaced: embedded images in a shared artifact's
content would otherwise break for an anonymous visitor, since asset_view
was owner-only. It now also allows access when the asset's parent
artifact is currently shared - unattached (orphaned) assets stay private
regardless.
- serialize_artifact now reports is_shared/share_url so the API reflects
sharing status (read-only for now; toggling stays UI-only).
- Feed UI: Share button (mirrors Keep's toggle interaction), a copyable
share-link box that appears when shared, and a "shared" badge on the
artifact row - distinct blue accent from Keep's green so both can be
visible on the same row without confusion.
Verified live end-to-end (toggle on/off, public access, fake-token 404,
old-link-dies-on-unshare, embedded-asset access before/after sharing)
before writing the 12 automated tests in tests/views/test_sharing.py.
Also fixed two unrelated stale spots noticed in docs/api.md while adding
the Sharing section: the example response was missing is_kept/size_bytes,
and the raw-content table didn't list json/csv (added earlier this
session but never documented there).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
TEXT_KINDS and TEXT_SUFFIXES now include json/csv, with matching
sniff_text_kind branches so both explicit --kind and positional auto-mode
work. Added validate_payload's JSON shape check mirroring the server.
Updated SKILL.md/docs/llm_posting_helper.md with real usage examples,
replacing the stale --as-markdown workaround guidance (kept only for
genuinely unstructured code, which still has no dedicated kind). Also
fixed an unrelated stale doc line found in the same table: html_full's
validation was documented as requiring <!doctype AND <html, but only
<html is actually required.
Verified via dry-run (explicit --kind, auto-mode, invalid-JSON rejection)
and one real end-to-end post through the script against the live dev
instance.
Added tests/scripts/test_daybook_post.py - 17 tests, plain unittest, zero
new dependencies, loaded via importlib since the script isn't a package.
Closes the test-coverage question the issue raised.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
#009: html_fragment's content-shape check is substring-based, so it can
false-positive on inert text inside <script>/<pre> (e.g. a JS string
literal that happens to contain "<html>"). Documented as a known
limitation in docs/api.md rather than implementing real HTML parsing - a
proper fix is a much bigger change than the narrow trigger justifies, and
substring matching is the same approach the client helper already uses.
#011: added a curated CDN Snippets section to SKILL.md for html_full
(Tailwind, Chart.js, with an offline/air-gapped caveat), scoped explicitly
to that kind only. Fixed daybook_post.py's --theme help text, which named
a non-existent "bootstrap" theme (written ahead of a vendored-Bootstrap
feature that was planned but never built, not a typo).
Also updated SKILL.md's kind cheat sheet, which never mentioned the
json/csv kinds and still pointed at the old --as-markdown workaround for
them - and filed issue #012 for a real gap that surfaced doing so: the
posting script itself doesn't support --kind json/csv yet, only the raw API
does.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
These accepted any JSON type on both POST and PATCH, silently mangled
into Python repr() strings at save time (a dict became the literal text
"{'a': 1}", unparseable as JSON) - and the immediate response echoed the
raw pre-save value, while a later GET returned the mangled one, an
internally inconsistent API.
Added validate_string_fields(), rejecting non-string values with 400
before they reach save(), matching #005's approach for tags. Verified live
and with new tests.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
make_room_for() read usage before any write lock was held, so concurrent
requests could each read stale data and both conclude "this fits" -
confirmed reproducible (measured 616 bytes over a 1551-byte quota with 5
truly concurrent posts).
Fix: force SQLite to acquire its write lock immediately via a no-op UPDATE
on the user's UserProfile row, before reading `used`, wrapped in
transaction.atomic() through the subsequent Artifact.objects.create() so
the lock is held across the whole eviction+insert sequence. Added a 20s
DB timeout as headroom for legitimate contention.
Deliberately no retry/backoff logic - this project's actual load (a
low-volume self-hosted tool) doesn't need it, and SQLite has no row-level
locking regardless, so quota-affecting writes serialize globally either
way. Did fix a real bug found while verifying this: exception handling
around artifact creation only caught QuotaExceeded, so a genuine lock
timeout left the just-written file orphaned on disk - both call sites now
clean up on any exception.
Verified live against the real dev instance (twice, with 5 truly
simultaneous posts against a tight quota) rather than via an automated
test - Django's in-memory shared-cache test database uses a different,
non-retryable SQLite locking model that fails for reasons unrelated to
this fix's correctness.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Markdown pipe-tables and hand-written html_fragment <table> markup already
rendered - the only existing rule was 'max-width: 100%', so they came out
as bare, unbordered browser-default tables. Adds borders, header
background, cell padding, and zebra striping, plus a display:block +
overflow-x:auto pattern so wide tables scroll horizontally instead of
squeezing columns or breaking page layout.
Lives in base.css, which report.css and tasks.css both @import, so it
applies to every existing and future markdown/html_fragment artifact
immediately - no agent-side change needed, since this is rendered live
from CSS rather than baked into stored HTML.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Gap 1 from planning/2026-09-02-artifact_kind_gaps_for_frequent_agent_output.md.
Chose new first-class kinds over riding on html_fragment, resolving that
doc's open design question in favor of filtering support, raw-source
preservation (like markdown's .md sidecar), and server-owned validation.
- json: validated with json.loads at post time (400 on invalid JSON),
rendered with renderjson (collapsible tree), vendored locally.
- csv: parsed server-side with the stdlib csv module into columns/rows and
handed to Grid.js as plain data - no client-side CSV-parsing plugin
needed, keeping normalization server-owned. Vendored locally (MIT).
- Both preserve their original text as a sidecar file, generalizing what
was previously markdown-only handling (storage.py's SIDECAR_SUFFIXES) so
disk-size accounting, eviction cleanup, and ?raw=1 all cover json/csv too.
- Raw JSON is embedded in a <script type="application/json"> block with
`</script` occurrences escaped (safe because JSON's \/ escape round-trips
correctly) - verified with a live </script>-injection payload against the
actual running server, not just unit tests.
New migration (0012) for the KIND_CHOICES change - SQLite doesn't enforce
choices at the schema level, but Django's migration state tracks it.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Enables codehilite on the markdown renderer (Pygments, 'friendly' style,
vendored as static/themes/pygments.css) so fenced code blocks get real
syntax highlighting with zero client-side JS - matching the "themes are
CSS-only" stance in framework_artifact_rendering.md. Verified live against
the running dev instance: a fenced python block renders with highlighted
keyword/string/function spans.
This was Gap 2 from planning/2026-09-02-artifact_kind_gaps_for_frequent_agent_output.md,
picked as the first fix since it needed no new kind or design decision -
just enabling an existing markdown extension.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Confirms all five are live on the local dev instance without a restart
(autoreload picked them up), and suggests where to probe next - boundary
cases on the new length/content-shape validation, other wrong-typed PATCH
fields beyond tags, and eviction race conditions #006 doesn't cover.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- #006 (high, data loss): make_room_for() no longer deletes unkept
artifacts on a doomed request - checks whether eviction can possibly
succeed before deleting anything. Also fixed a related latent bug found
while testing: the function returned evicted Artifact objects after
calling .delete() on them, so their .id was always None (Django clears
pk on delete); now returns ids captured beforehand.
- #003: added server-side field-length validation (source/title/
source_name/source_id/theme) on POST and PATCH, matching the model's
own max_length limits, which were previously unenforced on the ingest
path entirely.
- #004: ported daybook_post.py's content-shape validation
(html_fragment/html_full/deck/markdown) to the server, closing the gap
for any client that doesn't go through the helper script.
- #005: PATCH now rejects wrong-typed `tags` (previously silently wiped
existing tags) and rejects immutable fields (content/source/kind/day)
with 400 instead of silently no-op'ing them.
- #002: deleted the misleading skills/daybook-posting/.env.example (a
zero-dependency script that never reads .env files) and clarified
SKILL.md/docs/llm_posting_helper.md that these must be real
process/shell env vars.
Adds tests/unit/ for storage-layer tests. 46 tests passing (was 25).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
AGENT_DAYBOOK__LLM_TEST (a sibling project) always uses this account to
test the API. Documenting it here so a future reset of the local dev
database gets it recreated instead of silently breaking that project.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Peter needed a second user for throwaway testing without going through
Django admin each time. Superusers now get a "Users" tab in Settings
(hidden for regular users) with a create-user form and a list of existing
accounts. New users are regular (non-staff, non-superuser) by default, with
an optional "Grant admin access" checkbox; creating one immediately issues
their API token and displays it once, since that's the credential an agent
account needs and a human account can otherwise regenerate for themselves
later from their own Settings -> Account.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The app-side fix (CSRF_TRUSTED_ORIGINS, SECURE_PROXY_SSL_HEADER in
settings.py) was already committed as d458cd3 after the first Cloud-VM-1
deployment hit "CSRF verification failed" on login behind Apache. This
finishes the rest of that issue's checklist: documents
DJANGO_CSRF_TRUSTED_ORIGINS in deploy/env.example, adds the required
X-Forwarded-Proto header to the Apache/Caddy examples in
cloud_agent_deploy.md (plus a checklist item that verifies via an actual
login, not just an unauthenticated capabilities curl), and adds the same
proxy-agnostic guidance to docs/deployment.md.
Also introduces issues/, a tracked one-file-per-issue log for the project
going forward.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Django had no CSRF_TRUSTED_ORIGINS and no SECURE_PROXY_SSL_HEADER, so
it always treated requests as plain HTTP. Login over HTTPS behind
Apache/Nginx failed CSRF verification as a result.
A follow-up review of the previous fix caught that M1's DAYBOOK_HOST_DATA_DIR
override wasn't carried through to the backup step or checklist - both still
hardcoded deploy/data. Exactly when the override matters (a host with a
separate/larger volume), the backup would silently tar the wrong directory
while the checklist still read as complete.
Backup and checklist now derive the path from deploy/.env instead of
restating it. Also documents in docker-compose.yml why the .env
interpolation only resolves correctly when invoked as
`docker compose -f deploy/docker-compose.yml`, per a non-blocking note from
the same review.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Addresses a review of deploy/cloud_agent_deploy.md against Cloud-VM-1 (a
shared Hetzner VPS already running ~10 other services, including the
Forgejo instance this repo is hosted on):
- Blocking: git.cygnus.systems' git-SSH listens on port 2222, not 22 -
fixed the SSH config example and clone fallback, with a note not to
assume 22 if it changes again.
- High: container port now binds via configurable DAYBOOK_BIND_ADDR
(defaults to all interfaces, set to 127.0.0.1 when a reverse proxy fronts
it) instead of always publishing to 0.0.0.0; reverse-proxy+TLS is now the
only path presented for internet-facing hosts, not a coequal option with
direct exposure.
- High: replaced the Caddy-only hardening example with an Apache vhost +
certbot path (matching how this specific host already terminates TLS for
everything else on it), plus a check-before-assuming-80/443-are-free step.
- High: added mem_limit/mem_reservation to docker-compose.yml so this
doesn't risk OOM-killing other services on a memory-tight box.
- Medium: added configurable DAYBOOK_HOST_DATA_DIR so persistent data can
land on a separate/larger volume instead of a space-constrained main disk.
- Medium/Low: checklist now prompts checking the target's own
service-inventory/monitoring/changelog conventions, and .env guidance
notes it should stay low-stakes since a deploy agent reads/writes it.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Self-contained operational SOP for an autonomous agent deploying and
operating Daybook on a cloud VM: bootstrapping its own scoped SSH deploy
key, cloud-specific network/firewall setup, the git-pull redeploy loop with
rollback, TLS hardening for internet-facing hosts, and a completion
checklist.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds docs/deployment.md, a platform-agnostic guide (any Linux host with
Docker) built around a git-pull based fast-redeploy workflow, replacing the
Proxmox-specific deploy/proxmox_llm_deploy.md which was also stale (old
`persona` field, single-shared-token model predating multi-user auth).
Also: drops the unused DAYBOOK_API_TOKEN from deploy/env.example, documents
the new DELETE endpoint and capabilities `methods` field in docs/api.md and
docs/agent_quickstart.md, and fixes CLAUDE.md's OpenCode note now that
opencode.json is untracked (machine-specific MCP config).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Daybook: a self-hosted, multi-user Django + DRF app where agents, scripts,
or people publish artifacts and each logged-in user browses their own
time-ordered, label-filterable feed.
This squashes prior local development history (kept privately on a separate
branch) into a single clean starting point for the public repository.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>