#015 — deeply nested JSON raised RecursionError (a RuntimeError, not
ValueError) and became an unhandled 500 on two paths:
- json-kind content validation now catches RecursionError -> 400.
- Added a custom DRF EXCEPTION_HANDLER
(artifacts/exception_handlers.py, wired via REST_FRAMEWORK) mapping
otherwise-unhandled RecursionError -> 400, covering the parser-level
crash on deeply nested request bodies (metadata etc.) too.
#016 — wrap_csv_viewer handed Grid.js rows without reconciling width to the
header, so over-long rows silently dropped cells. Now normalizes every row
to the max width: long rows widen the columns (auto-named "Column N"),
short rows pad with "". No cell is dropped.
Both verified live; 4 regression tests added. 106 tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
markdown/deck/html_fragment rendered author-influenced content verbatim,
so <script>/<img onerror>/etc. executed on the Daybook origin — self-XSS
against the owner's session, and (post-sharing) unauthenticated via /s/.
Two layers, both in artifacts/views.py:
- Create-time allowlist sanitization via nh3 (maintained ammonia/Rust
sanitizer, not deprecated bleach). Keeps formatting/tables/links/images
(incl. relative /assets URLs) and codehilite markup; drops script, on*
handlers, style, unknown tags, and javascript:/data: URLs. html_full is
deliberately NOT sanitized (documented verbatim escape hatch).
- Serve-time CSP (script-src 'none') for the sanitized kinds, on both
/a/{id} and the unauthenticated /s/{token} route. Being a response
header, it also neutralizes scripts in artifacts stored before this fix.
Not applied to html_full or json/csv (json/csv data is already escaped;
their viewers need their own JS).
Verified live across all kinds; 7 regression tests added. 102 tests pass.
Not addressed (noted in the issue): CSV formula injection,
CSRF_COOKIE_HTTPONLY (moot with XSS closed), nonce CSP for json/csv/full.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Moves the 11 Fixed issues (#001-006, #008-012) into issues/_archive_/,
which is gitignored — they stay on local disk and in git history but drop
out of the committed tree, keeping the issues/ dir focused on what's open.
The sibling test project reads this dir by local path, so it still sees
the archived files. Adds the three new open issues found in round-3
testing: #014 (stored XSS in markdown/deck/html_fragment — High), #015
(RecursionError -> 500 on deeply nested JSON), #016 (ragged CSV rows lose
data in the rendered table). Documents the archive convention in the README.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Covers everything shipped since the #007 handoff: the json/csv kinds,
.xlsx conversion, public sharing, syntax highlighting/table styling, and
the #008-#012 fixes. Prioritizes round-3 probe suggestions by risk, with
sharing (the new unauthenticated surface) and the #010 quota race (no
automated coverage) called out as the highest-value targets.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
TEXT_KINDS and TEXT_SUFFIXES now include json/csv, with matching
sniff_text_kind branches so both explicit --kind and positional auto-mode
work. Added validate_payload's JSON shape check mirroring the server.
Updated SKILL.md/docs/llm_posting_helper.md with real usage examples,
replacing the stale --as-markdown workaround guidance (kept only for
genuinely unstructured code, which still has no dedicated kind). Also
fixed an unrelated stale doc line found in the same table: html_full's
validation was documented as requiring <!doctype AND <html, but only
<html is actually required.
Verified via dry-run (explicit --kind, auto-mode, invalid-JSON rejection)
and one real end-to-end post through the script against the live dev
instance.
Added tests/scripts/test_daybook_post.py - 17 tests, plain unittest, zero
new dependencies, loaded via importlib since the script isn't a package.
Closes the test-coverage question the issue raised.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
#009: html_fragment's content-shape check is substring-based, so it can
false-positive on inert text inside <script>/<pre> (e.g. a JS string
literal that happens to contain "<html>"). Documented as a known
limitation in docs/api.md rather than implementing real HTML parsing - a
proper fix is a much bigger change than the narrow trigger justifies, and
substring matching is the same approach the client helper already uses.
#011: added a curated CDN Snippets section to SKILL.md for html_full
(Tailwind, Chart.js, with an offline/air-gapped caveat), scoped explicitly
to that kind only. Fixed daybook_post.py's --theme help text, which named
a non-existent "bootstrap" theme (written ahead of a vendored-Bootstrap
feature that was planned but never built, not a typo).
Also updated SKILL.md's kind cheat sheet, which never mentioned the
json/csv kinds and still pointed at the old --as-markdown workaround for
them - and filed issue #012 for a real gap that surfaced doing so: the
posting script itself doesn't support --kind json/csv yet, only the raw API
does.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
These accepted any JSON type on both POST and PATCH, silently mangled
into Python repr() strings at save time (a dict became the literal text
"{'a': 1}", unparseable as JSON) - and the immediate response echoed the
raw pre-save value, while a later GET returned the mangled one, an
internally inconsistent API.
Added validate_string_fields(), rejecting non-string values with 400
before they reach save(), matching #005's approach for tags. Verified live
and with new tests.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
make_room_for() read usage before any write lock was held, so concurrent
requests could each read stale data and both conclude "this fits" -
confirmed reproducible (measured 616 bytes over a 1551-byte quota with 5
truly concurrent posts).
Fix: force SQLite to acquire its write lock immediately via a no-op UPDATE
on the user's UserProfile row, before reading `used`, wrapped in
transaction.atomic() through the subsequent Artifact.objects.create() so
the lock is held across the whole eviction+insert sequence. Added a 20s
DB timeout as headroom for legitimate contention.
Deliberately no retry/backoff logic - this project's actual load (a
low-volume self-hosted tool) doesn't need it, and SQLite has no row-level
locking regardless, so quota-affecting writes serialize globally either
way. Did fix a real bug found while verifying this: exception handling
around artifact creation only caught QuotaExceeded, so a genuine lock
timeout left the just-written file orphaned on disk - both call sites now
clean up on any exception.
Verified live against the real dev instance (twice, with 5 truly
simultaneous posts against a tight quota) rather than via an automated
test - Django's in-memory shared-cache test database uses a different,
non-retryable SQLite locking model that fails for reasons unrelated to
this fix's correctness.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Confirms all five are live on the local dev instance without a restart
(autoreload picked them up), and suggests where to probe next - boundary
cases on the new length/content-shape validation, other wrong-typed PATCH
fields beyond tags, and eviction race conditions #006 doesn't cover.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- #006 (high, data loss): make_room_for() no longer deletes unkept
artifacts on a doomed request - checks whether eviction can possibly
succeed before deleting anything. Also fixed a related latent bug found
while testing: the function returned evicted Artifact objects after
calling .delete() on them, so their .id was always None (Django clears
pk on delete); now returns ids captured beforehand.
- #003: added server-side field-length validation (source/title/
source_name/source_id/theme) on POST and PATCH, matching the model's
own max_length limits, which were previously unenforced on the ingest
path entirely.
- #004: ported daybook_post.py's content-shape validation
(html_fragment/html_full/deck/markdown) to the server, closing the gap
for any client that doesn't go through the helper script.
- #005: PATCH now rejects wrong-typed `tags` (previously silently wiped
existing tags) and rejects immutable fields (content/source/kind/day)
with 400 instead of silently no-op'ing them.
- #002: deleted the misleading skills/daybook-posting/.env.example (a
zero-dependency script that never reads .env files) and clarified
SKILL.md/docs/llm_posting_helper.md that these must be real
process/shell env vars.
Adds tests/unit/ for storage-layer tests. 46 tests passing (was 25).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The app-side fix (CSRF_TRUSTED_ORIGINS, SECURE_PROXY_SSL_HEADER in
settings.py) was already committed as d458cd3 after the first Cloud-VM-1
deployment hit "CSRF verification failed" on login behind Apache. This
finishes the rest of that issue's checklist: documents
DJANGO_CSRF_TRUSTED_ORIGINS in deploy/env.example, adds the required
X-Forwarded-Proto header to the Apache/Caddy examples in
cloud_agent_deploy.md (plus a checklist item that verifies via an actual
login, not just an unauthenticated capabilities curl), and adds the same
proxy-agnostic guidance to docs/deployment.md.
Also introduces issues/, a tracked one-file-per-issue log for the project
going forward.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
