From 4cacd8f64eb02f71820237814e62104eead3131f Mon Sep 17 00:00:00 2001 From: peter Date: Fri, 4 Sep 2026 11:41:03 +0100 Subject: [PATCH] Issues: file in-app guide (020); archive completed issues - Add issue 020: in-app user guide (Settings -> Guide), Open, for later. - Archive to _archive_/ (gitignored, local only): fixed bugs 014/015/016 and the completed LLM-test handover docs 007/013/018/019. - Tracked issues now: 017 (open), 020 (open), README. Co-Authored-By: Claude Opus 4.8 --- issues/007-handover-to-llm-test-2026-09-02.md | 54 ------ issues/013-handover-to-llm-test-round-2.md | 121 ------------- ...4-stored-xss-in-rendered-text-artifacts.md | 169 ------------------ ...5-deeply-nested-json-recursionerror-500.md | 111 ------------ ...-ragged-rows-silent-data-loss-in-render.md | 94 ---------- issues/018-handover-to-llm-test-round-3.md | 93 ---------- ...-handover-to-llm-test-round-4-predeploy.md | 77 -------- issues/020-in-app-user-guide.md | 43 +++++ 8 files changed, 43 insertions(+), 719 deletions(-) delete mode 100644 issues/007-handover-to-llm-test-2026-09-02.md delete mode 100644 issues/013-handover-to-llm-test-round-2.md delete mode 100644 issues/014-stored-xss-in-rendered-text-artifacts.md delete mode 100644 issues/015-deeply-nested-json-recursionerror-500.md delete mode 100644 issues/016-csv-ragged-rows-silent-data-loss-in-render.md delete mode 100644 issues/018-handover-to-llm-test-round-3.md delete mode 100644 issues/019-handover-to-llm-test-round-4-predeploy.md create mode 100644 issues/020-in-app-user-guide.md diff --git a/issues/007-handover-to-llm-test-2026-09-02.md b/issues/007-handover-to-llm-test-2026-09-02.md deleted file mode 100644 index 1303fb9..0000000 --- a/issues/007-handover-to-llm-test-2026-09-02.md +++ /dev/null @@ -1,54 +0,0 @@ -# 007 — Handover to AGENT_DAYBOOK__LLM_TEST: issues #002-#006 fixed - -**Status:** For information — not a bug report, a handback after fixing -what #002-#006 found. - -## What Changed - -All five issues your last testing round filed are fixed, committed -(`07328a3` on `main`, pushed to `origin`), and confirmed live on the running -local dev instance (`http://127.0.0.1:8001`, same one you test against — -no restart needed on your end, Django's autoreloader already picked it up): - -- **#002** — `.env.example` deleted; `SKILL.md`/`docs/llm_posting_helper.md` - now say explicitly these must be real process/shell env vars. -- **#003** — `source`/`title`/`source_name`/`source_id`/`theme` now - enforce their model length limits on both `POST` and `PATCH`, returning - `400` naming the field. -- **#004** — server now validates content shape per `kind` - (`html_fragment`/`html_full`/`deck`/`markdown`), mirroring - `daybook_post.py`'s existing client-side checks. -- **#005** — `PATCH` now rejects wrong-typed `tags` (400, instead of - silently wiping existing tags) and rejects immutable fields - (`content`/`source`/`kind`/`day`) with 400 instead of silently no-op'ing. -- **#006** — the quota-eviction data-loss bug: a doomed `POST` no longer - destroys unkept artifacts on its way to still failing with `413`. - -Full detail, root cause, and exact fix for each is in its own issue file -(`002` through `006`) — this doc is just the summary + what to do next. - -## Suggested Next Round - -Worth specifically re-probing (not just re-running the original repros — -try to break the fixes themselves): - -- **#003/#004 boundary/edge cases**: exact-limit-plus-one values, unicode - characters (does length-checking count characters or bytes?), mixed valid - content with edge-case whitespace/casing on the ``, `< html`, content - containing those strings inside a code block/string literal rather than as - real markup). -- **#005**: other wrong-typed fields on `PATCH` beyond `tags` — e.g. - `source_prompt`/`source_name`/`source_id` as non-strings (currently - accepted via implicit coercion, noted as untested in #005's original - writeup). -- **#006**: multi-request race conditions around eviction (two posts near - the quota boundary at once) — not covered by the fix, which only handles - the single-request case cleanly. -- General: anything from the original `issues/002-006` reproductions that - might have a sibling case not explicitly covered by the tests added - alongside each fix (`tests/api/test_artifacts.py`, `tests/unit/test_storage.py`). - -As always: use the `test`/`testpwd` account, clean up what you create, and -file anything real as a new numbered issue rather than reopening a closed -one (link back to the original if it's a variant of the same root cause). diff --git a/issues/013-handover-to-llm-test-round-2.md b/issues/013-handover-to-llm-test-round-2.md deleted file mode 100644 index 4cae7ac..0000000 --- a/issues/013-handover-to-llm-test-round-2.md +++ /dev/null @@ -1,121 +0,0 @@ -# 013 — Handover to AGENT_DAYBOOK__LLM_TEST (round 2) - -**Status:** For information — a handback covering everything shipped since -the round-1 handoff (#007). - -Round 1 (#007) closed out issues #002–#006. Since then, your round-2 -testing filed #008–#012 (all now fixed), and several new capabilities -landed. This doc summarizes all of it and points at where the fresh -surface area is worth probing. - -All of this is committed on `main` (pushed to Forgejo) and live on the -local dev instance you test against (`http://127.0.0.1:8001`) — Django's -autoreloader has already picked it up, no restart needed on your end. - -## New Capabilities To Exercise - -These are new since #007 and largely untested by you yet: - -1. **`json` and `csv` artifact kinds** (commit `e5d2f19`). Text kinds like - markdown. `json` renders as a collapsible tree (renderjson), `csv` as a - sortable/searchable table (Grid.js). Both validate server-side (`json` - must parse; `csv` only needs non-blank content) and preserve original - text as a `?raw=1` sidecar. Both vendored libs are served from - `/static/vendor/…`. - -2. **`.xlsx` upload → `csv`** (commit `4b5ec09`). Uploading an `.xlsx` file - (multipart, like any file) is converted server-side via openpyxl into a - `csv` artifact rather than stored as `binary`. Active sheet only; cached - formula results only; `.xls` unsupported; corrupt file → `400`. - -3. **Per-artifact public sharing** (commit `91ea19d`). Owner toggles a - share link from the feed UI; `GET /s/{share_token}` then serves that one - artifact with **no authentication at all**. `is_shared`/`share_url` now - appear in every artifact API response. Embedded images in a shared - artifact are also served unauthenticated while it's shared. **This is - the highest-value target for adversarial testing** — see below. - -4. **Syntax-highlighted code + styled tables** (commits `af9ecf2`, - `696cfe7`). Markdown fenced code blocks get Pygments highlighting; - tables in markdown and `html_fragment` render styled. Cosmetic, low-risk. - -5. **Posting script supports `json`/`csv`/`.xlsx`** (commit `e137725`). - `daybook_post.py --kind json/csv`, positional auto-detect of - `.json`/`.csv`/`.xlsx`. - -6. **Light/dark UI toggle** (commit `9a3b381`). UI-chrome only, no API - surface — not relevant to API testing. - -## Fixes Since #007 (issues you filed) - -- **#008** — `source_name`/`source_id`/`source_prompt` now reject - non-string values with `400` instead of silently mangling them. -- **#009** — the `html_fragment` content-shape check is substring-based; - documented as a known limitation rather than switched to real HTML - parsing (deliberate — matches the client helper's own approach). -- **#010** — concurrent posts could race past quota; fixed with a - write-lock-before-read + atomic transaction. Verified live but **not** - covered by an automated test (SQLite shared-cache test DB behaves - differently — see the issue). Worth re-probing. -- **#011** — CDN snippets documented for `html_full`; `--theme` help typo - fixed. -- **#012** — posting script gained `json`/`csv` support. - -## Suggested Round-3 Probes - -Prioritized by where the new surface area is riskiest. - -### Sharing (#91ea19d) — security boundary, probe hardest - -- Can an **unshared** artifact ever be reached via `/s/`? Brute/ - guessed/empty/malformed tokens should all `404`. -- Does **unsharing immediately** kill the old link? (Confirmed manually, - but re-verify — and try racing an unshare against an in-flight request.) -- Is `share_token` ever leaked in a response for an artifact you *don't* - own, or in the list endpoint, or anywhere a non-owner could see it? -- Can a shared artifact's **embedded assets** be reached unauthenticated — - and, importantly, can an *un*shared artifact's assets be reached that way - (they must not)? Also: does an orphaned/unattached asset stay private - even when some unrelated artifact is shared? -- Does `?raw=1` on `/s/` correctly serve raw source for the kinds - that support it, and not leak anything for kinds that don't? -- Cross-user: user A shares an artifact; can user B toggle/​unshare it, or - only view it? - -### json / csv (#e5d2f19) - -- `json`: valid-but-not-an-object payloads (bare array, string, number, - `null`); deeply nested / very large; unicode; the `` injection - vector (confirmed handled for json — verify **csv** escapes its embedded - data block the same way, it's a separate code path). -- `csv`: ragged rows (fewer/more columns than the header), embedded commas/ - quotes/newlines, duplicate column names, header-only, empty, a single - giant cell, BOM/encoding oddities. -- Quota accounting: json/csv now write a sidecar file — confirm size - accounting and eviction include/delete the sidecar (this generalized the - markdown-only logic; worth verifying it didn't regress markdown either). - -### .xlsx conversion (#4b5ec09) - -- Multi-sheet workbook — verify only the active sheet survives and the - others are silently dropped (documented behavior, confirm it's true). -- Formula cells never calculated by a real spreadsheet app (openpyxl - doesn't evaluate) → should be blank, not error. -- A non-`.xlsx` file renamed to `.xlsx`; a real `.xlsx` sent with a wrong/ - generic content type; a huge sheet; empty workbook; `.xls` (must not - convert). -- Field mangling regression: this path rebuilds the payload dict manually - because multipart `QueryDict` + `{**data}` was silently list-wrapping - values (the bug that made this a multi-fix). Re-probe that title/tags/ - source_* all come through correctly on `.xlsx` posts specifically. - -### #010 quota race (no automated coverage) - -- Fire N truly-concurrent posts at a tight quota; confirm final usage never - exceeds quota and no artifact is left half-created (row without files or - files without row). - -As always: use the `test` / `testpwd` account, clean up what you create, -check `artifacts/views.py` before calling a surprise a bug (some, like -#009, are documented-by-design), and file anything real as a new numbered -issue (next is #014), linking back if it's a variant of a closed one. diff --git a/issues/014-stored-xss-in-rendered-text-artifacts.md b/issues/014-stored-xss-in-rendered-text-artifacts.md deleted file mode 100644 index 88edbd9..0000000 --- a/issues/014-stored-xss-in-rendered-text-artifacts.md +++ /dev/null @@ -1,169 +0,0 @@ -# 014 — Stored XSS: markdown (and deck/html_fragment) render unsanitized script; no CSP; now reachable unauthenticated via public share links - -**Status:** Fixed -**Severity:** High — stored XSS executing on the Daybook origin, with a -concrete escalation chain against the owner's session, and (since the -sharing feature shipped in `91ea19d`) reachable by unauthenticated third -parties. - -Round-3 probe of content validation ("put malformed/bad content in each -kind, is it caught?"). Short answer: JSON and CSV are well-defended; -markdown is not; deck and html_fragment inject script too. - -## What was tested and what happened - -All posted to the `test` account and fetched back rendered from `/a/{id}`. - -| Kind | Injected | Result | -|---|---|---| -| `markdown` | ``, ``, `` | **All pass through verbatim into rendered HTML — executes.** | -| `deck` | `
…` | **Script passes through into the themed deck page — executes.** | -| `html_fragment` | `

hi

` | **Script/`svg onload` pass through into the themed page — executes.** (The #004 content-shape check blocks `//` but never `"}` | **Defended.** `escape_script_body` (`re.compile(r"` data block; renderjson shows it as inert text. Case-variation bypass (``) correctly caught. | -| `csv` | `` in a cell | **Defended.** Same escaping via `json.dumps` + `escape_script_body`. | -| `csv` | `=cmd|/c calc`, `+SUM(...)` as cell values | Passes through into the `?raw=1` sidecar verbatim (classic CSV/formula injection). Low severity / debatable — see below. | - -## Root cause (markdown specifically) - -`create_text_artifact` renders markdown with: - -```python -markdown.markdown(content, extensions=["extra", "sane_lists", "codehilite"], ...) -``` - -Python-Markdown does **no** HTML sanitization, and the `extra` extension -enables raw-HTML passthrough. Its own docs warn output must be sanitized -(bleach/nh3) before display if the input is at all untrusted. The rendered -HTML is then written to the artifact's `.html` file and served as-is. - -Unlike `html_full`/`html_fragment`, **markdown is not an "author your own -HTML" kind** — nobody expects a Markdown note to be a script-execution -vector. In an agent-driven system this is exactly the kind most likely to -carry *indirectly* untrusted content: an agent that summarizes a scraped -page, an email, or an ingested document can emit a summary containing an -attacker-controlled `