Commit graph

2 commits

Author SHA1 Message Date
4b5ec09ae2 Convert uploaded .xlsx files to csv artifacts server-side
Uploading a .xlsx file no longer becomes a generic downloadable binary -
it's read server-side with openpyxl and converted to a csv artifact,
rendering in the same interactive grid as a native CSV post. Detected by
content type or .xlsx extension (some clients send generic/incorrect
content types for multipart uploads). Only the active sheet converts;
formula cells use whatever cached result the spreadsheet software last
saved (openpyxl doesn't evaluate formulas itself). Corrupt/unreadable
files return 400.

Since the existing posting script already forwards any non-text file via
multipart with no special-casing, `daybook_post.py balance_sheet.xlsx`
already works with zero script changes - verified live.

Two real bugs found and fixed while building this, both now covered by
regression tests:
- {**a_querydict, ...} silently wrapped every value (including title) in
  a list - Python's ** spread on a dict subclass bypasses QueryDict's
  __getitem__ override and exposes its internal one-list-per-key storage
  directly. Fixed by building the delegated dict explicitly via .get().
- Pre-setting None for absent optional fields (e.g. source_prompt) defeated
  create_text_artifact's own data.get(field, default) calls, since .get()
  only falls through to its default when the key is *missing*, not when
  the value is None - caused a NOT NULL constraint failure. Fixed by only
  setting keys that were actually provided.
- csv.writer's default "\r\n" terminator was getting doubled by
  Path.write_text()'s own newline translation on Windows, producing a
  blank line between every row. Fixed with lineterminator="\n".

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-02 16:29:39 +01:00
07328a3412 Fix issues #002-#006 found by the LLM test harness
- #006 (high, data loss): make_room_for() no longer deletes unkept
  artifacts on a doomed request - checks whether eviction can possibly
  succeed before deleting anything. Also fixed a related latent bug found
  while testing: the function returned evicted Artifact objects after
  calling .delete() on them, so their .id was always None (Django clears
  pk on delete); now returns ids captured beforehand.
- #003: added server-side field-length validation (source/title/
  source_name/source_id/theme) on POST and PATCH, matching the model's
  own max_length limits, which were previously unenforced on the ingest
  path entirely.
- #004: ported daybook_post.py's content-shape validation
  (html_fragment/html_full/deck/markdown) to the server, closing the gap
  for any client that doesn't go through the helper script.
- #005: PATCH now rejects wrong-typed `tags` (previously silently wiped
  existing tags) and rejects immutable fields (content/source/kind/day)
  with 400 instead of silently no-op'ing them.
- #002: deleted the misleading skills/daybook-posting/.env.example (a
  zero-dependency script that never reads .env files) and clarified
  SKILL.md/docs/llm_posting_helper.md that these must be real
  process/shell env vars.

Adds tests/unit/ for storage-layer tests. 46 tests passing (was 25).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-02 13:38:59 +01:00