peter/daybook
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add json and csv artifact kinds with interactive viewers

Browse source 
Gap 1 from planning/2026-09-02-artifact_kind_gaps_for_frequent_agent_output.md.
Chose new first-class kinds over riding on html_fragment, resolving that
doc's open design question in favor of filtering support, raw-source
preservation (like markdown's .md sidecar), and server-owned validation.

- json: validated with json.loads at post time (400 on invalid JSON),
  rendered with renderjson (collapsible tree), vendored locally.
- csv: parsed server-side with the stdlib csv module into columns/rows and
  handed to Grid.js as plain data - no client-side CSV-parsing plugin
  needed, keeping normalization server-owned. Vendored locally (MIT).
- Both preserve their original text as a sidecar file, generalizing what
  was previously markdown-only handling (storage.py's SIDECAR_SUFFIXES) so
  disk-size accounting, eviction cleanup, and ?raw=1 all cover json/csv too.
- Raw JSON is embedded in a <script type="application/json"> block with
  `</script` occurrences escaped (safe because JSON's \/ escape round-trips
  correctly) - verified with a live </script>-injection payload against the
  actual running server, not just unit tests.

New migration (0012) for the KIND_CHOICES change - SQLite doesn't enforce
choices at the schema level, but Django's migration state tracks it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
peter 2026-09-02 14:56:20 +01:00
parent af9ecf28a9
commit e5d2f192df
12 changed files with 476 additions and 18 deletions
Download patch file Download diff file Expand all files Collapse all files

29
artifacts/storage.py
View file

@ -1,8 +1,9 @@
"""Disk-usage accounting, per-user quota, and automatic eviction.
An artifact's on-disk footprint is the primary stored file plus, for markdown,
the preserved `.md` sidecar. We record it on the row (`size_bytes`) at ingest so
usage is a cheap SUM rather than a stat of every file.
An artifact's on-disk footprint is the primary stored file plus, for kinds
that preserve their original source (see `SIDECAR_SUFFIXES`), the sidecar
file. We record it on the row (`size_bytes`) at ingest so usage is a cheap
SUM rather than a stat of every file.
"""
import shutil
@ -19,22 +20,31 @@ class QuotaExceeded(Exception):
"""Raised when a post cannot fit even after evicting every unkept artifact."""
# Kinds that preserve their original source alongside the rendered .html file.
SIDECAR_SUFFIXES = {
Artifact.MARKDOWN: ".md",
Artifact.JSON: ".json",
Artifact.CSV: ".csv",
}
def artifact_disk_size(relative_path, kind):
"""Bytes an artifact occupies on disk, including the markdown sidecar."""
"""Bytes an artifact occupies on disk, including any sidecar file."""
root = settings.ARTIFACTS_ROOT
total = 0
primary = root / relative_path
if primary.exists():
total += primary.stat().st_size
if kind == Artifact.MARKDOWN:
sidecar = primary.with_suffix(".md")
sidecar_suffix = SIDECAR_SUFFIXES.get(kind)
if sidecar_suffix:
sidecar = primary.with_suffix(sidecar_suffix)
if sidecar.exists():
total += sidecar.stat().st_size
return total
def delete_artifact_files(artifact):
"""Remove an artifact's files from disk (primary, markdown sidecar, assets).
"""Remove an artifact's files from disk (primary, sidecar, assets).
Assets are cascade-deleted from the DB by the FK, but their files must be
removed here explicitly.
@ -42,8 +52,9 @@ def delete_artifact_files(artifact):
root = settings.ARTIFACTS_ROOT
primary = root / artifact.path
primary.unlink(missing_ok=True)
if artifact.kind == Artifact.MARKDOWN:
primary.with_suffix(".md").unlink(missing_ok=True)
sidecar_suffix = SIDECAR_SUFFIXES.get(artifact.kind)
if sidecar_suffix:
primary.with_suffix(sidecar_suffix).unlink(missing_ok=True)
for asset in artifact.assets.all():
delete_asset_file(asset)