Add json/csv support to the posting script (issue #012)
TEXT_KINDS and TEXT_SUFFIXES now include json/csv, with matching sniff_text_kind branches so both explicit --kind and positional auto-mode work. Added validate_payload's JSON shape check mirroring the server. Updated SKILL.md/docs/llm_posting_helper.md with real usage examples, replacing the stale --as-markdown workaround guidance (kept only for genuinely unstructured code, which still has no dedicated kind). Also fixed an unrelated stale doc line found in the same table: html_full's validation was documented as requiring <!doctype AND <html, but only <html is actually required. Verified via dry-run (explicit --kind, auto-mode, invalid-JSON rejection) and one real end-to-end post through the script against the live dev instance. Added tests/scripts/test_daybook_post.py - 17 tests, plain unittest, zero new dependencies, loaded via importlib since the script isn't a package. Closes the test-coverage question the issue raised. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
parent
532b4bd4eb
commit
e137725cab
6 changed files with 177 additions and 8 deletions
|
|
@ -56,10 +56,19 @@ python skills/daybook-posting/scripts/daybook_post.py --file diagram.svg --title
|
||||||
python skills/daybook-posting/scripts/daybook_post.py --file report.pdf --title "Generated Report"
|
python skills/daybook-posting/scripts/daybook_post.py --file report.pdf --title "Generated Report"
|
||||||
```
|
```
|
||||||
|
|
||||||
Render CSV/JSON/code as Markdown today:
|
Post CSV or JSON as an interactive table/tree (auto-mode detects the kind
|
||||||
|
from the extension; explicit `--kind` works too):
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
python skills/daybook-posting/scripts/daybook_post.py --file results.csv --as-markdown --title "Results CSV"
|
python skills/daybook-posting/scripts/daybook_post.py results.csv
|
||||||
|
python skills/daybook-posting/scripts/daybook_post.py --kind json --title "Run Summary" --content-file summary.json
|
||||||
|
```
|
||||||
|
|
||||||
|
Render other code as a Markdown fenced block instead (no dedicated `code`
|
||||||
|
kind yet):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python skills/daybook-posting/scripts/daybook_post.py --file script.py --as-markdown --title "Generated Script"
|
||||||
```
|
```
|
||||||
|
|
||||||
Dry run:
|
Dry run:
|
||||||
|
|
@ -78,8 +87,9 @@ The helper validates common LLM mistakes before sending:
|
||||||
|---|---|
|
|---|---|
|
||||||
| `deck` | Must contain `<section`. |
|
| `deck` | Must contain `<section`. |
|
||||||
| `html_fragment` | Must not contain `<!doctype`, `<html`, or `<body`. |
|
| `html_fragment` | Must not contain `<!doctype`, `<html`, or `<body`. |
|
||||||
| `html_full` | Must contain `<!doctype` and `<html`. |
|
| `html_full` | Must contain `<html` (a `<!doctype` is recommended, not required). |
|
||||||
| `markdown` | Must not be blank. |
|
| `markdown` | Must not be blank. |
|
||||||
|
| `json` | Must be valid JSON (`json.loads` must succeed). |
|
||||||
| `.svg` upload | Forces `image/svg+xml`. |
|
| `.svg` upload | Forces `image/svg+xml`. |
|
||||||
| `.pdf` upload | Forces `application/pdf`. |
|
| `.pdf` upload | Forces `application/pdf`. |
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,6 @@
|
||||||
# 012 — `daybook_post.py` doesn't know about the `json`/`csv` kinds
|
# 012 — `daybook_post.py` doesn't know about the `json`/`csv` kinds
|
||||||
|
|
||||||
**Status:** Open
|
**Status:** Fixed
|
||||||
|
|
||||||
## Description
|
## Description
|
||||||
|
|
||||||
|
|
@ -43,4 +43,26 @@ kind at all.
|
||||||
|
|
||||||
## Resolution
|
## Resolution
|
||||||
|
|
||||||
_(pending)_
|
Added `"json"`/`"csv"` to `TEXT_KINDS` and `.json`/`.csv` to `TEXT_SUFFIXES`
|
||||||
|
with matching `sniff_text_kind` branches, so both explicit `--kind
|
||||||
|
json`/`csv` and plain positional auto-mode (`daybook_post.py data.json`)
|
||||||
|
work. Added a `json` branch to `validate_payload()` mirroring the server's
|
||||||
|
`json.loads` check (no branch needed for `csv` - matches the server's own
|
||||||
|
lenient stance). Updated `SKILL.md` and `docs/llm_posting_helper.md` with
|
||||||
|
real usage examples, replacing the stale `--as-markdown` workaround
|
||||||
|
guidance for CSV/JSON (kept for genuinely unstructured "code" only, which
|
||||||
|
still has no dedicated kind). Also fixed an unrelated pre-existing doc
|
||||||
|
inaccuracy noticed in the same table: `html_full`'s validation was
|
||||||
|
documented as requiring both `<!doctype` and `<html`, but the code (and
|
||||||
|
server) only require `<html`.
|
||||||
|
|
||||||
|
Verified via dry-run for both explicit and auto-mode paths, invalid-JSON
|
||||||
|
rejection, and one real end-to-end post through the script against the
|
||||||
|
live dev instance.
|
||||||
|
|
||||||
|
Added `tests/scripts/test_daybook_post.py` (plain `unittest`, zero new
|
||||||
|
dependencies, loaded via `importlib` since the script isn't a package) -
|
||||||
|
17 tests covering `sniff_text_kind`, `validate_payload`, and
|
||||||
|
`resolve_auto_path`. Answers this issue's open question about test
|
||||||
|
coverage: yes, worth having now that the script's logic has real
|
||||||
|
per-kind branches worth protecting against regression.
|
||||||
|
|
|
||||||
|
|
@ -93,7 +93,18 @@ python skills/daybook-posting/scripts/daybook_post.py report.md
|
||||||
The image is linked to the artifact that references it and is deleted along with it.
|
The image is linked to the artifact that references it and is deleted along with it.
|
||||||
Unreferenced uploads are cleaned up after 24 hours.
|
Unreferenced uploads are cleaned up after 24 hours.
|
||||||
|
|
||||||
For CSV, JSON, or code that should render today, use `--file path --as-markdown` to post it as a Markdown fenced code block. Without `--as-markdown`, these upload as downloadable binary files.
|
For CSV and JSON, use `--kind csv`/`--kind json` with `--content-file` — or
|
||||||
|
just post the file positionally; auto-mode detects `.csv`/`.json` and picks
|
||||||
|
the right kind automatically:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python skills/daybook-posting/scripts/daybook_post.py results.csv
|
||||||
|
python skills/daybook-posting/scripts/daybook_post.py --kind json --title "Run Summary" --content-file summary.json
|
||||||
|
```
|
||||||
|
|
||||||
|
For other code that should render today, use `--file path --as-markdown` to
|
||||||
|
post it as a Markdown fenced code block. Without `--as-markdown`, it uploads
|
||||||
|
as a downloadable binary file.
|
||||||
|
|
||||||
## Rules
|
## Rules
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -18,8 +18,8 @@ from urllib.parse import urljoin
|
||||||
from urllib.request import Request, urlopen
|
from urllib.request import Request, urlopen
|
||||||
|
|
||||||
|
|
||||||
TEXT_KINDS = {"markdown", "html_fragment", "html_full", "deck"}
|
TEXT_KINDS = {"markdown", "html_fragment", "html_full", "deck", "json", "csv"}
|
||||||
TEXT_SUFFIXES = {".md", ".markdown", ".txt", ".html", ".htm"}
|
TEXT_SUFFIXES = {".md", ".markdown", ".txt", ".html", ".htm", ".json", ".csv"}
|
||||||
DEFAULT_URL = "http://127.0.0.1:8001"
|
DEFAULT_URL = "http://127.0.0.1:8001"
|
||||||
DEFAULT_SOURCE = "agent"
|
DEFAULT_SOURCE = "agent"
|
||||||
|
|
||||||
|
|
@ -156,6 +156,10 @@ def resolve_auto_path(args):
|
||||||
def sniff_text_kind(suffix, text):
|
def sniff_text_kind(suffix, text):
|
||||||
if suffix in (".md", ".markdown", ".txt"):
|
if suffix in (".md", ".markdown", ".txt"):
|
||||||
return "markdown"
|
return "markdown"
|
||||||
|
if suffix == ".json":
|
||||||
|
return "json"
|
||||||
|
if suffix == ".csv":
|
||||||
|
return "csv"
|
||||||
lower = text.lower()
|
lower = text.lower()
|
||||||
if "<!doctype" in lower or "<html" in lower:
|
if "<!doctype" in lower or "<html" in lower:
|
||||||
return "html_full"
|
return "html_full"
|
||||||
|
|
@ -301,6 +305,11 @@ def validate_payload(payload, args):
|
||||||
raise ValueError("html_full must be a complete HTML document with <html> (and ideally <!doctype html>)")
|
raise ValueError("html_full must be a complete HTML document with <html> (and ideally <!doctype html>)")
|
||||||
if kind == "markdown" and not payload["content"].strip():
|
if kind == "markdown" and not payload["content"].strip():
|
||||||
raise ValueError("markdown content cannot be blank")
|
raise ValueError("markdown content cannot be blank")
|
||||||
|
if kind == "json":
|
||||||
|
try:
|
||||||
|
json.loads(payload["content"])
|
||||||
|
except json.JSONDecodeError as exc:
|
||||||
|
raise ValueError(f"json content must be valid JSON: {exc}") from exc
|
||||||
|
|
||||||
|
|
||||||
def print_dry_run(base_url, token, payload, args):
|
def print_dry_run(base_url, token, payload, args):
|
||||||
|
|
|
||||||
0
tests/scripts/__init__.py
Normal file
0
tests/scripts/__init__.py
Normal file
117
tests/scripts/test_daybook_post.py
Normal file
117
tests/scripts/test_daybook_post.py
Normal file
|
|
@ -0,0 +1,117 @@
|
||||||
|
import importlib.util
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
SCRIPT_PATH = (
|
||||||
|
Path(__file__).resolve().parents[2]
|
||||||
|
/ "skills"
|
||||||
|
/ "daybook-posting"
|
||||||
|
/ "scripts"
|
||||||
|
/ "daybook_post.py"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _load_module():
|
||||||
|
spec = importlib.util.spec_from_file_location("daybook_post", SCRIPT_PATH)
|
||||||
|
module = importlib.util.module_from_spec(spec)
|
||||||
|
spec.loader.exec_module(module)
|
||||||
|
return module
|
||||||
|
|
||||||
|
|
||||||
|
daybook_post = _load_module()
|
||||||
|
|
||||||
|
|
||||||
|
class SniffTextKindTestCase(unittest.TestCase):
|
||||||
|
def test_markdown_suffixes(self):
|
||||||
|
for suffix in (".md", ".markdown", ".txt"):
|
||||||
|
self.assertEqual(daybook_post.sniff_text_kind(suffix, "anything"), "markdown")
|
||||||
|
|
||||||
|
def test_json_suffix(self):
|
||||||
|
self.assertEqual(daybook_post.sniff_text_kind(".json", '{"a": 1}'), "json")
|
||||||
|
|
||||||
|
def test_csv_suffix(self):
|
||||||
|
self.assertEqual(daybook_post.sniff_text_kind(".csv", "a,b\n1,2"), "csv")
|
||||||
|
|
||||||
|
def test_html_full_sniffed_from_content(self):
|
||||||
|
self.assertEqual(
|
||||||
|
daybook_post.sniff_text_kind(".html", "<!doctype html><html></html>"), "html_full"
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_deck_sniffed_from_content(self):
|
||||||
|
self.assertEqual(daybook_post.sniff_text_kind(".html", "<section>slide</section>"), "deck")
|
||||||
|
|
||||||
|
def test_html_fragment_is_the_fallback(self):
|
||||||
|
self.assertEqual(daybook_post.sniff_text_kind(".html", "<p>hi</p>"), "html_fragment")
|
||||||
|
|
||||||
|
|
||||||
|
class ValidatePayloadTestCase(unittest.TestCase):
|
||||||
|
def test_valid_json_passes(self):
|
||||||
|
daybook_post.validate_payload({"kind": "json", "content": '{"a": 1}'}, None) # no raise
|
||||||
|
|
||||||
|
def test_invalid_json_raises(self):
|
||||||
|
with self.assertRaises(ValueError):
|
||||||
|
daybook_post.validate_payload({"kind": "json", "content": "{not json"}, None)
|
||||||
|
|
||||||
|
def test_deck_without_section_raises(self):
|
||||||
|
with self.assertRaises(ValueError):
|
||||||
|
daybook_post.validate_payload({"kind": "deck", "content": "<h1>no slides</h1>"}, None)
|
||||||
|
|
||||||
|
def test_html_fragment_with_nested_html_raises(self):
|
||||||
|
with self.assertRaises(ValueError):
|
||||||
|
daybook_post.validate_payload(
|
||||||
|
{"kind": "html_fragment", "content": "<html><body>x</body></html>"}, None
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_html_full_without_html_tag_raises(self):
|
||||||
|
with self.assertRaises(ValueError):
|
||||||
|
daybook_post.validate_payload({"kind": "html_full", "content": "<p>x</p>"}, None)
|
||||||
|
|
||||||
|
def test_blank_markdown_raises(self):
|
||||||
|
with self.assertRaises(ValueError):
|
||||||
|
daybook_post.validate_payload({"kind": "markdown", "content": " "}, None)
|
||||||
|
|
||||||
|
def test_csv_has_no_shape_check(self):
|
||||||
|
daybook_post.validate_payload({"kind": "csv", "content": "anything at all"}, None) # no raise
|
||||||
|
|
||||||
|
def test_upload_path_skips_all_checks(self):
|
||||||
|
daybook_post.validate_payload({"upload_path": Path("whatever.png")}, None) # no raise
|
||||||
|
|
||||||
|
|
||||||
|
class ResolveAutoPathTestCase(unittest.TestCase):
|
||||||
|
def _args(self, path):
|
||||||
|
class Args:
|
||||||
|
pass
|
||||||
|
|
||||||
|
args = Args()
|
||||||
|
args.path = str(path)
|
||||||
|
args.as_markdown = False
|
||||||
|
args.title = None
|
||||||
|
return args
|
||||||
|
|
||||||
|
def test_json_file_infers_json_kind(self):
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
path = Path(tmp) / "data.json"
|
||||||
|
path.write_text('{"a": 1}', encoding="utf-8")
|
||||||
|
args = daybook_post.resolve_auto_path(self._args(path))
|
||||||
|
self.assertEqual(args.kind, "json")
|
||||||
|
self.assertEqual(args.content_file, str(path))
|
||||||
|
|
||||||
|
def test_csv_file_infers_csv_kind(self):
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
path = Path(tmp) / "data.csv"
|
||||||
|
path.write_text("a,b\n1,2", encoding="utf-8")
|
||||||
|
args = daybook_post.resolve_auto_path(self._args(path))
|
||||||
|
self.assertEqual(args.kind, "csv")
|
||||||
|
|
||||||
|
def test_image_file_becomes_multipart_upload(self):
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
path = Path(tmp) / "chart.png"
|
||||||
|
path.write_bytes(b"\x89PNG\r\n")
|
||||||
|
args = daybook_post.resolve_auto_path(self._args(path))
|
||||||
|
self.assertIsNone(args.kind)
|
||||||
|
self.assertEqual(args.file, str(path))
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
Loading…
Add table
Add a link
Reference in a new issue