daybook/issues/009-content-shape-check-false-positive-in-script-strings.md

009 — html_fragment content-shape check false-positives on safe text inside <script> string literals

Status: Fixed (documented as a known limitation)

Follow-up to the #007 handover, which specifically asked to probe the <html/<!doctype/<body/<section substring checks for content containing those strings "inside a code block/string literal rather than as real markup."

Description

validate_content_shape (added for #004) rejects html_fragment content containing the literal substrings <html, <!doctype, or <body anywhere in the text, case-insensitively — with no awareness of HTML structure/context. This correctly catches genuinely dangerous cases (see #004's original report and the escaped-vs-unescaped <pre> test below), but also rejects content where those substrings are inert — e.g. a JS string literal inside <script>, which can never become a real DOM element:

{
  "kind": "html_fragment",
  "content": "<p>Config example:</p><script>const tpl = \"<html><body></body></html>\";</script>"
}

→ 400 {"detail":"html_fragment must be body-only HTML; do not include doctype/html/body"}

This content is legitimate, safe, body-only HTML (a <script> block whose string contents are never parsed as markup by the browser) — a completely reasonable pattern for an artifact demonstrating/generating HTML client-side. It's rejected purely because the substring check has no idea it's inside a JS string.

For contrast/completeness, confirmed correct behavior on two adjacent cases while investigating this:

• Properly escaped literal HTML text in a <pre> block (<html>) is accepted, as it should be — the check only fires on unescaped <.
• Unescaped <html> inside a <pre> block is correctly rejected — that's not actually a false positive, since unescaped tags inside <pre> still get parsed as real nested elements by the browser (same root cause as #004), so blocking it is correct.
• Uppercase <HTML>/<!DOCTYPE HTML> is correctly still caught (case-insensitive check works both directions — no case-based bypass), and uppercase <SECTION> in a deck is correctly still accepted.

So the check's case-insensitivity and escaping-awareness are both solid; the gap is specifically "text inside script/string context is indistinguishable from real markup" to a substring check.

Severity / Scope

Low-to-moderate. Narrow trigger (an artifact literally about generating or documenting HTML client-side), and there's a workaround (escape the inner < as < even inside <script>, since JS accepts HTML entities... actually it doesn't — entities are only decoded by the HTML parser, so escaping breaks the JS string value itself; the real workaround is string concatenation, e.g. "<" + "html>", which is awkward). Worth a conscious call on whether this is worth fixing (real HTML parsing) or documenting as a known limitation, rather than treating it as a must-fix — a proper fix means parsing the content as HTML and checking node structure instead of substring matching, which is a bigger change than the substring-check approach #004 shipped.

Suggested Fix (pick one)

• Document as a known limitation in docs/api.md's kind table: "content-shape validation is substring-based, not a real HTML parse — avoid the literal strings <html, <!doctype, <body anywhere in html_fragment content, including inside <script>/<pre> text." Cheapest, matches the spirit of the #004 fix (mirrors the client helper's own substring approach).
• Real fix: parse content with an HTML parser (e.g. Python's html.parser or lxml) and check for actual top-level <html>/ <body> elements rather than substrings. More correct, more work, and adds a dependency if not using stdlib html.parser.

Resolution

Took the cheap option, per the issue's own framing: documented as a known limitation in docs/api.md rather than implementing real HTML parsing. The narrow trigger (an artifact literally about generating/documenting HTML client-side) doesn't justify the bigger, riskier change a proper fix would need (parse content and check actual node structure instead of substrings) — substring matching is already the deliberate approach #004 shipped and that daybook_post.py's own client-side check uses, so this keeps server/client behavior consistent rather than making the server stricter in a way the client can't warn about ahead of time.