I have a pipeline that generates document scaffolds — skeletons with blanks for a human to
fill in. Every generated file looks roughly like this:
<!-- draft-stage: scaffold -->
## Hook
- [✏️ one line that stops the scroll]
## Caption
- [✏️ 1-2 sentences, facts only]
## Before you publish
- [ ] Zero `[✏️]` left → delete the `<!-- draft-stage: scaffold -->` line above
Two markers carry the state. [✏️] marks a blank nobody filled yet. The HTML comment marks
"this file is still a skeleton." A dashboard reads both to decide whether a document is done:
SCAFFOLD_MARK = "<!-- draft-stage: scaffold -->"
PEN = "✏️"
def stage_of(path) -> str:
text = path.read_text(encoding="utf-8")
return "scaffold" if (SCAFFOLD_MARK in text or PEN in text) else "draft"
Simple enough. Read the file, look for either marker, done.
Except every single file came back scaffold. All 90 of them. Files that had been filled in
weeks earlier, reviewed, and published still showed up as unfinished work.
The last line of the file
Look at the checklist again:
- [ ] Zero `[✏️]` left → delete the `<!-- draft-stage: scaffold -->` line above
That line — the one telling you how to mark the file complete — contains both markers
verbatim. So the moment you follow the instruction and delete the HTML comment, stage_of()
still finds [✏️]… in the sentence explaining that there should be no [✏️].
The exit condition was unsatisfiable. Not "hard to satisfy" — unsatisfiable, because the
document could not describe its own completion criteria without violating them.
The generator wrote that line. Every file it produced was born permanently unfinished.
Why nobody noticed for weeks
This is the part I find more interesting than the bug.
The dashboard showed a big number of pending documents. That number never went down. And a
number that never goes down stops being information — people stop reading it. When I finally
asked why it was stuck, I'd already been ignoring it for a while, which is exactly what
always-red signals train you to do.
The fix took one line. Rewrite the instruction so it describes the markers instead of
containing them:
- [ ] No pencil slots left → delete the `draft-stage` comment at the top of the file
Re-running the check flipped 37 of 90 files from "unfinished" to "done" instantly. Nothing
about those files changed. They had been done the whole time.
The general shape
This is in-band signaling: control
information travelling in the same channel as the payload. Telephone networks hit it in the
1960s — the 2600 Hz tone that told a trunk line it
was idle could be whistled into the handset by a person, and the network could not tell the
difference between "the switch says this line is free" and "a human made that sound." Same
class of problem, seventy years apart.
Once you have the shape in your head, you see it everywhere:
- A log parser that greps for
ERROR— and logsERRORwhen it fails to parse a line. - A linter whose README documents the pattern it flags, and which is run over its own repo.
- A test that asserts "no
TODOin source" while its own source explains what aTODOis. - Markdown-based feature flags where the flag name appears in the docs describing the flag.
The common failure mode isn't that the marker is a bad choice. [✏️] is fine. It's that the
same file serves two readers — a parser and a human — and the humans need to talk about
the marker while the parser only knows how to find it.
What I'd do differently
Three options, roughly in order of how much I'd trust them:
1. Move state out of the payload. A sidecar file, frontmatter field, or database column
means the prose can say whatever it wants. This is the real fix; the rest are mitigations.
2. Make the sentinel unspeakable. Pick something a human writing documentation would never
type, and if you must mention it, mention it by name (the draft-stage comment) rather than by
value. This is what I actually shipped, because the marker also has to be visually obvious in a
rendered document — that was the whole point of using an emoji.
3. Exclude the region. Skip fenced code blocks, or everything after a <!-- parser:stop -->
line. Works, but now you have two parsers that must agree, and the failure is silent when they
drift.
There's a fourth option I want to flag because it's tempting and wrong: parse "smarter" —
only count [✏️] when it's inside brackets at the start of a list item, etc. Every heuristic
you add makes the rule harder to state, and a state machine you can't state in one sentence is
one nobody can reason about. The bug came from a rule that was too clever by half already.
The check that would have caught it
The cheapest guard is a self-test on the generator, not the documents:
def test_generated_scaffold_can_be_completed():
"""A freshly generated file must be completable by following its own instructions."""
text = build_scaffold(sample_item)
# do what the instructions say: fill every slot, drop the marker
filled = re.sub(r"\[✏️[^\]]*\]", "x", text).replace(SCAFFOLD_MARK, "")
assert stage_of_text(filled) == "draft", "scaffold cannot be completed as instructed"
That test fails on the old template and passes on the new one. It encodes the property that
actually matters — the exit condition is reachable — instead of testing the parser against
inputs I thought of.
For context: the pipeline generates publishing copy for a set of small browser-based tools I
build (toolio.pongvn.com), which is why the documents have slots
for hooks and captions in the first place. The bug had nothing to do with the tools and
everything to do with letting a document describe its own state — but it did mean a couple of
months of "you have 90 things to do" that were, in fact, 37 things already done.
If you have a marker-based state machine anywhere, go check whether your docs mention the
marker. It takes about thirty seconds and the answer is occasionally embarrassing.
SOCIAL SHARE CARD GENERATOR