Most health services in Australia are sitting on several years of patient comments that nobody has read from start to finish. The Likert scores get tabulated and put in the quarterly pack. The free text gets skimmed by whoever is assembling that pack, and the rest is archived. This post is part of our , the ACSQHC's 12-item patient experience question set, which includes a free-text comment field. One implementation study covered 86,180 surveys across 36 private hospitals over 18 months. Nothing new needs to be collected for a theme-mining project. The corpus exists.
The volumes are worth knowing before you scope anything. In a provincial health system's inpatient survey analysis, . Short comments, high volume. Trivial for a machine to process, impractical for a person to read.
Complaints are a separate stream and a more sensitive one. Ahpra received found that consultees considered the current regulatory system not fit for purpose, and canvassed mandatory guardrails for high-risk settings. Analysing what patients wrote about waiting times and car parking is not a high-risk setting. There is no diagnosis, no triage, no treatment recommendation, and no patient-facing output.
It also lands on an accreditation hook you already have to satisfy. NSQHS (Gillespie and Reader, BMJ Qual Saf 2016), which was built from a systematic review of 59 studies covering 88,069 complaints and distils 729 codes into three domains and seven problem categories: quality, safety, environment, institutional processes, listening, communication, and respect and patient rights. Each is graded across four severity levels, plus stage of care and harm.
Two practical consequences. First, raters found an average of 1.94 problems per complaint letter, so your classifier has to be multi-label. Single-label classification will quietly lose half the signal. Second, a published taxonomy means your numbers can be compared against other sites and across years, which an invented cluster set cannot.
The closest published analogue to what you would be building is a 2025 cross-sectional study where for this kind of multi-source integration work, but any pipeline that gives you an immutable staging copy is fine.
A per-comment output looks something like this:
{
"comment_id": "c-2026-04-118322",
"domain": "management",
"categories": ["institutional_processes", "communication"],
"severity": "medium",
"stage_of_care": "in_practice",
"harm": "minimal",
"confidence": 0.62,
"evidence": ["waited over two hours past my appointment time"]
}
Human sign-off is a design requirement here, not a caveat. No item reaches the risk register, a named individual, or a governing body report without a quality officer confirming the classification. The model decides what gets read first. A person decides what it means.
Privacy and governance
The OAIC's . That is the most likely way this project dies here too.
Costs and honest limits
The model bill is the smallest line item. A corpus of a million or two tokens costs very little to classify at current API prices, even if you re-run it several times while tuning prompts. The real cost is human: a few hundred comments double-coded by two staff to build a validation set, which is two to three weeks of a quality officer's time, plus the PIA and vendor review.
Some limits worth stating up front. Sentiment on 17-word comments is noisy and should be read as a trend, never as a per-comment score. Feedback volumes skew toward strong opinions, so theme counts are not incidence rates for the patient population. Severity classification is where model agreement is weakest, so treat it as a sorting aid only. And if you change model versions, re-run the back history, otherwise your trend line is measuring the model rather than the hospital.
Done properly, what you end up with is a quarterly view of what patients actually said, ranked by volume and severity, traceable back to individual comments, and defensible in an accreditation conversation.
PicNet builds production AI systems for Australian organisations. .
SOCIAL SHARE CARD GENERATOR