The gate
No prompt ships without passing the eval
The clinical voice is the product. So it is tested like one. A golden set of 37 caregiver scenarios — crisis-adjacent messages, medical-boundary questions, boundary-testing prompts, misinformation, IEP frustration, family conflict — runs against any change to a prompt or a model.
Every scenario is judged on three universal dimensions — tone (acknowledge before advise, warmth, no minimisation), safety (no diagnosis, no dosing, crisis resources surfaced when indicated), and boundaries (out-of-scope requests declined supportively) — plus its own must and must-not criteria. A safety failure fails the run outright.
npm run eval:tone · eval/scenarios.json · 37 scenarios, 10 categories
The routing
Changing models is a config change, not a rewrite
Every model call goes through a provider abstraction, so the system can run on Vertex AI or the Anthropic API, and route different work to different tiers — a fast model for classification and titles, a stronger one for reasoning that matters.
This is what makes the eval above worth having. When a new model generation lands, adopting it is a measured decision rather than a leap of faith: run the golden set against the candidate, compare tone and safety, keep whichever holds the voice. Prompt caching is wired through the same layer, so long conversations read their own history at a fraction of full price.
src/lib/ai/ — provider abstraction, tiered routing, prompt caching
The economics
Every conversation has a known cost
Token usage is logged per call — input, output, cache reads, cache writes, latency, provider, route — and priced against a maintained rate table. Budget thresholds raise alerts.
A non-profit giving a product away has to know what the product costs, or the promise expires quietly. Metering it per conversation is also how we can say a new capability is affordable before shipping it, rather than discovering the answer on an invoice.
Per-call usage logging · cost calculation · budget alerting
The memory
Memory is opt-in, staged, and yours to delete
Remembering matters in caregiving — nobody wants to re-explain their child every session. It is also the part most worth being careful with, so it is off until you turn it on.
When enabled, the pipeline runs in four separable stages: extract what was shared, decompose it into atomic facts, embed those for retrieval, and detect patterns across time. Separable stages mean each one can be inspected, tested and reasoned about on its own — and everything it holds can be viewed, exported or deleted from Settings.
Four-service pipeline: extraction → decomposition → embedding → pattern detection
The floor
Accessibility is audited, not asserted
An automated axe-core audit runs the public surface against WCAG 2.1 Level A and AA. It exits non-zero on any violation, so it can gate a release rather than decorate a slide.
Seven public routes are covered today, including the tools a caregiver reaches first. Anyone can reproduce the run against production. This is the difference between a compliance claim and a compliance position — and for a product used by people who may themselves be neurodivergent or disabled, it is the floor, not a feature.
npm run audit:a11y · axe-core · 7 public routes · WCAG 2.1 A/AA