Comparing Chatbot APIs for Healthtech Triage: Quality, Latency, and Portability

The article argues that evaluating chatbot APIs for healthtech support should be a controlled test of triage quality, latency, and provider portability rather than a public leaderboard. It recommends keeping the provider behind a small, provider-neutral interface so that switching vendors does not require rewriting queue logic or handoff code. Triage is framed as routing and escalation decisions, with the model not allowed to invent clinical answers. Evaluation should use a reviewed set of synthetic tickets, scoring routing accuracy, unsafe escalation, and human-review flags separately, alongside time to first token and time to a usable decision. The author also cautions against comparing streamed output from one provider against fully buffered output from another, since that measures the interface rather than the model.