THE ESSENTIALS IN 30 SECONDS

  • Nine widely used models released in 2026, which Doximity does not name, answered more than 8,000 dosing questions covering 2,204 drugs.
  • At least 2.3% of doses were wrong; 43.8% of detected errors were rated serious and 3.4% potentially lethal, with newer drugs far more error-prone.
  • The findings come from Doximity's own research and have not been independently verified; it is unclear whether they have been peer-reviewed.

Doximity, a US professional network for physicians that also offers an AI assistant to clinicians, published research on October 29 suggesting that general-purpose language models make a meaningful number of mistakes when asked about drug doses. The company released a checking tool alongside the findings.

What was tested

According to Doximity, nine widely used language models, all released in 2026, were asked more than 8,000 dosing questions modelled on real physician queries, covering 2,204 drugs and 261,508 individual dosages. The company does not name the models.

The models got at least 2.3% of doses wrong, a figure Doximity presents as a conservative lower bound. Using one severity scale, nearly 20 of every 1,000 answers contained a serious dosing error; a second scale put answers containing errors that caused harm or required intervention at about 30.5 per 1,000. Of all detected errors, 43.8% were classed as serious and 3.4% as potentially lethal.

The most common failures were not adjusting doses for patient factors such as age, weight, kidney or liver function and drug interactions, even when that information was in the question, and unit or decimal mistakes. Drugs approved in 2024 or later had about five times the odds of an error, which Doximity links to limited real-world data on them.

The tool

Doximity also launched DoseCheck, a free layer within its Doximity Ask assistant. It extracts each dose claim from an AI answer and compares it with the company's structured drug database, built from sources including FDA labelling, rather than asking another language model. Verified doses get a badge linking to their source. Doximity says the tool does not replace clinical judgement and does not yet cover the most complex regimens.

Caveats

The research comes from a company that sells a competing clinical AI product, and Healthcare Dive notes it is not clear whether it has been published in a journal or as a preprint. Studies disagree on whether specialised clinical tools outperform general models. Still, the error patterns described, especially around patient-specific adjustments, point to a practical risk as doctors and patients increasingly ask chatbots about medication.