By Allison Sikorsky, DNP, PMHNP-BC
Founder & CEO, PMHScribe
When a behavioral health practice begins testing an AI scribe, the first question is usually predictable: How much time will this save our providers?
It is a fair question. After-hours documentation affects clinicians, families, practice operations, and eventually retention. But time alone cannot tell a practice whether the technology improved its documentation process.
A note generated quickly may still require extensive correction. A clinician may close charts earlier but continue carrying the same cognitive burden throughout the day. Strong usage numbers may reflect enthusiasm without telling leadership whether the resulting notes are clinically useful.
A better question is: Did this improve the way our clinicians document care?
Before testing an AI scribe, a practice should be able to name the problem it wants to solve.
Are clinicians completing notes at night? Is there a growing backlog? Are documentation styles inconsistent across the practice? Are providers mentally composing notes during appointments instead of remaining fully present? Is the review process itself taking too long?
These problems are related, but each requires a different measure of success.
The Peterson Health Technology Institute found that healthcare organizations are adopting ambient scribes to address several goals, including clinician burnout, cognitive load, patient experience, documentation quality, and operational capacity. Its review also found that evidence regarding direct time savings and financial impact remains mixed.
That makes a clear starting point essential. A practice trying to reduce after-hours charting should evaluate different outcomes from one trying to improve documentation consistency across a large clinical team.
Note-generation speed is easy to demonstrate. What matters is what happens after the draft appears. Practices should compare:
Same-day note completion
Unsigned-note backlogs
Time spent documenting outside scheduled hours
Time required to review and revise drafts
Time required to move completed notes into the EHR
A draft generated in seconds is helpful only if it shortens the path to a signed, clinically sound note.
Collecting a short baseline before the pilot makes this comparison possible. Without knowing what the documentation process looked like before implementation, it is difficult to determine whether the tool produced a meaningful change.
Usage data can show how often clinicians opened the platform. It cannot show whether the resulting notes were accurate or useful.
A 2026 UC Davis Health pilot evaluated 356 AI-generated notes for accidental omissions, hallucinations, accidental inclusions, and bias. Researchers found that 94.7% were free from significant errors. However, accidental omissions appeared in 18% of the notes reviewed, and 5.3% contained an error rated as posing a serious or imminent risk if left uncorrected.
The study involved multiple ambulatory specialties and was not specific to behavioral health, so its percentages should not be assumed to represent psychiatric documentation. Its broader lesson is still relevant: a polished draft requires meaningful clinician review.
During a behavioral health pilot, practices should examine whether notes:
Accurately reflect the encounter
Preserve the clinician’s assessment and treatment rationale
Capture relevant changes in symptoms, functioning, medication, and risk
Omit clinically important information
Introduce information that was not stated
Include sensitive details that are unnecessary for the medical record
Read like useful psychiatric documentation rather than a transcript
Behavioral health documentation is particularly sensitive to both omission and over-inclusion. A note can be factually accurate but fail to explain why a diagnosis changed, why medication was continued, or why a specific follow-up plan was chosen. It can also capture more of a patient’s personal story than the medical record requires.
A small, structured note review during the pilot can reveal issues that usage reports will not.
Documentation burden often begins before a clinician starts typing.
During an appointment, providers may be remembering details for the note, organizing the assessment, or mentally tracking what belongs in the risk assessment and treatment plan. Part of their attention is already working on documentation.
That burden does not appear on a timestamp. Practices should ask clinicians:
Were you more attentive during the encounter?
Did you spend less mental energy organizing the note while the patient was speaking?
Did reviewing the draft feel easier than writing the note from the beginning?
Were you less mentally depleted at the end of the day?
Did you carry fewer unfinished notes home?
These responses are subjective, but they address an important part of clinical work. A tool may reduce documentation time without meaningfully reducing documentation strain.
Adoption should not be reduced to a single percentage. Some clinicians may use an AI scribe for psychiatric evaluations but not for brief medication follow-ups. Others may find it most useful during complex visits. A therapist may decide that a particularly sensitive session is not an appropriate setting for it.
A provider may also stop using a platform because the drafts require too many changes, the templates do not fit, or the workflow feels cumbersome. Low use does not always indicate resistance. High use does not automatically indicate value.
A useful pilot includes clinicians with different documentation styles, workloads, levels of experience, and degrees of comfort with technology. It should also include different encounter types. A platform that performs well during a straightforward follow-up may behave differently during an evaluation involving diagnostic uncertainty, substance use, medication changes, psychotherapy, or safety concerns.
The goal is to learn where the technology helps, where it does not, and why.
An AI scribe does not operate separately from the rest of the practice. Its value depends on how it fits into existing clinical and administrative workflows.
The pilot should consider:
How clinicians begin and end sessions
How drafts are reviewed and edited
How completed notes move into the EHR
How different templates and note types are handled
What happens when the tool is not used
What training and support clinicians need
Whether the process creates additional work elsewhere
Practices should also pay attention to the encounter itself. Did the clinician feel more present? Was the use of the technology explained according to practice policy and applicable requirements? Did the patient appear comfortable?
Behavioral healthcare depends on observation, tone, pacing, silence, and trust. A documentation tool should create more room for that attention, not quietly disrupt it.
A thoughtful pilot does not require a complicated research study. It does require consistent measures.
Documentation workflow
Same-day note completion
After-hours documentation
Unsigned-note backlog
Review and revision time
Documentation quality
Important omissions
Unsupported additions
Unnecessary sensitive detail
Preservation of clinical reasoning
Usefulness of the finalized note
Clinician experience
Cognitive load during visits
End-of-day mental fatigue
Ease of reviewing drafts
Confidence in finalized documentation
Adoption and fit
Usage by clinician and encounter type
Reasons for using or not using the tool
Patient comfort
Training and support needs
Compatibility with existing workflows
The final decision should answer one complete question: Does this tool help our clinicians produce accurate, useful documentation with less burden, in a way that fits the reality of our practice?
Behavioral health practices should expect an AI scribe to save time. They should also expect it to preserve clinical reasoning, support thoughtful documentation, reduce rather than relocate cognitive work, and fit naturally into patient care.
The stopwatch can measure part of the change. It cannot measure all of it.
1.What should a behavioral health practice measure during an AI scribe pilot?
Practices should evaluate note-completion time, after-hours documentation, revision time, documentation quality, cognitive workload, clinician adoption, patient comfort, and fit with existing workflows.
2.How long should an AI scribe pilot last?
There is no universal pilot length. It should give clinicians enough time to move beyond the novelty of the platform and test it across representative encounter types. Practices should collect baseline information before the pilot so results can be compared meaningfully.
3.Who should participate in an AI scribe pilot?
A pilot should include clinicians with different documentation styles, workloads, experience levels, and comfort with technology. Testing only with enthusiastic early adopters may not reveal barriers that appear during a broader rollout.
4.How should practices review AI-generated notes?
Practices can review a manageable sample for omissions, unsupported additions, unnecessary information, clinical reasoning, accuracy, and usefulness. Every AI-generated draft should be carefully reviewed and approved by the responsible clinician before entering the medical record.
4. Is time saved the best measure of AI scribe success?
Time saved is important, but it does not measure documentation quality, cognitive burden, clinical presence, patient comfort, or workflow fit. These areas should be evaluated alongside documentation time.
Allison Sikorsky, DNP, PMHNP-BC, is the Founder and CEO of PMHScribe and a board-certified Psychiatric Mental Health Nurse Practitioner. Her career spans clinical practice, telepsychiatry, healthcare leadership, medical documentation, and behavioral health technology.
She founded PMHScribe after experiencing documentation burden firsthand and focuses on helping psychiatrists, PMHNPs, therapists, and behavioral health organizations use AI responsibly while preserving clinician oversight, thoughtful documentation, and human connection.
Learn how PMHScribe works or explore AI-assisted documentation for behavioral health practices.
This article is intended for educational purposes and reflects the author’s professional perspective together with current published research. It is not legal, compliance, billing, or medical advice.
Sources
