I’m Deaf and I use cochlear implants. Captioning is not a side feature for me. It is often how I get the information everyone else already heard.
That makes a dropped name, time, dosage, address, or warning different from an ordinary typo. The rest of a transcript can be perfect and the caption can still fail at the exact moment I need it.
A transcript is useful when it preserves what matters, not only when most of its words match.
The gap I kept seeing
Word error rate is useful, but it treats every word as part of one average. It does not know that “15 milligrams, not 50” is more important than a missing filler word. It also does not tell us whether an alarm, knock, or ringing phone was captioned at all.
I made DeafBench to test those failures directly. It measures word errors, but it also checks critical information, environmental sounds, speaker attribution, and latency.
What I want from it
I want people building speech systems to see where a clean score hides an accessibility problem. I also want the work to be reproducible, so a claim points back to the samples, model revision, predictions, and report that produced it.
This is still early work. I’m learning, testing, and correcting it as I go. That is part of the point: make the evidence visible enough that other people can help improve it.