Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

whisper : add WER tests #2454

Open
ggerganov opened this issue Oct 5, 2024 · 0 comments
Open

whisper : add WER tests #2454

ggerganov opened this issue Oct 5, 2024 · 0 comments
Labels
help wanted Extra attention is needed research🔬

Comments

@ggerganov
Copy link
Owner

It would be nice to start measuring the word error rate (WER) of whisper.cpp across some representative dataset:

  • short audio
  • long audio
  • english
  • non-english
  • etc.

This will help us catch regressions in the future. I'm not familiar with what is typically used for TTS WER benchmarks, so looking for help from the community.
@ggerganov ggerganov added help wanted Extra attention is needed research🔬 labels Oct 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
help wanted Extra attention is needed research🔬
Projects
ggml : roadmap
Status: Todo
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ggerganov