High Latency in TTS Synthesis on Android with Screen Readers

[divineDev-dotcom]

Hello,

I am using your TTS as an Android Text-to-Speech (TTS) engine for offline use, but I have encountered an issue with the audio synthesis. It is taking approximately 500 ms to speak the text on the screen when using screen readers on Android in the onSynthesize method.

Is there any option or solution to reduce this latency? I am trying to create a TTS system for Android that blind/low-vision users can use effectively with their screen readers, so minimizing latency is critical for accessibility.

[yuyun2000]

1. Use the piper model, which has a faster inference speed; 2. Adjust the inference thread, if the thread is 1; 3. Use a better mobile phone

[csukuangfj]

Which model are you using and which kind of android phone, i.e., the CPU of your phone, are you using?

[nanaghartey]

From experience, latency depends on the kind of model, the device's processing power (CPU), and the length of the text being processed. Piper's "medium" quality models (around 60mb) have lower latency , compared to other models.

I was able to speed up inference by splitting the input text into batches, using punctuation as natural sentence boundaries, This allows for quicker synthesis of smaller chunks of text. On more powerful devices, this step may not be necessary, as they can handle larger texts efficiently. You can adjust the batching based on the device's capabilities.

[divineDev-dotcom]

Which model are you using and which kind of android phone, i.e., the CPU of your phone, are you using?

here are the details:
phone: Motorola edge 40(processor: MediaTek Dimensity 7030 (6nm))
Model: vits-piper-en_US-lessac-medium

[csukuangfj]

phone: Motorola edge 40(processor: MediaTek Dimensity 7030 (6nm))

This phone has
2 Cortex-A78 and 6 Cortex-A55 CPUs.

If it uses Cortex A78 during synthesis, then it should be very fast.

If it uses Cortex A55, then it would be slow.