Basic Bash Piper Book Reader.

[clort81]

Piper's monophonic output can be promoted to stereo automatically by alsa, but as soon as something else is playing, alsa dumps the mono channel to left speaker.

Investigating how to work around this, I went down a rabbit hole of audio processing raw pcm streams.

Using the awesome ffmpeg swiss army knife, I got promotion to stereo, equalization to improve the sound (on my speakers). I added a 'reader' that will try to group an input text into paragraphs which it then sends to piper.

I was unable to avoid buffer underrunns 100% on my setup (AC97 motherboard, alsa).

This probably won't work 'out of the box' for you, but it is shared in hope that some of the work invested might be useful to others.

If anyone has questions or improvements, please share and discuss.

#!/bin/bash
# Read a book with 'piper' https://github.com/rhasspy/piper
#
# skips over non-paragraph blank lines emitted by pdf2txt by detecting if the line before the blank had a sentence-ending punctuation mark.
#
# requires:
#       - piper
#       - buffer
#       - ffmpeg
#       - bash
#
my_paragraph=""
lastsentenceended="false"
lastline=""

while IFS= read -r line; do
    if [[ $lastline =~ [\.!?]$ ]]; then
        lastsentenceended="true"
    fi
    if [ -n "$line" ]; then
        if [ "$lastsentenceended" = "true" ]; then  # paragraph complete; speak it.

            #echo "Found empty line with preceeding line ending with punctuation: "
            #echo "$lastline"
            #echo "$line"

            # Call external program with $my_paragraph as parameter
            echo "$my_paragraph"
            # Call piper with raw output piped to a player program.  This is a bit complex.
            # - piper outputs a single channel stream at the bitrate specified by the voice file
            # - aplay can't promote that to dual channel sensibly (duplicating the channels) instead it sends samples alternately to left and right, which 
            #   doubles frequency and playback speed.  If you can fix this, tell us how.
            #   aplay -r 22050 -f S16_LE -t raw -c 2 --chmap=mono,mono -
            # - Tried promoting to dual-channel with ffplay, but that didn't work either, so I inserted a ffmpeg before ffplay, which 
            #   does manage to split the mono into stereo
            # - So i tried adding a buffer command just before ffplay, which ought to help, but it didn't fix it.
            # - Unfortunately I still get too-frequent buffer underruns with this setup
            #
            echo "$my_paragraph" | piper  --model /pr/TextToSpeech/Piper/Voices/en_US-ryan-high.onnx ---noise_scale 0.433 --length_scale 1.0 --noise_w 0.436 --output-raw 2>/dev/null | ffmpeg -v error  -f s16le -ar 22.05k -ac 1 -i - -af "pan=stereo|c0=c0|c1=c0,aresample=resampler=soxr:osr=22050,superequalizer=3b=0.6:4b=0.6:5b=0.7:6b=0.8:11b=1.1:12b=1.2:13b=1.3:14b=1.2:15b=1.1:16b=1.16:17b=1.03:18b=1.00" -f s16le - 2>/dev/null |  buffer -m 100000 -u 100 -p 50% | ffplay -nodisp -autoexit -v error -f s16le -af "channelsplit,amerge" - 2>/dev/null

            my_paragraph=""                     # clear the paragraph after playing it
            lastsentenceended="false"           # unset our last sentence ended flag
            sleep 0.2s                          # a bit of a delay between paragraphs sounds better
        fi

        if [ "$lastsentenceended" = "true" ]; then
            # Start a new paragraph after finding the last sentence
            my_paragraph=$line
        else
            my_paragraph+=" $line"
        fi
    fi
    lastline="$line"  # remember the current line for testing of last iteration.
done < "$1"

# Loop Tail: Call external program with $my_paragraph as parameter for the last paragraph stored
echo "$my_paragraph"
echo "$my_paragraph" | piper  --model /pr/TextToSpeech/Piper/Voices/en_US-ryan-high.onnx ---noise_scale 0.433 --length_scale 1.0 --noise_w 0.436 --output-raw 2>/dev/null | ffmpeg -v error  -f s16le -ar 22.05k -ac 1 -i - -af "pan=stereo|c0=c0|c1=c0,aresample=resampler=soxr:osr=22050,superequalizer=3b=0.6:4b=0.6:5b=0.7:6b=0.8:11b=1.1:12b=1.2:13b=1.3:14b=1.2:15b=1.1:16b=1.16:17b=1.03:18b=1.00" -f s16le - 2>/dev/null |  buffer -m 100000 -u 100 -p 50% | ffplay -nodisp -autoexit -v error -f s16le -af "channelsplit,amerge" - 2>/dev/null

# sample badfile that the program helps fix.
#
#This is a sentence that
#continues to the next line
#and ends with a period.
#
#This is a paragraph. It 
#suffers from a blank line
#
#inserted before the end.

[sweetbbak]

Where does the command buffer come from?

[ghost]

🤷 I don't remember doing that last night.

[clort81]

Where does the command buffer come from?

On my system, from the debian package 'buffer'. It is found in /usr/bin/buffer.

It was an experiment to see if the buffer underruns were caused by piper, in which case a buffer inserted into the pipeline can help to ameliorate the underruns.