-
Notifications
You must be signed in to change notification settings - Fork 33
Workshops
Spoken language is natively used by humans to communicate with each other, and a very important tool for social and cultural participation. For that purpose, acoustic speech signals, produced by physics with muscles and effortlessly steered by will, contain information that is transmitted by waves thought the air and can be heard, understood, and interpreted by other humans. The inability or difficulty to utter own thoughts or comprehend those of fellows human beings can severely compromise the quality of live. Linguists say that speech information is encoded in sentences, which consist of words, which in turn consist of phonemes, the smallest acoustic unit that can represent a difference in meaning. Although speech signals are usually very robust, i.e., the same information is encoded several times, which allows a degradation (e.g., when signal parts are removed) without a loss of information, eventually parts of the original message cannot be recovered anymore. Hearing loss causes a loss of (acoustic) information in the transmission of speech signals. The primary aim of hearing aids is to rehabilitate the speech communication abilities of their respective users without limiting them otherwise.
While hearing aids promise to improve the speech recognition of their users, they can't always hold this promise in all listening conditions. Hearing aids, among other signal processing, map the relevant acoustic levels to the available dynamic range of the user. Because the available dynamic range of listeners with impaired hearing is smaller that the dynamic range of relevant acoustic levels, the dynamic range has to be compressed. Particularly when listening in noise, where social participation often takes place, this can even negatively affect the speech speech recognition performance of their users. Further, the characterization of hearing loss by the audiogram is incomplete; listeners with the same audiogram can experience different benefits from wearing the same hearing aid. As a consequence, prospective hearing aid users have to try and evaluate several devices, which need to be manually adapted, i.e., fitted, to their individual needs. In addition, aesthetics, form factors and their corresponding limitations have to be taken into account. Hence, the perceived benefit---and hence the acceptance of---hearing aids depends also on many subjective factors.
- What is the audible range of frequencies and sound pressure levels?
- What are relevant sound pressure levels?
- What is an audiogram?
- What is dynamic compression?
Hearing aids are highly integrated, small, and optimized devices. Their development, miniaturization, and optimization are cost- and time-intensive. It is not possible to rapidly try new algorithms that were developed in the lab, where computational resources are abundant. However, in the lab, where new algorithm can be run in real-time, realistic communication conditions are very difficult to simulate. In fact, the relevant factors that influence speech recognition in realistic communication conditions are still subject to intense research. Mobile hearing aid prototypes allow to rapidly prototype new algorithms for hearing devices and test them in realistic conditions with low latency with relatively little effort. The increasing computational power and decreasing price of small embedded devices allows to build mobile hearing aid prototypes based on consumer hardware. Combined with the availability of an open source software platform for hearing aid algorithms, this allows to assemble a fully functional prototype of a hearing device.
- What is real-time processing? What is latency?
- Which latencies are relevant for hearing devices?
- Which properties best describe the relevant differences and similarities between a real hearing aid, a mobile hearing aid prototype, and an implementation on a lab PC?
- Roland CS-10EM Binaural Microphones/Earphones
- Semi-custom pre-amplifier design based on ELV circuit with 36dB amplification
- Audioinjector.net (stereo) £17.50(w/o shipping)
- Raspberry Pi 3 model B
- SanDisk Ultra 32Gb microSDHC UHS-I card
- Intenso Powerbank Q10000
- Belt bag
- Connect the hardware
- Turn fully to the right all knobs of the sound card
- Download, extract, and copy the image to the SD card: https://github.com/m-r-s/hearingaid-prototype/wiki#image-download
- Make sure that you don't wear the headphones!
- Boot the hardware
- Wait a minute (the first boot takes a bit longer)
- Try carefully if the loudspeakers of the headphones produce sound
- Carefully plug-in the in-ear headphones
- Take out the headphones immediately if you feel uncomfortable
The default wifi-password is "openmha-wifi". The default ssh user name is "pi" and the default password is "openmha". The default openMHA configuration includes:
- Soft clipping
- Binaural coherence filter
- Multi-band dynamic compression
- Frequency shifter
The default calibration assumes a pre-amplification of 36 dB. The sound card is by default limited to -25 dB full scale (FS) which should be roughly 100 dB SPL. The openMHA configuration is a modified version of the 2017 openMHA prerelease configuration for offline audio file processing, where the adaptive differential microphones (ADM) plugin has been removed and the default gain table (amplification scheme) has been set to "no amplification" (starting from SD card image version 0.2; to be released soon). In earlier versions, the default gain table was taken unmodified from the prerelease configuration, which applied high gains at some frequencies.
- Why is calibration important? How could the setup be calibrated?
- What is feedback and how can it be measured?
- What is an input-output (IO) function and how can it be measured?
There is a fitting tool with a graphical user interface (GUI) included in openMHA which calculates gain tables from prescription rules. The gain table describes the amplification that is provided depending on the frequency and input level.
You can try these presciption rules as follows:
- Clone openMHA to your PC
- Connect to the prototype forwarding the configuration port:
ssh -L33337:localhost:33337 pi@HOSTNAME - Using GNU/Octave (or Matlab) enter the subdirectory "mha/tools/mfiles"
- Edit the file "mhacontrol.m" and comment out lines 55-57 adding "%" to the beginning of those lines (this will prevent mhacontrol from resetting you calibration, only needs to be performed once)
- Add the java library:
javaaddpath 'mhactl_java.jar' - Run mhacontrol:
mhacontrol - Read the openMHA manual for further instructions: https://github.com/HoerTech-gGmbH/openMHA/blob/master/openMHA_gui_manual.pdf
- What are the differences between the different amplification schemes?
- Which perceptual dimensions should be considered?
- How would you map the input to the output levels given a certain audiogram?
OpenMHA uses a simple format to describe the gain table: openMHA gain table description. Implement your idea on how to map input to output levels and write a Matlab/Octave script that calculates the gain table in the openMHA format.
- How many frequency bands are considered for compression in the default configuration?
- How could the number of independent frequency bands play a role for speech recognition performance?
The main goal is to achieve high speech recognition performance in the different listening conditions that are relevant for hearing aid users.
- Which are relevant listening conditions?
- How can speech recognition performance of human listeners be measured?
- Which performance would you expect for the different conditions for listeners with normal hearing and impaired hearing with and without hearing aids?
Think about possible limitations of hearing aids in general and our mobile prototype in particular.
The instructions for building a mobile prototype of a hearing aid is available on line: https://github.com/m-r-s/hearingaid-prototype
OpenMHA is free software: http://www.openmha.org
The openMHA README file provides further pointers: https://github.com/HoerTech-gGmbH/openMHA/blob/master/README.md
Download link for one-page setup instructions: https://cs.uol.de/s/AqroQ2ZHmcr8qWC