iCub microphone + YARP interface

[Sohaib448506]

Hi, I am trying to access the microphones of iCub. I read a lot of doucmentation but not getting how to access the microphones of iCub and connect it to Yarp.
kindly guide me how to do this.

[randaz81]

Dear @Sohaib448506, we do not have an official tutorial yet.
You may also notice some ongoing PRs in yarp, because we are currently improving some aspects related to audio (e.g. robotology/yarp#2433)

However, I'm going to write here some hints to start.
Recording and playback is supported in yarp via libportaudio, which wraps on the top of the linux device drivers.
These yarp devices are portaudioPlayer (http://www.yarp.it/git-master/classPortAudioPlayerDeviceDriver.html) and portaudioRecorder (http://www.yarp.it/git-master/classPortAudioRecorderDeviceDriver.html), you need to enable them from ccmake when you compile yarp on your laptop (the option is avilable only if libportaudio needs to be installed). On icub, these devices are already enabled by default.

Then, yarp has two network wrappers to send received audio data to/from the network. These devices are AudioPlayerWrapper (http://www.yarp.it/git-master/classAudioPlayerWrapper.html) and AudioRecorderWrapper (http://www.yarp.it/git-master/classAudioRecorderWrapper.html) . Again, you need to check your yarp ccmake configuration to enable and compile them.

You can record audio from a microphone and stream it over the network with the command:

yarpdev --device AudioRecorderWrapper --subdevice  portaudioRecorder --start

On the other side, you can played some sound received from the network with:

yarpdev --device AudioPlayerWrapper --subdevice  portaudioPlayer --start

You can connect the two modules with the command:

yarp connect /audioPlayerRecorder/audio:o /audioPlayerWrapper/audio:i tcp_fast

The option --start will automatically enable the devices on startup. Otherwise you can start/stop the recording/playback by sending the command on the corresponding rpc ports.

Now, the difficult part. All these devices/subdevices require some specific (and complex) options to work properly. These options are documented in the yarp pages I linked before. The parameters required by portaudioRecorder/portaudioPlayer are specific to the sound format (e.g. frequency, sample format etc). The parameters required by AudioRecorderWrapper/AudioPlayerWrapper define the fragmentation of the sound packets over the network and the buffering options. I recommend to use the default options.
Please note that buffering introduces some inevitable delay (five seconds by default), and that realtime audio playback is not currently supported (so you might hear some clicks, when a buffer underrun occurs).

A different use case is processing audio in you application. To do this, you may:

• read the audio data directly from the /audioPlayerRecorder/audio:o port
• open an AudioPlayerWrapper in your module, using the standard polydriver.
  If you use the second option, the audio packets should be already assembled by the wrapper.

Finally, we also have some test devices:

yarp_fakeMicrophone
yarp_fakeSpeaker
yarp_audioToFileDevice
yarp_audioFromFileDevice

which can be used for testing/development purpose without the need of the real hardware.

As I was mentioning in the beginning, audio documentation is in progress and it is expected to be added on the yarp documentation page in the January 2021. I'll link the page here when it will be available. Please let me know if you have additional questions.

[Sohaib448506]

This information helps me a lot. Thanks

[Sohaib448506]

Hi. Thanks for your email. I wanted to access the ear of ICUB so kindly guide me about it. The whole scenario you send me is simply interfacing the yarp to my laptop microphone. So kindly guide me how can i access ICUB ears? or what is the port name of ears of ICUB and how can i enable them? Thanks.Waiting for your answer.

…

On Mon, Dec 21, 2020 at 9:50 PM Marco Randazzo ***@***.***> wrote: Dear @Sohaib448506 <https://github.com/Sohaib448506>, we do not have an official tutorial yet. You may also notice some ongoing PRs in yarp, because we are currently improving some aspects related to audio (e.g. robotology/yarp#2433 <robotology/yarp#2433>) However, I'm going to write here some hints to start. Recording and playback is supported in yarp via libportaudio, which wraps on the top of the linux device drivers. These yarp devices are portaudioPlayer ( http://www.yarp.it/git-master/classPortAudioPlayerDeviceDriver.html) and portaudioRecorder ( http://www.yarp.it/git-master/classPortAudioRecorderDeviceDriver.html), you need to enable them from ccmake when you compile yarp on your laptop (the option is avilable only if libportaudio needs to be installed). On icub, these devices are already enabled by default. Then, yarp has two network wrappers to send received audio data to/from the network. These devices are AudioPlayerWrapper ( http://www.yarp.it/git-master/classAudioPlayerWrapper.html) and AudioRecorderWrapper ( http://www.yarp.it/git-master/classAudioRecorderWrapper.html) . Again, you need to check your yarp ccmake configuration to enable and compile them. You can record audio from a microphone and stream it over the network with the command: yarpdev --device AudioRecorderWrapper --subdevice portaudioRecorder --start On the other side, you can played some sound received from the network with: yarpdev --device AudioPlayerWrapper --subdevice portaudioPlayer --start You can connect the two modules with the command: yarp connect /audioPlayerRecorder/audio:o /audioPlayerWrapper/audio:i tcp_fast The option --start will automatically enable the devices on startup. Otherwise you can start/stop the recording/playback by sending the command on the corresponding rpc ports. Now, the difficult part. All these devices/subdevices require some specific (and complex) options to work properly. These options are documented in the yarp pages I linked before. The parameters required by portaudioRecorder/portaudioPlayer are specific to the sound format (e.g. frequency, sample format etc). The parameters required by AudioRecorderWrapper/AudioPlayerWrapper define the fragmentation of the sound packets over the network and the buffering options. I recommend to use the default options. Please note that buffering introduces some inevitable delay (five seconds by default), and that realtime audio playback is not currently supported (so you might hear some clicks, when a buffer underrun occurs). A different use case is processing audio in you application. To do this, you may: - read the audio data directly from the /audioPlayerRecorder/audio:o port - open an AudioPlayerWrapper in your module, using the standard polydriver. If you use the second option, the audio packets should be already assembled by the wrapper. Finally, we also have some test devices: yarp_fakeMicrophone yarp_fakeSpeaker yarp_audioToFileDevice yarp_audioFromFileDevice which can be used for testing/development purpose without the need of the real hardware. As I was mentioning in the beginning, audio documentation is in progress and it is expected to be added on the yarp documentation page in the January 2021. Please let me know if you have additional questions. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#461 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ASG3EOAAV6KY53GRZCRJGLDSV54GPANCNFSM4VDCJ7LA> .

[randaz81]

The whole scenario you send me is simply interfacing the
yarp to my laptop microphone.
So kindly guide me how can i access ICUB ears?

If you execute:
yarpdev --device AudioRecorderWrapper --subdevice portaudioRecorder --start on the head pc104, you'll get audio from icub microphones.

If you want to simulate a microphone, i.e. play a file from disk:
yarpdev --device AudioRecorderWrapper --subdevice audioFromFileDevice --file_name myfile.wav --start

In both cases, the port opened by default by AudioRecorderWrapper is /audioRecorderWrapper/audio:o. You can check the options described in AudioRecorderWrapper.h if you want to choose a custom port name.

[farooqUsman]

Hello shoaib448506,
I hope you are fine...i am writing this to inquire about the above mention issue because i am also facing the same issue and need your help if you overcome this.Thanks

[Sohaib448506]

Hi @farooqUsman . They have not given the access to the ear port yet. Although the name has been mentioned at a lot of places and still they working on it.

[mshoaibb]

Dear Farooq bhai, This was a project worked by Sohaib. Thanks Sohaib for the update. Warmest regards, Mohammad Shoaib +92 309 5978370

…

On Thu, Oct 6, 2022, 7:33 PM Sohaib ***@***.***> wrote: Hi @farooqUsman <https://github.com/farooqUsman> . They have not given the access to the ear port yet. Although the name has been mentioned at a lot of places and still they working on it. — Reply to this email directly, view it on GitHub <#461 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ASCBWDJICLTTIC4AFXUL7BTWB3PKRANCNFSM4VDCJ7LA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.*** com>

[randaz81]

Some examples are provided in: https://yarp.it/latest/group__AudioDoc.html
On the PC104 (icub head:)

yarpdev --device audioRecorderWrapper --subdevice portaudioRecorder --start --min_samples_over_network 3200 --max_samples_over_network 3200 --AUDIO_BASE::rate 16000 --AUDIO_BASE::samples 6400 --AUDIO_BASE::channels 1

On the receiver laptop:

yarpdev --device audioPlayerWrapper --subdevice portaudioPlayer --start --playback_network_buffer_size 0.1 --AUDIO_BASE::samples 32000
connect /audioRecorderWrapper/audio:o /audioPlayerWrapper/audio:i

The exact values of the parameters to use may depend on your network configuration (e.g. ethernet/wifi).
The larger is the buffer, the higher is the delay.

[farooqUsman]

Hello @randaz81,
Is it possible to convert the above mentioned terminal commands into python?

[randaz81]

Can you explain better your goal?

In the above commands:

• yarpdev is an executable, it cannot be converted to python.
• audioRecorderWrapper, audioPlayerWrapper are plugins (.dll/.so) written in C++. They are responsible for the network (ie.yarp ports) management.
• portaudioRecorder, portaudioPlayer are plugins (written in C++) that link to a library called portAudio. They are responsible for communication with the hardware drivers (mics/speakers).

The C++ source code of the plugins can be found in the yarp/devices folder.
If you want to read the audio stream from your python application, you just need to read the yarp port opened by the audioRecorderWrapper plugin as a standard yarp port in python.

[farooqUsman]

Hello @randaz81
Firstly i want to send sound file to the ICUB environment and read that sound in the environment and then perform sound localization by using python language so that's why i want to convert the above mention commands in python language. So i want to know that is it possible to convert are use these commands in python?
plz guide