How to fine-tune pyannote/embedding?

[zh2336]

I wanted to fine tune pyannote/embedding for a classification task. I started with following the adapting_pretrained_pipeline.ipynb tutorial, but had issue setting up the task -- ValueError: No loader for files with '.rttm' suffix from pyannote/database/custom.py line 88. This issue seems to come out recently as some users reported on stackoverflow.

I also saw discussion 745 on fine-tuning the embedding model. It seems Hervé tried to make training speaker embedding easier (issue 749) but it didn't get implemented?

Wonder what's the feasible way to fine tune the embedding model, and I'm going to take a stab on how to do that, could you please take a look if I'm on the right track?

1. Define a speaker verification protocol based on my own dataset
2. Create a task using pyannote.audio.tasks (assumably the SpeakerEmbedding task?)
3. Freeze whatever layers I need
4. Train model with my dataset

Let me know if I'm missing something, I'm happy to learn more about the pyannote stuff!