Optimizing GPU and Environment Configuration in Pyannote 3.0.1 for Improved Speed

[rhenanbartels]

Hello! Thank you for the incredible work on Pyannote.audio. It has been invaluable in helping me identify speakers in audio files.
I started testing pyannote-audio 3.0.1 and the results are really good and better than version 2.1.1. However, I still couldn't prepare the environment to run pyannote 3.0.1 as fast as version 2.1.1

Issue:

Pyannote 3.0.1 is slower than version 2.1.1 and seems to be utilizing the GPU at an average of 5% capacity, and I'm encountering the following warnings:

[W:onnxruntime:, session_state.cc:1162 VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape 
related ops to CPU to improve perf.       

[W:onnxruntime:, session_state.cc:1164 VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-minimal build will show node assignments.

Question

Does anyone have insights into optimizing the configuration of the environment so Pyannote 3.0.1 achieves performance comparable to version 2.1.1? Any suggestions or solutions would be greatly appreciated.

Thanks in advance,

Rhenan

---

Code/Configuration:

VPS: AWS EC2 - g4dn.xlarge - NVIDIA T4 GPUs

Docker image

FROM nvidia/cuda:11.7.1-cudnn8-runtime-ubuntu20.04

Libraries version

Python 3.9.16

nvidia-cublas-cu12==12.1.3.1                                                                                                                             
nvidia-cuda-cupti-cu12==12.1.105                                                                                                                         
nvidia-cuda-nvrtc-cu12==12.1.105                                                                                                                         
nvidia-cuda-runtime-cu12==12.1.105                                                                                                                       
nvidia-cudnn-cu12==8.9.2.26                                                                                                                              
nvidia-cufft-cu12==11.0.2.54                                                                                                                             
nvidia-curand-cu12==10.3.2.106                                                                                                                           
nvidia-cusolver-cu12==11.4.5.107                                                                                                                         
nvidia-cusparse-cu12==12.1.0.106                                                                                                                         
nvidia-nccl-cu12==2.18.1                                                                                                                                 
nvidia-nvjitlink-cu12==12.3.52                                                                                                                           
nvidia-nvtx-cu12==12.1.105                                                                                                                               
omegaconf==2.4.0.dev0                                                                                                                                    
onnxruntime-gpu==1.16.1
...
pyannote.audio==3.0.1                                                                                                                                    
pyannote.core==5.0.0                                                                                                                                     
pyannote.database==5.0.1                                                                                                                                 
pyannote.metrics==3.2.1                                                                                                                                  
pyannote.pipeline==3.0.1
...
pytorch-lightning==2.1.1                                                                                                                                 
pytorch-metric-learning==2.3.0  

NVIDIA

Code

pipeline = Pipeline.from_pretrained(
    "pyannote/speaker-diarization-3.0",
      use_auth_token=token,
       cache_dir="/var/models/pyannote/cache"
)
 pipeline.to(torch.device("cuda"))
 diarization = pipeline(file_path)