Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RTPSender stops sending audio packets when alternating between nil and track with ReplaceTrack method #2623

Open
kenzoi opened this issue Nov 24, 2023 · 0 comments
Labels
bug Something isn't working difficulty:moderate

Comments

@kenzoi
Copy link

kenzoi commented Nov 24, 2023

Your environment.

  • Pion Receiver Version: v3.2.23 (also tested with v4.0.0-beta.7)
  • Pion Sender Version: v3.2.23
  • Browser Sender Version: Brave v1.60.118 Chromium: 119.0.6045.163 (Official Build) (64-bit)
  • Other Information - Fedora Linux 39, using pipewire audio stack

What did you do?

We've a server (Pion) which now always acts as a audio receiver and is the one which makes the offer.
We've two types of clients: browser (which uses the native WebRTC client) and a CLI (which uses Pion)

The server (always Receiver) adds an audio transceiver:

peerConnectionUser.AddTransceiverFromKind(webrtc.RTPCodecTypeAudio, webrtc.RTPTransceiverInit{
    Direction: webrtc.RTPTransceiverDirectionRecvonly,
})

After connection is established the Sender adds a track to Transceiver on its side.
RTPPackets are flowing normally
On Client/Sender we have a "mute" implementation which calls RTPSender.ReplaceTrack(nil) when we want to "mute" the audio and then it calls RTPSender.ReplaceTrack(track) to establish the audio again.

What did you expect?

  • When RTPSender.ReplaceTrack(nil) is called: the Receiver should stop receiving RTP packets.
  • When RTPSender.ReplaceTrack(track) is called: the Receiver should start receiving RTP packets again.

What happened?

From Browser Client -> Receiver (Pion): it is working flawless, every time that we replace the track with null, the TrackRemote of the RTPReceiver is not receiving the RTPPackets and when we replace the track with the local track (the same track or a new one get using getUserMedia), the RTPPackets start to flow again through the TrackRemote of the RTPReceiver.

From Pion Client -> Receiver (Pion): when we replace the track with nil, it looks okay and the TrackRemote stops on receiving the RTPPackets, but very frequently after we call ReplaceTrack with the local track (the same track or a new one using pion/mediadevices/getUserMedia) the RTPPackets do not start to flow again through the TrackRemote of the RTPReceiver.
And after that we call ReplaceTrack(nil) and ReplaceTrack(localTrack) again, sometimes we still don't have the RTPPackets, sometimes we do start receiving RTPPackets, we couldn't find a pattern

The same Receiver is being used for both clients.
I've tried to debug the Pion Client RTPSender.ReplaceTrack, we don't get any error when calling the method with nil or with the local track and the track looks like is being set by the RTPSender.ReplaceTrack in either case.

We now solved our problem by not calling ReplaceTrack with Pion Sender and using other channel like the signaling to communicate the state change, but we keep the RTPSender flowing the RTPPackets to the Receiver and is working correctly, which means that is not a problem with the Pion mediadevices or the audio driver.

@kenzoi kenzoi changed the title RTPSender stops sending audio packets when alternating between track and nil with ReplaceTrack method RTPSender stops sending audio packets when alternating between nil and track with ReplaceTrack method Nov 24, 2023
@Sean-Der Sean-Der added bug Something isn't working difficulty:moderate labels Mar 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working difficulty:moderate
Development

No branches or pull requests

2 participants