Receive stream with whep

[ditogam]

Question

Hello
Using

gst-launch-1.0 \
audiotestsrc is-live=true ! audioconvert ! audioresample ! opusenc bitrate=96000 ! rtpopuspay \
! 'application/x-rtp,media=audio,encoding-name=OPUS,payload=96,clock-rate=48000,encoding-params=(string)2' \
! whip.sink_0 \
videotestsrc is-live=true ! video/x-raw,width=1280,height=720,framerate=30/1,format="I420" ! timeoverlay halignment=right valignment=top text="CLOCK: " shaded-background=true font-desc="Sans, 36" \
! videoconvert ! x264enc tune=zerolatency pass=cbr speed-preset=fast bitrate=3000 ! rtph264pay config-interval=-1 \
! 'application/x-rtp,media=video,encoding-name=H264,payload=127,clock-rate=90000,packetization-mode=(string)1' \
! whip.sink_1 \
whipsink name=whip \
whip-endpoint="http://192.168.2.1:8889/mystream/whip"

and successfully viewed from browser
http://192.168.2.1:8889/mystream

But cannot receive stream with

gst-launch-1.0 whepsrc name=whep \
whep-endpoint="http://192.168.2.1:8889/mystream/whep" \
audio-caps="application/x-rtp,media=audio,encoding-name=OPUS,payload=96,clock-rate=48000" \
whep.src_0 ! rtpopusdepay ! opusdec ! audio/x-raw ! audioconvert ! audioresample ! autoaudiosink

Error is

net/webrtchttp/src/whepsrc/imp.rs(428): gstwebrtchttp::whepsrc::imp::WhepSrc::raise_error (): /GstPipeline:pipeline0/GstWhepSrc:whep:
Unexpected response: 400 - {\"error\":\"codecs not supported by client\"}

As I found codecs not supported by client in PeerConnection.CreateFullAnswer, which relays on answer from sdp addTransceiverSDP

debuged and look at code

codecs := t.getCodecs() returns nil.
it uses mediaEngineCodecs := t.api.mediaEngine.getCodecsByKind(t.kind)

func (m *MediaEngine) getCodecsByKind(typ RTPCodecType) []RTPCodecParameters {
	m.mu.RLock()
	defer m.mu.RUnlock()

	if typ == RTPCodecTypeVideo {
		if m.negotiatedVideo {
			return m.negotiatedVideoCodecs
		}

		return m.videoCodecs
	} else if typ == RTPCodecTypeAudio {
		if m.negotiatedAudio {
			return m.negotiatedAudioCodecs
		}

		return m.audioCodecs
	}

	return nil
}

audioCodecs and videoCodecs had values, but negotiatedVideoCodecs and negotiatedAudioCodecs were nils, as both negotiatedAudio & negotiatedAudio were true, function returns nil
Any Idea?

Thank you in advance

[casterplay]

I have the same issue. Tested all combinations I can think of in Gstreamer. The MediaMTX whep browser works fine, and if I look at the browser network traffic, I see WHEP SDP the payloadtype four audio is 111. WHIP with Gstreamer works fine.

gst-launch-1.0 whepsrc whep-endpoint="https://myserver.com:8889/mystream/whep" use-link-headers=true audio-caps="application/x-rtp,media=audio,encoding-name=OPUS,payload=111,clock-rate=48000" ! rtpjitterbuffer ! rtpopusdepay ! opusdec ! audioconvert ! audioresample ! autoaudiosink

Still the same error from Gstreamer (v 1.24.6):
ERROR: from element /GstPipeline:pipeline0/GstWhepSrc:whepsrc0: GStreamer encountered a general resource error.
Additional debug info:
net\webrtchttp\src\whepsrc\imp.rs(428): gstwebrtchttp::whepsrc::imp::WhepSrc::raise_error (): /GstPipeline:pipeline0/GstWhepSrc:whepsrc0:
Unexpected response: 400 - {"error":"codecs not supported by client"}
Execution ended after 0:00:00.278426600

Any ideas?

[aler9]

Hello, the error "codecs not supported by client" is triggered because whepsrc is not including the Opus channel count inside the SDP that is sent to the server, which you can see by setting logLevel: debug inside MediaMTX:

a=rtcp-rsize
a=recvonly
a=rtpmap:111 opus/48000
a=rtcp-fb:111 transport-cc
a=mid:audio0
a=fingerprint:sha-256

There's opus/48000 instead of opus/48000/2. If you take a look at GStreamer source code, you'll find out that the channel count is taken from the encoding-params property of audio capabilities:

https://gitlab.freedesktop.org/gstreamer/gstreamer/-/blob/main/subprojects/gst-plugins-base/gst-libs/gst/sdp/gstsdpmessage.c?ref_type=heads#L3861-3863

Therefore, it's enough to add encoding-params=2 to audio-caps:

gst-launch-1.0 whepsrc whep-endpoint="https://myserver.com:8889/mystream/whep" \
use-link-headers=true audio-caps="application/x-rtp,media=audio,encoding-name=OPUS,payload=111,clock-rate=48000,encoding-params=2" \
! rtpjitterbuffer ! rtpopusdepay ! opusdec ! audioconvert ! audioresample ! autoaudiosink

Resulting in

a=rtcp-rsize
a=recvonly
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 transport-cc
a=mid:audio0
a=fingerprint:sha-256

This works correctly.

[ditogam]

Thank you @aler9 , works perfect

[casterplay]

Perfect, it works. But don't forget to cast "encoding-params" as string. encoding-params=(string)2

Thanks!

[olivetom]

Hi, my question is somewhat related to this topic... Hopefully someone can give me a hand on how to consume from mediamtx using gstreamer whepsrc using basic user and password authentication. I've tryed using whep-endpoint=http://<user>:<password>@server:8889/mystream/ and whep-endpoint=http://server:8889/mystream?user=<user>&pass=<password> but the failed with message Unexpected response: 404 - 404 page not found

Despite this gstreamer playback issue, I'm able to do playback from chrome smoothly.