Video calls using HPB shows no video from android app to android app

[muchachagrande]

Steps to reproduce

1. Initiates a video call from an Android Talk app to another Android Talk app
2. Answer the video call from the receiving end

Expected behaviour

The call is established and both users starts to see and hear to each other.

Actual behaviour

The call establishes and both users starts to hear each other but can't see the others video.

Device brand and model

Tested on various smart phone models with Android using Talk GPlay version and Lineage using Talk F-Droid version.

Nextcloud Talk app version

Tested with various versions. The problem has persisted since version 15 or so.

Nextcloud server version

Tested with various versions. It has happened since version 25 or so.

Talk version

Tested with various versions. It has happened since version 15 or so.

Custom Signaling server configured

Yes (specify version in Additional Information)

Custom TURN server configured

Yes

Custom STUN server configured

Yes

Android logs

No response

Server log

Nothing on the server log that day

Additional information

This problem has been happening through various Signaling versions. I think I first saw this with Signaling v1.1.0.
Also have been experimenting this using many different versions of Janus and is actually happening with the new 1.x branch that recently was made compatible with Talk.

Other observed details:

• Video calls between android app and browser work as expected but calls from android app to android app show no video from the remote end.
• Then toggling video via the camera icon in the app makes the video stream visible on the other end. This has to be done on both ends of the conversation for video to be 2 way.
• I have tested a group call with some android clients and some web clients with the same results. Android clients can't see other android clients but can see web clients. As soon as one android client cycles the video off and on, the other android clients starts to see it. Web clients can see other clients without problem.
• I have experimented from time to time (randomly) that in some video calls, only one peer can see the other at the beginning, so only one side has to cycle the camera to make it appear on the other end.

[ASerbinski]

I can confirm this.

[pmarini-nc]

I can also confirm this

[himpierre]

Is this fixed with 18.0.0-rc1? If not, am I in the wrong movie? :)

[muchachagrande]

May be most users are not impacted by this problem or may be most users do not use HPB. I don't know.

[MarkPartlett]

I can confirm this

[muchachagrande]

Will this issue be addressed in the next version ? or in some future version ?

[skl256]

I can confrm, 18.0.1

[mircokam]

Connection type	Cameras work	Cameras only work if...
Android to Android	!	they are toggled off and on on both devices
Android to iOS	✓
Android to Web	✓
iOS to Web	✓

Tested versions:
Android app: 18.1.0
iOS app: 18.0.3
Nextcloud: 28.0.3
Talk: 18.0.5
nextcloud-spreed-signaling: 1.2.3
Janus: 1.1.2-1 (Debian Bookworm)

[muchachagrande]

I've been trying to find some clues in the signaling log and found nothing of interest.
Compared a web to android and a android to android video call log and in both cases the video room is opened correctly.
May be there is some handling issue on the code that prevents the video from appearing at first.
After that, if you cycle the camera off and on, the video starts to show on the other end.
The log doesn't show anything when you cycle the camera, so I think the channels are working correctly.

[muchachagrande]

Another detail about this issue. I have experimented from time to time that in some video calls, one peer can see the other at the beginning, so only one side has to cycle the camera to make it appear on the other end.
This is a random event. Most of the times the video calls starts with no video on both ends.

[jakobroehrl]

Tested it yesterday, can confirm the bug. HPB with android <-> android the video does not show up

[Byter3]

This issue is here with us for idk more than two years at this point?! Can't the devs just implement a script, too auto toggle the cameraas on the android app when a call started, just as a work around? Please
#3999

[SystemKeeper]

I tried to look into this issue, it seems to be a twofold issue.

Overwriting video state (problem on receiving side)

It seem the following is happening

1. We handle the stream change
   

   talk-android/app/src/main/java/com/nextcloud/talk/call/CallParticipant.java

   Lines 181 to 193 in 45b18a7

   	private void handleStreamChange(MediaStream mediaStream) {
   	if (mediaStream == null) {
   	callParticipantModel.setMediaStream(null);
   	callParticipantModel.setVideoAvailable(Boolean.FALSE);
   	return;
   	}
   	boolean hasAtLeastOneVideoStream = mediaStream.videoTracks != null && !mediaStream.videoTracks.isEmpty();
   	callParticipantModel.setMediaStream(mediaStream);
   	callParticipantModel.setVideoAvailable(hasAtLeastOneVideoStream);
   	}

   
   Here we determine that we have a media stream and if there's at least one video stream, we call callParticipantModel.setVideoAvailable(true);. This would be enough to show the video
2. We handle the ice connection state change
   After we already handled the stream change, we now handle the ice connection state change to CHECKING:
   

   talk-android/app/src/main/java/com/nextcloud/talk/call/CallParticipant.java

   Lines 171 to 179 in 45b18a7

   	private void handleIceConnectionStateChange(PeerConnection.IceConnectionState iceConnectionState) {
   	callParticipantModel.setIceConnectionState(iceConnectionState);
   	if (iceConnectionState == PeerConnection.IceConnectionState.NEW ||
   	iceConnectionState == PeerConnection.IceConnectionState.CHECKING) {
   	callParticipantModel.setAudioAvailable(null);
   	callParticipantModel.setVideoAvailable(null);
   	}
   	}

   
   This resets the available video again: callParticipantModel.setVideoAvailable(null);. Which results in no video.

I am not totally sure where this race condition comes from or wether it's expected. A simple fix would be to just checking state NEW here, from my POV:

diff --git a/app/src/main/java/com/nextcloud/talk/call/CallParticipant.java b/app/src/main/java/com/nextcloud/talk/call/CallParticipant.java
index 50c1ae669..3f9f83ed2 100644
--- a/app/src/main/java/com/nextcloud/talk/call/CallParticipant.java
+++ b/app/src/main/java/com/nextcloud/talk/call/CallParticipant.java
@@ -171,8 +171,7 @@ public class CallParticipant {
     private void handleIceConnectionStateChange(PeerConnection.IceConnectionState iceConnectionState) {
         callParticipantModel.setIceConnectionState(iceConnectionState);
 
-        if (iceConnectionState == PeerConnection.IceConnectionState.NEW ||
-                iceConnectionState == PeerConnection.IceConnectionState.CHECKING) {
+        if (iceConnectionState == PeerConnection.IceConnectionState.NEW) {
             callParticipantModel.setAudioAvailable(null);
             callParticipantModel.setVideoAvailable(null);
         }

Not sending initial media state (problem on sending side)

The problem above would not be an issue, if talk-android would correctly send the initial media state.

talk-android/app/src/main/java/com/nextcloud/talk/webrtc/PeerConnectionWrapper.java

Lines 301 to 315 in 45b18a7

	private void sendInitialMediaStatus() {
	if (localStream != null) {
	if (localStream.videoTracks.size() == 1 && localStream.videoTracks.get(0).enabled()) {
	sendChannelData(new DataChannelMessage("videoOn"));
	} else {
	sendChannelData(new DataChannelMessage("videoOff"));
	}
	if (localStream.audioTracks.size() == 1 && localStream.audioTracks.get(0).enabled()) {
	sendChannelData(new DataChannelMessage("audioOn"));
	} else {
	sendChannelData(new DataChannelMessage("audioOff"));
	}
	}
	}

This will basically do nothing when using an HPB, because the wrapper of the participant does not have a localStream attached (only the publisher does). Even then it would not work, because the data should be send via the publisher and not the individual wrappers. So the initial state should be correctly send as well, to fix both problems.

[MelBourbon]

Another support for my users would be if I could set on server side that at the beginning of a video call the camera is initially disabled and needs to be enabled. Is this possible?

[PH03N1X999]

I can also confirm this on mine too.
And I'm using the latest version with external signaling and custom turn server

[muchachagrande]

Don't know what is the problem with this issue. It's two years old.

[himpierre]

It's always a matter of prioritization. :) Am 8. November 2024 03:47:32 MEZ schrieb Muchacha Grande ***@***.***>:

…

Don&#39;t know what is the problem with this issue. It&#39;s two years old.

[mahibi]

This issue is being worked on and a fix should be available soon

[PH03N1X999]

@mahibi
Please fix this issue if you could and consider it a priority

[mahibi]

@mahibi Please fix this issue if you could and consider it a priority

It is a priority, but it is also very complex thus it takes time. A colleague is working on this and it's hopefully available in the next weeks