Skip to content

Commit

Permalink
feat: add cookbook and finalize UI
Browse files Browse the repository at this point in the history
  • Loading branch information
myandrienko committed Oct 1, 2024
1 parent 3d8eeb8 commit e3a4c19
Show file tree
Hide file tree
Showing 13 changed files with 298 additions and 48 deletions.
6 changes: 4 additions & 2 deletions packages/client/src/Call.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2278,7 +2278,8 @@ export class Call {
/**
* Specify preference for incoming video resolution. The preference will
* be matched as close as possible, but actual resolution will depend
* on the video source quality and client network conditions.
* on the video source quality and client network conditions. Will enable
* incoming video, if previously disabled.
*
* @param resolution preferred resolution, or `undefined` to clear preference
* @param sessionIds optionally specify session ids of the participants this
Expand All @@ -2301,7 +2302,8 @@ export class Call {
};

/**
* Enables or disables incoming video from other call participants.
* Enables or disables incoming video from all remote call participants,
* and removes any preference for preferred resolution.
*/
setIncomingVideoEnabled = (enabled: boolean) => {
this.dynascaleManager.setVideoTrackSubscriptionOverrides(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ We consider lobby to be place, where:
The approach to visualise the components will differ from application to application. Therefore, in this guide, we will focus only on the principles of building components and plugging them with right data sources
:::

### The Call data
## The Call data

We would like to show some basic information about the call, when users arrive to the lobby. For example:

Expand Down Expand Up @@ -63,7 +63,7 @@ const call =

These hooks make sure, that the information about call metadata or call members is updated in real time. The updates are made automatically in response to [Stream's WebSocket events](../../advanced/events) arriving from the backend.

### Video Input Preview
## Video Input Preview

We can show the local video preview in the Lobby view, before joining the call to check if everything is fine with your video. We can get the video stream from the camera using the media stream from the `call.camera` object and show it using the `RTCView` component from `@stream-io/react-native-webrtc` library. And by using the `useConnectedUser` hook we can get the user info.

Expand Down Expand Up @@ -177,7 +177,7 @@ const styles = StyleSheet.create({
});
```

### Media Stream Management
## Media Stream Management

To control audio or video mute status in the Lobby, you can use the `useCameraState` and `useMicrophoneState` hooks from the `useCallStateHooks`, that orchestrates the local state of the device within the SDK and handles streaming of the media effectively.

Expand Down Expand Up @@ -294,7 +294,7 @@ const styles = StyleSheet.create({
});
```

### Participants in a call
## Participants in a call

We can retrieve the list of members, that already joined the call (participants), by inspecting the call metadata object (`callMetadata.session.participants`). The object is provided and maintained up-to-date by `useCallMetadata` hook.

Expand Down Expand Up @@ -358,7 +358,7 @@ const styles = StyleSheet.create({
});
```

### Joining the call button
## Joining the call button

Lastly, to join a call we simply invoke call.join(). Learn more about the topic in the dedicated [Joining & Creating Calls guide](../../core/joining-and-creating-calls/).

Expand Down Expand Up @@ -400,7 +400,7 @@ const styles = StyleSheet.create({
});
```

### Assembling it all together
## Assembling it all together

<ImageShowcase
items={[
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ To do that, we follow these steps:
For the token generation, you can use our [Token Generator](https://getstream.io/chat/docs/token_generator/).
:::

### The call data
## The call data

We would like to show some basic information about the call, when users arrive to the lobby. For example:

Expand Down Expand Up @@ -112,7 +112,7 @@ Learn how to build a toggle button for call preview pages in [Call controls tuto
We build our custom sound detector in the [dedicated tutorial about Audio Volume Indicator](../audio-volume-indicator).
:::

### Device selection
## Device selection

Switching devices is done through a set of utility methods or hooks.
We speak a bit more about the function and the pre-built components in the [media devices guide](../../guides/camera-and-microphone).
Expand Down Expand Up @@ -193,7 +193,7 @@ export const AudioOutputDeviceSelector = () => {
};
```

### Participants in a call
## Participants in a call

We can retrieve the list of members, that already joined the call (participants), by inspecting the call session object (`session.participants`).
The object is provided and maintained up-to-date by `useCallSession` hook.
Expand Down Expand Up @@ -226,6 +226,6 @@ export const ParticipantsPreview = () => {
};
```

### Joining the call button
## Joining the call button

Lastly, to join a call we simply invoke `call.join()`. Learn more about the topic in the dedicated [Joining & Creating Calls guide](../../guides/joining-and-creating-calls).
Original file line number Diff line number Diff line change
@@ -0,0 +1,196 @@
---
id: manual-video-quality-selection
title: Manual Video Quality Selection
---

By default, our SDK chooses the incoming video quality that best matches the size of a video element for a given participant. It makes less sense to waste bandwidth receiving Full HD video when it's going to be displayed in a 320 by 240 pixel rectangle.

However, it's still possible to override this behavior and manually request higher resolution video for better quality, or lower resolution to save bandwidth. It's also possible to disable incoming video altogether for an audio-only experience.

:::note
Actual incoming video quality depends on a number of factors, such as the quality of the source video, and network conditions. Manual video quality selection allows you to specify your preference, while the actual resolution is automatically selected from the available resolutions to match that preference as closely as possible.
:::

In this article we'll build a UI control for manual video quality selection.

## Prerequisites

If you haven't
already bootstrapped a video calling application (our
[Video Calling Tutorial](https://getstream.io/video/sdk/react/tutorial/video-calling/)
is a great place to start!), here's a very simple application that we'll use as
a starting point:

```jsx
import {
SpeakerLayout,
StreamCall,
StreamVideo,
StreamVideoClient,
} from '@stream-io/video-react-sdk';
import '@stream-io/video-react-sdk/dist/css/styles.css';

const client = new StreamVideoClient({
apiKey: 'REPLACE_WITH_API_KEY',
user: {
id: 'REPALCE_WITH_USER_ID',
name: 'Jane Doe',
},
token: 'REPLACE_WITH_TOKEN',
});

const App = () => {
const [call, setCall] = useState(null);

useEffect(() => {
const newCall = client.call('appointment', callId);
newCall
.join({ create: true })
.then(() => setCall(newCall))
.catch(() => console.error('Failed to join the call'));

return () =>
newCall.leave().catch(() => console.error('Failed to leave the call'));
}, []);

if (!call) {
return <>Loading...</>;
}

return (
<div className="str-video">
<StreamVideo client={client}>
<StreamCall call={call}>
<SpeakerLayout />
</StreamCall>
</StreamVideo>
</div>
);
};
```

## Getting and Setting Incoming Video Settings

To get the current incoming video quality settings, we will use the `useIncomingVideoQualitySettings` call state hook. Most importantly, it returns the following two values:

- `enabled` - a boolean flag indicating whether incoming video is enabled.
- `preferredResolution` - if video is enabled, an object of the shape `{ width: number; height: number }` containing the current preferred resolution of the incoming video.

To modify the current setting, the following two methods are available on the Call object:

- `setIncomingVideoEnabled` - enables or disables incoming video, clearing any preferred resolution.
- `setPreferredIncomingVideoResolution` - sets the preference for the incoming video resolution, enabling video if it was previously disabled.

:::note
It's also possible to set a preferred resolution per call participant. There's an optional second parameter to the `setPreferredIncomingVideoResolution`, accepting an array of participant session ids. However, in this cookbook we assume that the preferences apply to all call participants.
:::

To combine these two settings into a single control, we'll need to do some mapping:

```typescript
import type { Call } from '@stream-io/video-react-sdk';

const incomingVideoSettings = ['auto', '1080p', '720p', '480p', 'off'] as const;
type IncomingVideoSetting = (typeof incomingVideoSettings)[number];
type VideoDimension = { width: number; height: number };

function applyIncomingVideoSetting(call: Call, setting: IncomingVideoSetting) {
if (setting === 'auto') {
call.setIncomingVideoEnabled(true);
} else if (setting === 'off') {
call.setIncomingVideoEnabled(false);
} else {
call.setPreferredIncomingVideoResolution(
incomingVideoSettingToResolution(setting),
);
}
}

function incomingVideoSettingToResolution(
setting: Exclude<IncomingVideoSetting, 'auto' | 'off'>,
): VideoDimension {
switch (setting) {
case '1080p':
return { width: 1920, height: 1080 };
case '720p':
return { width: 1280, height: 720 };
case '480p':
return { width: 640, height: 480 };
}
}

function incomingVideoResolutionToSetting(
resolution: VideoDimension,
): IncomingVideoSetting {
switch (true) {
case resolution.height >= 1080:
return '1080p';
case resolution.height >= 720:
return '720p';
case resolution.height >= 480:
return '480p';
default:
return 'auto';
}
}
```

## Building Incoming Video Quality Selector

Now we're ready to build a UI control to display and change the incoming video quality.

```tsx
import type { FormEvent } from 'react';
import { useCallStateHooks } from '@stream-io/video-react-sdk';

const IncomingVideoQualitySelector = () => {
const call = useCall();
const { useIncomingVideoSettings } = useCallStateHooks();
const { enabled, preferredResolution } = useIncomingVideoSettings();
let currentSetting: IncomingVideoSetting;

if (!preferredResolution) {
currentSetting = enabled ? 'auto' : 'off';
} else {
currentSetting = incomingVideoResolutionToSetting(preferredResolution);
}

const handleChange = (event: FormEvent<HTMLSelectElement>) => {
if (call) {
const setting = event.currentTarget.value as IncomingVideoSetting;
applyIncomingVideoSetting(call, setting);
}
};

return (
<div className="quality-selector">
<select
className="quality-selector-dropdown"
value={currentSetting}
onChange={handleChange}
>
{incomingVideoSettings.map((setting) => (
<option key={setting} value={setting}>
{setting}
</option>
))}
</select>
</div>
);
};
```

![Zoom in on the video quality selector](../assets/06-ui-cookbook/20-manual-video-quality-selection/video-quality-selector.png)

And now by adding this component inside of the `StreamCall`, we have a video quality selector in the call UI:

```jsx
<StreamVideo client={client}>
<StreamCall call={call}>
<IncomingVideoQualitySelector />
<SpeakerLayout />
</StreamCall>
</StreamVideo>
```

![Video quality selector component in use](../assets/06-ui-cookbook/20-manual-video-quality-selection/video-quality-ui.png)
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ const Select = (props: {
export type DropDownSelectOptionProps = {
label: string;
selected?: boolean;
icon: string;
icon?: string;
};

export const DropDownSelectOption = (props: DropDownSelectOptionProps) => {
Expand All @@ -179,7 +179,7 @@ export const DropDownSelectOption = (props: DropDownSelectOptionProps) => {
onClick: () => handleSelect(index),
})}
>
<Icon className="str-video__dropdown-icon" icon={icon} />
{icon && <Icon className="str-video__dropdown-icon" icon={icon} />}
<span className="str-video__dropdown-label">{label}</span>
</div>
);
Expand Down
2 changes: 1 addition & 1 deletion sample-apps/react/react-dogfood/components/ActiveCall.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ import { ChatUI } from './ChatUI';
import { CallStatsSidebar, ToggleStatsButton } from './CallStatsWrapper';
import { ClosedCaptions, ClosedCaptionsSidebar } from './ClosedCaptions';
import { ToggleSettingsTabModal } from './Settings/SettingsTabModal';
import { IncomingVideoSettingsButton } from './IncomingVideoSettings';
import { ToggleEffectsButton } from './ToggleEffectsButton';
import { ToggleNoiseCancellationButton } from './ToggleNoiseCancellationButton';
import { ToggleFeedbackButton } from './ToggleFeedbackButton';
Expand All @@ -48,7 +49,6 @@ import {

import { StepNames, useTourContext } from '../context/TourContext';
import { useNotificationSounds } from '../hooks/useNotificationSounds';
import { IncomingVideoSettingsButton } from './IncomingVideoSettingsButton';

export type ActiveCallProps = {
chatClient?: StreamChat | null;
Expand Down
Loading

0 comments on commit e3a4c19

Please sign in to comment.