Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(closed captions): Integration in the SDKs + UI Cookbooks #1508

Open
wants to merge 34 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
96ed9b9
docs: closed captions React cookbook
myandrienko Oct 3, 2024
1d26c9b
fix syntax highlighting
myandrienko Oct 3, 2024
ed47579
docs: closed captions (#1497)
szuperaz Oct 3, 2024
a32257a
Merge branch 'main' into docs/cc-react
oliverlaz Oct 3, 2024
4e0b750
feat: move closed caption handling to the SDK [wip]
oliverlaz Oct 3, 2024
933e754
feat: handle closed captions in the SDK
oliverlaz Oct 4, 2024
1895625
Merge branch 'main' into docs/cc-react
oliverlaz Oct 4, 2024
462c6fd
chore: more assertions
oliverlaz Oct 4, 2024
aaca354
chore: more tests
oliverlaz Oct 4, 2024
31ee11a
Update ts quickstart to use SDK state
szuperaz Oct 4, 2024
741e48c
feat: add cc start/stop endpoints
oliverlaz Oct 4, 2024
aa56be1
Update JS docs
szuperaz Oct 4, 2024
64eb786
Add start/stop endpoints
szuperaz Oct 4, 2024
10d5cb8
feat: enrich closed captions with speaker name
oliverlaz Oct 4, 2024
933cdbd
Merge branch 'main' into docs/cc-react
oliverlaz Oct 4, 2024
cf645da
Integrate speaker_name field to JS app
szuperaz Oct 4, 2024
da3bbbe
Remove unnecessary code
szuperaz Oct 4, 2024
7432c06
fix: use optimistic updates
oliverlaz Oct 7, 2024
1d0d2a6
fix: bubble the error up
oliverlaz Oct 7, 2024
a73c465
Merge branch 'main' into docs/cc-react
oliverlaz Oct 7, 2024
fb1e300
fix: update docs, missing hook, toggle button
oliverlaz Oct 7, 2024
c0fbeae
feat: add ClosedCaptions component for React Native
oliverlaz Oct 7, 2024
ee3720e
chore: missing imports
oliverlaz Oct 7, 2024
067c47e
chore: align cookbooks
oliverlaz Oct 7, 2024
6901fda
docs: update example, add screenshot
oliverlaz Oct 8, 2024
f09d1ec
feat: update to the newest OpenAPI schema; update the cookbooks
oliverlaz Oct 8, 2024
e44bfd6
fix: use correct types, update cookbook
oliverlaz Oct 8, 2024
f1cccbb
Merge branch 'refs/heads/main' into docs/cc-react
oliverlaz Oct 22, 2024
1dce3c2
Merge branch 'main' into docs/cc-react
oliverlaz Nov 1, 2024
026c1fa
feat: update schema
oliverlaz Nov 1, 2024
9e9bbbe
Merge branch 'main' into docs/cc-react
myandrienko Nov 7, 2024
e2e3828
update openapi types
myandrienko Nov 7, 2024
579950a
Merge branch 'main' into docs/cc-react
myandrienko Nov 14, 2024
740f195
revert doc updates
myandrienko Nov 14, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion packages/client/openapitools.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@
"$schema": "../../node_modules/@openapitools/openapi-generator-cli/config.schema.json",
"spaces": 2,
"generator-cli": {
"version": "7.5.0"
"version": "7.8.0"
}
}
51 changes: 48 additions & 3 deletions packages/client/src/Call.ts
Original file line number Diff line number Diff line change
Expand Up @@ -54,12 +54,14 @@ import type {
SendCallEventResponse,
SendReactionRequest,
SendReactionResponse,
StartClosedCaptionsResponse,
StartHLSBroadcastingResponse,
StartRecordingRequest,
StartRecordingResponse,
StartTranscriptionRequest,
StartTranscriptionResponse,
StatsOptions,
StopClosedCaptionsResponse,
StopHLSBroadcastingResponse,
StopLiveResponse,
StopRecordingResponse,
Expand All @@ -74,13 +76,14 @@ import type {
UpdateCallResponse,
UpdateUserPermissionsRequest,
UpdateUserPermissionsResponse,
VideoResolution,
VideoDimension,
} from './gen/coordinator';
import { OwnCapability } from './gen/coordinator';
import {
AudioTrackType,
CallConstructor,
CallLeaveOptions,
ClosedCaptionsSettings,
JoinCallData,
PublishOptions,
TrackMuteType,
Expand Down Expand Up @@ -545,6 +548,7 @@ export class Call {
this.dynascaleManager.setSfuClient(undefined);

this.state.setCallingState(CallingState.LEFT);
this.state.dispose();

// Call all leave call hooks, e.g. to clean up global event handlers
this.leaveCallHooks.forEach((hook) => hook());
Expand Down Expand Up @@ -1713,7 +1717,48 @@ export class Call {
};

/**
* Sends a `call.permission_request` event to all users connected to the call. The call settings object contains infomration about which permissions can be requested during a call (for example a user might be allowed to request permission to publish audio, but not video).
* Starts the closed captions of the call.
*/
startClosedCaptions = async (): Promise<StartClosedCaptionsResponse> => {
const trx = this.state.setCaptioning(true); // optimistic update
try {
return await this.streamClient.post<StartClosedCaptionsResponse>(
`${this.streamClientBasePath}/start_closed_captions`,
);
} catch (err) {
trx.rollback(); // revert the optimistic update
throw err;
}
};

/**
* Stops the closed captions of the call.
*/
stopClosedCaptions = async (): Promise<StopClosedCaptionsResponse> => {
const trx = this.state.setCaptioning(false); // optimistic update
try {
return await this.streamClient.post<StopClosedCaptionsResponse>(
`${this.streamClientBasePath}/stop_closed_captions`,
);
} catch (err) {
trx.rollback(); // revert the optimistic update
throw err;
}
};

/**
* Updates the closed caption settings.
*
* @param config the closed caption settings to apply
*/
updateClosedCaptionSettings = (config: Partial<ClosedCaptionsSettings>) => {
this.state.updateClosedCaptionSettings(config);
};

/**
* Sends a `call.permission_request` event to all users connected to the call.
* The call settings object contains information about which permissions can be requested during a call
* (for example, a user might be allowed to request permission to publish audio, but not video).
*/
requestPermissions = async (
data: RequestPermissionRequest,
Expand Down Expand Up @@ -2311,7 +2356,7 @@ export class Call {
* preference has effect on. Affects all participants by default.
*/
setPreferredIncomingVideoResolution = (
resolution: VideoResolution | undefined,
resolution: VideoDimension | undefined,
sessionIds?: string[],
) => {
this.dynascaleManager.setVideoTrackSubscriptionOverrides(
Expand Down
Loading
Loading