Skip to content

Commit

Permalink
Add transformer - draft
Browse files Browse the repository at this point in the history
  • Loading branch information
hamishwillee committed Aug 21, 2023
1 parent 82941de commit 9b4988f
Show file tree
Hide file tree
Showing 3 changed files with 40 additions and 15 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,13 @@ browser-compat: api.RTCRtpScriptTransformer.generateKeyFrame

{{APIRef("WebRTC Encoded Transform")}} <!-- {{APIRef("WebRTC")}} -->

The **`generateKeyFrame()`** method of the {{domxref("RTCRtpScriptTransformer")}} interface ...
The **`generateKeyFrame()`** method of the {{domxref("RTCRtpScriptTransformer")}} interface causes a video encoder generate a key frame.

https://w3c.github.io/webrtc-encoded-transform/#abstract-opdef-generate-key-frame-algorithm
This method is called by a transformer that is processing outgoing frames to force a new complete frame to be sent (rather than a delta frame).
It might be needed by a WebRTC Encoded Transform that encrypts frames, to ensure that if a new encryption key is added, a key frame encrypted with that key is sent as soon as possible.

The sender can specify a restriction identifier for the requested frame, which will restrict what encoders can be used.
The promise returned by the method will resolve just before enqueuing the corresponding key frame in a `RTCRtpScriptTransformer` readable.

## Syntax

Expand All @@ -23,15 +27,25 @@ generateKeyFrame(rid)

- `rid` {{optional_inline}}

- : A string ...
- : A string containing a "restriction identifier" ("rid"), which selects a video encoder to generate the frame based on capabilities such as: max-width, max-height, max-fps, max-fs (frame size), max-br (bitrate), max-pps (pixels per second), max-bpp (bits per pixel) depends (streams this steam depends on for interpretation).
The format is defined in [rfc8851](https://www.rfc-editor.org/rfc/rfc8851).

The RID helps select the video encoder to generate the key frame based on its capabilities.
The first encoder that matches the specified `rid` is used.
If no encoder matches the `rid` then the first encoder is used, and `rid` is set to the encoder's restrictions.

### Return value

`Promise<unsigned long long>`
A {{jsxref("Promise")}} that fulfills with the timestamp of the frame.

### Exceptions

InvalidStateError,
- `InvalidStateError`
- The encoder is not processing video frames or is `undefined`.
- `TypeError`
- The provided `rid` but does not conform to the grammar requirements specified in Section 10 of [RFC8851]
- `NotFoundError`
- There are no video encoders. This might be raised if the corresponding `RTCRtpSender` is not active or its track is ended.

## Examples

Expand Down
16 changes: 9 additions & 7 deletions files/en-us/web/api/rtcrtpscripttransformer/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,15 @@ browser-compat: api.RTCRtpScriptTransformer

{{APIRef("WebRTC Encoded Transform")}} <!-- {{APIRef("WebRTC")}} -->

The **`RTCRtpScriptTransformer`** interface of the [WebRTC API](/en-US/docs/Web/API/WebRTC_API) is an abstraction for a stream object that can be used to transform encoded video and audio frames as are sent or received.
The **`RTCRtpScriptTransformer`** interface of the [WebRTC API](/en-US/docs/Web/API/WebRTC_API) is the worker-side counterpart of a {{DOMxRef("RTCRtpScriptTransform")}}.

Like the {{DOMxRef("TransformStream")}} interface, this represents an abstraction a concrete implementation of the [pipe chain](/en-US/docs/Web/API/Streams_API/Concepts#pipe_chains) _transform stream_ concept.
The transformer exposes a `readable` and `writable` stream into the worker, along with an "options" object provided to the {{DOMxRef("RTCRtpScriptTransform")}} on construction.
When the associated `RTCRtpScriptTransform` is assigned to a {{DOMxRef("RTCRtpSender")}} or {{DOMxRef("RTCRtpReceiver")}}, encoded media frames from the WebRTC sender or receiver pipelines are enqueued on the `readable` stream.

This is only available in dedicated workers (not the main window).
A WebRTC Encoded Transform pipes the `readable` through one or more {{DOMxRef("TransformStream")}} instances to the `writable`, and pipes the readable.
The "options" allow the transform code to determine whether the encoded media frames are incoming or outgoing.

Objects of the type are only available in dedicated workers (not the main window), and are created when you construct a {{DOMxRef("RTCRtpScriptTransform")}}.

{{AvailableInWorkers}}

Expand All @@ -27,14 +31,12 @@ This is only available in dedicated workers (not the main window).
## Instance methods

- {{domxref("RTCRtpScriptTransformer.generateKeyFrame()")}}
- : ?
- : Requests a video encoder generate a key frame. Called by a transformer when processing outgoing frames.
- {{domxref("RTCRtpScriptTransformer.sendKeyFrameRequest()")}}
- : ?
- : Requests the sender send a key frame. Called by a transformer when processing incoming encoded video frames.

## Examples

### ?

In the following example ...

## Specifications
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,13 @@ browser-compat: api.RTCRtpScriptTransformer.sendKeyFrameRequest

{{APIRef("WebRTC Encoded Transform")}} <!-- {{APIRef("WebRTC")}} -->

The **`sendKeyFrameRequest()`** method of the {{domxref("RTCRtpScriptTransformer")}} interface ...
The **`sendKeyFrameRequest()`** method of the {{domxref("RTCRtpScriptTransformer")}} interface may be called by WebRTC Encoded Transform that is processing incoming encoded video frames, in order to request a key frame from the sender.

This method might be called when a new user joins a WebRTC conference, in order to reduce the time before the user can start decoding the video (delta frames unless the recipient has a key frame and all subsequent key frames).
It might also be needed by a WebRTC Encoded Transform that encrypts frames, to ensure that if a new encryption key is added, a key frame encrypted with that key is sent as soon as possible.

The message causes a Full Intra Request (FIR) to be sent back by the receiver of a video frame from a packetizer.
It should only be called when when receiving video and, for whatever reason, a user will not be able to decode a frame.

## Syntax

Expand All @@ -22,13 +28,16 @@ None

### Return value

`Promise<undefined>`
A {{jsxref("Promise")}} that fulfills with `undefined`.

### Exceptions

- `InvalidStateError`
- : The de-packetizer is not processing video packages, or is undefined.

## Examples

### ???
TBD.

## Specifications

Expand Down

0 comments on commit 9b4988f

Please sign in to comment.