Skip to content

Commit

Permalink
feat: use TextDecoder to decode buffer (#1699)
Browse files Browse the repository at this point in the history
<!--
**Thanks for contributing to Fury.**

**If this is your first time opening a PR on fury, you can refer to
[CONTRIBUTING.md](https://github.com/apache/fury/blob/main/CONTRIBUTING.md).**

Contribution Checklist

- The **Apache Fury (incubating)** community has restrictions on the
naming of pr titles. You can also find instructions in
[CONTRIBUTING.md](https://github.com/apache/fury/blob/main/CONTRIBUTING.md).

- Fury has a strong focus on performance. If the PR you submit will have
an impact on performance, please benchmark it first and provide the
benchmark result here.
-->

## What does this PR do?

<!-- Describe the purpose of this PR. -->

Use the browser builtin decode utils: TextDecoder

https://developer.mozilla.org/en-US/docs/Web/API/TextDecoder

This can improve a lot of performance. see the diff of the benchmark.

and the original implementation will cause many `minor gc` in
javascript, the longer the string, the longer the GC time.

## Related issues



## Does this PR introduce any user-facing change?

<!--
If any user-facing interface changes, please [open an
issue](https://github.com/apache/fury/issues/new/choose) describing the
need to do so and update the document if necessary.
-->

- [ ] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?


## Benchmark

<!--
When the PR has an impact on performance (if you don't know whether the
PR will have an impact on performance, you can submit the PR first, and
if it will have impact on performance, the code reviewer will explain
it), be sure to attach a benchmark data here.
-->

![CleanShot 2024-06-25 at 17 04
45@2x](https://github.com/apache/fury/assets/13938334/29517aa9-563f-46ba-9aec-563d49c8db9a)
  • Loading branch information
bytemain authored Jul 3, 2024
1 parent 0c9dcc2 commit d25ccbb
Show file tree
Hide file tree
Showing 2 changed files with 8 additions and 17 deletions.
25 changes: 8 additions & 17 deletions javascript/packages/fury/lib/platformBuffer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@

import { hasBuffer } from "./util";

let utf8Encoder: TextEncoder | null;
let textDecoder: TextDecoder | null;

export type SupportedEncodings = "latin1" | "utf8";

export interface PlatformBuffer extends Uint8Array {
Expand Down Expand Up @@ -96,22 +99,12 @@ export class BrowserBuffer extends Uint8Array implements PlatformBuffer {
if (end - start < 1) {
return "";
}
let str = "";
for (let i = start; i < end;) {
const t = this[i++];
if (t <= 0x7F) {
str += String.fromCharCode(t);
} else if (t >= 0xC0 && t < 0xE0) {
str += String.fromCharCode((t & 0x1F) << 6 | this[i++] & 0x3F);
} else if (t >= 0xE0 && t < 0xF0) {
str += String.fromCharCode((t & 0xF) << 12 | (this[i++] & 0x3F) << 6 | this[i++] & 0x3F);
} else if (t >= 0xF0) {
const t2 = ((t & 7) << 18 | (this[i++] & 0x3F) << 12 | (this[i++] & 0x3F) << 6 | this[i++] & 0x3F) - 0x10000;
str += String.fromCharCode(0xD800 + (t2 >> 10));
str += String.fromCharCode(0xDC00 + (t2 & 0x3FF));
}

if (!textDecoder) {
textDecoder = new TextDecoder("utf-8");
}
return str;

return textDecoder.decode(this.subarray(start, end));
}

copy(target: Uint8Array, targetStart?: number, sourceStart?: number, sourceEnd?: number) {
Expand Down Expand Up @@ -153,8 +146,6 @@ export const alloc = (hasBuffer ? Buffer.allocUnsafe : BrowserBuffer.alloc) as u

export const strByteLength = hasBuffer ? Buffer.byteLength : BrowserBuffer.byteLength;

let utf8Encoder: TextEncoder | null;

export const fromString
= hasBuffer
? (str: string) => Buffer.from(str) as unknown as PlatformBuffer
Expand Down
Binary file added platform-buffer.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit d25ccbb

Please sign in to comment.