Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(storage): Dict encoding for compaction #740

Merged

Conversation

yyin-dev
Copy link
Contributor

This PR addresses #674

Signed-off-by: Yue Yin [email protected]

Signed-off-by: Yue Yin <[email protected]>
@yyin-dev yyin-dev force-pushed the yueyin/dict_encoding_for_compaction branch from f64a216 to b3d3888 Compare December 13, 2022 00:22
Signed-off-by: Yue Yin <[email protected]>
@skyzh skyzh enabled auto-merge (squash) December 16, 2022 04:39
@skyzh skyzh merged commit 467e953 into risinglightdb:main Dec 16, 2022
MingjiHan99 pushed a commit that referenced this pull request Dec 22, 2022
* Dict encoding for compaction

Signed-off-by: Yue Yin <[email protected]>

* CI

Signed-off-by: Yue Yin <[email protected]>

* Add tests

Signed-off-by: Yue Yin <[email protected]>

Signed-off-by: Yue Yin <[email protected]>
Signed-off-by: MingjiHan <[email protected]>
MingjiHan99 pushed a commit that referenced this pull request Dec 22, 2022
* Dict encoding for compaction

Signed-off-by: Yue Yin <[email protected]>

* CI

Signed-off-by: Yue Yin <[email protected]>

* Add tests

Signed-off-by: Yue Yin <[email protected]>

Signed-off-by: Yue Yin <[email protected]>
Signed-off-by: MingjiHan <[email protected]>
wangrunji0408 added a commit that referenced this pull request Dec 23, 2022
* fix(storage): compaction type error (#737)

* Fix compaction type error

Signed-off-by: Yue Yin <[email protected]>

* assert empty

Signed-off-by: Alex Chi <[email protected]>

Signed-off-by: Yue Yin <[email protected]>
Signed-off-by: Alex Chi <[email protected]>
Co-authored-by: Alex Chi <[email protected]>
Signed-off-by: MingjiHan <[email protected]>

* chore: bump sqllogictest to 0.9.0 (#736)

Signed-off-by: MingjiHan <[email protected]>

* feat(storage): Dict encoding for compaction (#740)

* Dict encoding for compaction

Signed-off-by: Yue Yin <[email protected]>

* CI

Signed-off-by: Yue Yin <[email protected]>

* Add tests

Signed-off-by: Yue Yin <[email protected]>

Signed-off-by: Yue Yin <[email protected]>
Signed-off-by: MingjiHan <[email protected]>

* perf(expr): apply auto-vectorization and remove explicit SIMDs (#741)

* remove explicit simd

Signed-off-by: Runji Wang <[email protected]>

* apply auto-vectorization for all binary/unary ops

Signed-off-by: Runji Wang <[email protected]>

* add more bench for ops

Signed-off-by: Runji Wang <[email protected]>

* SIMD accelerate &[bool] to BitVec

Signed-off-by: Runji Wang <[email protected]>

* optimize const expression evaluation

Signed-off-by: Runji Wang <[email protected]>

* fix clippy

Signed-off-by: Runji Wang <[email protected]>

* recover push

Signed-off-by: Runji Wang <[email protected]>

Signed-off-by: Runji Wang <[email protected]>
Signed-off-by: MingjiHan <[email protected]>

* release: v0.2 (#742)

Signed-off-by: Alex Chi <[email protected]>
Signed-off-by: MingjiHan <[email protected]>

* perf(expr): further optimize performance (#744)

* optimize to string array

Signed-off-by: Runji Wang <[email protected]>

* optimize bitvec

Signed-off-by: Runji Wang <[email protected]>

* avoid zip_eq for performance

Signed-off-by: Runji Wang <[email protected]>

* array: add `is_null` and `get_raw`

Signed-off-by: Runji Wang <[email protected]>

* add bench for array filter

Signed-off-by: Runji Wang <[email protected]>

* optimize filter -30%

Signed-off-by: Runji Wang <[email protected]>

* optimize filter from bool array

Signed-off-by: Runji Wang <[email protected]>

* clear null data

Signed-off-by: Runji Wang <[email protected]>

* fix cardinality error

Signed-off-by: Runji Wang <[email protected]>

* remove array iterator

Signed-off-by: Runji Wang <[email protected]>

* introduce non-null iterator

Signed-off-by: Runji Wang <[email protected]>

* optimize bitmap &&

Signed-off-by: Runji Wang <[email protected]>

* optimize BitVec operations

Signed-off-by: Runji Wang <[email protected]>

* fix clippy and test

Signed-off-by: Runji Wang <[email protected]>

Signed-off-by: Runji Wang <[email protected]>
Signed-off-by: MingjiHan <[email protected]>

* add path option (#747)

Signed-off-by: MingjiHan <[email protected]>

Signed-off-by: MingjiHan <[email protected]>

* wtf

Signed-off-by: MingjiHan <[email protected]>

* updates

Signed-off-by: Mingji Han <[email protected]>
Signed-off-by: MingjiHan <[email protected]>

* change python compile configurations

Signed-off-by: Mingji Han <[email protected]>
Signed-off-by: MingjiHan <[email protected]>

* updates

Signed-off-by: Mingji Han <[email protected]>
Signed-off-by: MingjiHan <[email protected]>

* change python compile configurations

Signed-off-by: Mingji Han <[email protected]>
Signed-off-by: MingjiHan <[email protected]>

* add python type conversion

Signed-off-by: Mingji Han <[email protected]>
Signed-off-by: MingjiHan <[email protected]>

* add docs

Signed-off-by: Mingji Han <[email protected]>
Signed-off-by: MingjiHan <[email protected]>

* fix format

Signed-off-by: Mingji Han <[email protected]>
Signed-off-by: MingjiHan <[email protected]>

* fix macos

Signed-off-by: MingjiHan <[email protected]>

* fix linux...

Signed-off-by: MingjiHan <[email protected]>

* support macOS complication

Signed-off-by: MingjiHan <[email protected]>

* Update docs/07-python-extension.md

Co-authored-by: Runji Wang <[email protected]>
Signed-off-by: MingjiHan <[email protected]>

* Update docs/07-python-extension.md

Co-authored-by: Runji Wang <[email protected]>
Signed-off-by: MingjiHan <[email protected]>

* Update src/lib.rs

Co-authored-by: Runji Wang <[email protected]>
Signed-off-by: MingjiHan <[email protected]>

* Update src/lib.rs

Co-authored-by: Runji Wang <[email protected]>
Signed-off-by: MingjiHan <[email protected]>

* move files

Signed-off-by: MingjiHan <[email protected]>

Signed-off-by: Yue Yin <[email protected]>
Signed-off-by: Alex Chi <[email protected]>
Signed-off-by: MingjiHan <[email protected]>
Signed-off-by: Runji Wang <[email protected]>
Signed-off-by: Mingji Han <[email protected]>
Co-authored-by: Yue Yin <[email protected]>
Co-authored-by: Alex Chi <[email protected]>
Co-authored-by: xxchan <[email protected]>
Co-authored-by: Runji Wang <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants