Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

perf: Pipelined parallel extract #72

Closed
wants to merge 16 commits into from
Closed

perf: Pipelined parallel extract #72

wants to merge 16 commits into from

Commits on Sep 18, 2023

  1. and we got a benchmark

    @cosmicexplorer
    cosmicexplorer committed Sep 18, 2023
    Configuration menu
    Copy the full SHA
    47d2c39 View commit details
    Browse the repository at this point in the history

  2. significantly faster pipelined extraction!!!

    @cosmicexplorer
    cosmicexplorer committed Sep 18, 2023
    Configuration menu
    Copy the full SHA
    009c6dd View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2023

  1. using a secondary rayon threadpool is SLOWER than manual threading!!

    @cosmicexplorer
    cosmicexplorer committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    5d939b8 View commit details
    Browse the repository at this point in the history

  2. convert Vec<u8> to Bytes/BytesMut

    @cosmicexplorer
    cosmicexplorer committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    ddd7cf5 View commit details
    Browse the repository at this point in the history

  3. add once_cell

    @cosmicexplorer
    cosmicexplorer committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    e0afdd4 View commit details
    Browse the repository at this point in the history

  4. conver to rayon::join

    @cosmicexplorer
    cosmicexplorer committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    ebb9148 View commit details
    Browse the repository at this point in the history

  5. reuse the file handle to set permissions

    @cosmicexplorer
    cosmicexplorer committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    321ffca View commit details
    Browse the repository at this point in the history

  6. clarify randomness in bench (means incompressibility)

    @cosmicexplorer
    cosmicexplorer committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    24e76c7 View commit details
    Browse the repository at this point in the history

  7. move dir creation out of the hot path

    @cosmicexplorer
    cosmicexplorer committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    1922204 View commit details
    Browse the repository at this point in the history

  8. avoid the need to reverse by hand

    @cosmicexplorer
    cosmicexplorer committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    e4dbf6e View commit details
    Browse the repository at this point in the history

Commits on Sep 21, 2023

  1. ok the pipelining works perfectly and lucidrously fast now

    @cosmicexplorer
    cosmicexplorer committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    f4c11f1 View commit details
    Browse the repository at this point in the history

Commits on May 1, 2024

  1. Merge remote-tracking branch 'cosmicex/pipelined-extract' into oldpr407 

    # Conflicts:
#	Cargo.toml
#	src/read.rs
#	src/write.rs
    @Pr0methean
    Pr0methean committed May 1, 2024
    Configuration menu
    Copy the full SHA
    a9a182b View commit details
    Browse the repository at this point in the history

Commits on May 2, 2024

  1. Merge branch 'master' into oldpr407

    @Pr0methean
    Pr0methean authored May 2, 2024
    Configuration menu
    Copy the full SHA
    64fadca View commit details
    Browse the repository at this point in the history

Commits on May 6, 2024

  1. Merge branch 'master' into oldpr407 

    Signed-off-by: Chris Hennick <[email protected]>
    @Pr0methean
    Pr0methean authored May 6, 2024
    Configuration menu
    Copy the full SHA
    fdc7b2b View commit details
    Browse the repository at this point in the history

Commits on May 10, 2024

  1. Merge branch 'master' into oldpr407 

    Signed-off-by: Chris Hennick <[email protected]>
    @Pr0methean
    Pr0methean authored May 10, 2024
    Configuration menu
    Copy the full SHA
    e0bf245 View commit details
    Browse the repository at this point in the history

  2. chore: Partial fix for build errors 

    Signed-off-by: Chris Hennick <[email protected]>
    @Pr0methean
    Pr0methean authored May 10, 2024
    Configuration menu
    Copy the full SHA
    ff8a59a View commit details
    Browse the repository at this point in the history