Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[flink-runner] Improve Datastream for batch performances #32440

Open
wants to merge 27 commits into
base: master
Choose a base branch
from
Open

[flink-runner] Improve Datastream for batch performances #32440

wants to merge 27 commits into from

Commits on Oct 17, 2024

  1. [Flink] Set return type of bounded sources

    @jto
    jto committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    b817957 View commit details
    Browse the repository at this point in the history

  2. [Flink] Use a lazy split enumerator for bounded sources 

    [Flink] fix lazy enumerator package
    @jto
    jto committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    ca9630a View commit details
    Browse the repository at this point in the history

  3. [Flink] Default to maxParallelism = parallelism in batch

    @jto
    jto committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    866fb78 View commit details
    Browse the repository at this point in the history

  4. [Flink] Avoid re-serializing trigger on every element

    @jto
    jto committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    5c89f15 View commit details
    Browse the repository at this point in the history

  5. [Flink] Avoid re-evaluating options every time a new state is stored

    @jto
    jto committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    f2b2eb7 View commit details
    Browse the repository at this point in the history

  6. [Flink] Only serialize states namespace keys if necessary

    @jto
    jto committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    cf670c7 View commit details
    Browse the repository at this point in the history

  7. [Flink] Make ToKeyedWorkItem part of the DoFnOperator

    @jto
    jto committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    e805ad1 View commit details
    Browse the repository at this point in the history

  8. [Flink] Remove ToBinaryKV

    @jto
    jto committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    a96e8ed View commit details
    Browse the repository at this point in the history

  9. [Flink] Refactor CombinePerKeyTranslator

    @jto
    jto committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    490576e View commit details
    Browse the repository at this point in the history

  10. [Flink] Combine before Reduce (no side-input only) 

    [Flink] Implement partial reduce

[Flink] dead code cleanup

[Flink] spotless

[Flink] persistent PartialReduceBundleOperator operator state
    @jto
    jto committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    37847e7 View commit details
    Browse the repository at this point in the history

  11. [Flink] Combine before GBK

    @jto
    jto committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    04f3d68 View commit details
    Browse the repository at this point in the history

  12. [Flink] Combine before reduce (with side input)

    @jto
    jto committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    f38fd11 View commit details
    Browse the repository at this point in the history

  13. [Flink] Force slot sharing group in batch mode

    @jto
    jto committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    d96a464 View commit details
    Browse the repository at this point in the history

  14. [Flink] Disable bundling in batch mode

    @jto
    jto committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    d1de772 View commit details
    Browse the repository at this point in the history

  15. [Flink] Lower default max bundle size in batch mode

    @jto
    jto committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    4b205b7 View commit details
    Browse the repository at this point in the history

  16. [Flink] Code cleanup 

    * spotless
* checkstyle
* spotless
    @jto
    jto committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    e7699d0 View commit details
    Browse the repository at this point in the history

  17. [Flink] fix WindowDoFnOperatorTest

    @jto
    jto committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    ff9cb80 View commit details
    Browse the repository at this point in the history

  18. [Flink] spotless 

    [Flink] spotless
    @jto
    jto committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    6fddd1b View commit details
    Browse the repository at this point in the history

  19. [Flink] fix broken tests

    @jto
    jto committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    7d62bf5 View commit details
    Browse the repository at this point in the history

  20. [Flink] Remove 1.14 compat code

    @jto
    jto committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    98a99f4 View commit details
    Browse the repository at this point in the history

  21. [Flink] Fix flaky test

    @jto
    jto committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    21e83bd View commit details
    Browse the repository at this point in the history

  22. [Flink] Use a custom key type to better distribute load

    @jto
    jto committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    eaa08a5 View commit details
    Browse the repository at this point in the history

  23. [Flink] Add post commit triggers

    @jto
    jto committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    ac24d8f View commit details
    Browse the repository at this point in the history

  24. [Flink] licence

    @jto
    jto committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    a7b9023 View commit details
    Browse the repository at this point in the history

  25. [Flink] spotless

    @jto
    jto committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    ae0f3b4 View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2024

  1. Additional Flink github action trigger files

    @kennknowles
    kennknowles committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    9738b01 View commit details
    Browse the repository at this point in the history

Commits on Nov 5, 2024

  1. Merge pull request #236 from kennknowles/flink-datastream-tests 

    Additional Flink github action trigger files
    @jto
    jto authored Nov 5, 2024
    Configuration menu
    Copy the full SHA
    db09fd5 View commit details
    Browse the repository at this point in the history