Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Batches to zarr #40

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
Open

Batches to zarr #40

wants to merge 9 commits into from

Commits on Nov 17, 2021

  1. Functionality for storing batches with zarr 

    Add `BatchGenerator.to_zarr` and `BatchGenerator.from_zarr` to make it
possible to save generated batches to zarr and later load them from
zarr. By chunking along the batch dimension this enables fast
data-loading at training time.
    @leifdenby
    leifdenby committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    0e7b538 View commit details
    Browse the repository at this point in the history

  2. cleanup test

    @leifdenby
    leifdenby committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    45e20ec View commit details
    Browse the repository at this point in the history

  3. Apply linting etc with pre-commit

    @leifdenby
    leifdenby committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    9083881 View commit details
    Browse the repository at this point in the history

  4. add zarr to dev-requirements

    @leifdenby
    leifdenby committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    7b2341b View commit details
    Browse the repository at this point in the history

  5. Merge branch 'main' into batches-to-zarr

    @leifdenby
    leifdenby committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    66a9ec5 View commit details
    Browse the repository at this point in the history

Commits on Feb 3, 2022

  1. Merge branch 'main' into batches-to-zarr

    Joe Hamman authored Feb 3, 2022
    Configuration menu
    Copy the full SHA
    dd4108c View commit details
    Browse the repository at this point in the history

Commits on May 10, 2022

  1. Merge branch 'main' of https://github.com/pangeo-data/xbatcher into b… 

    …atches-to-zarr
    @leifdenby
    leifdenby committed May 10, 2022
    Configuration menu
    Copy the full SHA
    e7dca77 View commit details
    Browse the repository at this point in the history

  2. store init attrs and create BatchGeneratorBase

    @leifdenby
    leifdenby committed May 10, 2022
    Configuration menu
    Copy the full SHA
    1ce312f View commit details
    Browse the repository at this point in the history

Commits on May 11, 2022

  1. linting fixes

    @leifdenby
    leifdenby committed May 11, 2022
    Configuration menu
    Copy the full SHA
    08a9e94 View commit details
    Browse the repository at this point in the history