Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open kerchunk ref as virtual dataset, only json (from PR 119) #186

Closed

Conversation

kthyng
Copy link

@kthyng kthyng commented Jul 13, 2024

This started from #119 and is meant to finish an MVP of that work. All of the work here is essentially from there, which @norlandrhagen did!

@TomNicholas merged main into that branch. Most changes here were brought in from main branch to fix a few leftover errors. Also a change so that if a parquet file is input the NotImplemented exception is raised, and a docstring added for the kerchunk filetype.

  • Closes #xxxx
  • Tests added
  • Tests passing
  • Changes are documented in docs/releases.rst
  • New functions/methods are listed in api.rst
  • New functionality has documentation

Most changes brought in from main branch to fix a few errors. Also a change so that if a
parquet file is input the NotImplemented exception is raised, and a docstring added for the
kerchunk filetype.
@kthyng
Copy link
Author

kthyng commented Jul 15, 2024

Two concerns as I've been working with this more:

  1. I wasn't able to read in a json file I generated with the kerchunk library with ValueError: too many values to unpack (expected 3) (details below)
  2. When I generated json files and then read them back in to create a combined dataset, that combined dataset looked different than the dataset I had initially made before I had been saving them to file. In particular, w was a coordinate instead of a data_var, and some coordinates were data_vars instead of coordinates. This was also a problem with the original combined dataset, but some variables had changed classification.

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[19], [line 11](vscode-notebook-cell:?execution_count=19&line=11)
      [9](vscode-notebook-cell:?execution_count=19&line=9) for filepath in sorted(basedir.glob(f"{year}*.json")):
     [10](vscode-notebook-cell:?execution_count=19&line=10)     print(filepath)
---> [11](vscode-notebook-cell:?execution_count=19&line=11)     vdss.append(open_virtual_dataset(str(filepath), filetype="kerchunk"))
     [12](vscode-notebook-cell:?execution_count=19&line=12) combined_vds = xr.concat(vdss, dim='time', coords='minimal', compat='override')
     [13](vscode-notebook-cell:?execution_count=19&line=13) combined_vds.virtualize.to_kerchunk(f'combined_{year}.parq', format='parquet')

File ~/packages/VirtualiZarr/virtualizarr/xarray.py:150, in open_virtual_dataset(filepath, filetype, drop_variables, loadable_variables, cftime_variables, indexes, virtual_array_class, reader_options)
    [146](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:146)     import json
    [148](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:148)     refs_dict = json.loads(fpath.read().decode("utf-8"))
--> [150](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:150)     vds = dataset_from_kerchunk_refs(refs_dict)
    [151](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:151)     return vds
    [152](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:152) elif kerchunk_storage_ftype in [".parquet", ".parq"]:

File ~/packages/VirtualiZarr/virtualizarr/xarray.py:343, in dataset_from_kerchunk_refs(refs, drop_variables, virtual_array_class, indexes)
    [327](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:327) def dataset_from_kerchunk_refs(
    [328](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:328)     refs: KerchunkStoreRefs,
    [329](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:329)     drop_variables: list[str] = [],
    [330](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:330)     virtual_array_class: type = ManifestArray,
    [331](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:331)     indexes: MutableMapping[str, Index] | None = None,
    [332](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:332) ) -> xr.Dataset:
    [333](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:333)     """
    [334](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:334)     Translate a store-level kerchunk reference dict into an xarray Dataset containing virtualized arrays.
    [335](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:335) 
   (...)
    [340](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:340)         Currently can only be ManifestArray, but once VirtualZarrArray is implemented the default should be changed to that.
    [341](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:341)     """
--> [343](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:343)     vars = virtual_vars_from_kerchunk_refs(refs, drop_variables, virtual_array_class)
    [344](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:344)     ds_attrs = kerchunk.fully_decode_arr_refs(refs["refs"]).get(".zattrs", {})
    [345](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:345)     coord_names = ds_attrs.pop("coordinates", [])

File ~/packages/VirtualiZarr/virtualizarr/xarray.py:321, in virtual_vars_from_kerchunk_refs(refs, drop_variables, virtual_array_class)
    [315](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:315)     drop_variables = []
    [316](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:316) var_names_to_keep = [
    [317](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:317)     var_name for var_name in var_names if var_name not in drop_variables
    [318](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:318) ]
    [320](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:320) vars = {
--> [321](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:321)     var_name: variable_from_kerchunk_refs(refs, var_name, virtual_array_class)
    [322](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:322)     for var_name in var_names_to_keep
    [323](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:323) }
    [324](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:324) return vars

File ~/packages/VirtualiZarr/virtualizarr/xarray.py:369, in variable_from_kerchunk_refs(refs, var_name, virtual_array_class)
    [366](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:366) arr_refs = kerchunk.extract_array_refs(refs, var_name)
    [367](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:367) chunk_dict, zarray, zattrs = kerchunk.parse_array_refs(arr_refs)
--> [369](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:369) manifest = ChunkManifest._from_kerchunk_chunk_dict(chunk_dict)
    [371](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:371) # we want to remove the _ARRAY_DIMENSIONS from the final variables' .attrs
    [372](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/xarray.py:372) dims = zattrs.pop("_ARRAY_DIMENSIONS")

File ~/packages/VirtualiZarr/virtualizarr/manifests/manifest.py:288, in ChunkManifest._from_kerchunk_chunk_dict(cls, kerchunk_chunk_dict)
    [285](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/manifests/manifest.py:285) @classmethod
    [286](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/manifests/manifest.py:286) def _from_kerchunk_chunk_dict(cls, kerchunk_chunk_dict) -> "ChunkManifest":
    [287](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/manifests/manifest.py:287)     chunkentries = {
--> [288](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/manifests/manifest.py:288)         cast(ChunkKey, k): ChunkEntry.from_kerchunk(v).dict()
    [289](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/manifests/manifest.py:289)         for k, v in kerchunk_chunk_dict.items()
    [290](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/manifests/manifest.py:290)     }
    [291](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/manifests/manifest.py:291)     return ChunkManifest(entries=cast(ChunkDict, chunkentries))

File ~/packages/VirtualiZarr/virtualizarr/manifests/manifest.py:39, in ChunkEntry.from_kerchunk(cls, path_and_byte_range_info)
     [37](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/manifests/manifest.py:37) @classmethod
     [38](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/manifests/manifest.py:38) def from_kerchunk(cls, path_and_byte_range_info: list[str | int]) -> "ChunkEntry":
---> [39](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/manifests/manifest.py:39)     path, offset, length = path_and_byte_range_info
     [40](https://vscode-remote+ssh-002dremote-002blakes02-002eaxiomptk.vscode-resource.vscode-cdn.net/home/kristen/projects/virtualizarr_test/~/packages/VirtualiZarr/virtualizarr/manifests/manifest.py:40)     return ChunkEntry(path=path, offset=offset, length=length)

ValueError: too many values to unpack (expected 3)

@TomNicholas
Copy link
Member

TomNicholas commented Jul 16, 2024

I wasn't able to read in a json file I generated with the kerchunk library with ValueError: too many values to unpack (expected 3) (details below)

This is likely kerchunk's fault. I've seen similar things before when kerchunk unpredictably violates it's own specification. If you can provide a reproducible example we can see exactly what the problem was and work around it in VirtualiZarr / fix it upstream in kerchunk.

coordinate instead of a data_var, and some coordinates were data_vars instead of coordinates

@ayushnag also recently reported a bug in determining which variables are coordinate variables (#189), so you might be seeing that too. Again a reproducible example would help.

@maresb
Copy link
Contributor

maresb commented Jul 16, 2024

@kthyng, as @TomNicholas points out, I also hit this issue in #187. You might try if my workaround helps. Unless you're opening a ref created by ZarrtoZarr I suspect you'll hit my NotImplementedError though.

@kthyng
Copy link
Author

kthyng commented Jul 16, 2024

@TomNicholas Yeah sorry I didn't provide examples at the moment. I was at the end of my workday and just wanted to get some notes out to be clear I had found some issues in the PR.

  1. Reading in a kerchunk library json file and getting ValueError: too many values to unpack (expected 3). Here is such a file: 2000_0001.json I just tried adding the three lines from @maresb from the PR

    path = path_and_byte_range_info[0]
    offset = 0
    length = UPath(path).stat().st_size

    into my manifest.py to see if that would work and it did not, but I didn't dig in enough to know more about why.

  2. I did see that PR and thought I'd watch it move forward to try it out and see if it fixed my problem. Without the code from that PR, the coords are missing from this first example (I didn't check the second one again), but then I copied in the lines from @ayushnag's open_virtual_dataset returns some coordinates as data variables #189 to try them out and here is two examples with different outcomes:

# I am not clear why I can't just use the thredds file links directly (they work e.g. directly to xarray) so I downloaded them 
# from the thredds file server https://thredds.cencoos.org/thredds/catalog/cencoos/ccsnrt/catalog.html
# otherwise the links would have been
# loc1 = "http://thredds.cencoos.org/thredds/dodsC/cencoos/ccsnrt/2011/2011_01/ccsnrt_2011_01_02.nc"
# loc2 = "http://thredds.cencoos.org/thredds/dodsC/cencoos/ccsnrt/2011/2011_01/ccsnrt_2011_01_03.nc"

loc1 = "ccsnrt_2011_01_02.nc"
loc2 = "ccsnrt_2011_01_03.nc"

vds1 = open_virtual_dataset(loc1, indexes={}, reader_options={})
vds2 = open_virtual_dataset(loc2, indexes={}, reader_options={})

combined_vds = xr.concat([vds1, vds2], dim='ocean_time', coords='minimal', compat='override', data_vars='minimal')
combined_vds

gives correct coords both initially and after saving

<xarray.Dataset> Size: 15MB
Dimensions:  (time: 1, eta_v: 180, xi_v: 186, z: 11, eta_u: 181, xi_u: 185,
              eta_rho: 181, xi_rho: 186)
Coordinates:
    lat_v    (eta_v, xi_v) float64 268kB ManifestArray<shape=(180, 186), dtyp...
    lat_u    (eta_u, xi_u) float64 268kB ManifestArray<shape=(181, 185), dtyp...
    lon_u    (eta_u, xi_u) float64 268kB ManifestArray<shape=(181, 185), dtyp...
    lat_rho  (eta_rho, xi_rho) float64 269kB ManifestArray<shape=(181, 186), ...
    z        (z) float64 88B ManifestArray<shape=(11,), dtype=float64, chunks...
    lon_v    (eta_v, xi_v) float64 268kB ManifestArray<shape=(180, 186), dtyp...
    lon_rho  (eta_rho, xi_rho) float64 269kB ManifestArray<shape=(181, 186), ...
    time     (time) float64 8B ManifestArray<shape=(1,), dtype=float64, chunk...
Dimensions without coordinates: eta_v, xi_v, eta_u, xi_u, eta_rho, xi_rho
Data variables: (12/13)
    vbar     (time, eta_v, xi_v) float64 268kB ManifestArray<shape=(1, 180, 1...
    u        (time, z, eta_u, xi_u) float32 1MB ManifestArray<shape=(1, 11, 1...
    ubar     (time, eta_u, xi_u) float64 268kB ManifestArray<shape=(1, 181, 1...
    temp     (time, z, eta_rho, xi_rho) float32 1MB ManifestArray<shape=(1, 1...
    rho      (time, z, eta_rho, xi_rho) float32 1MB ManifestArray<shape=(1, 1...
    w        (time, z, eta_rho, xi_rho) float32 1MB ManifestArray<shape=(1, 1...
    ...       ...
    h        (eta_rho, xi_rho) float64 269kB ManifestArray<shape=(181, 186), ...
    v        (time, z, eta_v, xi_v) float32 1MB ManifestArray<shape=(1, 11, 1...
    urot     (time, z, eta_rho, xi_rho) float32 1MB ManifestArray<shape=(1, 1...
    angle    (eta_rho, xi_rho) float64 269kB ManifestArray<shape=(181, 186), ...
    vrot     (time, z, eta_rho, xi_rho) float32 1MB ManifestArray<shape=(1, 1...
    salt     (time, z, eta_rho, xi_rho) float32 1MB ManifestArray<shape=(1, 1...
Attributes: (12/58)
    ADM_LBC:                         \nEDGE:  WEST   SOUTH  EAST   NORTH  \nz...
    CF%3afeature_type:               GRID
    CPP_options:                     WC12_CCSRA, ADJOINT, ANA_BSFLUX, ANA_BTF...
    Conventions:                     CF-1.4, _Coordinates
    DODS_EXTRA.Unlimited_Dimension:  ocean_time
    NCO:                             "4.6.1"
    ...                              ...
    summary:                         This West Coast ROMS model is one of two...
    svn_rev:                         exported
    svn_url:                         https://www.myroms.org/svn/src/trunk
    tiling:                          003x005
    title:                           UCSC California Current System ROMS Nowc...
    type:                            ROMS/TOMS history file

then I save and reopen

combined_vds.virtualize.to_kerchunk(f'combined.parq', format='parquet')

combined_ds = xr.open_dataset('combined.parq', engine="kerchunk")
combined_ds

to get

Dimensions:  (eta_rho: 181, xi_rho: 186, eta_u: 181, xi_u: 185, eta_v: 180,
              xi_v: 186, time: 1, z: 11)
Coordinates:
    lat_rho  (eta_rho, xi_rho) float64 269kB ...
    lat_u    (eta_u, xi_u) float64 268kB ...
    lat_v    (eta_v, xi_v) float64 268kB ...
    lon_rho  (eta_rho, xi_rho) float64 269kB ...
    lon_u    (eta_u, xi_u) float64 268kB ...
    lon_v    (eta_v, xi_v) float64 268kB ...
  * time     (time) datetime64[ns] 8B 2011-01-02
  * z        (z) float64 88B -250.0 -200.0 -150.0 -100.0 ... -10.0 -5.0 -2.0
Dimensions without coordinates: eta_rho, xi_rho, eta_u, xi_u, eta_v, xi_v
Data variables: (12/13)
    angle    (eta_rho, xi_rho) float64 269kB ...
    h        (eta_rho, xi_rho) float64 269kB ...
    rho      (time, z, eta_rho, xi_rho) float32 1MB ...
    salt     (time, z, eta_rho, xi_rho) float32 1MB ...
    temp     (time, z, eta_rho, xi_rho) float32 1MB ...
    u        (time, z, eta_u, xi_u) float32 1MB ...
    ...       ...
    urot     (time, z, eta_rho, xi_rho) float32 1MB ...
    v        (time, z, eta_v, xi_v) float32 1MB ...
    vbar     (time, eta_v, xi_v) float64 268kB ...
    vrot     (time, z, eta_rho, xi_rho) float32 1MB ...
    w        (time, z, eta_rho, xi_rho) float32 1MB ...
    zeta     (time, eta_rho, xi_rho) float64 269kB ...

Ok I have a second example that does not work out the same with coords vs. data_vars but it is pathological and I am giving up for now. I work with this model all the time but I can't get consistent behavior to show an example for this and also have a time error only sometimes pop up in this work (the cf-time thing fixes it but it wasn't necessary before and only now despite working with this model and virtualizarr for a few weeks) and now just hit the same ValueError: too many values to unpack (expected 3) as with the kerchunk-library files. I think I need to work on something else for awhile.

@TomNicholas
Copy link
Member

Okay no worries @kthyng - sorry you've run into a few bugs here! Thanks for being intrepid enough to try VirtualiZarr out.

Now we have reproducible examples we can track down these bugs and fix them, and I'll let you know once we've done that.

@TomNicholas
Copy link
Member

Superceded by #251

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants