Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add capability to created wanted variables list with an explicit list of paths #508

Open
wants to merge 15 commits into
base: development
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/source/user_guide/documentation/classes_dev_uml.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
90 changes: 45 additions & 45 deletions doc/source/user_guide/documentation/classes_user_uml.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
62 changes: 53 additions & 9 deletions icepyx/core/variables.py
Original file line number Diff line number Diff line change
Expand Up @@ -429,7 +429,14 @@ def _iter_paths(self, sum_varlist, req_vars, vgrp, beam_list, keyword_list):
return req_vars

# DevGoal: we can ultimately add an "interactive" trigger that will open the not-yet-made widget. Otherwise, it will use the var_list passed by the user/defaults
def append(self, defaults=False, var_list=None, beam_list=None, keyword_list=None):
def append(
self,
defaults=False,
var_list=None,
beam_list=None,
keyword_list=None,
path_list=None,
):
"""
Add to the list of desired variables using user specified beams and variable list.
A pregenerated default variable list can be used by setting defaults to True.
Expand Down Expand Up @@ -457,6 +464,9 @@ def append(self, defaults=False, var_list=None, beam_list=None, keyword_list=Non
the product that include that keyword in their path. A list of availble keywords can be obtained by
entering `keyword_list=['']` into the function.

path_list : list of strings, default None
A list of variable path names given as a the full path from the H5 root path.
andypbarrett marked this conversation as resolved.
Show resolved Hide resolved

Notes
-----
See also the `IS2_data_access2-subsetting
Expand All @@ -482,25 +492,59 @@ def append(self, defaults=False, var_list=None, beam_list=None, keyword_list=Non
To add all variables and paths in ancillary_data

>>> reg_a.order_vars.append(keyword_list=['ancillary_data']) # doctest: +SKIP

To add variables as a list of explicit paths

>>> path_list = ['gt1l/heights/delta_time',
'gt1l/heights/dist_ph_across',
'gt1l/heights/dist_ph_along',
'gt1l/heights/h_ph',
'gt1l/heights/lat_ph',
'gt1l/heights/lon_ph']
andypbarrett marked this conversation as resolved.
Show resolved Hide resolved
>>> region_a.order_vars.append(path_list=path_list) # doctest: +SKIP

"""

assert not (
defaults == False
and var_list == None
and beam_list == None
and keyword_list == None
), "You must enter parameters to add to a variable subset list. If you do not want to subset by variable, ensure your is2.subsetparams dictionary does not contain the key 'Coverage'."
# Check that at least one allowable combination of keywords are set
if (
not defaults
and not (var_list or beam_list or keyword_list)
and not path_list
):
raise ValueError(
"Either default or path_list, or at least one of var_list, "
"beam_list or keyword_list must be set\n"
"If you do not want to subset by variable, "
"ensure your is2.subsetparams dictionary does "
"not contain the key 'Coverage'."
andypbarrett marked this conversation as resolved.
Show resolved Hide resolved
)

# Check that only path_list, or only var_list, beam_list or keyword_list
# are set.
if (var_list or beam_list or keyword_list) and path_list:
raise ValueError(
"path_list cannot be set if var_list or "
"beam_list or keyword_list are set"
)

final_vars = {}

if path_list:
_, (beams, keywords, variables) = self.parse_var_list(
path_list, tiered_vars=True
)
var_list = list(set(variables))
beam_list = list(set(beams))
keyword_list = list(set(keywords))
Comment on lines +533 to +535
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went to add the new example in the docstring to the variables tutorial notebook it raised 2 issues:

  1. The example fails because it's for ATL03, and the examples are on ATL06 (and in the notebook, there's a second one for ATL09)
  2. Trying to use a path_list input on ATL06 fails because it's got four layers of variables, not the three we collect here.

I haven't reminded myself of how we handle this in general in the module, so that may recommend a solution for (2). (1) is easy if we just sub in another list.


vgrp, allpaths = self.avail(options=True, internal=True)
self._check_valid_lists(vgrp, allpaths, var_list, beam_list, keyword_list)

# Instantiate self.wanted to an empty dictionary if it doesn't exist
# Instantiate self.wanted to an empty dictionaryto an if it doesn't exist
andypbarrett marked this conversation as resolved.
Show resolved Hide resolved
if not hasattr(self, "wanted") or self.wanted == None:
self.wanted = {}

# DEVGOAL: add a secondary var list to include uncertainty/error information for lower level data if specific data variables have been specified...
# DEVGOAL: add a secondary var list to include uncertainty/error information for lower level data if specific data variables have been specified...

# generate a list of variable names to include, depending on user input
sum_varlist = self._get_sum_varlist(var_list, vgrp.keys(), defaults)
Expand Down