Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Queries that return empty results can fail #129

Closed
3 tasks
dougbrn opened this issue Aug 2, 2024 · 0 comments · Fixed by #128
Closed
3 tasks

Queries that return empty results can fail #129

dougbrn opened this issue Aug 2, 2024 · 0 comments · Fixed by #128
Assignees
Labels
bug Something isn't working

Comments

@dougbrn
Copy link
Collaborator

dougbrn commented Aug 2, 2024

Bug report
@AndyTza ran into an issue recently, where he was applying a query on his nested-dask workflow. This query resulting in empty dataframes for a small subset of partitions in the dataset. The nested-pandas implementation of query has issues with this: https://github.com/lincc-frameworks/nested-pandas/blob/main/src/nested_pandas/nestedframe/core.py#L265

These issues relate to the downstream function query_flat, having this exception for an empty result: https://github.com/lincc-frameworks/nested-pandas/blob/main/src/nested_pandas/series/accessor.py#L226. The empty series seems to not work correctly in some situations, and can produce this error: TypeError: incompatible index of inserted column with frame index

Interestingly, this doesn't come up in a simple toy case:
Screen Shot 2024-08-02 at 3 42 13 PM

There's a notebook on epyc (message me for the path) that can reproduce the error. So far, I've seen that simply deleting the if len(flat) == 0: section seems effective at removing the error, but we should look deeper at this and make sure we find a robust solution.

A draft PR with the rough fix is here: #128

Before submitting
Please check the following:

  • I have described the situation in which the bug arose, including what code was executed, information about my environment, and any applicable data others will need to reproduce the problem.
  • I have included available evidence of the unexpected behavior (including error messages, screenshots, and/or plots) as well as a descriprion of what I expected instead.
  • If I have a solution in mind, I have provided an explanation and/or pseudocode and/or task list.
@dougbrn dougbrn added the bug Something isn't working label Aug 2, 2024
@dougbrn dougbrn mentioned this issue Aug 9, 2024
21 tasks
@dougbrn dougbrn self-assigned this Aug 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant