Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bugfix for incorrect dimension-accessors when initializing from pandas #763

Merged
merged 6 commits into from
Aug 4, 2023

Conversation

danielhuppmann
Copy link
Member

Please confirm that this PR has done the following:

  • Tests Added
  • Documentation Added
  • Name of contributors Added to AUTHORS.rst
  • Description in RELEASE_NOTES.md Added

Description of PR

This PR introduces an explicit removal of unused levels when initializing from a pandas object. This bug could lead to inconsistent return-values of the dimension-accessors.

closes #762

@danielhuppmann danielhuppmann changed the title Bugfix/762 Bugfix for incorrect dimension-accessors when initializing from pandas Aug 3, 2023
@danielhuppmann danielhuppmann self-assigned this Aug 3, 2023
@codecov
Copy link

codecov bot commented Aug 3, 2023

Codecov Report

Merging #763 (98f59f0) into main (4bfcec0) will increase coverage by 0.0%.
The diff coverage is 100.0%.

@@          Coverage Diff          @@
##            main    #763   +/-   ##
=====================================
  Coverage   94.4%   94.4%           
=====================================
  Files         59      59           
  Lines       6080    6086    +6     
=====================================
+ Hits        5744    5750    +6     
  Misses       336     336           
Files Changed Coverage Δ
pyam/utils.py 92.7% <100.0%> (+<0.1%) ⬆️
tests/test_core.py 100.0% <100.0%> (ø)

@danielhuppmann danielhuppmann marked this pull request as ready for review August 3, 2023 20:29
Copy link
Collaborator

@znicholls znicholls left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, one suggestion for a more defensive way to write the test (not an essential change)

tests/test_core.py Outdated Show resolved Hide resolved
Copy link
Contributor

@phackstock phackstock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beautiful thanks a lot @danielhuppmann 🎉

@danielhuppmann danielhuppmann merged commit c902b2c into IAMconsortium:main Aug 4, 2023
17 checks passed
@danielhuppmann danielhuppmann deleted the bugfix/762 branch August 4, 2023 08:24
@coroa
Copy link
Collaborator

coroa commented Sep 13, 2023

ok, thanks, indeed, that is the issue i encountered today.

i would have suggested to replace calls to get_index_levels method with its pandas version, ie something like:

def get_index_levels(index, level):
     return index.unique(level)

so they can decide what is the most efficient solution, but your fix is also fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Helpful accessors confounded by pandas regression
4 participants