Hotfix/stability #10
Hotfix/stability #10
Conversation
steffencruz
commented
Aug 21, 2023
•
steffencruz
commented
- Adds workaround for known wandb decode error by forcing JSON parsing in non-strict mode
- Ensures expected metagraph block list is sorted before creating parquet dataframe
|
|
||
| def relaxed_json_loads(s, *args, **kwargs): | ||
| kwargs.pop("strict", None) # remove the strict argument if it exists | ||
| return original_json_loads(s, strict=False, *args, **kwargs) |
There was a problem hiding this comment.
why not call it explicitly like this in every json.loads call ?? or use this relaxed_json_loads.
Even though the approach is very nice in shape. I think it obscures what's really happening, and that's important for maintenance and readability of the code.
From my point of view, I'd say, let's make the call explicit as it is in this line, or use the function relaxed_json_loads, so we know something different is happening
What do you think about this?
There was a problem hiding this comment.
Yea it's not my preferred solution at all, however the issue actual originates inside the wandb codebase - they enforce strict mode when pulling assets from their databases. We never actually call json decide ourselves..
I propose we put it either here or shamefully in a Util file for now
There was a problem hiding this comment.
So it's a way of overriding json globally.
Floppy actually came up with this hack because anyone who wants to analyze wandb data has to do something similar
|
very nice catch! LGTM after solving comments :) |
Co-authored-by: Eduardo García <[email protected]>
