Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Script 3d sometimes tries to read the wrong json file #465

Open
dtebaykin opened this issue Aug 13, 2024 · 1 comment · May be fixed by #467
Open

Script 3d sometimes tries to read the wrong json file #465

dtebaykin opened this issue Aug 13, 2024 · 1 comment · May be fixed by #467
Labels
bug Something isn't working

Comments

@dtebaykin
Copy link

Please refer to our FAQ and look at our known issues before opening a bug report.

Describe the bug
Toffy script 3d_compute_median_pulse_height.ipynb first functional code block contains this function:

mph_comp.compute_mph_metrics(bin_file_dir, mph_out_dir, fov)

Inside the python code it runs get_estimated_time(bin_file_dir, fov)
-> (inside the above function) json_file = io_utils.list_files(bin_file_dir, fov + ".json")
This line finds the first json file that starts with fov name. In my dataset there are 2 json files per fov like that:
Example: fov-1-scan-1.json (the one we want) and fov-1-scan-1_processing.json (the one we get sometimes).
I am uncertain where that second json came from, but it is there for each fov.

Expected behavior
The script needs to find the correct json file for each fov. Possible solution: do not invoke list_files() function, instead read the correct json directly since we know it's name should be fov + ".json"

To Reproduce
`TypeError Traceback (most recent call last)
File ~/Documents/Bendall lab projects/toffy/src/toffy/mph_comp.py:36, in get_estimated_time(bin_file_dir, fov)
35 time = run_metadata.get("dwellTimeMillis")
---> 36 estimated_time = int(size**2 * time)
37 except TypeError:

TypeError: unsupported operand type(s) for ** or pow(): 'NoneType' and 'int'

During handling of the above exception, another exception occurred:

KeyError Traceback (most recent call last)
Cell In[19], line 9
7 if not os.path.exists(os.path.join(mph_out_dir, '%s-mph_pulse.csv' % fov)):
8 print("Computing MPH for fov %s" % fov)
----> 9 mph_comp.compute_mph_metrics(bin_file_dir, mph_out_dir, fov)

File ~/Documents/Bendall lab projects/toffy/src/toffy/mph_comp.py:108, in compute_mph_metrics(bin_file_dir, csv_dir, fov, mass, mass_start, mass_stop)
105 raise FileNotFoundError(f"The FOV name supplied doesn't have a JSON file: {fov}")
107 count = count_dict[fov]
--> 108 time = get_estimated_time(bin_file_dir, fov)
110 out_df = pd.DataFrame({"fov": [fov], "MPH": [median], "total_count": [count], "time": [time]})
112 # saves individual .csv files to csv_dir

File ~/Documents/Bendall lab projects/toffy/src/toffy/mph_comp.py:38, in get_estimated_time(bin_file_dir, fov)
36 estimated_time = int(size**2 * time)
37 except TypeError:
---> 38 raise KeyError(
39 "The FOV json file is missing one of the necessary keys (frameSize or dwellTimeMillis)"
40 )
42 return estimated_time

KeyError: 'The FOV json file is missing one of the necessary keys (frameSize or dwellTimeMillis)'`

@dtebaykin dtebaykin added the bug Something isn't working label Aug 13, 2024
@camisowers
Copy link
Contributor

Can be fixed by setting exact_match=True in alpineer.io_utils.list_files().

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants