Erase features in 3D from 3D mask #35
Conversation
Co-authored-by: Alessio d'Acapito <[email protected]>
|
|
||
| grid = CubicBSplineGrid3d(resolution=resolution) | ||
| optimiser = torch.optim.Adam(grid.parameters(), lr=0.01) | ||
|
|
||
| for i in range(500): | ||
| # what does the model predict for our observations? | ||
| prediction = grid(foreground_sample_idx).squeeze() | ||
|
|
||
| # zero gradients and calculate loss between observations and model prediction | ||
| optimiser.zero_grad() | ||
| loss = torch.sum((prediction - w)**2)**0.5 | ||
|
|
||
| # backpropagate loss and update values at points on grid | ||
| loss.backward() | ||
| optimiser.step() |
There was a problem hiding this comment.
Not convinced I'm using this right, here...
|
Hi guys!!! If I understood well the goal, IMHO it would be simpler to adapt/change/retrain membrain-seg to have a "--2d" option so it can segment membranes in 2d (at tilt series level) and then apply fidder. Am I wrong? (of course, easier said than done). |
There was a problem hiding this comment.
This looks great! Minor usage error on the spline grid but otherwise looks good
I'm not convinced this should live here though - it's about enabling an application that isn't anything to do with fiducials...
How about we part out the erasing code for both 2D + 3D into a cryoeraser package? That way these could live together and fidder could depend on just the 2D bit
@estrozi nice to see you here! I agree 2D erasing of the membranes prior to tomogram reconstruction would be a better solution overall. There might be some value in erasing in 3D though 🤷🏼♂️
|
|
||
| for i in range(500): | ||
| # what does the model predict for our observations? | ||
| prediction = grid(foreground_sample_idx).squeeze() |
There was a problem hiding this comment.
this looks like it's about right except when sampling the model coordinates should be rescaled to [0, 1] where 0 is the center of the first element in the dimension and 1 is the center of the last element in the dimension
There was a problem hiding this comment.
Mhmh... I see, I'll look into it :)
There was a problem hiding this comment.
@alisterburt does this make sense now? Though unfortunately it doens't look better @alessiodacapito :/ I think the bulk of the work here is being done by the noise generation, not the local mean...
Yes, I was thinking the same! I think fidder now is really 2 different things in one and the eraser would be useful as standalone. I also think that in that case we should try to merge the paths between 2D and 3D a bit, all this duplication is fragile. But for now I went for separate because it was quicker and easier to review.
Yes, that's exactly what I suggested as well, though it's a significantly harder work so we went for the low-hanging fruit first. One major drawback of working in 3D is that the noise pattern is way different (very much not normal distribution, I suspect...) which makes the masking pretty bad. Though, maybe the mistake I made in the grid is also to blame... |
Yes great idea !
I am currently working on it, it's showing good results for now ! |
First attempt at using fidder to do feature erasure on a volume instead of a stack of 2D images.
To avoid unnecessary refactoring, for now we just made a secondary code path for 3D which is essentially a copy-paste of the 2D one with minor changes.
estimate_local_meanis now replaced byestimate_local_mean_3d, which uses a badly implementedtorch_cubic_spline_grids.b_spline_grids.CubicBSplineGrid3d(@alisterburt help pls :P) in order to interpolate in 3D instead of 2D.Then, we also added an
estimate_background_std_3dwhich is trying to be a bit smarter about noise (since Z noise is quite different from XY noise in tomos).Any thoughts/suggestions on this approach?
For now, we don't see an obvious improvement on the erased subvolumes (at least visually), but we're testing some downstream processing to see if it affects the STA.
cc @alessiodacapito