20x Scale Bugs

[KompressorSC]

Describe the bug
When using CellViT to segment some WSIs under 20x magnification scale, the result showed in QuPath is incorrect. However, when under 40x magnification scale, they are correct. But for those WSIs that doesn't s support 40x magnification scale, segment result under 20x scale is correct. I think there are some bugs in the post-process procedure of the result, making it recognize the result of the model as under 40x magnification scale (which is actually under 20x) and sett he wrong rescale factor.

To Reproduce

1. Command
   Patch Extraction YAML

# dataset paths
wsi_paths: /path/to/wsi
output_path: /path/to/output

# basic setups
patch_size: 1024
patch_overlap: 6.25
target_mag: 20
processes: 8
overwrite: True

# macenko stain normalization
normalize_stains: True

# finding patches
min_intersection_ratio: 0.05

Patch Extraction
python3 ./preprocessing/patch_extraction/main_extraction.py --config ./preprocessing.yaml
Inference

python3 ./cell_segmentation/inference/cell_detection.py \
   --model ./weights/CellViT-256-x20.pth \
   --gpu 0 \
   --geojson \
   --magnification 20 \
   process_wsi \
   --wsi_path path/to/wsi \
   --patched_slide_path path/to/output

2. File
   Including WSI and segment results.
   https://drive.google.com/drive/folders/1MxZIGvaws8HIdhrVq-nT4F2wvvNyszrZ?usp=drive_link

3. Error Traceback
   No error during the code running, but error appears in QuPath.

Expected behavior
Fix the bugs.

Screenshots

Additional context
None

[FabianHoerst]

Hi, we do not recommend to use the x20 models, as the performance is inferior. We always load the models on the largest WSI level and therefore rescaling to x20 magnification most probably leads to the shift. We are aware of this, but currently do not have the capacity to implement a fix. However, we are very glad for contributors to solve this issue. Sorry that I cannot help you further!

Best, Fabian

[FabianHoerst]

Have you tried using 512 pixels for the x20 model?

[KompressorSC]

Have you tried using 512 pixels for the x20 model?

Not yet, I will try it soon. Thanks for the reply!

[Adriushum]

Hi @FabianHoerst , I have a follow up question ,
is there a config that lets me run the 40x model with a 20x wsi?
I assume i have to patch the wsi with 512x512 patch size and then resize to 1024x1024.
do i need to edit the patch extractor script? or is there a way around it?
i tried passing "target_mag: 40" in patch extractor config file for a 20x slide , but it throws an error.
and do you in general recommend using 20x model for 20x WSI and 40x model for 40x WSI , instead of resizing?

[ddyang1]

Hi @FabianHoerst, I have tried using 512 x 512 pixel patches on 20x but still having the same issue. I'm hoping to run CellViT on some samples from interesting clinical cohorts which are digitized at 20x only, so any tips you have on getting around this issue would be great, thanks!

[ddyang1]

I believe the solution for correct global coordinates is to add in the wsi_scaling_factor (=2 for 20x) for the following (which allows for the correct x/y offset), at approximately lines 346 and 350 in cell_segmentation/inference/cell_detection.py:

x_global = int(patch_metadata["row"] * patch_size * wsi_scaling_factor - (patch_metadata["row"] + 0.5) * overlap * wsi_scaling_factor)
y_global = int(patch_metadata["col"] * patch_size * wsi_scaling_factor - (patch_metadata["col"] + 0.5) * overlap * wsi_scaling_factor)

For the correct spacing between nuclei at 20x, you would need to add in the wsi_scaling_factor around lines 358-360:
centroid_global = cell["centroid"] * wsi_scaling_factor + np.flip(offset_global)
contour_global = cell["contour"] * wsi_scaling_factor + np.flip(offset_global)
bbox_global = cell["bbox"] * wsi_scaling_factor + offset_global