Not same color intensities between pydicom and highdicom

[lucas02061]

Hi,

I am running a segmentation model on a WSI opened with pydicom. However when opening using pixelarray it seems to be quite memory intensive and even with 96gb of memory I run out of memory for larger WSI's.

I tried to mitigate this by using highdicom to load a single frame at a time, but even though the data gathered from pydicom and highdicom contains the "same" image, the color intensity seems to be quite different. Essentially the contrast is higher when using pydicom and lower when using highdicom.

This leads to problems during training since the lower contrast for highdicom seems to decrease performance quite drastically. Models trained on pydicom images are way better than the opposite. Is there a way to load the image using highdicom but still get the same intensity as pydicom? If no, is there a less memory-intensive way of loading the image using pydicom?

Example of how I retrieve the data:

if isinstance(dicom, str):
    img = []
    dicom_image = ImageFileReader(dicom)
    dicom_image.open()
    for i in tqdm(range(dicom_image.number_of_frames), desc="Extracting frames from DICOM", leave=False):
        img.append(dicom_image.read_frame(i))
    img = np.array(img)
else:
    img = dicom.pixel_array

[mrbean-bremen]

If you just use the raw pixel data in pydicom, no LUT is applied if you don't call apply_modality_lut/apply_voi_lut yourself.
I'm guessing that the ImageFileReader (is that from ITK? or something from highdicom?) is already applying this LUT, so that the image is displayed as a radiologist would expect it. You can confirm if this is the case by applying the DICOM LUT in the data from pydicom and see if you get the same (less contrast, but more correct) image.

Also note that pydicom supports iterating over single frames if that is what you need (and not some special handling for WSI).

[scaramallion]

The pixel data tutorial has a section about using iter_pixels() or pixel_array() to retrieve specific frames while minimising memory usage.