Skip to content

Commit

Permalink
Merge branch 'main' into invalid-image-size-handling
Browse files Browse the repository at this point in the history
  • Loading branch information
erikogabrielsson authored Aug 9, 2023
2 parents 358bef8 + fe91d24 commit 1de77bb
Show file tree
Hide file tree
Showing 32 changed files with 166 additions and 145 deletions.
17 changes: 13 additions & 4 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

- Override image size for labels and overviews if it is likely to be wrong.

### Changed

- Use logger instead of issuing warnings.

### Fixed

- Error in documentation of read_overview() in readme.
- Spellings.

## [0.10.0] - 2023-06-01

### Added
Expand All @@ -35,11 +44,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

- Support for opening DICOM WSI using DICOMWeb.
- save() now takes the optional parameter add_missing_levels, that enables adding missing pyramid levels up to the single-tile level.
- read_region(), read_region_mm(), and read_region_mpp() takes an optional parameter threads, that allows multiple threads to be used for stitching togheter the region.
- read_region(), read_region_mm(), and read_region_mpp() takes an optional parameter threads, that allows multiple threads to be used for stitching together the region.

### Changed

- WsiDicom is now initalized using a Source, that is responsible for provides the instances to view.
- WsiDicom is now initialized using a Source, that is responsible for provides the instances to view.
- Saving a WsiDicom is now handled by WsiDicomFileTarget.
- Refactoring due to adding support for DICOMWeb and opening instances using Source and saving using Target.
- Frame positions and tiling for levels are now parsed lazily, e.g. on first tile access.
Expand Down Expand Up @@ -85,7 +94,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

- Focal planes are considered equal if configurable withing threshold distance, see config.focal_plane_distance_threshold
- Focal planes are considered equal if configurable within threshold distance, see config.focal_plane_distance_threshold
- Option to read region defined in slide coordinate system for get_region_mm().

### Changed
Expand Down Expand Up @@ -113,7 +122,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Fixed

- Order of paramaters for ConceptCode matches pydicom Code.
- Order of parameters for ConceptCode matches pydicom Code.

## [0.3.0] - 2022-04-20

Expand Down
14 changes: 7 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ region_mpp = slide.read_region_mpp((0, 0), 0.01, (3, 3))
***Read a thumbnail of the whole slide with maximum dimensions 200x200 px.***

```python
thumbnail = slide.read_thumbnail(200, 200)
thumbnail = slide.read_thumbnail((200, 200))
```

***Read an overview image (if available).***
Expand Down Expand Up @@ -146,7 +146,7 @@ slide.close()
An opened WsiDicom instance can be saved to a new path using the save()-method. The produced files will be:

- Fully tiled. Any sparse tiles will be replaced with a blank tile with color depending on the photometric interpretation.
- Have a basic offset table (or optionally an exteded offset table or no offset table).
- Have a basic offset table (or optionally an extended offset table or no offset table).
- Not be concatenated.

The frames are copied as-is, i.e. without re-compression.
Expand All @@ -156,7 +156,7 @@ with WsiDicom.open(path_to_folder) as slide:
slide.save(path_to_output)
```

The output folder must already exists. Be careful to specify a unique folder folder to avoid mixing files from diferent images.
The output folder must already exists. Be careful to specify a unique folder folder to avoid mixing files from different images.

## Settings

Expand All @@ -175,7 +175,7 @@ Annotations are structured in a hierarchy:
- AnnotationInstance
Represents a collection of AnnotationGroups. All the groups have the same frame of reference, i.e. annotations are from the same wsi stack.
- AnnotationGroup
Represents a group of annotations. All annotations in the group are of the same type (e.g. PointAnnotation), have the same label, description and category and type. The category and type are codes that are used to define the annotated feature. A good resource for working with codes is avaiable [here](https://qiicr.gitbook.io/dcmqi-guide/opening/coding_schemes).
Represents a group of annotations. All annotations in the group are of the same type (e.g. PointAnnotation), have the same label, description and category and type. The category and type are codes that are used to define the annotated feature. A good resource for working with codes is available [here](https://qiicr.gitbook.io/dcmqi-guide/opening/coding_schemes).
- Annotation
Represents a annotation. An Annotation has a geometry (currently Point, Polyline, Polygon) and an optional list of Measurements.
- Measurement
Expand Down Expand Up @@ -257,7 +257,7 @@ To watch unit tests use:
poetry run pytest-watch -- -m unittest
```

The integration tests uses test images from nema.org thats needs to be downloaded. The location of the test images can be changed from the default tests\testdata\slides using the enviroment variable WSIDICOM_TESTDIR. Download the images using the supplied script:
The integration tests uses test images from nema.org that's needs to be downloaded. The location of the test images can be changed from the default tests\testdata\slides using the environment variable WSIDICOM_TESTDIR. Download the images using the supplied script:

```console
python .\tests\download_test_images.py
Expand Down Expand Up @@ -285,7 +285,7 @@ A WSI DICOM pyramid is in *wsidicom* represented by a hierarchy of objects of di

Labels and overviews are structured similarly to levels, but with somewhat different properties and restrictions. For DICOMWeb the WsiDicomFile\* classes are replaced with WsiDicomWeb\* classes.

A Source is used to create WsiInstances, either from files (*WsiDicomFileSource*) or DICOMWeb (*WsiDicomWebSource*), and can be used to to initate a *WsiDicom* object. A source is easiest created with the open() and open_web() helper functions, e.g.:
A Source is used to create WsiInstances, either from files (*WsiDicomFileSource*) or DICOMWeb (*WsiDicomWebSource*), and can be used to to Initiate a *WsiDicom* object. A source is easiest created with the open() and open_web() helper functions, e.g.:

```python
slide = WsiDicom.open(path_to_folder)
Expand Down Expand Up @@ -342,4 +342,4 @@ Our aim is to provide constructive and positive code reviews for all submissions

*wsidicom*: Copyright 2021 Sectra AB, licensed under Apache 2.0.

This project is part of a project that has received funding from the Innovative Medicines Initiative 2 Joint Undertaking under grant agreement No 945358. This Joint Undertaking receives support from the European Union’s Horizon 2020 research and innovation programme and EFPIA. IMI website: www.imi.europa.eu
This project is part of a project that has received funding from the Innovative Medicines Initiative 2 Joint Undertaking under grant agreement No 945358. This Joint Undertaking receives support from the European Union’s Horizon 2020 research and innovation programme and EFPIA. IMI website: <www.imi.europa.eu>
24 changes: 21 additions & 3 deletions poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ pycodestyle = "^2.8.0"
black = "^23.1.0"
flake8 = "^4.0.1"
parameterized = "^0.8.1"
codespell = "^2.2.5"

[build-system]
requires = ["poetry-core>=1.2.0"]
Expand Down
92 changes: 45 additions & 47 deletions tests/download_test_images.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,62 +19,62 @@
from typing import Any, Dict


FILESERVER = 'medical.nema.org'
FILESERVER_SLIDE_PATH = Path('MEDICAL/Dicom/DataSets/WG26')
FILESERVER = "medical.nema.org"
FILESERVER_SLIDE_PATH = Path("MEDICAL/Dicom/DataSets/WG26")

SLIDES: Dict[str, Dict[str, Any]] = {
'FULL_WITH_BOT': {
'name': r'Histech^Samantha [1229631]',
'parentpath': r'WG26Demo2020_PV',
'subpath': r'20190104 140000 [Case S - Colon polyps]/Series 000 [SM]',
'files': {
'2.25.173648596820938096199028939965251554503.dcm': '865538d55fce37ae6d91d85aebe29029', # NOQA
'2.25.181487944453580109633363498147571426374.dcm': '7ff4acf3c71572236ce968e06e79d8db', # NOQA
'2.25.191907898033754830752233761154920949936.dcm': '596183ec0444fedaba981de2f94652cb', # NOQA
'2.25.209236321826383427842899333369775338594.dcm': 'f7fe907553f036ec6d2a68443f006fd4', # NOQA
'2.25.222943316513786317622687980099326639180.dcm': 'ec1db6ca69c7d6fe8c4dacb04051ff11', # NOQA
'2.25.251277550657103455222613143487830679046.dcm': '03399b8332c967a9a97e67bffdd70fb9', # NOQA
'2.25.253973508129488054885063915385651983009.dcm': 'ac85a6f618ed0ca2bdf921be429fa185', # NOQA
'2.25.259312801889857550164526960213815274816.dcm': 'b7c64d1ed975b42f2b1b4d917c4ba8c0', # NOQA
'2.25.264278491200307498225194538752823371217.dcm': '0cc906dbeb22e10bff65f9b6d8961fa7', # NOQA
'2.25.290884199110265475119989552423006928136.dcm': 'b8faf60dc44e4cb9aa5e7ab92a706b88', # NOQA
'2.25.315427219625170954090644500893748466389.dcm': '84cca002af2e6263f25913b2896e36db', # NOQA
'2.25.339652625381363485545547336547695948130.dcm': '52ea8ec41f3f86368d50542c9ed41975', # NOQA
'2.25.98447601926716844575918590187306802549.dcm': '382e1b6780404efb7f44c65444694b05' # NOQA
}
"FULL_WITH_BOT": {
"name": r"Histech^Samantha [1229631]",
"parentpath": r"WG26Demo2020_PV",
"subpath": r"20190104 140000 [Case S - Colon polyps]/Series 000 [SM]",
"files": {
"2.25.173648596820938096199028939965251554503.dcm": "865538d55fce37ae6d91d85aebe29029", # NOQA
"2.25.181487944453580109633363498147571426374.dcm": "7ff4acf3c71572236ce968e06e79d8db", # NOQA
"2.25.191907898033754830752233761154920949936.dcm": "596183ec0444fedaba981de2f94652cb", # NOQA
"2.25.209236321826383427842899333369775338594.dcm": "f7fe907553f036ec6d2a68443f006fd4", # NOQA
"2.25.222943316513786317622687980099326639180.dcm": "ec1db6ca69c7d6fe8c4dacb04051ff11", # NOQA
"2.25.251277550657103455222613143487830679046.dcm": "03399b8332c967a9a97e67bffdd70fb9", # NOQA
"2.25.253973508129488054885063915385651983009.dcm": "ac85a6f618ed0ca2bdf921be429fa185", # NOQA
"2.25.259312801889857550164526960213815274816.dcm": "b7c64d1ed975b42f2b1b4d917c4ba8c0", # NOQA
"2.25.264278491200307498225194538752823371217.dcm": "0cc906dbeb22e10bff65f9b6d8961fa7", # NOQA
"2.25.290884199110265475119989552423006928136.dcm": "b8faf60dc44e4cb9aa5e7ab92a706b88", # NOQA
"2.25.315427219625170954090644500893748466389.dcm": "84cca002af2e6263f25913b2896e36db", # NOQA
"2.25.339652625381363485545547336547695948130.dcm": "52ea8ec41f3f86368d50542c9ed41975", # NOQA
"2.25.98447601926716844575918590187306802549.dcm": "382e1b6780404efb7f44c65444694b05", # NOQA
},
},
"SPARSE_NO_BOT": {
"name": r"MoticWangjie^Professer [100001]",
"parentpath": r"WG26Demo2019_PV",
"subpath": r"20190903 102029 [200001]\Series 000 [SM]",
"files": {
"1.2.276.0.7230010.3.1.4.1145362585.2096.1567753444.778.dcm": "4463e99bea080b14591f0665a6e84559", # NOQA
"1.2.276.0.7230010.3.1.4.1145362585.2096.1567753447.783.dcm": "0b48b76e46c754fc75b13d6fbf682a19", # NOQA
"1.2.276.0.7230010.3.1.4.1145362585.2096.1567753450.788.dcm": "75452d5f811f5c8c8f4475fbac6665bb", # NOQA
"1.2.276.0.7230010.3.1.4.1145362585.2096.1567753451.793.dcm": "3bf4fee344571bda7005d9735b5bd699", # NOQA
"1.2.276.0.7230010.3.1.4.1145362585.2096.1567753451.798.dcm": "84f2b3ba111ec09beb8846d17ddcd9e5", # NOQA
"1.2.276.0.7230010.3.1.4.1145362585.2096.1567753451.803.dcm": "d2e565d600a573685ca1be48d96a0ef4", # NOQA
"1.2.276.0.7230010.3.1.4.1145362585.2096.1567753451.808.dcm": "aedbe1f7ba9f15754078e3c8a9077fee", # NOQA
"1.2.276.0.7230010.3.1.4.1145362585.2096.1567753451.813.dcm": "3533d30ef5e9abe3b8ee6e1c7bc0fb17", # NOQA
},
},
'SPARSE_NO_BOT': {
'name': r'MoticWangjie^Professer [100001]',
'parentpath': r'WG26Demo2019_PV',
'subpath': r'20190903 102029 [200001]\Series 000 [SM]',
'files': {
'1.2.276.0.7230010.3.1.4.1145362585.2096.1567753444.778.dcm': '4463e99bea080b14591f0665a6e84559', # NOQA
'1.2.276.0.7230010.3.1.4.1145362585.2096.1567753447.783.dcm': '0b48b76e46c754fc75b13d6fbf682a19', # NOQA
'1.2.276.0.7230010.3.1.4.1145362585.2096.1567753450.788.dcm': '75452d5f811f5c8c8f4475fbac6665bb', # NOQA
'1.2.276.0.7230010.3.1.4.1145362585.2096.1567753451.793.dcm': '3bf4fee344571bda7005d9735b5bd699', # NOQA
'1.2.276.0.7230010.3.1.4.1145362585.2096.1567753451.798.dcm': '84f2b3ba111ec09beb8846d17ddcd9e5', # NOQA
'1.2.276.0.7230010.3.1.4.1145362585.2096.1567753451.803.dcm': 'd2e565d600a573685ca1be48d96a0ef4', # NOQA
'1.2.276.0.7230010.3.1.4.1145362585.2096.1567753451.808.dcm': 'aedbe1f7ba9f15754078e3c8a9077fee', # NOQA
'1.2.276.0.7230010.3.1.4.1145362585.2096.1567753451.813.dcm': '3533d30ef5e9abe3b8ee6e1c7bc0fb17' # NOQA
}
}
}


def cwd_to_folder(ftp, folder: Path):
ftp.cwd('/')
ftp.cwd("/")
for subfolder in folder.parts:
ftp.cwd(str(subfolder))


def download_file(ftp: FTP, file: str, filename: Path):
with open(filename, 'wb') as fp:
ftp.retrbinary(f'RETR {file}', fp.write)
with open(filename, "wb") as fp:
ftp.retrbinary(f"RETR {file}", fp.write)


def get_slide_dir() -> Path:
DEFAULT_DIR = 'tests/testdata'
SLIDE_DIR = 'slides'
DEFAULT_DIR = "tests/testdata"
SLIDE_DIR = "slides"
test_data_path = os.environ.get("DICOM_TESTDIR")
if test_data_path is None:
test_data_dir = Path(DEFAULT_DIR)
Expand All @@ -89,15 +89,13 @@ def get_slide_dir() -> Path:


def get_or_check_slide(slide_dir: Path, slide: Dict[str, Any], ftp: FTP):
path = slide_dir.joinpath(slide['name'], slide['subpath'])
path = slide_dir.joinpath(slide["name"], slide["subpath"])
ftp_path = FILESERVER_SLIDE_PATH.joinpath(
slide['parentpath'],
slide['name'],
slide['subpath']
slide["parentpath"], slide["name"], slide["subpath"]
)
os.makedirs(path, exist_ok=True)
cwd_to_folder(ftp, ftp_path)
for file, checksum in slide['files'].items():
for file, checksum in slide["files"].items():
file_path = path.joinpath(file)
if not file_path.exists():
print(
Expand All @@ -111,7 +109,7 @@ def get_or_check_slide(slide_dir: Path, slide: Dict[str, Any], ftp: FTP):


def check_checksum(file_path: Path, checksum: str):
with open(file_path, 'rb') as saved_file:
with open(file_path, "rb") as saved_file:
data = saved_file.read()
file_checksum = md5(data).hexdigest()
if checksum != file_checksum:
Expand Down
3 changes: 1 addition & 2 deletions tests/test_save.py
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,6 @@ def _get_encoded_tile(self, tile: Point, z: float, path: str) -> bytes:


@pytest.mark.save
@pytest.mark.filterwarnings("ignore")
class WsiDicomFileSaveTests(unittest.TestCase):
@classmethod
def setUpClass(cls):
Expand Down Expand Up @@ -177,7 +176,7 @@ def _get_relative_path(slide_path: Path) -> Path:
def create_test_dataset(frame_count: int, image_data: WsiDicomTestImageData):
dataset = Dataset()
dataset.SOPClassUID = WSI_SOP_CLASS_UID
dataset.ImageType = ["ORGINAL", "PRIMARY", "VOLUME", "NONE"]
dataset.ImageType = ["ORIGINAL", "PRIMARY", "VOLUME", "NONE"]
dataset.NumberOfFrames = frame_count
dataset.SOPInstanceUID = generate_uid()
dataset.StudyInstanceUID = generate_uid()
Expand Down
1 change: 0 additions & 1 deletion tests/test_wsidicom_file_target.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,6 @@


@pytest.mark.integration
@pytest.mark.filterwarnings("ignore")
class WsiDicomFileTargetIntegrationTests(unittest.TestCase):
@classmethod
def setUpClass(cls):
Expand Down
1 change: 0 additions & 1 deletion tests/test_wsidicom_integration.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@


@pytest.mark.integration
@pytest.mark.filterwarnings("ignore")
@parameterized_class(
[
{"input_type": WsiInputType.FILE},
Expand Down
3 changes: 2 additions & 1 deletion wsidicom/conceptcode.py
Original file line number Diff line number Diff line change
Expand Up @@ -228,7 +228,7 @@ def _from_cid(cls, meaning: str) -> Code:
for code in cls.cid.values():
if code.meaning == meaning:
return code
raise ValueError("Unsupported code")
raise ValueError(f"Unsupported code with meaning {meaning}.")

@classmethod
def list(cls) -> List[str]:
Expand All @@ -254,6 +254,7 @@ def from_code_value(
code.meaning, code.value, code.scheme_designator, code.scheme_version
)


class UnitCode(SingleConceptCode):
"""Code for concepts representing units according to UCUM scheme"""

Expand Down
6 changes: 3 additions & 3 deletions wsidicom/errors.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,14 +38,14 @@ def __str__(self):


class WsiDicomMatchError(Exception):
"""Raised if item in group that should match doesnt match."""
"""Raised if item in group that should match does not match."""

def __init__(self, item: str, group: str):
self.item = item
self.group = group

def __str__(self):
return f"{self.item} doesnt match {self.group}"
return f"{self.item} does not match {self.group}"


class WsiDicomUidDuplicateError(Exception):
Expand Down Expand Up @@ -89,7 +89,7 @@ class WsiDicomRequirementError(Exception):
"""Raised if required attribute is missing."""


class WsiDicomNoResultionError(Exception):
class WsiDicomNoResolutionError(Exception):
"""Raised if method is not possible as resolution is missing in image
data."""

Expand Down
6 changes: 3 additions & 3 deletions wsidicom/file/wsidicom_file.py
Original file line number Diff line number Diff line change
Expand Up @@ -366,7 +366,7 @@ def _read_sequence_delimiter(self):
TAG_BYTES = 4
self._file.seek(-TAG_BYTES, 1)
if self._file.read_tag() != SequenceDelimiterTag:
raise WsiDicomFileError(self._file, "No sequence delimeter tag")
raise WsiDicomFileError(self._file, "No sequence delimiter tag")

def read_frame(self, frame_index: int) -> bytes:
"""Return frame data from pixel data by frame index.
Expand Down Expand Up @@ -411,11 +411,11 @@ def _parse_pixel_data(self) -> Tuple[List[Tuple[int, int]], OffsetTableType]:
then not be empty. A BOT most always be the first item in the Pixel
data, but can be empty (zero length). If EOT is used BOT should be empty.
First seach to pixel data position, which is either EOT tag or PixelData tag.
First search to pixel data position, which is either EOT tag or PixelData tag.
If EOT read the EOT. For all cases validate that the filepointer now is at the
PixelData tag. If BOT read the BOT, otherwise skip the BOT. If EOT nor BOT has
been read, parse frame positions from pixel data. Otherwise parse frame
positions from EOT or BOT. Finaly check that the number of read frames equals
positions from EOT or BOT. Finally check that the number of read frames equals
the specified number of frames, otherwise frames are fragmented which we dont
support.
Expand Down
2 changes: 1 addition & 1 deletion wsidicom/file/wsidicom_file_image_data.py
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ def transfer_syntax(self) -> UID:

def _get_file(self, frame_index: int) -> WsiDicomFile:
"""
Return file contaning frame index.
Return file containing frame index.
Raises WsiDicomNotFoundError if frame is not found.
Expand Down
Loading

0 comments on commit 1de77bb

Please sign in to comment.