Reading .spm as ISO 28600:2011 not Bruker

[st-te]

Hi,

Currently have data stored as .spm (ISO 28600:2011 SPM data transfer format) but TopoStats freezes at 'Loading Image'. If I interrupt I can trace back to pySPM utilising Bruker.py (presumably instead of SPM.py?)

I'm wondering if there's an easy workaround in config.yaml > file_ext to recognise it as this format or whether this requires something configuring/editing in either the TopoStats or pySPM source - I've not been able to see anything obvious in your documentation (but maybe I've been staring at it too long!)

TopoStats package works (processes .jpk no problem) and .spm files are fine (opening in Gwyddion).

Thanks

[Tue, 25 Jul 2023 16:45:22] [INFO ] [topostats] Loading image from : C:\Users\m46654st\Desktop\sample_preproc_test\cuts\sample_preproc_test_1.spm
Traceback (most recent call last):
File "c:\users\m46654st\appdata\local\programs\python\python39\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "c:\users\m46654st\appdata\local\programs\python\python39\lib\runpy.py", line 87, in run_code
exec(code, run_globals)
File "C:\Users\m46654st\AppData\Local\Programs\Python\Python39\Scripts\run_topostats.exe_main.py", line 7, in
File "c:\users\m46654st\appdata\local\programs\python\python39\lib\site-packages\topostats\run_topostats.py", line 226, in main
all_scan_data.get_data()
File "c:\users\m46654st\appdata\local\programs\python\python39\lib\site-packages\topostats\io.py", line 752, in get_data
self.image, self.pixel_to_nm_scaling = suffix_to_loadersuffix
File "c:\users\m46654st\appdata\local\programs\python\python39\lib\site-packages\topostats\io.py", line 386, in load_spm
scan = pySPM.Bruker(self.img_path)
File "c:\users\m46654st\appdata\local\programs\python\python39\lib\site-packages\pySPM\Bruker.py", line 28, in init
line = self.file.readline().rstrip().replace(b'\', b"")
KeyboardInterrupt

[ns-rse]

Hi @st-te

Thanks for getting in touch.

The traceback shows that hanging occurs with the Bruker module of pySPM (pySPM/Bruker.py).

I've noticed that pySPM has just had a major update for the first time in two and a half years (v0.2.2 > v0.3.4) and wonder if that might be an underlying cause. Having only just clocked this we've not tested it at all.

What does pip show pySPM report in your virtual environment?

Are you able to share the .spm file with us so we can investigate?

[st-te]

Hi @ns-rse,

I had been using v0.2.2. Attached are the a few of the sample files I'm working with - crystal surfaces with terraces. I guess it could be the data prep itself, since we're going from multi-channel .jpk to cropped singe-channel .spm which Gywddion exports as ISO28600 as far as I'm aware, but I also suspect this format is different to a Bruker .spm file.

Interestingly, I updated pySPM to v0.3.4 to test which throws up the following - might have to downgrade!

PS C:\Users\m46654st\Desktop\sample_preproc_test\cuts> run_topostats --config config.yaml
Traceback (most recent call last):
File "c:\users\m46654st\appdata\local\programs\python\python39\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "c:\users\m46654st\appdata\local\programs\python\python39\lib\runpy.py", line 87, in run_code
exec(code, run_globals)
File "C:\Users\m46654st\AppData\Local\Programs\Python\Python39\Scripts\run_topostats.exe_main.py", line 4, in
File "c:\users\m46654st\appdata\local\programs\python\python39\lib\site-packages\topostats\run_topostats.py", line 19, in
from topostats.io import find_files, read_yaml, save_folder_grainstats, write_yaml, LoadScans
File "c:\users\m46654st\appdata\local\programs\python\python39\lib\site-packages\topostats\io.py", line 13, in
from igor import binarywave
File "c:\users\m46654st\appdata\local\programs\python\python39\lib\site-packages\igor\binarywave.py", line 110, in
1:numpy.complex, # NT_CMPLX, makes number complex.
File "c:\users\m46654st\appdata\local\programs\python\python39\lib\site-packages\numpy_init.py", line 305, in getattr
raise AttributeError(former_attrs[attr])
AttributeError: module 'numpy' has no attribute 'complex'.
np.complex was a deprecated alias for the builtin complex. To avoid this error in existing code, use complex by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use np.complex128 here.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations

sample_preproc_test.zip

[ns-rse]

Thanks @st-te for sharing the files, we'll have a look at this and see if we can work out what is going on.

I've encountered that numpy error before, can't remember when now but it may be why TopoStats pins the version of numpy==1.23.4. In a similar vein pySPM pins its version dependency on Numpy to 1.24.2.

There is early and not widely tested support for .jpk and .gwy files in TopoStats if you wanted to try those in the mean time.

EDIT : Requires TopoStats v2.1.0.

[st-te]

Thanks for this @ns-rse - I'll give 2.1.0 a try with my .jpk files