Can BigStitcher-Spark do fusion on a downsampled resolution (s1)?

[FangmingXie]

Can BigStitcher-Spark do fusion based on s1 instead of s0 resolution?

For large data (20 tiles; 2TB), this would be a very useful feature, as in many cases s1, and even s2 s3 are enough for downstream tasks. I know this can be done by first fusing a s0 image and then generating a multi-resolution pyramid. But it is not clear to me that if we can directly generate a downsampled fused image.

Thanks!

[StephanPreibisch]

Hi, sorry, I missed that ... yes, it can do it multiple ways. The easiest would be to define your own downsampling steps (see here):

BigStitcher-Spark/src/main/java/net/preibisch/bigstitcher/spark/AffineFusion.java

Lines 107 to 108 in 7eaf5be

	@Option(names = { "-ds", "--downsampling" }, split = ";", required = false, description = "Manually define steps to create of a multi-resolution pyramid (e.g. -ds 2,2,1; 2,2,1; 2,2,2; 2,2,2)")
	private List<String> downsampling = null;

You could also simply apply a transformation to all views in BigStitcher that sets a downsampling as you want (e.g. 2,2,2 or 2,2,1) and then normally fuse knowing s0 is actually s1.

Please let me know if I can help with the details of both approaches ...

[FangmingXie]

Thanks Stephan! It is very straightfoward and I will try this. I have noticed this ds parameter but thought it does something different.

[FangmingXie]

@StephanPreibisch I tried your suggestion by adding -ds and found it doesn't work on my old version (August this year). So I updated to the latest commit and reinstalled the package. It now does recognizes -ds, but the program terminated immediately after I ran it. Any idea why? I suspect it is some installation issue.

I am able to revert to the older version and run without -ds.

Below is my command and the log. Thanks so much!

./affine-fusion \
        -x './r1_test_flatfield_v8.xml' \
        -o './r1_flatfield_export.n5' \
        -d "/c0_tryds/s0" \
        -ds "16,16,8" \
        --channelId 0 \
        --preserveAnisotropy \
        --UINT16 \
        --minIntensity 0 \
        --maxIntensity 65535 \
        --blockSize "128,128,128"

23/12/13 13:45:21 INFO SparkContext: Running Spark version 3.3.2
23/12/13 13:45:21 INFO ResourceUtils: ==============================================================
23/12/13 13:45:21 INFO ResourceUtils: No custom resources configured for spark.driver.
23/12/13 13:45:21 INFO ResourceUtils: ==============================================================
23/12/13 13:45:21 INFO SparkContext: Submitted application: AffineFusion
23/12/13 13:45:21 INFO ResourceProfile: Default ResourceProfile created, executor resources: Map(cores -> name: cores, amount: 1, script: , vendor: , memory -> name: memory, amount: 1024, script: , vendor: , offHeap -> name: offHeap, amount: 0, script: , vendor: ), task resources: Map(cpus -> name: cpus, amount: 1.0)
23/12/13 13:45:21 INFO ResourceProfile: Limiting resource is cpu
23/12/13 13:45:21 INFO ResourceProfileManager: Added ResourceProfile id: 0
23/12/13 13:45:22 INFO SecurityManager: Changing view acls to: f7xiesnm
23/12/13 13:45:22 INFO SecurityManager: Changing modify acls to: f7xiesnm
23/12/13 13:45:22 INFO SecurityManager: Changing view acls groups to:
23/12/13 13:45:22 INFO SecurityManager: Changing modify acls groups to:
23/12/13 13:45:22 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(f7xiesnm); groups with view permissions: Set(); users  with modify permissions: Set(f7xiesnm); groups with modify permissions: Set()```