Problem reproducing synthesis

[simonpcastillo]

Hi, I downloaded and tested your pipeline with the data you provide here and it seems that something is failing in the synthesis, please check this out for some of the tiles in the sample TMA_40x_1. I maintained all default statements, except the corresponding paths. What do you recommend?

[binli123]

Do you have the weights downloaded and loaded to the model without error?
It also looks like an issue associated with the image data type. Though I have no problem on my side, here is my output example. This might be due to the package compatibility, I have updated the environment specifics with packages of newer versions. Please let me know if that works for you.

[simonpcastillo]

Thanks for your quick reply!
It has helped a bit (I had to downgrade pythorch). Unfourtenately, the results remain different. This is yours stiched and this is the one I obtain.. If you notice it seems it is synthesising on another channel (?), and if you look at the tiles that should be with absence of prediction (like top/left) they reproduce a pattern with a particular signal.

[binli123]

I suspect somehow the channel order of the RGB input is messed up there. Could you add these lines after line 159

he_shg_synth_workflow/main.py

Line 159 in 0219e64

input = batch['image'].float().to(device).view(1, 3, window_shape[0], window_shape[1])

                    if iteration == 0:
                        np.savetxt('0_0_c1.txt', input.squeeze().cpu().numpy()[0], delimiter=',')
                        np.savetxt('0_0_c2.txt', input.squeeze().cpu().numpy()[1], delimiter=',')
                        np.savetxt('0_0_c3.txt', input.squeeze().cpu().numpy()[2], delimiter=',')
                        break

and compare to what I got? https://drive.google.com/drive/folders/1JuW-xhBkRGhA9r2HVQzxtz4SvQwT8UXX?usp=sharing

[simonpcastillo]

yeap, same.

Do you mind to verify this line, please? If the input to the model is ok, maybe the output are not correctly stated?

he_shg_synth_workflow/main.py

Line 163 in 0219e64

result_patch = prediction[i, :, :, :,].cpu().detach().numpy().transpose((1, 2, 0))*255

[binli123]

That line is the same as what I have. I also uploaded the prediction prints to the same google drive folder. You could obtain the output using the following lines (after prediction = model(input))

                    if iteration == 0:
                        np.savetxt('0_0_c1.txt', prediction.squeeze().cpu().numpy()[0], delimiter=',')
                        np.savetxt('0_0_c2.txt', prediction.squeeze().cpu().numpy()[1], delimiter=',')
                        np.savetxt('0_0_c3.txt', prediction.squeeze().cpu().numpy()[2], delimiter=',')
                        break

Also, please comment out line 17 warnings.simplefilter("ignore", UserWarning) to see if there are any informative warnings.

[binli123]

Hi, I've tried this also on another Windows machine, I didn't experience the same issue as you did. I have uploaded an 'env.yml' file, you could try to create a new environment using conda env create --name syn --file env.yml to see if the issue still persists.

[simonpcastillo]

Solved! I have geforce RTX-3090 CUDA sm_86 which wasn't compatible with the PyTorch version. Replaced with conda install pytorch torchvision torchaudio cudatoolkit=11.1 -c pytorch -c conda-forge and now it works. Thanks!!