Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue with current reporting #341

Open
man-shu opened this issue Jan 5, 2021 · 6 comments
Open

Issue with current reporting #341

man-shu opened this issue Jan 5, 2021 · 6 comments

Comments

@man-shu
Copy link
Contributor

man-shu commented Jan 5, 2021

Problem:

By running the easy_start examples, I found that the major issue with pyprocess's current reporting is that the reports show plots by embedding locally saved images into an HTML template. Most of the browsers these days have a security feature which prevents loading locally saved images through HTML. As a result, even though pypreprocess creates and saves the plots on the disk properly, they do not appear in the HTML reports at all.

The HTML reports actually mention this issue - "RENDERING OF QA THUMBNAILS IS ENSURED BY LOCAL JQUERY SCRIPTS. THIS MAY NOT BE POSSIBLE ON CHROME DUE TO FILE-ACCESS RESTRICTIONS!"

Solutions:

  • In Google Chrome browser, this security feature can be disabled by running a terminal command google-chrome --allow-file-access-from-files on Linux. A browser window would open now and loading the HTML report in this window would render all the thumbnails and plots properly.

  • Reporting in nilearn provides another (more efficient) solution. It bypasses this security feature by creating SVG URLs from matplotlib figure objects and then embeds those SVG URLs into the HTML reports. This means that the plots are not saved as raster images on the disk at all and the browsers simply render them as vector graphics whenever HTML reports are opened up. This could be implemented for pypreprocess as well. But that could be a bit complicated because there are several functions and class methods in pypreprocess named generate_x_thumbnails (where x = various preprocessing steps such as realignment, segmentation etc.) which:

    1. plot and save images corresponding to each preprocessing step
    2. implement various event handlers which ensure that correct images are being plotted according to the provided input
    3. create clickable thumbnail objects for each plot that are eventually added to HTML reports

    If we were to implement nilearn-style reporting, iii. would be completely useless and could be removed. But i. and ii. would still be important and adapting these two to nilearn-style reporting would be a bit complex.

@man-shu
Copy link
Contributor Author

man-shu commented Jan 7, 2021

So keeping i. and ii. would mean that there would still be some remnants of current reporting, but the plots would be embedded in HTML reports as they are in the nilearn reporting module.

@bthirion
Copy link
Member

bthirion commented Jan 7, 2021

@man-shu thx for the detailed analysis !
I think e should keep i and ii indeed. If I understand correctly, this amounts to adapting current functions to give a different output.
It can be a bit tedious, but not too complex I hope ?
Best,

@man-shu
Copy link
Contributor Author

man-shu commented Jan 8, 2021

Thanks, @bthirion ! Yes, I am on it!

@NicolasGensollen
Copy link
Contributor

Hi @man-shu
I thought a little bit about the size issue of the generated reports you mentioned. If I understand correctly, you have several SVG images that end up weighting around 24MB which is way too heavy.
Maybe I understood it wrong (and we can discuss this more tomorrow during our meeting then), but could you have a look at the following example code I tried to see if this could solve your problem:

from io import StringIO
from string import Template
import matplotlib.pyplot as plt

# Vectorized fig
figV, axesV = plt.subplots(nrows=10, ncols=5)
# Rasterized fig
figR, axesR = plt.subplots(nrows=10, ncols=5)

# Plot garbage data
for ax in figV.axes:
    ax.scatter(range(3000), range(3000))
    
for ax in figR.axes:
    ax.set_rasterized(True)
    ax.scatter(range(3000), range(3000))

# Encode figures
with StringIO() as bufferV:
    figV.savefig(bufferV, format='svg', dpi=150)
    figV_encoded = bufferV.getvalue()
    
with StringIO() as bufferR:
    figR.savefig(bufferR, format='svg', dpi=150)
    figR_encoded = bufferR.getvalue()

# Define a super basic HTML template
html_template = Template("<html>\n<body>\n$figure\n</body>\n</html>")

# Save to html files
for name,figenc in zip(["heavy", "light"],
                       [figV_encoded, figR_encoded]):
    with open(f"{name}_report.html", 'w') as fp:
        fp.write(html_template.substitute(figure=figenc))

@bthirion
Copy link
Member

bthirion commented Feb 3, 2021

If svgs are too heavy, you may want to reduce the dpi.
Best,

@man-shu
Copy link
Contributor Author

man-shu commented Feb 4, 2021

Thanks, @NicolasGensollen @bthirion. Yes, this is exactly the problem. I will try this and let you know.

man-shu added a commit that referenced this issue Mar 24, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants