JOSS review: Runtime errors after setting up an environment

[jhagerer]

Dear @WarmCyan, I am the reviewer of your JOSS submission.

I run into problems after setting up the environment. When running icat.initialize(offline=True), the following errors occur:

One way to reproduce:

git clone https://github.com/ORNL/icat.git
cd icat
conda env create --file environment.yml  # to set the right python version in a fresh virtual environment
conda activate icat
conda install pip
pip install icat-iml
conda install jupyter_bokeh  # necessary after ModuleNotFoundError
pip install ipywidgets-bokeh  # necessary after ModuleNotFoundError
pip install -r requirements.txt  # tried that for debugging, didn't help

As a first suggestion, you might want to add ipywidgets-bokehto requirements.txt and jupyter_bokeh to environment.yml (or both to requirements.txt). Secondly, I would recommend to run the setup by yourself from scratch the way I did it, which is quite close to your documentation. Alternatively, you might want to give me feedback where I made something wrong during the setup.

I put the JOSS submission review on hold and try again next weekend. Please tag me here or in the review thread if you have updates or suggestions. Thanks!

[WarmCyan]

Hey @jhagerer! Thanks for agreeing to review this!

Just a quick update, I started taking a look at this yesterday, go figure in the couple months since I last tested setting this up from scratch, various package updates have somehow broken it 😩

I don't think either ipywidgets-bokeh or jupyter_bokeh should strictly be necessary for running this in JupyterLab, or at least in my (working) development environment I don't have either installed. As far as I'm aware, those libraries are used depending on what context you're actually running stuff in (putting this here for my own future reference: https://panel.holoviz.org/explanation/comms.html#libraries-modules-with-support-for-communication-in-these-contexts), but you raise a good point that it'd probably be better to just go ahead and include them to maximally cover various contexts. Moving forward I'll add them into the setup.

(Notably I haven't actually tried running this from vscode before. I know every now and then that can do odd things with widgets, but given it's not even working in JupyterLab in a new environment either, that shouldn't be the issue here)

I'm still working to figure out the most recent magical combination of version numbers that work (I think the issue is somewhere in the interplay between bokeh, panel, and the recent pyviz>3). If I don't get something satisfactory by the weekend, I'll check that the exact versions I have in my development environment work in a new env and list those here so you at least have something to play with.

[WarmCyan]

Still continuing to work on this, sorry I didn't get anything to you by the weekend, I've been jumping from fire to fire!

I've figured out at least one of the issues, I think there's some sort of a breaking change between the latest ipywidgets 8.1.3 version and panel when used together in JupyterLab, I've separately opened up an issue holoviz/pyviz_comms#129 because I don't think it's something I can fix over here. I've also (finally) added a playwright test to automate checking that one of the notebooks actually runs and displays the interface to hopefully catch these sorts of problems sooner moving forward.

Restricting only ipywidgets and jupyterlab_widgets versions still has some issue going on that I'm trying to figure out, but for what it's worth I do now have a very strict list of dependencies in setup.py that sets exact versions for most relevant things in my dev environment (and nominally works, at least in JupyterLab)

In principle, to get a currently working environment with those, you should be able to run:

git clone https://github.com/ORNL/icat.git
cd icat
conda create -n icat python
conda activate icat
pip install -e .

I'll update again once I've narrowed down any other dependency issues and have a less restrictive setup.py

[WarmCyan]

I found and fixed a related bug that had started presenting with a recent ipyvue update as well, I believe setup.py in the updated 0.7.3 version now correctly reflects everything needed to get a currently working environment for JupyterLab

@jhagerer in theory you should now be able to run

git clone https://github.com/ORNL/icat.git
cd icat
conda create -n icat python
conda activate icat
conda install pip
pip install icat-iml

and have a functioning environment, let me know if it works for you and I'll close the issue

[jhrinv]

@WarmCyan I clearly see you made some substantial effort. Either tomorrow or next weekend I will take a deeper look into your updates and proceed with the review. Thank you for your efforts, and you'll hear back from me soon!

[jhagerer]

Dear @WarmCyan , I was able to run the installation successfully. Thank you for the improvements! However, I am getting stuck in the execution of ICAT when I run icat.initialize(offline=True), which is given by notebooks/simple_example.ipynb from the repository:

Here you can inspect the full error message:
error_stack.txt

The same exception occurs when I run icat.initialize(offline=True) given by lm_similarity_example.ipynb.

What would be your remark on this? Thanks for your time and effort.

[jhagerer]

PS: I am using Python 3.12.4. Attached you see my installed packages.
packages.txt

[jhagerer]

PPS: I just tried executing it as described before, but this time using Python 3.10.0. I receive the exact same error message MultipleInstanceError: An incompatible sibling of 'Kernel' is already instantiated as singleton: str.

[WarmCyan]

I think this has to do with running it in vscode, this is likely related to a separate issue (about ipywidgets inside panel inside vscode) posted on the panel repo: holoviz/panel#6175

Have you tried running this directly in Jupyter Lab? There are unfortunately compatibility issues sometimes when running widget-based stuff within vscode, so I'm only explicitly trying to support Jupyter Lab (I can add a note about this in the docs/README)

[jhagerer]

@WarmCyan Forget my last post (I just removed it, since I had another problem setting up my conda environment). I got icat running successfully. I still have some feedback regarding usability which IMHO should be addressed. I will document my feedback over here. Feel free to close the current issue.

[WarmCyan]

Thanks for the update! Going to leave this issue open and note for myself that I'll close this once I add to the README/docs that this doesn't work in VSCode