Developer install help

[ndmlny-qs]

I'm installing a development copy of ArviZ in order to make future contributions, and have run into a few issues I need help with. I will narrate what I've done below with questions and observations at the end.

Fork/Clone and testing

Following along with the contributor documentation, I've forked ArviZ, and cloned my fork locally. I've updated the remote of my local-cloned repo as shown in the documentation with

git remote add upstream https://github.com/arviz-devs/arviz.git

I then created a virtual environment with mamba (https://mamba.readthedocs.io/en/latest/installation.html) using Python 3.8, since the ArviZ tests indicate that Python 3.8 is the minimum install.

arviz/.azure-pipelines/azure-pipelines-base.yml

Line 25 in 5ca7643

Python_38:

mamba create --name arviz-py38 python=3.8 pip

After the environment has been created, I activate it with

mamba activate arviz-py38

I then installed the requirements defined in both the requirements and requirements-dev.txt files using

pip install -r requirements.txt
pip install -r requirements-dev.txt

The installation works just fine up to this point. I typically run tests next before I work on a contribution so I ran them using

pytest arviz/tests/base_tests/

This leads to my first error.

ImportError while loading conftest 'arviz/arviz/tests/conftest.py'.
arviz/__init__.py:32: in <module>
    from .data import *
arviz/data/__init__.py:4: in <module>
    from .datasets import clear_data_home, list_datasets, load_arviz_data
arviz/data/datasets.py:21: in <module>
    with open(os.path.join(_EXAMPLE_DATA_DIR, "data_local.json"), "r", encoding="utf-8") as f:
E   FileNotFoundError: [Errno 2] No such file or directory: 'arviz/arviz/data/example_data/data_local.json'

When I look in the arviz/arviz/data/example_data/ folder I see that there are no data and no JSON files in it. So I assumed that I needed to download the data using the ArviZ package in an interactive session, which meant I started up Python in my terminal and attempted to import arviz. Doing this gives the same FileNotFoundError

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "arviz/arviz/__init__.py", line 32, in <module>
    from .data import *
  File "arviz/arviz/data/__init__.py", line 4, in <module>
    from .datasets import clear_data_home, list_datasets, load_arviz_data
  File "arviz/arviz/data/datasets.py", line 21, in <module>
    with open(os.path.join(_EXAMPLE_DATA_DIR, "data_local.json"), "r", encoding="utf-8") as f:
FileNotFoundError: [Errno 2] No such file or directory: 'arviz/arviz/data/example_data/data_local.json'

I didn't fully understand this since the data_local.json and data_remote.json files do not exist in a normal pip installed version of ArviZ. There is, however, another repo in the arviz-devs org called arviz_example_data found at https://github.com/arviz-devs/arviz_example_data. Inside this repo both the data JSON files exist. So I cloned and copied those files to my forked repo.

git clone https://github.com/arviz-devs/arviz_example_data
cp arviz_example_data/data_{local,remote}.json arviz/arviz/data/example_data

I started a new Python session in my terminal and imported ArviZ with no errors. With no more errors due to files not being found, I ran the tests again.

pytest arviz/tests/base_tests

The tests run, with a few creating matplotlib images. Once the tests completed I ended up with

• 27 failed tests
• 1986 passed
• 30 skipped
• 1329 errors

Inspecting the errors shows a few FileNotFoundError associated with the *.nc files found in the arviz_example_data repo. So I copied those files over to my dev cloned ArviZ repo and reran the tests. This time I got

• 1 failed
• 3341 passed
• 30 skipped

with the failures coming from not having bokeh or zarr installed. I saw that requirements-optional.txt have both bokeh and zarr listed, so I pip install that requirements file next,

pip install -r requirements-optional.txt

and reran the tests. This then leads to the following results

• 2 failed
• 3735 passed

with the failed tests coming from

=========================================================== FAILURES ============================================================
_______________________________________________________ test_cov[shape0] ________________________________________________________

shape = (8,)

    @pytest.mark.parametrize("shape", [(8,), (8, 8), (8, 8, 8)])
    def test_cov(shape):
        x = np.random.randn(*shape)
        if x.ndim <= 2:
>           assert np.allclose(_cov(x), np.cov(x))

arviz/tests/base_tests/test_plots_matplotlib.py:466:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
arviz/utils.py:417: in _cov
    return _cov_1d(data)
arviz/utils.py:192: in __call__
    return self.numba_fn(*args, **kwargs)
mambaforge/envs/arviz-py38/lib/python3.8/site-packages/numba/core/dispatcher.py:468: in _compile_for_args
    error_rewrite(e, 'typing')
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

e = TypingError('Failed in nopython mode pipeline (step: nopython frontend)\n- Resolution failure for literal arguments:\n...rviz/arviz/utils.py (409)\n\n\nFile "arviz/utils.py", line 409:\ndef _cov_1d(x):\n    x = x - x.mean(axis=0)\n    ^\n')
issue_type = 'typing'

    def error_rewrite(e, issue_type):
        """
        Rewrite and raise Exception `e` with help supplied based on the
        specified issue_type.
        """
        if config.SHOW_HELP:
            help_msg = errors.error_extras[issue_type]
            e.patch_message('\n'.join((str(e).rstrip(), help_msg)))
        if config.FULL_TRACEBACKS:
            raise e
        else:
>           raise e.with_traceback(None)
E           numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
E           - Resolution failure for literal arguments:
E           AssertionError()
E           - Resolution failure for non-literal arguments:
E           AssertionError()
E
E           During: resolving callee type: BoundFunction(array.mean for array(float64, 1d, C))
E           During: typing of call at arviz/arviz/utils.py (409)
E
E
E           File "arviz/utils.py", line 409:
E           def _cov_1d(x):
E               x = x - x.mean(axis=0)
E               ^

mambaforge/envs/arviz-py38/lib/python3.8/site-packages/numba/core/dispatcher.py:409: TypingError
__________________________________________________ test_plot_posterior_boolean __________________________________________________

    def test_plot_posterior_boolean():
        data = np.random.choice(a=[False, True], size=(4, 100))
        axes = plot_posterior(data)
        assert axes
        plt.draw()
        labels = [label.get_text() for label in axes.get_xticklabels()]
>       assert all(item in labels for item in ("True", "False"))
E       assert False
E        +  where False = all(<generator object test_plot_posterior_boolean.<locals>.<genexpr> at 0x7f1fac4037b0>)

arviz/tests/base_tests/test_plots_matplotlib.py:1059: AssertionError

I didn't really look into the above errors closely, before I reran the tests using the --save flag in pytests, as stated in the documentation; https://python.arviz.org/en/latest/contributing/pr_checklist.html#pull-request-checklist. Doing so did not fix any of the errors.

So this is where I've stopped. I do not know if these tests are supposed to pass or not, but there is a lot of failed jobs from the open PRs that I've seen. My questions are as follows:

• are these known tests that should fail?
• should I have known about the data needing to be installed from a different repo as referenced in Replace example data submodule with subtree #2105?

With the above issues I'm facing, I would like to hear about how other devs create a working ArviZ environment.

At this point I also tried to install the packages from requirements-docs.txt and ran into a different issue

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed.
This behaviour is the source of the following dependency conflicts.
distributed 2022.9.0 requires tornado<6.2,>=6.0.3,
but you have tornado 6.2 which is incompatible.

Similarly, when I try to install packages from requirements-external.txt I get the following warning and error.

  WARNING: Generating metadata for package pymc3 produced metadata for project name pymc. Fix your #egg=pymc3 fragments.
Discarding git+https://github.com/pymc-devs/pymc3: Requested pymc from git+https://github.com/pymc-devs/pymc3 (from -r requirements-external.txt (line 3)) has inconsistent name: filename has 'pymc3', but metadata has 'pymc'
Collecting emcee
  Downloading emcee-3.1.2-py2.py3-none-any.whl (46 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 46.2/46.2 kB 410.4 kB/s eta 0:00:00
Collecting pyjags
  Downloading pyjags-1.3.8.tar.gz (184 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 184.8/184.8 kB 2.3 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
ERROR: Could not find a version that satisfies the requirement pymc3 (unavailable) (from versions: 3.0rc1, 3.0rc2, 3.0rc4, 3.0rc5, 3.0rc6, 3.0, 3.1rc1, 3.1rc2, 3.1rc3, 3.1, 3.2rc1, 3.2, 3.3rc1, 3.3rc2, 3.3, 3.4rc1, 3.4rc2, 3.4.1, 3.5rc1, 3.5, 3.6, 3.7rc1, 3.7, 3.8, 3.9.0, 3.9.1, 3.9.2, 3.9.3, 3.10.0, 3.11.0, 3.11.1, 3.11.2, 3.11.3, 3.11.4, 3.11.5)
ERROR: No matching distribution found for pymc3 (unavailable)

Obviously I'm missing something, but I just don't know what it is, and any help would be greatly appreciated.

[ahartikainen]

Data stuff went to submodule (so you need to add --recursive while cloning your repo) and everything with submodules complicates things. That's why we have a PR to use subtree instead, which is less complicated and works normally.

So for now (until the PR is merged) just do (remember to change to your fork)

git clone --recursive https://github.com/arviz-devs/arviz

[ndmlny-qs]

Thanks @ahartikainen I can confirm that adding the --recursive flag when cloning fixes the FileNotFoundErrors. I'm still faced with the failed tests for test_cov and test_plot_posterior_boolean, but one mystery solved!

[OriolAbril]

We have already merged the subtree instead of submodule PR which should simplify things regarding cloning.

Re running tests, I think we should add a "How to run the test suite" page in https://python.arviz.org/en/latest/contributing/index.html that covers things like

• requirements needed to run tests
• requirements needed to run all tests (I generally run only the relevant tests locally and often don't have all optional or external dependencies installed)
• how to run specific tests (specific file, specific function within file and -k flag), and how to run all tests (both base and external and their differences)
• how to run tests while saving the output of the generate mpl images

We can then link there from the PR checklist and PR step-by-step pages.

Re failing tests, all test should pass, and all have passed in the latest PRs I merged today: https://dev.azure.com/ArviZ/ArviZ/_build/results?buildId=5981&view=results . Do the versions you have installed differ significantly from those or from those other ones? (both of these run on python 3.9)