Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Launch emu in a process during tests run #86

Draft
wants to merge 17 commits into
base: master
Choose a base branch
from

Conversation

davidcaron
Copy link
Collaborator

@davidcaron davidcaron commented Nov 30, 2018

Overview

Related to #67

So at the beginning of the test run, a Process is started in the same way as if we would call emu start. And it's terminated at the end of the test run.

I think this would allow running the tests on travis, but I haven't tested it.

I'm open to suggestions...

@davidcaron
Copy link
Collaborator Author

well... it doesn't work yet in Travis. Maybe it's something about creating a process in their environment. I will try to find a fix.

In the meantime, let me know if you think this is a good idea in general.

@cehbrecht
Copy link
Member

Can't say what goes wrong. Maybe it is an issue with the ip address? See the dockerfile in emu:

emu start -b 0.0.0.0

I'm a bit worried about the eggs in the requirements_dev.txt ...

@cehbrecht
Copy link
Member

@davidcaron I'm still in favor for mock tests :) But that does not exclude your pytest approach. We would have options to choose.

As a follow-up of this we could think about a pytest extension. I have opened a ticket for discussion:
bird-house/bird-house.github.io#28

@davidcaron
Copy link
Collaborator Author

We could have @pytest.mark.emu for these tests that need a running emu server and for everything else have mocks... So we could test against a live server but also have more targeted tests using mocks.

I'm still going to try to make emu work on travis, because I don't see why it shouldn't.

@cehbrecht
Copy link
Member

I'm still going to try to make emu work on travis, because I don't see why it shouldn't.

Did you check the IP address? On travis we are running in container ...

https://github.com/bird-house/emu/blob/d8abda60b9840abd56023279c2dd336127f31005/Dockerfile#L28

@davidcaron
Copy link
Collaborator Author

Debugging travis is really painful... Any idea why conda install -c birdhouse emu installs pywps 4.0.0 and conda install -c birdhouse "pywps>=4.1.22" returns an error?

here is the link to the latest build: https://travis-ci.org/bird-house/birdy/jobs/463009078

@cehbrecht
Copy link
Member

@davidcaron I thinks you are missing the conda-forge channel:

conda install -c conda-forge -c birdhouse pywps>=4.1

The emu conda package has not been updated for awhile, it is version 0.8.0. But it is also using pywps>=4.1 ... unless you may have got an even older version.

@cehbrecht
Copy link
Member

@davidcaron there was anther issue with the pywps conda package ... version 4.1.23 didn't include the pathlib dependeny, I have overseen that. This is fixed now with version 4.1.24.

I have made a new release and conda package for emu: 0.9.1.

@davidcaron
Copy link
Collaborator Author

davidcaron commented Dec 4, 2018

Thank you for the emu release.

I think it's possible there could be something wrong with the eggshell release on conda... Here are the files that are installed:

(birdy) ➜  eggshell conda list | grep eggshell
birdhouse-eggshell        0.3.0                      py_3    birdhouse
(birdy) ➜  eggshell pwd
/home/ubuntu/.conda/envs/birdy/lib/python3.6/site-packages/eggshell
(birdy) ➜  eggshell tree
.
├── esgf
│   ├── __init__.py
│   ├── __pycache__
│   │   ├── __init__.cpython-36.pyc
│   │   └── utils.cpython-36.pyc
│   └── utils.py
├── general
│   ├── __init__.py
│   ├── __pycache__
│   │   ├── __init__.cpython-36.pyc
│   │   ├── calculation.cpython-36.pyc
│   │   ├── datafetch.cpython-36.pyc
│   │   └── utils.cpython-36.pyc
│   ├── calculation.py
│   ├── datafetch.py
│   └── utils.py
├── nc
│   ├── __init__.py
│   ├── __pycache__
│   │   ├── __init__.cpython-36.pyc
│   │   └── utils.cpython-36.pyc
│   └── utils.py
├── ocg
│   ├── __init__.py
│   ├── __pycache__
│   │   ├── __init__.cpython-36.pyc
│   │   └── utils.cpython-36.pyc
│   └── utils.py
└── visual
    ├── __init__.py
    ├── __pycache__
    │   ├── __init__.cpython-36.pyc
    │   └── visualisation.cpython-36.pyc
    └── visualisation.py

10 directories, 24 files

emu is looking for eggshell.config and can't find it.

Also, maybe the requirement for emu should be birdhouse-eggshell>=0.3 instead of birdhouse-eggshell=0.3?

@cehbrecht
Copy link
Member

eggshell is too fragile and breaks from release to release ... I either need to pin or remove it. I didn't notice issues with emu/eggshell when I installed it from conda.

You may also try this setup in a local docker container ... that gets closer to the travis environment.

Travis is using ubuntu/trusty (14.04?):

Build dist: trusty

Not sure if the travis containers are available for download.

@Zeitsperre
Copy link
Contributor

Just chiming in that Ubuntu 16.04 (xenial) builds are available on travis as of last month.

@huard
Copy link
Contributor

huard commented Jul 23, 2019

I think it is worthwhile to pursue this objective and I like @davidcaron 's approach. It lets us tag the version of emu birdy should be compatible with so we can keep them both in sync, and avoids duplication inherent in mock objects.

@davidcaron
Copy link
Collaborator Author

I think I will have some time to check this soon...

I started looking at running a local travis docker image for faster debugging with travis. I think it will help a lot in this case.

@cehbrecht cehbrecht marked this pull request as draft April 1, 2021 15:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants