Skip to content
This repository has been archived by the owner on Jan 21, 2020. It is now read-only.

Feature/docker for testing #308

Open
wants to merge 35 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
2afe41b
Added docker-compose for development
TomHAnderson Apr 20, 2018
857298a
Instructions for Docker
TomHAnderson Apr 20, 2018
e6c6465
proofreading DOCKER.md
TomHAnderson Apr 20, 2018
97ca995
Added docker-compose and unit tests to travis
TomHAnderson Apr 22, 2018
a6b1272
The results in this test have changed order but are still valid. Rew…
TomHAnderson Apr 2, 2018
32cec04
Incorporated suggestions from @thomasvargiu for Docker
TomHAnderson Apr 22, 2018
c7b4d02
Added docker tests to travis
TomHAnderson Apr 22, 2018
6894562
Organized .travis.yml
TomHAnderson Apr 23, 2018
05aa000
Made changes based on feedback on github
TomHAnderson Apr 23, 2018
a2c7bdb
First test run of non-php specific travis test
TomHAnderson Apr 24, 2018
50dfd85
none = language?
TomHAnderson Apr 24, 2018
08fd71a
Added all tests back in
TomHAnderson Apr 24, 2018
26f53c0
Added build to install script
TomHAnderson Apr 24, 2018
e62c69a
composer-build works; extenting to .travis.yml
TomHAnderson Apr 24, 2018
91f7580
Added xdebug
TomHAnderson Apr 25, 2018
00faa66
Contributing Docker docs
TomHAnderson Apr 25, 2018
6661ae8
Tidy contributing
TomHAnderson Apr 25, 2018
0ee75b0
Removed DOCKER.md
TomHAnderson Apr 26, 2018
76bbabe
Removed ini debugging line
TomHAnderson Apr 26, 2018
8c767ad
Use aliases and simplify installing php extensions
michalbundyra Apr 26, 2018
3a2ce6b
Merge pull request #1 from webimpress/feature/docker-for-testing
TomHAnderson Apr 26, 2018
7fb7d96
Minor tweeks and documentation for Docker
TomHAnderson Apr 26, 2018
79ae7f3
Moved XDEBUG arg back below FROM
TomHAnderson Apr 26, 2018
7954365
Re-added COMPOSER_ARGS
TomHAnderson Apr 26, 2018
7a07548
Implemented all ideas from @webimpress
TomHAnderson Apr 27, 2018
b6f19ce
Put brackets back around variables
TomHAnderson Apr 27, 2018
f91132a
Put brackets back around variables
TomHAnderson Apr 27, 2018
8173b69
Move docker-compose to dist; move entrypoint to file; add ascii art
TomHAnderson Apr 27, 2018
acf0cf0
Made entrypoint.sh executable
TomHAnderson Apr 27, 2018
3043a23
Added ascii art
TomHAnderson Apr 27, 2018
a7c0657
Updated docs and ascii art
TomHAnderson Apr 27, 2018
142233a
Removed .dist in favor of Docker override
TomHAnderson Apr 28, 2018
4469bfb
Add override to .gitignore
TomHAnderson Apr 30, 2018
1090652
Added install and update hooks for comopser for MongoDB Shim
TomHAnderson Apr 30, 2018
78cdf71
Added JSON_UNESCAPED_SLASHES
TomHAnderson Apr 30, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@ test/data/
test/assets/module/*/config/module.config.php
clover.xml
coveralls-upload.json
docker/data/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see any usage of this folder, can we remove it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was a remnant; removed

3 changes: 3 additions & 0 deletions .travis.docker
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
cd /docker
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With previous changes, you don't need this anymore

echo `host mongo` | awk '{print $4, "localhost"}' > /etc/hosts
vendor/bin/phpunit
6 changes: 6 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ language: php

services:
- mongodb
- docker

cache:
directories:
Expand Down Expand Up @@ -59,17 +60,21 @@ matrix:
env:
- DEPS=lowest
- MONGO_DRIVER=mongodb
- DOCKER=true
- php: 7.2
env:
- DEPS=locked
- MONGO_DRIVER=mongodb
- DOCKER=true
- php: 7.2
env:
- DEPS=latest
- MONGO_DRIVER=mongodb
- DOCKER=true

before_install:
- if [[ $TEST_COVERAGE != 'true' ]]; then phpenv config-rm xdebug.ini || return 0 ; fi
- if [[ $DOCKER == 'true' ]]; then docker-compose up -d; fi
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need this


install:
- yes '' | pecl -q install -f $MONGO_DRIVER
Expand All @@ -85,6 +90,7 @@ install:
script:
- if [[ $TEST_COVERAGE == 'true' ]]; then composer test-coverage ; else composer test ; fi
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should use docker here too. We need to install xdebug on the docker image and then we can run tests with coverage:

$ docker-compose run --rm php ./vendor/bin/phpunit --colors=always --coverage-clover clover.xml

or, for test only:

$ docker-compose run --rm php ./vendor/bin/phpunit --colors=always

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Coverage is only tested on one matrix element. The output of running coverage on Docker would not be used. I don't think Docker coverage is necessary. Is there a place the output would be used I'm not seeing?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@TomHAnderson We need to find a way to load xdebug extension only for coverage test.

I think should be better to create a .docker directory to put there docker files as @webimpress suggested.

Then we could:

  • Write the entry point file and use COPY or ADD in the Dockerfile, without to write it "on the fly" building the image
  • Install xdebug extension, without enabling it
  • Create sh commands and copy them in the Dockerfile image

Then we can use a test-with-coverage script that will enable xdebug first:

#!/bin/sh

docker-php-ext-enable xdebug \
&& ./vendor/bin/phpunit --colors=always --coverage-clover clover.xml

What do you think?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand the need for coverage inside Docker. What benefit do we get?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@TomHAnderson My idea is that if we are implementing docker, we should use it for everything.

It's not just a coverage, but are tests too. Right now tests are executed 2 times. First in the travis-ci environment (that it's different from the container) with coverage, then from the container.

I think we should use every command from the container, even composer install and php-coveralls.

After all these changes, we don't need to do anything on travis-ci environment, and then we can use a simple image on travis.

It's just my opinion and a suggestion :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand that, thanks.

In this same thread @Ocramius suggests doing inside (Docker) and out (Travis) testing: #308 (comment)

@webimpress I need sign-off from you too before I'm going to go with a Docker-only approach to unit tests.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get the reasoning from @Ocramius , and generally agree. The idea should be to run the tests via docker always, for every environment, and not using the travis-ci environment; the rationale is to ensure that the CI environment mimics the environment maintainers use.

That said: only do it if it can be feasibly setup. If this is going to take you days to accomplish, then I'm not 100% convinced it's worth it; in that case, the docker setup is simply so contributors do not need to have the required services (mongo, RDBMS systems, etc) running on their own machine, nor need to configure a phpunit.xml with the specifics of their own services.

If it is possible to do easily, let's do it. Use build arguments to do things like enable xdebug and indicate test coverage should be run.

- if [[ $CS_CHECK == 'true' ]]; then composer cs-check ; fi
- if [[ $DOCKER == 'true' ]]; then docker exec -it $(docker ps -f name=zfapigilitydoctrine_php_1 | awk '{print $1;}' | tail -n 1) bash /docker/.travis.docker; fi
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can directly run:
docker-compose run --rm php with the Docker file in my previous comment


after_script:
- if [[ $TEST_COVERAGE == 'true' ]]; then travis_retry php vendor/bin/php-coveralls -v ; fi
Expand Down
57 changes: 57 additions & 0 deletions DOCKER.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Using Docker for Development

This library requires a running instance of Mongo in order to run and pass
the unit tests. It is not expected for each developer to configure their
individual machine to match this environment so Docker is provided.


## Running docker-compose

You will need docker-compose installed on your machine.

From the root directory of this project run

```
docker-compose build
```

This will build the php container. Next run

```
docker-compose up
```

This will spin up the php container and a mongo container.

To connect to the php container and run the unit tests
run

```
docker/connect
```

You will connect to the php container a the root directory.
`cd` to `docker` to work with the mapped local files.


## Configuration

Because you're in a Docker environment with a differnet IP address and name for each
container you need to either change the config files in the project to point to `mongo`
or with a simple hack map `localhost` to the `mongo` container.

* To edit the config file to point to the `mongo` container instead of `localhost` edit
`test/config/ODM/local.php` and change the configuration.

* Optionally you can map localhost to mongo with
```
echo `host mongo` | awk '{print $4, "localhost"}' > /etc/hosts
```

## Unit Tests

Having run `composer install` you may now run the unit tests

```
vendor/bin/phpunit
```
15 changes: 15 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
version: "3"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docker-compose cam be changed with:

version: "3"
services:
  php:
    build:
      context: .
      args:
        - PHP_VERSION=${PHP_VERSION:-7.2}
      dockerfile: docker/Dockerfile
    depends_on:
      - mongo
    volumes:
      - ./:/docker

  mongo:
    image: mongo:latest

You can optionally pass PHP_VERSION as environment variable in .travis.yml, using it for different PHP versions

services:
php:
build:
context: .
dockerfile: docker/Dockerfile
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should have .docker directory instead. I've seen it somewhere and I think it's better convention.
In other repos we usually uses .ci dir for travis config.

links:
- mongo
ports:
- "9000:9000"
volumes:
- ./:/docker
tty: true
mongo:
image: mongo:latest
11 changes: 11 additions & 0 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
FROM php:latest
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use an alpine image to speed up tests, something like this:

ARG PHP_VERSION=7.2
FROM php:${PHP_VERSION}-alpine

RUN apk add --no-cache \
        autoconf \
        make \
        g++
RUN pecl install mongodb && docker-php-ext-enable mongodb
RUN set -o pipefail && curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
RUN echo -e '#!/bin/sh' >> /usr/local/bin/entrypoint.sh \
    && echo -e 'while ! nc -z ${MONGO_HOST:-mongo} ${MONGO_PORT:-9000}; do sleep 1; done' >> /usr/local/bin/entrypoint.sh \
    && echo -e 'exec "$@"' >> /usr/local/bin/entrypoint.sh \
    && chmod +x /usr/local/bin/entrypoint.sh
WORKDIR /docker
ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
CMD ["./vendor/bin/phpunit"]

You can specify a build argument to say which image to build.
I also added an entry-point to wait mongodb service to be up.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Obviously you should change code where mongodb host is localhost, using environment variables $MONGO_HOST and $MONGO_PORT 

RUN apt-get update
RUN apt-get --yes install vim
RUN apt-get --yes install git
RUN apt-get --yes install inetutils-ping
RUN apt-get --yes install host
RUN pecl install mongodb && docker-php-ext-enable mongodb
RUN php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
RUN php composer-setup.php
RUN php -r "unlink('composer-setup.php');"
RUN mv composer.phar /bin/composer
1 change: 1 addition & 0 deletions docker/connect
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
docker exec -it $(docker ps -f name=zfapigilitydoctrine_php_1 | awk '{print $1;}' | tail -n 1) bash
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With previous changes, you don't need this

Empty file added docker/data/.gitkeep
Empty file.
35 changes: 26 additions & 9 deletions test/src/Admin/Model/DoctrineAutodiscoveryModelTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -29,17 +29,34 @@ public function testORMAutodiscoveryEntitiesWithFields()
$this->assertInternalType('array', $result);
$this->assertCount(3, $result);

$this->assertEquals(Album::class, $result[0]['entity_class']);
$this->assertEquals('Album', $result[0]['service_name']);
$this->assertCount(2, $result[0]['fields']);
$resultProcessed = [];
foreach ($result as $row) {
switch ($row['entity_class']) {
case Album::class:
$this->assertEquals('Album', $row['service_name']);
$this->assertCount(2, $row['fields']);
break;
case Artist::class:
$this->assertEquals('Artist', $row['service_name']);
$this->assertCount(2, $row['fields']);
break;
case Product::class:
$this->assertEquals('Product', $row['service_name']);
$this->assertCount(1, $row['fields']);
break;
default:
throw new \Exception("Unexpected result: " . $row['entity_class']);
}

$this->assertEquals(Artist::class, $result[1]['entity_class']);
$this->assertEquals('Artist', $result[1]['service_name']);
$this->assertCount(2, $result[1]['fields']);
if (isset($resultProcessed[$row['entity_class']])) {
$resultProcessed[$row['entity_class']] ++;
} else {
$resultProcessed[$row['entity_class']] = 1;
}
}

$this->assertEquals(Product::class, $result[2]['entity_class']);
$this->assertEquals('Product', $result[2]['service_name']);
$this->assertCount(1, $result[2]['fields']);
$this->assertEquals(3, sizeof($resultProcessed));
$this->assertEquals(3, array_sum($resultProcessed));
}

public function testODMAutodiscoveryEntitiesWithFields()
Expand Down