Tests for REST controllers fail when the size of the test data set exceeds the page request size in PageRequestVM

[GeraldScott]

Describe the bug
Tests for REST controllers fail when the size of the test data set exceeds the page request size in PageRequestVM when pagination is implemented.

The tests fail because they use the size of the array returned by the .get("/api/<%= entityApiUrl %>") clause as a proxy for the size of the dataset in the database.

This is not correct because the size of the array is limited to the page request size in PageRequestVM if pagination is implemented.

If the number of test data items exceeds the page request size (default is 50), the tests that use the array fail because the size is either larger than expected, or the entity cannot be found because it is outside the page that is returned.

To Reproduce

1. Create a test dataset that is larger than the page request size in PageRequestVM (by default this is 50, so create a dataset with more than 50 entries)
2. Run the REST controller test on the entity
3. The update, create and delete tests fail

Expected behavior
All tests should pass irrespective of the test dataset size.

Screenshots
For example, the REST test for delete failes with:

java.lang.AssertionError: 
Expected size: 49 but was: 50 in:
[Gender{id=1, code='Toys Viaduct proactive', description='Incredible'},
    Gender{id=2, code='Cotton', description='Communications Tasty'},
    Gender{id=3, code='Kids', description='haptic generation'},
    Gender{id=4, code='sensor Account', description='Sausages Home'},

Desktop (please complete the following information):
Not relevant

Smartphone (please complete the following information):
Not relevant

Additional context
None

[GeraldScott]

The pagination shows the correct number of items in the HTML. It gets the total from the response header (see this.totalItems = Number(headers.get('X-Total-Count'));), so this could be used in the tests that check the number of database entries instead of testing the size of the array that is returned by the /get/api.

Note also that this problem also affects tests that check for the existence of an entity by getting the whole array and then searching it for the entity. If the entity is outside the page boundary, the test fails even though it is actually in the database.

[mraible]

Added bug bounty to get this fixed. Please submit a PR if you have time.

[emilpaw]

I am currently working on this issue.

Using X-Total-Length instead of the length of the response array for determining the count of entities is straightforward.

The thing about retrieving the entity from a paged response is more tricky.
An alternative way to test that an entity was successfully added and the count of entities increased by one correctly would be to get the entity by ID and do another request to get the total count of entities.

e.g.
Current implementation, but I moved the logic to get the total count from the request to the function getDatabaseSize().

    @Test
    public void createMyEntityA() {
        var databaseSizeBeforeCreate = getDatabaseSize();

        // Create the MyEntityA
        myEntityA =
            given()
                .auth()
                .preemptive()
                .oauth2(adminToken)
                .contentType(APPLICATION_JSON)
                .accept(APPLICATION_JSON)
                .body(myEntityA)
                .when()
                .post("/api/my-entity-as")
                .then()
                .statusCode(CREATED.getStatusCode())
                .extract()
                .as(ENTITY_TYPE);

        // Validate the MyEntityA in the database
        var myEntityAList = given()
            .auth()
            .preemptive()
            .oauth2(adminToken)
            .accept(APPLICATION_JSON)
            .when()
            .get("/api/my-entity-as")
            .then()
            .statusCode(OK.getStatusCode())
            .contentType(APPLICATION_JSON)
            .extract()
            .as(LIST_OF_ENTITY_TYPE);

        assertThat(myEntityAList).hasSize(databaseSizeBeforeCreate + 1);
        var testMyEntityA = myEntityAList.stream().filter(it -> myEntityA.id.equals(it.id)).findFirst().get();
        assertThat(testMyEntityA.fieldName).isEqualTo(DEFAULT_FIELD_NAME);
    }

Alternative:

    @Test
    public void createMyEntityA() {
        var databaseSizeBeforeCreate = getDatabaseSize();

        // Create the MyEntityA
        myEntityA =
            given()
                .auth()
                .preemptive()
                .oauth2(adminToken)
                .contentType(APPLICATION_JSON)
                .accept(APPLICATION_JSON)
                .body(myEntityA)
                .when()
                .post("/api/my-entity-as")
                .then()
                .statusCode(CREATED.getStatusCode())
                .extract()
                .as(ENTITY_TYPE);

        // Validate the MyEntityA in the database
        var testMyEntityA = given()
            .auth()
            .preemptive()
            .oauth2(adminToken)
            .accept(APPLICATION_JSON)
            .when()
            .get("/api/my-entity-as/{id}", myEntityA.id)
            .then()
            .statusCode(OK.getStatusCode())
            .contentType(APPLICATION_JSON)
            .extract()
            .as(ENTITY_TYPE);

        var databaseSizeAfterCreate = getDatabaseSize();

        assertThat(databaseSizeAfterCreate).isEqualTo(databaseSizeBeforeCreate + 1);
        assertThat(testMyEntityA.fieldName).isEqualTo(DEFAULT_FIELD_NAME);
    }

This would introduce an extra request, to get databaseSizeAfterCreate though.

Thoughts on this?