Method invocation count breaks when debugging in Java

[JCass149]

Describe the bug

When breakpoints are triggered using a Java debugger the method invocation count becomes broken.

To Reproduce

Running the following without a debugger attached passes:

    def "test"() {
        given:
        def myItem = Mock(Object)
        def myList = new ArrayList()
        myList.add(myItem)

        when:
        myList.toString()

        then:
        1 * myItem.toString()
        noExceptionThrown()
    }

However, running the code with a debugger attached, with breakpoints at each line, fails returning the following:

Too many invocations for:

1 * myItem.toString() (2 invocations)

Matching invocations (ordered by last occurrence):

2 * myItem.toString() <-- this triggered the error

Expected behavior

The test to pass when both debugging and not debugging.

Actual behavior

The test fails.

Java version

$ java -version
openjdk version "11.0.2" 2019-01-15
OpenJDK Runtime Environment 18.9 (build 11.0.2+9)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.2+9, mixed mode)

Buildtool version

$ mvn --version
Apache Maven 3.8.1

What operating system are you using

Windows

Dependencies

...
+- org.spockframework:spock-core:jar:2.1-groovy-3.0:test
| - org.junit.platform:junit-platform-engine:jar:1.8.2:test
| +- org.opentest4j:opentest4j:jar:1.2.0:test
| +- org.junit.platform:junit-platform-commons:jar:1.8.2:test
| - org.apiguardian:apiguardian-api:jar:1.1.2:test
+- org.spockframework:spock-spring:jar:2.1-groovy-3.0:test
...
+- org.codehaus.groovy:groovy:jar:3.0.12:test
- org.codehaus.groovy:groovy-sql:jar:3.0.12:test

Additional context

I attached the debugger using the IntelliJ debug tool. The debug command:

-agentlib:jdwp=transport=dt_socket,address=127.0.0.1:<port>,suspend=y,server=n -javaagent:<path>/IntelliJIdea2022.2/captureAgent/debugger-agent.jar

IntelliJ version:

IntelliJ IDEA 2022.2.3 (Ultimate Edition)

pom.xml plugin config:

            <plugin>
                <!-- gmavenplus plugin is used to compile Groovy code -->
                <groupId>org.codehaus.gmavenplus</groupId>
                <artifactId>gmavenplus-plugin</artifactId>
                <version>1.13.1</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>addTestSources</goal>
                            <goal>compile</goal>
                            <goal>compileTests</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

[leonard84]

Well, debuggers usually call toString() on the variables, so it is just reflecting what is actually happening. What would you expect happening, when you manually evaluate something in the debugger, should that also not trigger the mocking behavior?

[JCass149]

Thanks for your response @leonard84

toString() was just an example. I’ve found it happens for other methods too.

I would expect either:

• method invocations to count the same whether breakpoints are triggered or not, or
• the docs to clarify that method invocation counts may differ if a breakpoint is triggered

I don’t know anything about how debuggers work, so perhaps this is expected behaviour.

[leonard84]

Do you have an example that doesn't involve the toString() special case?