[test-runner] Cannot read property 'decl' of undefined in coverage UI

[web-padawan]

Two files in the PR for vaadin-date-picker have this problem:

Cannot read property 'decl' of undefined
TypeError: Cannot read property 'decl' of undefined
    at /Users/sergey/vaadin/vaadin-date-picker-new/node_modules/istanbul-reports/lib/html/annotator.js:89:31
    at Array.forEach ()
    at annotateFunctions (/Users/sergey/vaadin/vaadin-date-picker-new/node_modules/istanbul-reports/lib/html/annotator.js:86:29)
    at annotateSourceCode (/Users/sergey/vaadin/vaadin-date-picker-new/node_modules/istanbul-reports/lib/html/annotator.js:234:9)
    at HtmlReport.onDetail (/Users/sergey/vaadin/vaadin-date-picker-new/node_modules/istanbul-reports/lib/html/index.js:409:33)
    at LcovReport. [as onDetail] (/Users/sergey/vaadin/vaadin-date-picker-new/node_modules/istanbul-reports/lib/lcov/index.js:28:23)
    at Visitor.value (/Users/sergey/vaadin/vaadin-date-picker-new/node_modules/istanbul-lib-report/lib/tree.js:38:38)
    at ReportNode.visit (/Users/sergey/vaadin/vaadin-date-picker-new/node_modules/istanbul-lib-report/lib/tree.js:88:21)
    at /Users/sergey/vaadin/vaadin-date-picker-new/node_modules/istanbul-lib-report/lib/tree.js:92:19
    at Array.forEach ()

As you can see the numbers are there:

But when opening the files, I get this:

[LarsDenBakker]

Seems related to istanbuljs/v8-to-istanbul#121

[mario-aleo]

This is an awkward error, when I run the test for all components in the project, I get this error on some of them, but when I run the test for only one of those components that have the error it generates the coverage report properly.

Running for all components in the project:

Running only for golfleet-filter-calendar:

I've noticed that the branch coverage isn't the same for the GolfleetFilterCalendar.js when the decl error happens and when it doesn't.

[LarsDenBakker]

It seems to happen when using typescript (or perhaps generally source maps). You can opt into doing coverage instrumentation with babel: https://modern-web.dev/docs/test-runner/writing-tests/code-coverage/#coverage-browser-support though this is slower.

[mario-aleo]

Thank you @LarsDenBakker, it did work, but it's kinda strange since I'm not using typescript in this project and it was running just fine with open-wc's testing-karma (I'm just upgrading my project dependencies to the new open-wc recommendations).

Also if anyone is going to try the babel aproach you may need to write it a little bit different as stated here #840.

[LarsDenBakker]

The karma config used babel for instrumenting code as well. We switched to using chromium for speed.

[LarsDenBakker]

@mario-aleo @web-padawan looks like we were not on the latest version of the library. Can you try updating to the versions listed here: #935 and see if the problem is still there? And if so report a reproduction to istanbuljs/v8-to-istanbul#121 ?

[mario-aleo]

@LarsDenBakker, sorry for the late response.
I've just tried and didn't work, as I've reported at istanbuljs/v8-to-istanbul#121, I'll create a repository with this error so you can easily reproduce it :)

[dwilson6]

We just ran into this on a project set up with snowpack, typescript, svelte and web test runner. We have svelte component A that imports svelte component B which in turn imports typescript module C. We had 100% code coverage and then component B and module C were moved to a subdirectory inside the dir where component A is. After that module C's code coverage is 71% for functions despite no changes in the file or the test suite.

When we look at the test report, we see the Cannot read property 'decl' of undefined error. After some investigation I noticed that if I ran the test suite for component A before module C, the error happens but if I run module C's tests first and then the ones for component A, there is no error.

I tried to create a simpler setup to reproduce the error but was unable to reproduce it. It is also worth mentioning that there are some issues with snowpack and svelte sourcemaps which cause the html report to not work for svelte components but this is a typescript file seeing the issue so I am not sure if the svelte issue is to blame.

[stephenwade]

@web-padawan @mario-aleo @LarsDenBakker I ran into this issue today with @web/test-runner. I was able to narrow it down to one class file and two test files.

If any of these characteristics are changed, the coverage report works properly:

• The class function assigns an anonymous function to an instance property.

export class MyClass {
  fn() {
    this._ = function () {};
  }
}

• There are two test files, both of which import the same class file.
• Only one of the test files calls the class function.

https://github.com/stephenwade/coverage-issue-repro

To reproduce, run npm test and open coverage/lcov-report/my-class.js.html.

[Stuk]

I have a minimal repro repository here: https://github.com/Stuk/wtr-coverage-repro. The details of what I believe causes this issue are listed in the readme.

To summarize, metadata for functions in a file is only collected the first time that file is loaded, however some methods might only be defined when more code executed. Hence the above error when trying to access metadata for a covered function that doesn't always exist.