You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
There is no testing strategy in place for grapevne modules (as opposed to the grapevne application which is tested). Modules are user-provided units of work that reside on a range of github repositories. The testing stategy should therefore be flexible and scalable.
Describe the solution you'd like
Modules should be self-testing to provide quality assurance and allow periodic checks. Standard entry point should be prescribed under the grapevne eco-system, with software solutions made available to install in repositories for users to test their own modules.
A workable solution would be the following:
Standard entry point for testing (module-side):
Provide a .test.sh script to execute any tests. This provides flexibility for module-specific factors, or could simply remove old output files before executing the tests.
The recommended approach to self-test is the following:
Provide a _test rule as part of the module's Snakemake definition. The _test rule can simply list the required inputs (output of the main rule(s)), and can include additional logic to validate the contents of these files.
Provide a config/.test.yaml file that is configured for testing purposes (include any results/in files that are required within the repository)
Provide repository mechanisms to launch module tests when files are modified (e.g. github actions)
This should isolate modules that have been added/amended and only run tests on the relevant modules (designed to run on PRs).
initially a script & a local github action definition file [migrating to a fully-fledged and installable github action for ease of installation]
This issue is largely resolved by kraemer-lab/vneyard#26, although the testing strategy also needs to be recorded in the grapevne documentation. The documentation is due to be refreshed as part of #245 .
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
There is no testing strategy in place for grapevne modules (as opposed to the grapevne application which is tested). Modules are user-provided units of work that reside on a range of github repositories. The testing stategy should therefore be flexible and scalable.
Describe the solution you'd like
Modules should be self-testing to provide quality assurance and allow periodic checks. Standard entry point should be prescribed under the grapevne eco-system, with software solutions made available to install in repositories for users to test their own modules.
A workable solution would be the following:
.test.sh
script to execute any tests. This provides flexibility for module-specific factors, or could simply remove old output files before executing the tests._test
rule as part of the module's Snakemake definition. The_test
rule can simply list the required inputs (output of the main rule(s)), and can include additional logic to validate the contents of these files.config/.test.yaml
file that is configured for testing purposes (include anyresults/in
files that are required within the repository)This issue is largely resolved by kraemer-lab/vneyard#26, although the testing strategy also needs to be recorded in the grapevne documentation. The documentation is due to be refreshed as part of #245 .
The text was updated successfully, but these errors were encountered: