Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide a standard mechanism for Unit testing modules #300

Closed
jsbrittain opened this issue Jul 9, 2024 · 0 comments
Closed

Provide a standard mechanism for Unit testing modules #300

jsbrittain opened this issue Jul 9, 2024 · 0 comments
Assignees
Labels
enhancement New feature or request

Comments

@jsbrittain
Copy link
Collaborator

jsbrittain commented Jul 9, 2024

Is your feature request related to a problem? Please describe.
There is no testing strategy in place for grapevne modules (as opposed to the grapevne application which is tested). Modules are user-provided units of work that reside on a range of github repositories. The testing stategy should therefore be flexible and scalable.

Describe the solution you'd like
Modules should be self-testing to provide quality assurance and allow periodic checks. Standard entry point should be prescribed under the grapevne eco-system, with software solutions made available to install in repositories for users to test their own modules.

A workable solution would be the following:

  • Standard entry point for testing (module-side):
    • Provide a .test.sh script to execute any tests. This provides flexibility for module-specific factors, or could simply remove old output files before executing the tests.
    • The recommended approach to self-test is the following:
      • Provide a _test rule as part of the module's Snakemake definition. The _test rule can simply list the required inputs (output of the main rule(s)), and can include additional logic to validate the contents of these files.
      • Provide a config/.test.yaml file that is configured for testing purposes (include any results/in files that are required within the repository)
  • Provide repository mechanisms to launch module tests when files are modified (e.g. github actions)
    • This should isolate modules that have been added/amended and only run tests on the relevant modules (designed to run on PRs).
    • initially a script & a local github action definition file [migrating to a fully-fledged and installable github action for ease of installation]

This issue is largely resolved by kraemer-lab/vneyard#26, although the testing strategy also needs to be recorded in the grapevne documentation. The documentation is due to be refreshed as part of #245 .

@jsbrittain jsbrittain added the enhancement New feature or request label Jul 9, 2024
@jsbrittain jsbrittain self-assigned this Jul 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant