Improve automatic testing strategy

[ns-rse]

The use of GitFlow Workflow workflow to address the work that has been done on improving tracing and untangling molecules means that because the test suite we have in place to ensure changes do not break existing functionality and expected output do not run on pull requests to the target development branch (maxgamill-sheffield/800-better-tracing) since currently the workflow is setup to only run on pull requests that target the main branch...

.github/workflows/tests.yaml

name: Tests (pytest)

on:
  pull_request:
    branches:
      - main

This has led to problems as the changes introduced in this branch have broken a number of tests (see #850).

We should be catching these much earlier in the development cycle and there are a couple of approaches which I would like to consider.

• Run CI on all pull requests not just main. #922
• Run tests with pre-commit on only the changed/affected files using pytest-testmon. #923
• Speed up GH Action test suite by caching virtualenv #924

I would also advocate against the overly complicated GitFlow Workflow which uses development branches and instead think we should use either GitHub or GitLab Flow instead. I would personally favour the former (GitHub Flow as attempts so far to define clear sets of features and establish Milestones under which release branches can be developed in the Gitlab Flow approach.

To date we have mostly used the GitHub Flow approach of merging all features to main and periodically tagging commits to trigger releases. This approach requires that tasks are broken down into smaller components which makes development and feedback considerably faster.