[Testing] Implementation of an updated testing framework

[dsikka]

Summary:

1. Update an existing unit test and an existing integration test, both for export such that they are using the unittest.TestCase base class and parameterized module
2. The integration tests are also updated to use config files to define the different combination of inputs that are meant to be tested for a particular integration test. Each config must include a cadence field, which dictates how frequently the test is run (set through an environment variables) and the test_type (sanity, regression, or smoke) which for now is a config variable but we can easily swap to just use decorators, as being done with the sample smoke test added
3. Introduce a base CustomIntegrationTest class which other integration tests can inherit and use to test custom scripts and custom test classes.
4. Add examples of decorators to mark unit , custom, and integration tests. Also introduce decorators to mark tests with certain package or compute environments requirements (e.g requires_torch, requires_gpu). These are described in a new testing_utils.py file

For a summary of the changes, see tests/docs/testing_framework.md

Testing

• An example of a class which uses the CustomIntegrationTest class is the TestGenerationExportIntegrationCustom and it can be found in test_generation_export.py. This test class was tested using sample config/python files added under tests/sparseml/export/transformers/generation_configs
• An existing integration test was updated to use config files (as mentioned above) and was tested using the newly added
  tests/sparseml/export/transformers/generation_configs/tiny_stories.yaml

Using unittest

• The existing tests have been updated to use unittest package. We can definitely keep pytest for the existing tests if there are strong feelings although I do find it easier to read and understand and we can easily use the package parameterized to get the same functionality (or even expanded functionality for classes, compared to pytest fixtures).
• Custom testing should keep the unittest framework as it makes testing custom scripts/tests fairly easy/straightforward

[bfineran]

is this meant to be a migration doc, or long standing demonstrating the new framework? if the latter probably don't need as much framing around what changed

[dsikka]

More so a summary of the new framework and what we want to establish going forward in terms of testing structure. Maybe makes more sense in internal docs as the target are MLE?

[bfineran]

any reason we wouldn't want to use relative path here? hardcoding this would require us to always run from sparseml/ not that this is not always the case though...

[dsikka]

yah I kept the path from sparseml as that is the directory used to run all gha tests

[rahul-tuli]

I'm cool with using a mix of unit test and pytest, specially cause pytest can pick up unittest style tests. Good work on the documentation, really appreciate it.