Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend existing unit tests using cover-agent #2408

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

mrT23
Copy link

@mrT23 mrT23 commented Nov 13, 2024

The PR content was generated automatically using cover-agent

Methodology

Cover-agent automatically scans a repository to detect existing unit test files. It gathers relevant context for each file, then triggers an AI-based workflow to enhance these tests by adding new cases that increase code coverage and address uncovered behaviors.

Reliability

But it's AI-generated code. How can we know it's reliable ? How can we know it's effective ?

Answer:

All AI-generated tests have met these four essential criteria:

  1. Execute successfully
  2. Pass all assertions
  3. Increase overall code coverage
  4. Test previously uncovered behaviors (as specified in the LLM prompt)

Increased coverage

Code coverage comparison of the relevant source files before and after these changes:
image

In total, 129 new lines were covered with the tests added in this PR

Appendix: unit tests - what are they good for?

Answer:

Personal opinion - writing unit testing is not fun. It becomes even less appealing as your codebase grows and maintaining tests becomes a time-consuming chore.

However, the benefits of comprehensive unit tests are real:

  • Reliability: They create a more reliable codebase where developers can make changes confidently

  • Speed: Teams can move quickly without fear of breaking existing functionality

  • Safe Refactoring: Code improvements and restructuring become significantly safer when backed by thorough tests

  • Living Documentation: Tests serve as clear documentation of your code's behavior:

    • They show exactly what happens for each input
    • They present changes in a human-readable format: "for this input → expect this output"
    • They run quickly and are easy to execute
    • This immediate feedback loop is beneficial during development

@mrT23 mrT23 changed the title Extend existing unit tests Extend existing unit tests using cover-agent Nov 13, 2024
@nfcampos
Copy link
Contributor

hi, I'm not opposed to merging some of these, but there's some things you'll have to fix before you can do that

  • files are not formatted using our formatter
  • you're adding an anyio marker in some tests, but we add it top-level in the file so not needed

@mrT23
Copy link
Author

mrT23 commented Nov 14, 2024

@nfcampos
thanks for the quick feedback.

i passed the files through a linter, and removed direct usage of anyio marker

(I will also use this feedback to further improve the auto tool, giving him more 'encouragement' to maintain the current used style)

@mrT23
Copy link
Author

mrT23 commented Nov 15, 2024

@nfcampos
Can you run the checks on the updated version ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants