Compare CI/CD workflow results across multiple runs

[purva-thakre]

Based on the behaviors in #2442 and #2431, do we want to add a workflow that logs the results after each run and compares them?

Flaky tests are a given due to some of the unit tests utilizing randomly generated data. The current method of logging through screenshots is not ideal.

Rerunning the unit tests until they all pass is also not the best solution.

[natestemen]

I think given the frequency with which we have flaky tests pop up don't quite warrant an automated system that we then have to maintain.

The current method of logging through screenshots is not ideal.

I agree, especially given the accessibility concerns surrounding screenshots full of text. IMO a good way of logging these is to open an issue with

1. A permalink to the test that is failing
2. A link to the github actions workflow, with line number included
3. The actual error itself copy and pasted from the github actions, or better yet from a terminal where the bug was recreated.

IMO that's a good recipe, and relatively simple to put together when a flaky issue arises. WDYT?

[purva-thakre]

an automated system that we then have to maintain

I agree.