Add integration test with Manifolds

[sethaxen]

We should have an integration test with Manifolds, that is, an action that runs Manifolds's test suite using the latest changes to ManifoldsBase. This will ensure that potentially breaking changes are identified before being merged. This workflow is really nice: JuliaDiff/ChainRulesTestUtils.jl#105

[kellertuer]

This means on a PR we run a complete test suite on Manifolds? That would increase test time like a lot. What do you think about making it optional (similar to documentation building over on Manifolds.jl)? Or does that act against what the integration should do?

[sethaxen]

No, it would be a separate workflow. See for example the one linked above. So it would not alter the time it takes to run the current test suite, and we would only need to pay attention to the integration test result before merging a PR. It provides extra security against making breaking changes, and for that reason, I don't think it should be optional.

[kellertuer]

Ok, but what I mean is, if you open a PR, then all tests run, including the integration test. All tests here in total take about 4 minutes or something, but the integration test would run the whole Manifolds.jl tests, right? So that test would take 35 minutes.
That's why I asked. Since I did the adaption for Manopt integration with Manifolds, for sure I can easily do this one, too. I just wanted to clarify this point beforehand.

[sethaxen]

All tests here in total take about 4 minutes or something, but the integration test would run the whole Manifolds.jl tests, right? So that test would take 35 minutes.

That's right, yes.

[kellertuer]

Could. we maybe introduce a short integration test over on Manifolds that we could test against? I do not want to test all manifolds with all variants and different static arrays and such.

[kellertuer]

A short update: I started reworking our test suite at JuliaManifolds/Manifolds.jl#548 so that we avoid a few double checks we are currently doing, but also such that the tests get more modular. Then we can maybe also do an integration test setup (maybe testing some major manifolds Rn, Sphere, Hyperbolic, Stiefel, SPD?) which to together do not run too long and still provide a good integration test.

[mateuszbaran]

I personally wouldn't mind a long-running integration test in ManifoldsBase.jl but a smaller version would be better, yes.

[kellertuer]

I think it depends, when the integration takes the current hour the Manifolds.jl tests run compared to the few minutes ManifoldsBase.jl runs its a little annoying.