ci: Run smoke tests in CI #16
Conversation
pkanal
added
no-changelog
| version: "17.5" | ||
| platform: "iOS" | ||
| device: "iPhone 15" |
There was a problem hiding this comment.
do we want to use some matrix tests to cover more versions/platforms/devices?
There was a problem hiding this comment.
if we do test on older iOS versions, we'll have to change the test expectations, because various MetricKit signals were added in different iOS versions. it's probably not trivial
There was a problem hiding this comment.
I think we should make this a follow up task, but overall yes this sounds like a good idea!
| environment: | ||
| COLLECTOR_VERSION: "0.111.0" | ||
| command: | | ||
| curl --proto '=https' --tlsv1.2 -fOL https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v${COLLECTOR_VERSION}/otelcol_${COLLECTOR_VERSION}_darwin_arm64.tar.gz |
There was a problem hiding this comment.
I was going to suggest using docker or caching the download could be good but the smoke tests take 10mins to run so probably not worth it 😂
There was a problem hiding this comment.
That was the original plan 😭 but I found it was really hard to get docker working on the CI mac machine and the collector setup itself is super fast. The thing that takes the longest is running the iOS app, which we can probably make faster with some CircleCI tricks.
pkanal
force-pushed
the
purvi/ci-smoke-tests
branch
from
336d743
to
b49f871
Compare
Which problem is this PR solving?
Run smoke tests in CI on every branch/PR and nightly on main.
Short description of the changes
How to verify that this has the expected result