Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up CI by caching compiled WASMs in Github Actions cache #952

Open
vigoo opened this issue Sep 19, 2024 · 2 comments
Open

Speed up CI by caching compiled WASMs in Github Actions cache #952

vigoo opened this issue Sep 19, 2024 · 2 comments
Labels
build-ci Build and CI related tasks

Comments

@vigoo
Copy link
Contributor

vigoo commented Sep 19, 2024

We could probably significantly speed up the CI test tasks by caching the test component's compiled native images in a Github Action cache.

This is not straightforward to do, as we cannot use Golem's built-in compilation cache - it is per component, and many tests are intentionally creating new components even though they use the same source WASM file.
So this should be some kind of wasm hash-based file system cache, enabled only for tests, and explicitly saved/restored on CI.

Note: The wasmtime CLI supports something like this (https://docs.wasmtime.dev/cli-cache.html), so maybe there is something available in wasmtime that makes implementing this easy - but even if not, it can be done the same way as the live golem compilation cache works.

@vigoo vigoo added the build-ci Build and CI related tasks label Sep 19, 2024
@vigoo
Copy link
Contributor Author

vigoo commented Sep 19, 2024

The cache key for this must be separate and only calculated from the test component WASMs, which are changing relatively infrequently.

@ologbonowiwi
Copy link

The aim is for multiple pipelines to use the same built component, or from within a given run, should we use the component N times without recompiling it?

Mb about the possibly naive question, but I am currently trying to gain experience around this specific project/codebase

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build-ci Build and CI related tasks
Projects
None yet
Development

No branches or pull requests

2 participants