Persistent storage for module extension evaluation.

[aignas]

Description of the feature request:

This has been discussed in this Slack thread
And it is outlined in this proposal document

Which category does this issue belong to?

This is about bzlmod extension evaluation.

What underlying problem are you trying to solve with this feature?

I would like to store URL, sha256 values that I can fetch from the PyPI index. That is something that is a by product of the extension evaluation and I would like to reuse that in future evaluations.

---

Just to be clear, my intention would be to use this in rules_python when we download these facts available through PyPI server:

1. e.g. Distribution a for version b has artifacts downloadable at X with hash Y.
2. Artifacts downloadable at X have further METADATA files associated with other hashes.

I would like to mark the extension as reproducible so that only the facts are persisted in the MODULE.bazel.lock file. For my usecase arch/os dependent facts are not needed.

[fmeum]

This could be as simple as (expanding on what I wrote in the doc back then):

1. adding a new facts parameter to module_ctx.extension_metadata that accepts an arbitrary Starlark value that can be converted to JSON. This is persisted in the lockfile, even for reproducible extensions (in that case, only the facts). Facts can help make more extensions reproducible. In lockfile mode error, a reproducible extension is expected to produce the exact same fact object that it received, otherwise it is considered out of date and fails.
2. if a lockfile entry for the extension (and OS/arch, if dependent on those) exists and its bzlTransitiveDigest matches (but not necessarily its usages or recorded inputs hashes), provide that object back to the implementation in module_ctx.facts.

Extensions are expected to only store universally true statements in facts, e.g. the immutable foo registry package [email protected] has download URL X and hash Y. It's essentially a generic go.sum file.

@Wyverald @meteorcloudy

Edit: Edited to better explain how reproducible extensions work with facts.

[aignas]

Added extra information to the description about the usecase. I think @fmeum has described what I need.