Automatically AOT compile dependencies (starting with clojure.core)

[jeaye]

This is the start of automatic AOT compilation for dependencies, which I think will be a very important feature for development iteration speed. Due to C++ compilation speeds, JIT compiling jank modules can be very slow. Loading them from AOT compiled modules, though, is a significant speedup. For a sense of scale, clojure.core takes 12 seconds to JIT compile, but after a 2 minute AOT compilation, it takes 0.3 seconds to load.

Now, I want us to AOT compile all dependencies, but the problem here is:

1. Knowing all dependencies
2. Knowing the dependencies of those dependencies (and building a graph out of this)

Generally, in Clojure, you can't know these things without evaluating the code. This is especially since a require call can be hidden anywhere. But maybe clj-kondo or other Clojure static analyzers get away with scraping this information from the common places and that'll be good enough. The reason why we want all of the information at once is that we need to parallelize the compilation of dependencies, rather than do it on demand. This is because each dependency may take multiple minutes to compile. So, the process would be:

1. Identify all dependencies (i.e. modules being required which aren't under the source paths)
2. Based on the dependency graph, compile all modules in parallel (may make sense to generate a Makefile from our graph and let make do the hard work)

AOT compiled dependencies can still have vars redefined and are still just as dynamic as if they were JIT compiled. So there should be no harm in doing all of this up front. The intended UX is that you need to wait a few minutes on your first run, or after updating your project's dependencies, but after that every startup will be very speedy.

Note that this is separate from AOT compilation of entire programs. We're just talking about jank's concept of modules here, which are loosely tied to Clojure namespaces.