Benchmark importing all standard library modules #338
Conversation
Note this will change if we add or remove modules from the standard library. We could alternatively hardcode a list of standard library modules.
There was a problem hiding this comment.
This is a good idea in general -- the standard library is likely to cover a lot of ground, and is definitely "real code people actually use".
However, this benchmark is likely to change even when the contents of the stdlib changes (not just when files are added/removed). So I think the only way to make a stable benchmark out of this would be to actually include a copy of the stdlib here. That's probably reasonably forward-compatible to new versions of Python (at least as much as any random library).
Given that would be kind of a weird thing to do, I guess I'd like to get some other feedback to see what others think before merging this.
|
As a user, I definitely care about stdlib import time, so in my mind that was maybe more of a feature. |
|
You make some good points. I think there's a couple of different frames of reference here. If the goal is "how fast is CPython at importing all this code?", copying the code makes sense. If the goal is "how fast does the stdlib take to import for CPython version X.Y", this PR as-is makes a lot of sense. I suppose as long as we document what it's good for to avoid confusion, that's fine. (We don't really have a good place to document those kind of things yet, but a comment in the |
| @@ -0,0 +1,29 @@ | |||
| import os | |||
There was a problem hiding this comment.
| import os | |
| """ | |
| Measures the time it takes to import all of the modules in the stdlib. | |
| This benchmark is expected to change as the stdlib changes, so is not | |
| suitable for measuring compilation time. | |
| """ | |
| import os |
Note this will change if we add or remove modules from the standard library. We could alternatively hardcode a list of standard library modules.