I want to know exactly how different programming languages perform. However, nowadays, there aren't many open projects on the Internet for this purpose.
This repository currently supports comparisons between Scheme and JavaScript.
- Nix is a tool that takes a unique approach to package management and system configuration. I use it to create a reproducible environment for running benchmarks. You may need to install it on your system.
- flake provides a standard way to write Nix expressions (and therefore packages) whose dependencies are version-pinned in a lock file, improving reproducibility of Nix installations. We use it to manage environments and run benchmarks
- The
time
command. The Bash-defaulttime
command is a shell keyword. This project requires the realtime
command, which some Linux distributions don't install by default. On my NixOS laptop, I install thetime
command with:
environment.systemPackages = [
pkgs.time
];
bash run.sh
All results are in the ./output
directory.
Note that
run.sh
does not really test all languages, due to that quickjs will failed on serveral benchmarks. To test all languages, you can runnix-bench all
instead.
To run benchmarks for a specific implementation, please use
bash nix-bench "<lang>/<impl>"
The result will be in ./output/once-time/
directory. All available implementations are listed with bash nix-bench --list
.
If you don't want to use Nix, you can use bench
instead. That will try to call the existed binary executable on your system.
- javascript/nodejs
- javascript/bun: I don't know why Bun achieves such fast results. Google suggests that it uses some unsafe behaviors, so by default, I won't rank it as the winner in the following table. But if you want Bun's performance, you can refer to here.
- javascript/quickjs: Thanks to Flamme Shadow, he contributes this environment to this repository. Though he mentioned in this pr that many benchmarks couldn't work, personally, I think his experience makes a baseline: if any languages cannot reach it, applying them in actual practice should be considerated more than two times.
- scheme/chezscheme
- scheme/guile
- python/pypy
- python/cpython3.13 & python/cpython3.9: Python has significant performance improvements since 3.10, so two different versions are chosen to compare performance.
All benchmarks are listed in the src directory. I tried my best to ensure that different languages consistently perform similar operations on the same tasks. All benchmarks are migrated from here, and I'll gradually add others.
Benchmark | Description | NOTE |
---|---|---|
ack | A recursive function with more than one parameter, it seems to evaluate Scheme's tail-call optimization. | For many languages like JavaScript, tail-call optimization is usually not implemented. To do a detailed comparison, I call the ack function as ack(3,10) , which is too small to showcase Scheme's advantage. |
primes | A recursive function to calculate prime numbers. | For many languages like JavaScript, tail-call optimization is usually not implemented. |
string | String operations mainly on concatenation and splitting. | Different languages often have specific details in string handling, leading to significant performance differences for similar operations. |
sum | Sum up integers from 1 to n using recursion. | |
sumfp | Sum up the float representation of integers from 1.0 to n using recursion. | |
sumfp-ignore-setuptime | Performs the sum operation many times to ignore setup time. | Chez Scheme outperforms NodeJS, but clearly doesn't compete with Bun. Now, my question is: how does Bun do it? |
fib | Fibonacci calculation. | |
nqueens | N-queen problem. | |
triangl | I'm not sure what this does. | |
deriv | List derivation for symbolic computation. | |
deriv-ignore-setuptime | Performs list derivation many times to ignore setup time. | |
cpstak | CPS (continuation-passing style) computation. | |
cpstak-ignore-setuptime | Performs the cpstak benchmark many times to ignore setup time. |
- Chez Scheme seems to perform well with recursion. Techniques like tail-call optimization really work.
- In my view, for those benchmarks that take less than 0.06 seconds in Chez Scheme, setup time really matters.