Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use ankerl::unordered_dense::map instead of std::unordered_map in UnorderedMapStorage #6922

Open
wants to merge 35 commits into
base: master
Choose a base branch
from

Conversation

SiarheiFedartsou
Copy link
Member

@SiarheiFedartsou SiarheiFedartsou commented May 30, 2024

Benchmark Results

Benchmark Base PR
alias aliased u32: 1093.43
plain u32: 1098.18
aliased double: 958.18
plain double: 953.895
aliased u32: 1156.26
plain u32: 1141.77
aliased double: 1186.31
plain double: 1184.94
e2e_match_ch Ops: 43.52 ± 0.09 ops/s. Best: 43.35 ops/s
Total: 3009.58ms ± 6.07ms. Best: 3001.25ms
Min time: 2.16ms ± 0.04ms
Mean time: 22.98ms ± 0.05ms
Median time: 16.76ms ± 0.06ms
95th percentile: 76.00ms ± 0.60ms
99th percentile: 93.11ms ± 0.54ms
Max time: 96.66ms ± 0.61ms
Ops: 50.51 ± 0.06 ops/s. Best: 50.42 ops/s
Total: 2593.68ms ± 3.03ms. Best: 2588.96ms
Min time: 2.16ms ± 0.02ms
Mean time: 19.80ms ± 0.02ms
Median time: 16.01ms ± 0.06ms
95th percentile: 61.86ms ± 0.41ms
99th percentile: 76.62ms ± 0.69ms
Max time: 88.41ms ± 0.58ms
e2e_match_mld Ops: 62.13 ± 0.10 ops/s. Best: 61.99 ops/s
Total: 2108.46ms ± 3.20ms. Best: 2101.52ms
Min time: 1.84ms ± 0.04ms
Mean time: 16.09ms ± 0.02ms
Median time: 8.65ms ± 0.05ms
95th percentile: 53.41ms ± 0.17ms
99th percentile: 61.63ms ± 0.15ms
Max time: 71.46ms ± 0.10ms
Ops: 68.38 ± 0.20 ops/s. Best: 67.90 ops/s
Total: 1915.80ms ± 5.71ms. Best: 1910.08ms
Min time: 1.77ms ± 0.04ms
Mean time: 14.63ms ± 0.04ms
Median time: 7.94ms ± 0.15ms
95th percentile: 46.81ms ± 0.43ms
99th percentile: 54.61ms ± 0.13ms
Max time: 62.83ms ± 0.29ms
e2e_nearest_ch Ops: 803.39 ± 5.30 ops/s. Best: 791.79 ops/s
Total: 1244.64ms ± 8.38ms. Best: 1235.20ms
Min time: 1.06ms ± 0.01ms
Mean time: 1.24ms ± 0.01ms
Median time: 1.15ms ± 0.01ms
95th percentile: 1.64ms ± 0.01ms
99th percentile: 1.70ms ± 0.02ms
Max time: 4.28ms ± 2.52ms
Ops: 797.69 ± 6.35 ops/s. Best: 786.78 ops/s
Total: 1253.78ms ± 10.12ms. Best: 1237.27ms
Min time: 1.06ms ± 0.00ms
Mean time: 1.25ms ± 0.01ms
Median time: 1.16ms ± 0.01ms
95th percentile: 1.66ms ± 0.01ms
99th percentile: 1.77ms ± 0.09ms
Max time: 4.93ms ± 2.72ms
e2e_nearest_mld Ops: 796.71 ± 5.18 ops/s. Best: 784.28 ops/s
Total: 1255.17ms ± 8.49ms. Best: 1247.35ms
Min time: 1.07ms ± 0.01ms
Mean time: 1.26ms ± 0.01ms
Median time: 1.16ms ± 0.01ms
95th percentile: 1.66ms ± 0.01ms
99th percentile: 1.72ms ± 0.02ms
Max time: 4.66ms ± 2.90ms
Ops: 799.35 ± 7.95 ops/s. Best: 784.89 ops/s
Total: 1250.95ms ± 13.01ms. Best: 1229.96ms
Min time: 1.06ms ± 0.01ms
Mean time: 1.25ms ± 0.01ms
Median time: 1.16ms ± 0.01ms
95th percentile: 1.65ms ± 0.02ms
99th percentile: 1.70ms ± 0.02ms
Max time: 4.53ms ± 2.76ms
e2e_route_ch Ops: 338.37 ± 0.84 ops/s. Best: 337.22 ops/s
Total: 2955.62ms ± 7.34ms. Best: 2943.37ms
Min time: 1.28ms ± 0.01ms
Mean time: 2.96ms ± 0.01ms
Median time: 2.98ms ± 0.01ms
95th percentile: 3.91ms ± 0.02ms
99th percentile: 4.35ms ± 0.05ms
Max time: 7.02ms ± 2.16ms
Ops: 337.35 ± 1.08 ops/s. Best: 335.91 ops/s
Total: 2964.00ms ± 9.94ms. Best: 2942.82ms
Min time: 1.28ms ± 0.01ms
Mean time: 2.96ms ± 0.01ms
Median time: 2.98ms ± 0.01ms
95th percentile: 3.92ms ± 0.03ms
99th percentile: 4.35ms ± 0.08ms
Max time: 6.87ms ± 2.07ms
e2e_route_mld Ops: 281.88 ± 1.30 ops/s. Best: 279.82 ops/s
Total: 3547.91ms ± 16.34ms. Best: 3521.14ms
Min time: 1.27ms ± 0.01ms
Mean time: 3.55ms ± 0.02ms
Median time: 3.59ms ± 0.02ms
95th percentile: 4.85ms ± 0.04ms
99th percentile: 5.34ms ± 0.05ms
Max time: 7.72ms ± 1.50ms
Ops: 283.37 ± 1.61 ops/s. Best: 280.34 ops/s
Total: 3528.50ms ± 22.25ms. Best: 3490.20ms
Min time: 1.27ms ± 0.01ms
Mean time: 3.53ms ± 0.02ms
Median time: 3.57ms ± 0.02ms
95th percentile: 4.83ms ± 0.05ms
99th percentile: 5.35ms ± 0.08ms
Max time: 7.43ms ± 1.63ms
e2e_table_ch Ops: 315.88 ± 0.70 ops/s. Best: 314.51 ops/s
Total: 3165.68ms ± 7.64ms. Best: 3153.37ms
Min time: 1.68ms ± 0.03ms
Mean time: 3.17ms ± 0.01ms
Median time: 3.16ms ± 0.01ms
95th percentile: 4.35ms ± 0.02ms
99th percentile: 4.69ms ± 0.06ms
Max time: 7.79ms ± 2.41ms
Ops: 308.46 ± 0.76 ops/s. Best: 306.85 ops/s
Total: 3242.08ms ± 8.09ms. Best: 3232.11ms
Min time: 1.72ms ± 0.03ms
Mean time: 3.24ms ± 0.01ms
Median time: 3.25ms ± 0.02ms
95th percentile: 4.46ms ± 0.03ms
99th percentile: 4.80ms ± 0.03ms
Max time: 7.31ms ± 2.33ms
e2e_table_mld Ops: 109.20 ± 0.31 ops/s. Best: 108.47 ops/s
Total: 9157.52ms ± 27.44ms. Best: 9127.31ms
Min time: 3.72ms ± 0.02ms
Mean time: 9.16ms ± 0.03ms
Median time: 9.13ms ± 0.04ms
95th percentile: 13.99ms ± 0.06ms
99th percentile: 14.96ms ± 0.05ms
Max time: 16.97ms ± 1.45ms
Ops: 109.02 ± 0.38 ops/s. Best: 108.22 ops/s
Total: 9172.02ms ± 35.02ms. Best: 9123.53ms
Min time: 3.75ms ± 0.03ms
Mean time: 9.17ms ± 0.03ms
Median time: 9.13ms ± 0.03ms
95th percentile: 14.03ms ± 0.07ms
99th percentile: 15.07ms ± 0.08ms
Max time: 18.09ms ± 1.51ms
e2e_trip_ch Ops: 94.92 ± 0.37 ops/s. Best: 94.30 ops/s
Total: 10536.72ms ± 45.21ms. Best: 10467.86ms
Min time: 1.56ms ± 0.12ms
Mean time: 10.54ms ± 0.05ms
Median time: 10.04ms ± 0.05ms
95th percentile: 18.63ms ± 0.08ms
99th percentile: 20.49ms ± 0.16ms
Max time: 24.62ms ± 1.66ms
Ops: 95.27 ± 0.29 ops/s. Best: 94.69 ops/s
Total: 10495.09ms ± 29.57ms. Best: 10452.27ms
Min time: 1.56ms ± 0.10ms
Mean time: 10.50ms ± 0.03ms
Median time: 10.05ms ± 0.05ms
95th percentile: 18.56ms ± 0.06ms
99th percentile: 19.93ms ± 0.09ms
Max time: 21.72ms ± 0.53ms
e2e_trip_mld Ops: 56.97 ± 0.12 ops/s. Best: 56.78 ops/s
Total: 17553.86ms ± 37.14ms. Best: 17501.64ms
Min time: 1.77ms ± 0.17ms
Mean time: 17.55ms ± 0.04ms
Median time: 17.14ms ± 0.08ms
95th percentile: 28.60ms ± 0.09ms
99th percentile: 30.83ms ± 0.19ms
Max time: 34.76ms ± 1.63ms
Ops: 56.85 ± 0.16 ops/s. Best: 56.57 ops/s
Total: 17590.18ms ± 51.36ms. Best: 17491.31ms
Min time: 1.76ms ± 0.22ms
Mean time: 17.59ms ± 0.05ms
Median time: 17.13ms ± 0.06ms
95th percentile: 28.72ms ± 0.12ms
99th percentile: 30.63ms ± 0.14ms
Max time: 34.57ms ± 2.04ms
json-render String: 7.52865ms
Stringstream: 10.5456ms
Vector: 6.60269ms
String: 6.77649ms
Stringstream: 10.4456ms
Vector: 6.99657ms
match_ch Default radius:
4.59468ms/req at 82 coordinate
0.0560327ms/coordinate
Radius 10m:
15.992ms/req at 82 coordinate
0.195025ms/coordinate
Default radius:
4.137ms/req at 82 coordinate
0.0504512ms/coordinate
Radius 10m:
14.4467ms/req at 82 coordinate
0.17618ms/coordinate
match_mld Default radius:
3.14934ms/req at 82 coordinate
0.0384066ms/coordinate
Radius 10m:
11.7558ms/req at 82 coordinate
0.143363ms/coordinate
Default radius:
2.91127ms/req at 82 coordinate
0.0355033ms/coordinate
Radius 10m:
9.98744ms/req at 82 coordinate
0.121798ms/coordinate
osrm_contract Time: 94.41s Peak RAM: 194.00MB Time: 94.75s Peak RAM: 195.88MB
osrm_customize Time: 1.33s Peak RAM: 116.66MB Time: 1.34s Peak RAM: 116.85MB
osrm_extract Time: 12.07s Peak RAM: 412.25MB Time: 12.15s Peak RAM: 409.04MB
osrm_partition Time: 2.06s Peak RAM: 133.73MB Time: 2.01s Peak RAM: 131.83MB
packedvector random write:
std::vector 11235.4 ms
util::packed_vector 81957.7 ms
slowdown: 7.29462
random read:
std::vector 11135.3 ms
util::packed_vector 33715.2 ms
slowdown: 3.02777
random write:
std::vector 11220.5 ms
util::packed_vector 74023.7 ms
slowdown: 6.5972
random read:
std::vector 11136.5 ms
util::packed_vector 30500.8 ms
slowdown: 2.73882
random_match_ch 500 matches, default radius
ops: 217.40 ± 0.82 ops/s. best: 215.68ops/s.
total: 262.19 ± 1.00ms. best: 261.01ms.
avg: 4.60 ± 0.02ms
min: 0.15 ± 0.01ms
max: 23.76 ± 0.06ms
p99: 23.76 ± 0.06ms

500 matches, radius=10
ops: 65.44 ± 0.07 ops/s. best: 65.36ops/s.
total: 977.95 ± 1.00ms. best: 976.40ms.
avg: 15.28 ± 0.02ms
min: 0.16 ± 0.00ms
max: 223.55 ± 0.49ms
p99: 223.55 ± 0.49ms

500 matches, radius=20
ops: 15.70 ± 0.03 ops/s. best: 15.66ops/s.
total: 4138.90 ± 7.60ms. best: 4129.29ms.
avg: 63.68 ± 0.12ms
min: 0.35 ± 0.01ms
max: 1112.51 ± 5.17ms
p99: 1112.51 ± 5.17ms
500 matches, default radius
ops: 246.64 ± 1.14 ops/s. best: 243.96ops/s.
total: 231.11 ± 1.08ms. best: 230.08ms.
avg: 4.05 ± 0.02ms
min: 0.14 ± 0.01ms
max: 22.42 ± 0.09ms
p99: 22.42 ± 0.09ms

500 matches, radius=10
ops: 72.83 ± 0.06 ops/s. best: 72.73ops/s.
total: 878.75 ± 0.78ms. best: 877.81ms.
avg: 13.73 ± 0.01ms
min: 0.15 ± 0.00ms
max: 218.57 ± 0.42ms
p99: 218.57 ± 0.42ms

500 matches, radius=20
ops: 17.37 ± 0.04 ops/s. best: 17.29ops/s.
total: 3742.35 ± 8.56ms. best: 3729.86ms.
avg: 57.57 ± 0.13ms
min: 0.30 ± 0.00ms
max: 1108.68 ± 6.78ms
p99: 1108.68 ± 6.78ms
random_match_mld 500 matches, default radius
ops: 303.88 ± 2.14 ops/s. best: 298.69ops/s.
total: 187.59 ± 1.33ms. best: 186.39ms.
avg: 3.29 ± 0.02ms
min: 0.14 ± 0.00ms
max: 19.49 ± 0.08ms
p99: 19.49 ± 0.08ms

500 matches, radius=10
ops: 106.34 ± 0.18 ops/s. best: 106.08ops/s.
total: 601.83 ± 1.01ms. best: 600.12ms.
avg: 9.40 ± 0.02ms
min: 0.16 ± 0.00ms
max: 114.52 ± 0.12ms
p99: 114.52 ± 0.12ms

500 matches, radius=20
ops: 21.26 ± 0.04 ops/s. best: 21.21ops/s.
total: 3057.73 ± 6.06ms. best: 3045.70ms.
avg: 47.04 ± 0.09ms
min: 0.21 ± 0.01ms
max: 601.53 ± 2.34ms
p99: 601.53 ± 2.34ms
500 matches, default radius
ops: 344.04 ± 1.78 ops/s. best: 339.79ops/s.
total: 165.69 ± 0.87ms. best: 164.83ms.
avg: 2.91 ± 0.02ms
min: 0.13 ± 0.00ms
max: 17.28 ± 0.06ms
p99: 17.28 ± 0.06ms

500 matches, radius=10
ops: 122.37 ± 0.17 ops/s. best: 121.97ops/s.
total: 523.01 ± 0.72ms. best: 522.40ms.
avg: 8.17 ± 0.01ms
min: 0.15 ± 0.00ms
max: 100.03 ± 0.12ms
p99: 100.03 ± 0.12ms

500 matches, radius=20
ops: 23.68 ± 0.02 ops/s. best: 23.66ops/s.
total: 2745.25 ± 1.80ms. best: 2742.73ms.
avg: 42.23 ± 0.03ms
min: 0.19 ± 0.00ms
max: 534.07 ± 0.54ms
p99: 534.07 ± 0.54ms
random_nearest_ch 10000 nearest, number_of_results=1
ops: 24821.82 ± 55.09 ops/s. best: 24699.61ops/s.
total: 402.87 ± 0.90ms. best: 401.97ms.
avg: 0.04 ± 0.00ms
min: 0.01 ± 0.00ms
max: 0.16 ± 0.02ms
p99: 0.10 ± 0.00ms

10000 nearest, number_of_results=5
ops: 18450.79 ± 54.42 ops/s. best: 18322.24ops/s.
total: 541.99 ± 1.64ms. best: 540.56ms.
avg: 0.05 ± 0.00ms
min: 0.02 ± 0.00ms
max: 0.15 ± 0.00ms
p99: 0.11 ± 0.00ms

10000 nearest, number_of_results=10
ops: 14618.31 ± 19.30 ops/s. best: 14584.83ops/s.
total: 684.07 ± 0.92ms. best: 682.73ms.
avg: 0.07 ± 0.00ms
min: 0.03 ± 0.00ms
max: 0.18 ± 0.00ms
p99: 0.13 ± 0.00ms
10000 nearest, number_of_results=1
ops: 24838.50 ± 62.27 ops/s. best: 24749.96ops/s.
total: 402.60 ± 1.01ms. best: 401.21ms.
avg: 0.04 ± 0.00ms
min: 0.01 ± 0.00ms
max: 0.15 ± 0.02ms
p99: 0.10 ± 0.00ms

10000 nearest, number_of_results=5
ops: 18590.76 ± 31.58 ops/s. best: 18520.87ops/s.
total: 537.90 ± 0.90ms. best: 536.88ms.
avg: 0.05 ± 0.00ms
min: 0.02 ± 0.00ms
max: 0.15 ± 0.00ms
p99: 0.11 ± 0.00ms

10000 nearest, number_of_results=10
ops: 14714.05 ± 25.53 ops/s. best: 14676.14ops/s.
total: 679.63 ± 1.19ms. best: 678.15ms.
avg: 0.07 ± 0.00ms
min: 0.03 ± 0.00ms
max: 0.18 ± 0.01ms
p99: 0.13 ± 0.00ms
random_nearest_mld 10000 nearest, number_of_results=1
ops: 24611.17 ± 77.73 ops/s. best: 24420.43ops/s.
total: 406.33 ± 1.29ms. best: 405.30ms.
avg: 0.04 ± 0.00ms
min: 0.01 ± 0.00ms
max: 0.16 ± 0.03ms
p99: 0.10 ± 0.00ms

10000 nearest, number_of_results=5
ops: 18398.80 ± 25.67 ops/s. best: 18362.76ops/s.
total: 543.52 ± 0.76ms. best: 542.39ms.
avg: 0.05 ± 0.00ms
min: 0.02 ± 0.00ms
max: 0.15 ± 0.00ms
p99: 0.11 ± 0.00ms

10000 nearest, number_of_results=10
ops: 14630.95 ± 30.73 ops/s. best: 14596.71ops/s.
total: 683.49 ± 1.43ms. best: 680.67ms.
avg: 0.07 ± 0.00ms
min: 0.03 ± 0.00ms
max: 0.18 ± 0.01ms
p99: 0.13 ± 0.00ms
10000 nearest, number_of_results=1
ops: 24709.78 ± 61.47 ops/s. best: 24609.34ops/s.
total: 404.70 ± 1.01ms. best: 403.45ms.
avg: 0.04 ± 0.00ms
min: 0.01 ± 0.00ms
max: 0.15 ± 0.02ms
p99: 0.10 ± 0.00ms

10000 nearest, number_of_results=5
ops: 18579.01 ± 26.68 ops/s. best: 18524.61ops/s.
total: 538.24 ± 0.77ms. best: 537.25ms.
avg: 0.05 ± 0.00ms
min: 0.02 ± 0.00ms
max: 0.15 ± 0.00ms
p99: 0.11 ± 0.00ms

10000 nearest, number_of_results=10
ops: 14684.76 ± 68.70 ops/s. best: 14541.40ops/s.
total: 681.00 ± 3.20ms. best: 678.21ms.
avg: 0.07 ± 0.00ms
min: 0.03 ± 0.00ms
max: 0.21 ± 0.05ms
p99: 0.13 ± 0.00ms
random_route_ch 1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
ops: 493.75 ± 2.22 ops/s. best: 488.83ops/s.
total: 1992.95 ± 9.01ms. best: 1983.99ms.
avg: 2.03 ± 0.01ms
min: 0.33 ± 0.00ms
max: 3.65 ± 0.22ms
p99: 2.95 ± 0.06ms

1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
ops: 571.18 ± 1.81 ops/s. best: 567.02ops/s.
total: 1750.78 ± 5.56ms. best: 1746.17ms.
avg: 1.75 ± 0.01ms
min: 0.05 ± 0.00ms
max: 4.36 ± 0.27ms
p99: 3.62 ± 0.03ms

1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
ops: 1009.61 ± 7.08 ops/s. best: 997.21ops/s.
total: 974.69 ± 6.84ms. best: 963.95ms.
avg: 0.99 ± 0.01ms
min: 0.22 ± 0.00ms
max: 1.55 ± 0.03ms
p99: 1.38 ± 0.03ms

1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
ops: 1125.58 ± 4.65 ops/s. best: 1117.31ops/s.
total: 888.45 ± 3.68ms. best: 884.92ms.
avg: 0.89 ± 0.00ms
min: 0.04 ± 0.00ms
max: 2.90 ± 0.03ms
p99: 1.86 ± 0.00ms
1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
ops: 497.01 ± 1.84 ops/s. best: 494.77ops/s.
total: 1979.89 ± 7.32ms. best: 1966.36ms.
avg: 2.01 ± 0.01ms
min: 0.32 ± 0.00ms
max: 3.45 ± 0.22ms
p99: 2.93 ± 0.04ms

1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
ops: 615.18 ± 3.11 ops/s. best: 608.78ops/s.
total: 1625.59 ± 8.59ms. best: 1615.68ms.
avg: 1.63 ± 0.01ms
min: 0.05 ± 0.00ms
max: 3.93 ± 0.16ms
p99: 3.58 ± 0.04ms

1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
ops: 1019.94 ± 8.93 ops/s. best: 1002.75ops/s.
total: 964.86 ± 8.52ms. best: 953.75ms.
avg: 0.98 ± 0.01ms
min: 0.23 ± 0.00ms
max: 1.66 ± 0.13ms
p99: 1.40 ± 0.07ms

1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
ops: 1232.06 ± 2.67 ops/s. best: 1227.03ops/s.
total: 811.65 ± 1.76ms. best: 809.52ms.
avg: 0.81 ± 0.00ms
min: 0.04 ± 0.00ms
max: 2.26 ± 0.05ms
p99: 1.69 ± 0.02ms
random_route_mld 1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
ops: 243.59 ± 1.20 ops/s. best: 241.60ops/s.
total: 4039.65 ± 19.69ms. best: 4012.10ms.
avg: 4.11 ± 0.02ms
min: 0.33 ± 0.00ms
max: 8.90 ± 0.27ms
p99: 6.94 ± 0.13ms

1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
ops: 234.13 ± 1.96 ops/s. best: 231.87ops/s.
total: 4271.44 ± 35.60ms. best: 4211.25ms.
avg: 4.27 ± 0.04ms
min: 0.05 ± 0.00ms
max: 10.28 ± 0.71ms
p99: 8.64 ± 0.15ms

1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
ops: 324.20 ± 1.06 ops/s. best: 322.57ops/s.
total: 3035.20 ± 9.88ms. best: 3016.20ms.
avg: 3.08 ± 0.01ms
min: 0.29 ± 0.00ms
max: 7.28 ± 0.08ms
p99: 5.30 ± 0.07ms

1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
ops: 291.86 ± 0.77 ops/s. best: 290.28ops/s.
total: 3426.38 ± 9.09ms. best: 3414.94ms.
avg: 3.43 ± 0.01ms
min: 0.04 ± 0.00ms
max: 7.80 ± 0.18ms
p99: 6.77 ± 0.10ms
1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
ops: 250.78 ± 0.58 ops/s. best: 249.97ops/s.
total: 3923.85 ± 8.74ms. best: 3906.22ms.
avg: 3.99 ± 0.01ms
min: 0.32 ± 0.00ms
max: 8.64 ± 0.12ms
p99: 6.76 ± 0.07ms

1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
ops: 243.25 ± 0.66 ops/s. best: 241.90ops/s.
total: 4111.10 ± 11.19ms. best: 4096.61ms.
avg: 4.11 ± 0.01ms
min: 0.05 ± 0.00ms
max: 9.41 ± 0.13ms
p99: 8.30 ± 0.03ms

1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
ops: 335.15 ± 1.53 ops/s. best: 331.96ops/s.
total: 2936.13 ± 14.20ms. best: 2919.65ms.
avg: 2.98 ± 0.01ms
min: 0.27 ± 0.00ms
max: 7.19 ± 0.11ms
p99: 5.14 ± 0.07ms

1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
ops: 306.40 ± 0.99 ops/s. best: 305.31ops/s.
total: 3263.80 ± 10.53ms. best: 3247.96ms.
avg: 3.26 ± 0.01ms
min: 0.04 ± 0.00ms
max: 7.19 ± 0.09ms
p99: 6.48 ± 0.06ms
random_table_ch 250 tables, 3 coordinates
ops: 1552.43 ± 14.31 ops/s. best: 1522.22ops/s.
total: 161.06 ± 1.50ms. best: 159.71ms.
avg: 0.64 ± 0.01ms
min: 0.43 ± 0.01ms
max: 0.98 ± 0.23ms
p99: 0.82 ± 0.03ms

250 tables, 25 coordinates
ops: 182.37 ± 0.07 ops/s. best: 182.25ops/s.
total: 1370.86 ± 0.55ms. best: 1370.19ms.
avg: 5.48 ± 0.00ms
min: 4.96 ± 0.01ms
max: 5.98 ± 0.03ms
p99: 5.92 ± 0.01ms

250 tables, 50 coordinates
ops: 89.65 ± 0.06 ops/s. best: 89.57ops/s.
total: 2788.53 ± 1.72ms. best: 2785.98ms.
avg: 11.15 ± 0.01ms
min: 10.37 ± 0.01ms
max: 12.50 ± 0.71ms
p99: 11.90 ± 0.03ms
250 tables, 3 coordinates
ops: 1463.74 ± 11.68 ops/s. best: 1441.58ops/s.
total: 170.81 ± 1.37ms. best: 169.13ms.
avg: 0.68 ± 0.01ms
min: 0.45 ± 0.01ms
max: 1.02 ± 0.26ms
p99: 0.86 ± 0.04ms

250 tables, 25 coordinates
ops: 173.00 ± 0.27 ops/s. best: 172.55ops/s.
total: 1445.13 ± 2.29ms. best: 1441.91ms.
avg: 5.78 ± 0.01ms
min: 5.15 ± 0.01ms
max: 6.41 ± 0.09ms
p99: 6.30 ± 0.05ms

250 tables, 50 coordinates
ops: 84.76 ± 0.26 ops/s. best: 84.30ops/s.
total: 2949.42 ± 9.00ms. best: 2935.47ms.
avg: 11.80 ± 0.04ms
min: 11.04 ± 0.07ms
max: 13.34 ± 0.89ms
p99: 12.81 ± 0.32ms
random_table_mld 250 tables, 3 coordinates
ops: 341.76 ± 0.85 ops/s. best: 339.92ops/s.
total: 731.51 ± 1.83ms. best: 729.59ms.
avg: 2.93 ± 0.01ms
min: 2.32 ± 0.01ms
max: 4.04 ± 0.02ms
p99: 3.86 ± 0.04ms

250 tables, 25 coordinates
ops: 37.75 ± 0.03 ops/s. best: 37.73ops/s.
total: 6621.92 ± 4.67ms. best: 6612.09ms.
avg: 26.49 ± 0.02ms
min: 23.70 ± 0.06ms
max: 30.05 ± 0.16ms
p99: 29.28 ± 0.09ms

250 tables, 50 coordinates
ops: 17.73 ± 0.02 ops/s. best: 17.69ops/s.
total: 14099.39 ± 19.19ms. best: 14067.81ms.
avg: 56.40 ± 0.08ms
min: 52.13 ± 0.23ms
max: 62.76 ± 2.53ms
p99: 60.11 ± 0.17ms
250 tables, 3 coordinates
ops: 340.67 ± 0.84 ops/s. best: 339.61ops/s.
total: 733.85 ± 1.81ms. best: 730.52ms.
avg: 2.94 ± 0.01ms
min: 2.32 ± 0.01ms
max: 4.08 ± 0.08ms
p99: 3.86 ± 0.04ms

250 tables, 25 coordinates
ops: 37.64 ± 0.05 ops/s. best: 37.56ops/s.
total: 6641.70 ± 9.01ms. best: 6627.78ms.
avg: 26.57 ± 0.04ms
min: 23.81 ± 0.06ms
max: 30.53 ± 0.18ms
p99: 29.42 ± 0.14ms

250 tables, 50 coordinates
ops: 17.42 ± 0.01 ops/s. best: 17.41ops/s.
total: 14349.22 ± 9.08ms. best: 14335.05ms.
avg: 57.40 ± 0.04ms
min: 53.10 ± 0.21ms
max: 65.21 ± 2.96ms
p99: 61.53 ± 0.18ms
random_trip_ch 250 trips, 3 coordinates
ops: 484.35 ± 3.38 ops/s. best: 477.75ops/s.
total: 516.19 ± 3.62ms. best: 511.77ms.
avg: 2.06 ± 0.01ms
min: 1.12 ± 0.01ms
max: 3.00 ± 0.32ms
p99: 2.71 ± 0.06ms

250 trips, 5 coordinates
ops: 322.03 ± 1.82 ops/s. best: 318.19ops/s.
total: 776.35 ± 4.41ms. best: 771.07ms.
avg: 3.11 ± 0.02ms
min: 2.05 ± 0.01ms
max: 4.14 ± 0.31ms
p99: 3.92 ± 0.21ms
250 trips, 3 coordinates
ops: 477.38 ± 6.97 ops/s. best: 463.36ops/s.
total: 523.84 ± 7.90ms. best: 515.13ms.
avg: 2.10 ± 0.03ms
min: 1.18 ± 0.01ms
max: 3.03 ± 0.34ms
p99: 2.83 ± 0.23ms

250 trips, 5 coordinates
ops: 322.11 ± 2.56 ops/s. best: 317.34ops/s.
total: 776.20 ± 6.17ms. best: 766.18ms.
avg: 3.10 ± 0.02ms
min: 2.02 ± 0.02ms
max: 4.01 ± 0.13ms
p99: 3.81 ± 0.06ms
random_trip_mld 250 trips, 3 coordinates
ops: 166.74 ± 1.20 ops/s. best: 164.76ops/s.
total: 1499.39 ± 10.83ms. best: 1480.84ms.
avg: 6.00 ± 0.04ms
min: 3.89 ± 0.00ms
max: 8.65 ± 0.61ms
p99: 7.89 ± 0.16ms

250 trips, 5 coordinates
ops: 108.97 ± 0.83 ops/s. best: 107.18ops/s.
total: 2294.31 ± 17.63ms. best: 2273.69ms.
avg: 9.18 ± 0.07ms
min: 6.33 ± 0.06ms
max: 11.71 ± 0.46ms
p99: 11.11 ± 0.14ms
250 trips, 3 coordinates
ops: 169.41 ± 0.91 ops/s. best: 168.64ops/s.
total: 1475.77 ± 7.89ms. best: 1457.82ms.
avg: 5.90 ± 0.03ms
min: 3.83 ± 0.01ms
max: 8.09 ± 0.26ms
p99: 7.68 ± 0.14ms

250 trips, 5 coordinates
ops: 109.40 ± 0.71 ops/s. best: 108.15ops/s.
total: 2285.21 ± 14.94ms. best: 2269.04ms.
avg: 9.14 ± 0.06ms
min: 6.32 ± 0.12ms
max: 12.17 ± 0.78ms
p99: 11.24 ± 0.19ms
route_ch 1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
435.478ms
0.435478ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
534.848ms
0.534848ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
159.094ms
0.159094ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
141.357ms
0.141357ms/req
1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
429.783ms
0.429783ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
529.222ms
0.529222ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
147.141ms
0.147141ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
128.186ms
0.128186ms/req
route_mld 1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
579.184ms
0.579184ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
735.042ms
0.735042ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
278.53ms
0.27853ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
308.042ms
0.308042ms/req
1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
560.844ms
0.560844ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
694.157ms
0.694157ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
249.912ms
0.249912ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
264.737ms
0.264737ms/req
rtree 1 result:
201.604ms -> 0.0201604 ms/query
10 results:
236.648ms -> 0.0236648 ms/query
1 result:
195.749ms -> 0.0195749 ms/query
10 results:
231.714ms -> 0.0231714 ms/query

@SiarheiFedartsou
Copy link
Member Author

@DennisOSRM WDYT about it? Our current benchmarks are not exhaustive, so not sure it is okay to 100% rely on them when making such decisions... On another hand I don't remember any examples when std::unordered_map was actually better than any other popular alternative hash map...

@DennisOSRM
Copy link
Collaborator

@DennisOSRM WDYT about it? Our current benchmarks are not exhaustive, so not sure it is okay to 100% rely on them when making such decisions... On another hand I don't remember any examples when std::unordered_map was actually better than any other popular alternative hash map...

I don't have major objections, yet I'd love to see some benchmark that shows the benefits. 😏

@SiarheiFedartsou SiarheiFedartsou changed the title Try to use ankerl::unordered_dense::map instead of std::unordered_map in UnorderedMapStorage Use ankerl::unordered_dense::map instead of std::unordered_map in UnorderedMapStorage Jun 26, 2024
@SiarheiFedartsou
Copy link
Member Author

@DennisOSRM could you please take a look now? After recent benchmark improvements I can be quite sure this change indeed improve things: it is either better or ~the same.Benchmarks are still "jumpy" though, but after multiple runs I am confident enough in it.

@SiarheiFedartsou SiarheiFedartsou marked this pull request as ready for review June 26, 2024 18:00
Copy link
Collaborator

@DennisOSRM DennisOSRM left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am a bit on the fence on this change. Pretty sure the map implementation is faster than the one from std namespace in micro benchmarks. Then it pulls in a 15k LOC dependency without moving the needle too much.

@SiarheiFedartsou
Copy link
Member Author

I am a bit on the fence on this change. Pretty sure the map implementation is faster than the one from std namespace in micro benchmarks. Then it pulls in a 15k LOC dependency without moving the needle too much.

Let’s test it on more stable benchmarks on newly added self-hosted runner and then make a decision 👍 Actually there are some other things to optimize here even with map from std - e.g. allocations eat a lot…

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants