Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mean.piar_index() is slow for large indexes #2

Open
marberts opened this issue Oct 3, 2023 · 1 comment
Open

mean.piar_index() is slow for large indexes #2

marberts opened this issue Oct 3, 2023 · 1 comment
Labels
enhancement New feature or request needs further thought I need to think about this...

Comments

@marberts
Copy link
Owner

marberts commented Oct 3, 2023

Aggregating an index over subperiods seems slow given that it should be a simple operation. The constraint appears to be the loop.

bench::press(
  levels = c(1e2, 1e3, 1e4),
  periods = c(3, 12, 120),
  {
    index <- as_index(matrix(runif(levels * periods), levels))
    bench::mark(mean(index), min_iterations = 10)
  }
)

  expression levels periods      min   median `itr/sec` mem_alloc `gc/sec` n_itr  n_gc total_time 
  <bch:expr>  <dbl>   <dbl> <bch:tm> <bch:tm>     <dbl> <bch:byt>    <dbl> <int> <dbl>   <bch:tm> 
1 mean(inde100       3   1.05ms   1.18ms   727.       8.81KB     7.35   297     3   408.44ms 
2 mean(inde1000       3   7.69ms   8.87ms   113.      79.06KB     7.90    43     3   379.68ms 
3 mean(inde10000       3  74.34ms  75.09ms    13.3    881.56KB    31.1      3     7   225.18ms 
4 mean(inde100      12   3.48ms   3.96ms   252.      25.83KB    10.8     93     4   369.46ms 
5 mean(inde1000      12  30.57ms  31.63ms    31.2    233.09KB     9.35    10     3   320.98ms 
6 mean(inde10000      12  338.2ms 398.73ms     2.42      2.5MB     7.74    10    32      4.14s 
7 mean(inde100     120  35.56ms  39.78ms    25.0    233.06KB    10.7      7     3   279.66ms 
8 mean(inde1000     120  389.5ms 415.32ms     2.44     2.04MB     6.34    10    26       4.1s 
9 mean(inde10000     120    4.19s    8.93s     0.116   22.12MB     3.82    10   329      1.44m 
@marberts marberts added the enhancement New feature or request label Oct 3, 2023
@marberts
Copy link
Owner Author

These timings have improved a bit over the last year.

  levels periods      min   median mem_alloc n_itr
   <dbl>   <dbl> <bch:tm> <bch:tm> <bch:byt> <int>
1    100       3 473.32µs 504.85µs   11.44KB   921
2   1000       3   2.68ms   2.96ms  102.78KB   154
3  10000       3  25.28ms  26.86ms    1.09MB    14
4    100      12 685.86µs 746.54µs   19.31KB   623
5   1000      12   4.87ms   5.22ms  173.94KB    84
6  10000      12  47.56ms   49.9ms    1.78MB     6
7    100     120   3.35ms   3.63ms  218.22KB   123
8   1000     120  31.68ms  32.99ms    1.97MB    10
9  10000     120  339.1ms 356.81ms   19.65MB    10

@marberts marberts added the needs further thought I need to think about this... label Sep 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request needs further thought I need to think about this...
Projects
None yet
Development

No branches or pull requests

1 participant