Issues while benchmarking performance

[hamzahsaleem]

I am trying to benchmark the performance of Private-ID by conducting experiments. However, the running times I am getting are much higher as compared to those reported in the paper: "Private Matching for Compute". Do you have any ideas on why my times are significantly higher than those reported in the paper for similar experiments. Following are the details of the experiments I conducted:

Machine: Google Cloud Instance (e2-standard-32) (32 vCPUs and 128GB RAM)
Datasets size: 10,000 (Identifiers: 128 bit strings, values: 64 bit integers)
Intersection Size: 50%
The client and server processes run on a single machine.

Experiment 1:
Algorithm: Private-ID
Total time: 70.03021 sec (Single core), 5.16027 sec (Multi-core)

Experiment 2:
Algorithm: PS3I
Total time: 1259.84367 sec (multi-core)

These numbers are significantly higher than those reported by the paper for similar settings. i.e. For Private-ID, Table 1 shows that for 10,000 dataset size and single core, it takes 4.2s in contrast to 70.03 s that I get. Similarly, for PS3I, I get 1259.84s for 32 cores whereas the paper reports 247s for a single core. Do you have any thoughts why my benchmarking numbers are significantly higher?

Additionally, here is the log trace printed by the PS3I run:

[2022-07-12T18:57:36Z WARN rpc::connect::create_client] Connecting to company without TLS, avoid in production
[2022-07-12T18:57:36Z INFO rpc::connect::create_client] Connecting to host: http://localhost:10010/
[2022-07-12T18:57:36Z INFO rpc::connect::create_client] Client connected!
[2022-07-12T18:57:39Z INFO protocol::fileio] Data initialised with dimensions: cols: 1, rows: 10000, keys len: 10000
[2022-07-12T18:57:39Z INFO cross_psi_client] Number of company features 1
[2022-07-12T18:57:39Z INFO common::timer] [u_company_keys | received u_company_keys size: 10000] elapsed: 0.46777 sec [qps: 21378]
[2022-07-12T18:57:39Z INFO common::timer] [u_company_keys] elapsed: 0.93586 sec
[2022-07-12T18:57:39Z INFO common::timer] [e_company | keys EC enc + srlz size: 10000] elapsed: 0.00002 sec [qps: 571493885]
[2022-07-12T19:06:10Z INFO common::timer] [u_company_feature | received u_company_feature size: 10000] elapsed: 510.25326 sec [qps: 20]
[2022-07-12T19:06:10Z INFO common::timer] [u_company_feature] elapsed: 1020.50688 sec
[2022-07-12T19:08:37Z INFO cross_psi_client] e_company_ack feature index 0
[2022-07-12T19:08:37Z INFO common::timer] [u_partner | keys EC enc size: 10000] elapsed: 0.00002 sec [qps: 502537816]
[2022-07-12T19:17:20Z INFO common::timer] [u_partner | column 0 HE enc size: 10000] elapsed: 518.63875 sec [qps: 19]
[2022-07-12T19:17:20Z INFO cross_psi_client] e_company_ack feature_index 0
[2022-07-12T19:17:20Z INFO common::timer] [shares_company_indices | received shares_company_indices size: 5000] elapsed: 0.03449 sec [qps: 144985]
[2022-07-12T19:17:20Z INFO common::timer] [shares_company_indices] elapsed: 0.06919 sec
[2022-07-12T19:17:20Z INFO common::timer] [indices | recv company intersection indices size: 5000] elapsed: 0.03621 sec [qps: 138093]
[2022-07-12T19:17:20Z INFO cross_psi_client] company intersection size: 5000
[2022-07-12T19:17:21Z INFO common::timer] [shares_feature | received shares_feature size: 5000] elapsed: 0.02990 sec [qps: 167238]
[2022-07-12T19:17:21Z INFO common::timer] [shares_feature] elapsed: 0.05998 sec
[2022-07-12T19:17:21Z INFO protocol::cross_psi::partner] Saving self-shares for feature 0
[2022-07-12T19:18:36Z INFO common::timer] [indices | recv shares for feature 0 size: 5000] elapsed: 75.41446 sec [qps: 66]
[2022-07-12T19:18:36Z INFO protocol::cross_psi::partner] revealing columns to output file
[2022-07-12T19:18:36Z INFO common::timer] [global | total time size: 10000] elapsed: 1259.84367 sec [qps: 8]