Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set number of jobs in rust build using Cargo's parallel level #3826

Merged
merged 1 commit into from
Jul 16, 2024

Conversation

benjaminwinger
Copy link
Collaborator

Fixes #3784.
Sets the bundled CMake build's parallelism level using the NUM_JOBS environment variable provided by Cargo in case the jobserver isn't available (as appears to be the case on macOS).

Copy link

Benchmark Result

Master commit hash: 433c8d303a1face982a62d4f69c0a51b5b6879e5
Branch commit hash: 2a1570a22de85a0ad41895262ca6a86f01929d81

Query Group Query Name Mean Time - Commit (ms) Mean Time - Master (ms) Diff
aggregation q24 647.23 696.38 -49.15 (-7.06%)
aggregation q28 11846.43 11904.72 -58.29 (-0.49%)
filter q14 127.27 134.59 -7.32 (-5.44%)
filter q15 122.88 131.13 -8.25 (-6.29%)
filter q16 305.09 311.56 -6.47 (-2.08%)
filter q17 455.53 459.52 -3.98 (-0.87%)
filter q18 1872.68 1959.31 -86.63 (-4.42%)
fixed_size_expr_evaluator q07 566.07 572.65 -6.58 (-1.15%)
fixed_size_expr_evaluator q08 806.58 797.34 9.24 (1.16%)
fixed_size_expr_evaluator q09 797.63 795.80 1.83 (0.23%)
fixed_size_expr_evaluator q10 243.64 249.57 -5.93 (-2.38%)
fixed_size_expr_evaluator q11 242.41 243.04 -0.63 (-0.26%)
fixed_size_expr_evaluator q12 236.49 244.56 -8.08 (-3.30%)
fixed_size_expr_evaluator q13 1505.21 1481.48 23.73 (1.60%)
fixed_size_seq_scan q23 112.72 123.10 -10.37 (-8.43%)
join q31 5.19 6.37 -1.19 (-18.60%)
ldbc_snb_ic q35 831.64 905.93 -74.29 (-8.20%)
ldbc_snb_ic q36 38.73 31.97 6.76 (21.15%)
ldbc_snb_is q32 2.22 4.91 -2.69 (-54.76%)
ldbc_snb_is q33 10.56 10.98 -0.42 (-3.79%)
ldbc_snb_is q34 1.12 1.71 -0.59 (-34.41%)
multi-rel multi-rel-large-scan 3068.07 2652.58 415.49 (15.66%)
multi-rel multi-rel-lookup 55.64 36.16 19.48 (53.87%)
multi-rel multi-rel-small-scan 80.62 71.97 8.65 (12.02%)
order_by q25 130.95 133.18 -2.22 (-1.67%)
order_by q26 447.72 454.45 -6.72 (-1.48%)
order_by q27 1399.20 1409.85 -10.65 (-0.76%)
scan_after_filter q01 174.83 181.42 -6.58 (-3.63%)
scan_after_filter q02 159.59 165.65 -6.06 (-3.66%)
shortest_path_ldbc100 q39 71.73 67.96 3.76 (5.54%)
var_size_expr_evaluator q03 2070.81 2058.36 12.45 (0.60%)
var_size_expr_evaluator q04 2204.45 2232.28 -27.83 (-1.25%)
var_size_expr_evaluator q05 2663.58 2734.05 -70.47 (-2.58%)
var_size_expr_evaluator q06 1345.79 1374.75 -28.96 (-2.11%)
var_size_seq_scan q19 1439.17 1459.67 -20.49 (-1.40%)
var_size_seq_scan q20 3074.82 3048.31 26.52 (0.87%)
var_size_seq_scan q21 2406.81 2387.51 19.30 (0.81%)
var_size_seq_scan q22 128.02 132.67 -4.65 (-3.50%)

@benjaminwinger benjaminwinger merged commit 33c2ecb into master Jul 16, 2024
20 checks passed
@benjaminwinger benjaminwinger deleted the rust-parallel branch July 16, 2024 07:54
mewim added a commit that referenced this pull request Jul 18, 2024
* Operator printing (#3805)

* Completed UseDatabase and TableFunctionCall Operators

* Fixed Issues on TableFunctionCall and UseDatabase

* Used correct function in printInfo TableFunctionCall

* Changed "Database Name: " to "Database: "

* Used clang-format on src

* Marked single-argument constructors explicit

* Implemented more Physical Operators - with bugs

* Added setProperty and fixed transaction

* Fixed formatting issues

* Fixed problems with my commit

* Fixed formatting issues

* Fixed last minute requirements

* Changed titles of Skip and Limit

* Add arm64 macOS CI workflow (#3825)

* Add arm64 macOS CI workflow

* Fix dependency

* Try to repair pip install

* Set CMAKE_BUILD_PARALLEL_LEVEL dynamically; do not use ninja

* Set CXX and CC

* Install ninja

* Include the installation of the "openssl@3" and "duckdb"

* Add ccache to the list of dependencies for CI workflow

* Fix python struct and map type interpretation (#3824)

* Fix lambda dependency analyze (#3829)

* Fix lambda dependency analyze

* Run clang-format

---------

Co-authored-by: CI Bot <[email protected]>

* Change max line width to 16k (#3833)

* Set number of jobs in rust build using Cargo's parallel level (#3826)

* Copy to parquet options (#3834)

* Add error message when kuzu detects numpy version at or above 2.0.0 (#3828)

* Fix depandabot security alert (#3837)

* Add DECIMAL conversion for Node.js API (#3835)

* Add DECIMAL conversion for Node.js API

* Fix Node.js DECIMAL test to use sorted result

* Pass transaction in hash index (#3822)

* rework transaction passing in hash index

* add const to passed transaction param

* fix (#3838)

* NetworkX multi-edge support (#3836)

* Address #3640

* Refactor get_as_networkx method signature to remove type hinting error

* Fix demo-db test (#3843)

* Issue 3744 (#3841)

* Implement copy from external table (#3844)

* Update pybind11 version (#3839)

* Optimize small rel table lookups

Adds a random access version of OnDiskGraph::scanFwd which reads only the CSR header information necessary for the requested nodes
Also fixes the caching in OnDiskGraph scans so that the cache isn't
regenerated for every scan

---------

Co-authored-by: Hamza Ammar <[email protected]>
Co-authored-by: ziyi chen <[email protected]>
Co-authored-by: andyfeng <[email protected]>
Co-authored-by: CI Bot <[email protected]>
Co-authored-by: Benjamin Winger <[email protected]>
Co-authored-by: Maxwell <[email protected]>
Co-authored-by: Guodong Jin <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Rust crate compile times on macOS arm64 are incredibly slow
2 participants