Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

libct/int: add exec benchmark #4432

Merged
merged 2 commits into from
Oct 25, 2024
Merged

Conversation

kolyshkin
Copy link
Contributor

Related to: #3181

This is a benchmark which checks how fast we can execute /bin/true
inside a container.

Results from my machine are below. As you can see, in default setup
about 70% of exec time is spent for CVE-2019-5736 (copying runc binary),
and using either RUNC_DMZ=true or memfd-bind helps a lot.

This can also be used for profiling (using -test.cpuprofile option).

Default setup

[kir@kir-tp1 integration]$ sudo ./integration.test -test.run xxx -test.v -test.benchtime 5s -test.count 5 -test.bench . .
goos: linux
goarch: amd64
pkg: github.com/opencontainers/runc/libcontainer/integration
cpu: 12th Gen Intel(R) Core(TM) i7-12800H
BenchmarkExecTrue
BenchmarkExecTrue-20                 327          24475677 ns/op
BenchmarkExecTrue-20                 244          25242718 ns/op
BenchmarkExecTrue-20                 232          26187174 ns/op
BenchmarkExecTrue-20                 237          26780030 ns/op
BenchmarkExecTrue-20                 318          18487219 ns/op
PASS

With DMZ enabled

[kir@kir-tp1 integration]$ sudo -E RUNC_DMZ=true ./integration.test -test.run xxx -test.v -test.benchtime 5s -test.count 5 -test.bench . .
goos: linux
goarch: amd64
pkg: github.com/opencontainers/runc/libcontainer/integration
cpu: 12th Gen Intel(R) Core(TM) i7-12800H
BenchmarkExecTrue
BenchmarkExecTrue-20                 694           8263744 ns/op
BenchmarkExecTrue-20                 778           8483228 ns/op
BenchmarkExecTrue-20                 784           8456018 ns/op
BenchmarkExecTrue-20                 732           8160239 ns/op
BenchmarkExecTrue-20                 769           8236972 ns/op
PASS

With memfd-bind

[kir@kir-tp1 integration]$ sudo systemctl start  memfd-bind@$(systemd-escape -p $PWD/integration.test)
[kir@kir-tp1 integration]$ sudo ./integration.test -test.run xxx -test.v -test.benchtime 5s -test.count 5 -test.bench . .
goos: linux
goarch: amd64
pkg: github.com/opencontainers/runc/libcontainer/integration
cpu: 12th Gen Intel(R) Core(TM) i7-12800H
BenchmarkExecTrue
BenchmarkExecTrue-20                 800           7538839 ns/op
BenchmarkExecTrue-20                 717           7424755 ns/op
BenchmarkExecTrue-20                 848           7747787 ns/op
BenchmarkExecTrue-20                 800           7668740 ns/op
BenchmarkExecTrue-20                 751           7304373 ns/op
PASS

Signed-off-by: Kir Kolyshkin [email protected]

Copy link
Member

@cyphar cyphar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kolyshkin
Copy link
Contributor Author

@AkihiroSuda @rata PTAL. It's just a benchmark so not risky to merge but it's helpful to have.

Copy link
Member

@rata rata left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This mostly LGTM, just left a question of a possible race.

This might be a silly question, but how are you running this? Where is the integration.test binary coming from? I run this with go test -bench, but I couldn't find how to build that binary you are using.

libcontainer/integration/bench_test.go Show resolved Hide resolved
@kolyshkin
Copy link
Contributor Author

This might be a silly question, but how are you running this?

The easiest way is

[kir@kir-tp1 runc]$ go test -v -run 234 -bench . -benchmem -exec sudo ./libcontainer/integration/
goos: linux
goarch: amd64
pkg: github.com/opencontainers/runc/libcontainer/integration
cpu: 12th Gen Intel(R) Core(TM) i7-12800H
BenchmarkExecTrue
BenchmarkExecTrue-20    	      57	  23171808 ns/op	   27448 B/op	     242 allocs/op
PASS
ok  	github.com/opencontainers/runc/libcontainer/integration	1.601s

Where is the integration.test binary coming from?

[kir@kir-tp1 runc]$ go test -c ./libcontainer/integration/

I was using the compiled version in the description since sudo makes it complicated to pass environment variables needed (you'd need something like go test -v -run 234 -bench . -benchmem -exec 'sudo -E RUNC_DMZ=true' ./libcontainer/integration/ and it becomes messy).

...so that they can be used for benchmarks, too.

Signed-off-by: Kir Kolyshkin <[email protected]>
This is a benchmark which checks how fast we can execute /bin/true
inside a container.

Results from my machine are below. As you can see, in default setup
about 70% of exec time is spent for CVE-2019-5736 (copying runc binary),
and using either RUNC_DMZ=true or memfd-bind helps a lot.

This can also be used for profiling (using -test.cpuprofile option).

=== Default setup ===

[kir@kir-tp1 integration]$ sudo ./integration.test -test.run xxx -test.v -test.benchtime 5s -test.count 5 -test.bench . .
goos: linux
goarch: amd64
pkg: github.com/opencontainers/runc/libcontainer/integration
cpu: 12th Gen Intel(R) Core(TM) i7-12800H
BenchmarkExecTrue
BenchmarkExecTrue-20    	     327	  24475677 ns/op
BenchmarkExecTrue-20    	     244	  25242718 ns/op
BenchmarkExecTrue-20    	     232	  26187174 ns/op
BenchmarkExecTrue-20    	     237	  26780030 ns/op
BenchmarkExecTrue-20    	     318	  18487219 ns/op
PASS

=== With DMZ enabled ===

[kir@kir-tp1 integration]$ sudo -E RUNC_DMZ=true ./integration.test -test.run xxx -test.v -test.benchtime 5s -test.count 5 -test.bench . .
goos: linux
goarch: amd64
pkg: github.com/opencontainers/runc/libcontainer/integration
cpu: 12th Gen Intel(R) Core(TM) i7-12800H
BenchmarkExecTrue
BenchmarkExecTrue-20    	     694	   8263744 ns/op
BenchmarkExecTrue-20    	     778	   8483228 ns/op
BenchmarkExecTrue-20    	     784	   8456018 ns/op
BenchmarkExecTrue-20    	     732	   8160239 ns/op
BenchmarkExecTrue-20    	     769	   8236972 ns/op
PASS

=== With memfd-bind ===

[kir@kir-tp1 integration]$ sudo systemctl start  memfd-bind@$(systemd-escape -p $PWD/integration.test)
[kir@kir-tp1 integration]$ sudo ./integration.test -test.run xxx -test.v -test.benchtime 5s -test.count 5 -test.bench . .
goos: linux
goarch: amd64
pkg: github.com/opencontainers/runc/libcontainer/integration
cpu: 12th Gen Intel(R) Core(TM) i7-12800H
BenchmarkExecTrue
BenchmarkExecTrue-20    	     800	   7538839 ns/op
BenchmarkExecTrue-20    	     717	   7424755 ns/op
BenchmarkExecTrue-20    	     848	   7747787 ns/op
BenchmarkExecTrue-20    	     800	   7668740 ns/op
BenchmarkExecTrue-20    	     751	   7304373 ns/op
PASS

Signed-off-by: Kir Kolyshkin <[email protected]>
Copy link
Member

@rata rata left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kolyshkin thanks! LGTM

@rata rata merged commit 4ad9f7f into opencontainers:main Oct 25, 2024
42 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants