Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: MO oom during stability test #4845

Closed
1 task done
aressu1985 opened this issue Sep 2, 2022 · 11 comments
Closed
1 task done

[Bug]: MO oom during stability test #4845

aressu1985 opened this issue Sep 2, 2022 · 11 comments
Assignees
Labels
kind/bug Something isn't working priority/p0 Critical feature that should be implemented in this version triage/accepted
Milestone

Comments

@aressu1985
Copy link
Contributor

Is there an existing issue for the same bug?

  • I have checked the existing issues.

Environment

- Version or commit-id (e.g. v0.1.0 or 8b23a93):0.6.0
- Hardware parameters:
- OS type:
- Others:

Actual Behavior

The mo was killed by oom when executing stability test
the test is as followings:
1、bvt test loop run, 1000 times
2、tpch(0.5G) query loop run ,1000 times
3、tp mixed cases run,20 hours

and nearly 12 hours after test start,the mo oom:
[74218.421881] [ 559341] 0 559341 19144 225 303104 0 0 sshd
[74218.421884] [ 559344] 0 559344 8586 39 151552 0 0 sshd
[74218.421886] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0-1,global_oom,task_memcg=/user.slice/user-1029.slice/session-16.scope,task=mo-service,pid=68923,uid=1029
[74218.422002] Out of memory: Killed process 68923 (mo-service) total-vm:277634136kB, anon-rss:251547972kB, file-rss:0kB, shmem-rss:0kB, UID:1029 pgtables:530216kB oom_score_adj:0
[74223.476360] oom_reaper: reaped process 68923 (mo-service), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB

Expected Behavior

No response

Steps to Reproduce

No response

Additional information

No response

@aressu1985 aressu1985 added kind/bug Something isn't working needs-triage severity/s0 Extreme impact: Cause the application to break down and seriously affect the use labels Sep 2, 2022
@aressu1985 aressu1985 added this to the v0.6.0 milestone Sep 2, 2022
@florashi181 florashi181 assigned yingfeng and unassigned jinqinghui Nov 4, 2022
@yingfeng yingfeng modified the milestones: v0.6.0, v0.7.0 Nov 8, 2022
@yingfeng
Copy link
Contributor

yingfeng commented Nov 8, 2022

Dedicated issue to track OOM, will continuously track in 0.7

@yingfeng
Copy link
Contributor

Remove S0 temporarily and need to reassign after restarting

@yingfeng yingfeng removed the severity/s0 Extreme impact: Cause the application to break down and seriously affect the use label Nov 12, 2022
@fengttt fengttt assigned nnsgmsone and unassigned yingfeng Nov 15, 2022
@nnsgmsone nnsgmsone assigned aressu1985 and unassigned nnsgmsone Jan 17, 2023
@nnsgmsone
Copy link
Contributor

same as #7613

@aressu1985
Copy link
Contributor Author

tracking for one more day

@nnsgmsone
Copy link
Contributor

no process

3 similar comments
@nnsgmsone
Copy link
Contributor

no process

@nnsgmsone
Copy link
Contributor

no process

@nnsgmsone
Copy link
Contributor

no process

@nnsgmsone
Copy link
Contributor

I will give a plan to take care of the memory problem.

@fengttt fengttt removed their assignment Feb 1, 2023
@fengttt fengttt modified the milestones: v0.7.0, V0.8.0-Backlog Feb 1, 2023
@fengttt
Copy link
Contributor

fengttt commented Feb 1, 2023

Taking this off from S-1. @nnsgmsone has his grand plan of addressing this in a general, systematic way.

@florashi181 florashi181 modified the milestones: V0.8.0-Backlog, Backlog Feb 9, 2023
@fengttt fengttt added the priority/p0 Critical feature that should be implemented in this version label Feb 9, 2023
@fengttt fengttt modified the milestones: Backlog, V0.8.0-Backlog Feb 9, 2023
@nnsgmsone nnsgmsone modified the milestones: V0.8.0-Backlog, V0.8.0 Feb 13, 2023
@aressu1985
Copy link
Contributor Author

I closed this , and tracked it with #8327

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Something isn't working priority/p0 Critical feature that should be implemented in this version triage/accepted
Projects
None yet
Development

No branches or pull requests

7 participants