Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: select count(*) error after loading data on eks with 3*cn(occasional) #10213

Closed
1 task done
aressu1985 opened this issue Jun 25, 2023 · 35 comments
Closed
1 task done

[Bug]: select count(*) error after loading data on eks with 3*cn(occasional) #10213

aressu1985 opened this issue Jun 25, 2023 · 35 comments
Assignees
@daviszhen
Labels
kind/bug Something isn't working severity/s0 Extreme impact: Cause the application to break down and seriously affect the use
Milestone
1.0.0

Comments

@aressu1985
Copy link
Contributor

aressu1985 commented Jun 25, 2023 •
edited
Loading

Is there an existing issue for the same bug?

  • I have checked the existing issues.

Environment

- Version or commit-id (e.g. v0.1.0 or 8b23a93): 6c14bc3
- Hardware parameters:
  2*CN : 16C  64G
  1*DN : 16C  64G
  3*LOG : 16C  64G
- OS type:
- Others:
  MEMCACHE: 16G
  DISKCACHE: 100G

Actual Behavior

on daily regression on eks JUN 25th, there was a error that result of select count(*) is 0 after loading tpch 100G lineitem data.

image

the job link:
https://github.com/matrixorigin/mo-nightly-regression/actions/runs/5367083303/jobs/9737259242

the mo log can be get from loki by ns: mo-nightly-regression and the time nearly at : 2023-06-25 02:17:51

Expected Behavior

result of select count(*) should be equal to the real count of table

Steps to Reproduce

It is very hard to repro.

Additional information

No response
@aressu1985 aressu1985 added kind/bug Something isn't working needs-triage severity/s0 Extreme impact: Cause the application to break down and seriously affect the use labels Jun 25, 2023
@aressu1985 aressu1985 added this to the V0.8.0 milestone Jun 25, 2023
@xzxiong xzxiong assigned jianwan0214 and unassigned matrix-meow Jun 25, 2023
@xzxiong
Copy link
Contributor

xzxiong commented Jun 25, 2023

@jianwan0214 ptal

@aressu1985
Copy link
Contributor Author

repro again in this job link:
https://github.com/matrixorigin/mo-nightly-regression/actions/runs/5370645833/jobs/9743466828

image

log can be get from loki by ns:mo-nightly-regression-20230625

@domingozhang domingozhang modified the milestones: V0.8.0, V0.8.0-Backlog Jun 30, 2023
@jianwan0214
Copy link
Contributor

未复现，还在定位中

@domingozhang domingozhang modified the milestones: V0.8.0-Backlog, 1.0.0 Jun 30, 2023
@jianwan0214
Copy link
Contributor

本地未复现

@jianwan0214
Copy link
Contributor

未复现

1 similar comment
@jianwan0214
Copy link
Contributor

未复现

@daviszhen daviszhen assigned daviszhen and unassigned jianwan0214 Jul 26, 2023
@daviszhen
Copy link
Contributor

还没看

6 similar comments
@daviszhen
Copy link
Contributor

还没看

@daviszhen
Copy link
Contributor

还没看

@daviszhen
Copy link
Contributor

还没看

@daviszhen
Copy link
Contributor

还没看

@daviszhen
Copy link
Contributor

还没看

@daviszhen
Copy link
Contributor

还没看

@daviszhen
Copy link
Contributor

@aressu1985 这个问题最近有复现吗?

@daviszhen
Copy link
Contributor

与@aressu1985 沟通后，先走给他

@daviszhen daviszhen assigned aressu1985 and unassigned daviszhen Aug 7, 2023
@heni02
Copy link
Contributor

heni02 commented Aug 10, 2023

It no longer exists, confirm closed
commit id:8af14455f059e657890f4249ca63699476bb7f63
date 8.9 eks regression job:https://github.com/matrixorigin/mo-nightly-regression/actions/runs/5811389178/job/15755992500
企业微信截图_8ff5f0b3-ea46-47ba-acba-5b8ee95ececb

@sukki37 sukki37 closed this as completed Aug 10, 2023
@Ariznawlll
Copy link
Contributor

Load data on eks with 4*cn：
image
job url：https://github.com/matrixorigin/mo-auto-test/actions/runs/6032290632/job/16367409706
log：http://abb38738d9c5441738d2e8e3854fe740-1814584411.us-west-2.elb.amazonaws.com/explore?orgId=1&left=%7B%22datasource%22:%22P8E80F9AEF21F6940%22,%22queries%22:%5B%7B%22refId%22:%22A%22,%22expr%22:%22%7Bnamespace%3D%5C%22mo-tpch%5C%22,%20pod%3D%5C%22mo-tpch-tp-cn-7qnhb%5C%22%7D%20%7C%3D%20%60%60%22,%22queryType%22:%22range%22,%22datasource%22:%7B%22type%22:%22loki%22,%22uid%22:%22P8E80F9AEF21F6940%22%7D,%22editorMode%22:%22builder%22%7D%5D,%22range%22:%7B%22from%22:%22now-24h%22,%22to%22:%22now%22%7D%7D

@Ariznawlll Ariznawlll reopened this Sep 1, 2023
@Ariznawlll Ariznawlll assigned daviszhen and unassigned aressu1985 Sep 1, 2023
@daviszhen
Copy link
Contributor

还没看

4 similar comments
@daviszhen
Copy link
Contributor

还没看

@daviszhen
Copy link
Contributor

还没看

@daviszhen
Copy link
Contributor

还没看

@daviszhen
Copy link
Contributor

还没看

@Ariznawlll
Copy link
Contributor

Load 1T data on eks with 3CN：
image
job：https://github.com/matrixorigin/mo-auto-test/actions/runs/6140373265/job/16660058495

@daviszhen
Copy link
Contributor

还没看

1 similar comment
@daviszhen
Copy link
Contributor

还没看

@daviszhen
Copy link
Contributor

还没看。

@daviszhen
Copy link
Contributor

还没看

7 similar comments
@daviszhen
Copy link
Contributor

还没看

@daviszhen
Copy link
Contributor

还没看

@daviszhen
Copy link
Contributor

还没看

@daviszhen
Copy link
Contributor

还没看

@daviszhen
Copy link
Contributor

还没看

@daviszhen
Copy link
Contributor

还没看

@daviszhen
Copy link
Contributor

还没看

@daviszhen
Copy link
Contributor

还没看

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@daviszhen daviszhen

Labels
kind/bug Something isn't working severity/s0 Extreme impact: Cause the application to break down and seriously affect the use
Projects
None yet
Milestone
1.0.0
Development

No branches or pull requests
9 participants
@xzxiong @jianwan0214 @aressu1985 @daviszhen @sukki37 @domingozhang @Ariznawlll @matrix-meow @heni02