[fix](scan) Incorrect scan keys lead to wrong query results. #40814
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thank you for your contribution to Apache Doris. Since 2024-03-18, the Document has been moved to doris-website. |
|
run buildall |
TPC-H: Total hot run time: 43301 ms |
|
TeamCity be ut coverage result: |
TPC-DS: Total hot run time: 194906 ms |
ClickBench: Total hot run time: 31.98 s |
mrhhsg
force-pushed
the
fix_scan_keys
branch
from
f189976
to
4a27742
Compare
|
run buildall |
|
TeamCity be ut coverage result: |
TPC-H: Total hot run time: 43265 ms |
mrhhsg
force-pushed
the
fix_scan_keys
branch
from
4a27742
to
dd3b0a1
Compare
|
run buildall |
TPC-H: Total hot run time: 43042 ms |
|
TeamCity be ut coverage result: |
TPC-DS: Total hot run time: 195840 ms |
ClickBench: Total hot run time: 30.86 s |
|
does 2.1 and 2.0 also have this problem? |
yiguolei
reviewed
Sep 18, 2024
| """ | ||
|
|
||
| qt_select1 " select * from test_scan_keys_with_bool_type order by 1, 2, 3, 4, 5, 6, 7; " | ||
| qt_select2 " select * from test_scan_keys_with_bool_type where col1 <= -100 and col2 in (true, false) and col3 = -82 order by 1, 2, 3, 4, 5, 6, 7; " |
There was a problem hiding this comment.
- col2 int, col2 in (1,2)
- col2 char, col2 in ('a','b')
yiguolei
reviewed
Sep 18, 2024
| `col7` datetime(3) REPLACE_IF_NOT_NULL NULL | ||
| ) ENGINE=OLAP | ||
| AGGREGATE KEY(`col1`, `col2`, `col3`) | ||
| DISTRIBUTED BY HASH(`col1`, `col2`, `col3`) BUCKETS 4 |
There was a problem hiding this comment.
set buckets to 1
disable compaction
mrhhsg
force-pushed
the
fix_scan_keys
branch
from
dd3b0a1
to
e4a07ab
Compare
|
run buildall |
|
TeamCity be ut coverage result: |
TPC-H: Total hot run time: 41695 ms |
TPC-DS: Total hot run time: 198545 ms |
ClickBench: Total hot run time: 33.12 s |
github-actions
bot
added
the
approved
|
PR approved by at least one committer and no changes requested. |
|
PR approved by anyone and no changes requested. |
mrhhsg
added a commit
to mrhhsg/doris
that referenced
this pull request
Sep 19, 2024
…40814) ## Proposed changes ``` mysql [doris_14555]>select * from table_9436528_3; +------+------+------+------+------------------------+--------------------+------+ | col1 | col2 | col3 | col5 | col4 | col6 | col7 | +------+------+------+------+------------------------+--------------------+------+ | -100 | 1 | -82 | 1 | 2024-02-16 04:37:37.00 | -1299962421.904282 | NULL | | -100 | 1 | 92 | 1 | 2024-02-16 04:37:37.00 | 23423423.0324234 | NULL | | -100 | 0 | -82 | 0 | 2023-11-11 10:49:43.00 | 840968969.872149 | NULL | ``` wrong result: ``` mysql [doris_14555]>select * from table_9436528_3 where col1 <= -100 and col2 in (true, false) and col3 = -82; +------+------+------+------+------------------------+--------------------+------+ | col1 | col2 | col3 | col5 | col4 | col6 | col7 | +------+------+------+------+------------------------+--------------------+------+ | -100 | 1 | -82 | 1 | 2024-02-16 04:37:37.00 | -1299962421.904282 | NULL | | -100 | 1 | 92 | 1 | 2024-02-16 04:37:37.00 | 23423423.0324234 | NULL | +------+------+------+------+------------------------+--------------------+------+ ```
mrhhsg
added a commit
to mrhhsg/doris
that referenced
this pull request
Sep 19, 2024
…40814) ``` mysql [doris_14555]>select * from table_9436528_3; +------+------+------+------+------------------------+--------------------+------+ | col1 | col2 | col3 | col5 | col4 | col6 | col7 | +------+------+------+------+------------------------+--------------------+------+ | -100 | 1 | -82 | 1 | 2024-02-16 04:37:37.00 | -1299962421.904282 | NULL | | -100 | 1 | 92 | 1 | 2024-02-16 04:37:37.00 | 23423423.0324234 | NULL | | -100 | 0 | -82 | 0 | 2023-11-11 10:49:43.00 | 840968969.872149 | NULL | ``` wrong result: ``` mysql [doris_14555]>select * from table_9436528_3 where col1 <= -100 and col2 in (true, false) and col3 = -82; +------+------+------+------+------------------------+--------------------+------+ | col1 | col2 | col3 | col5 | col4 | col6 | col7 | +------+------+------+------+------------------------+--------------------+------+ | -100 | 1 | -82 | 1 | 2024-02-16 04:37:37.00 | -1299962421.904282 | NULL | | -100 | 1 | 92 | 1 | 2024-02-16 04:37:37.00 | 23423423.0324234 | NULL | +------+------+------+------+------------------------+--------------------+------+ ```
yiguolei
pushed a commit
that referenced
this pull request
Sep 19, 2024
…#40971) ## Proposed changes pick #40814 ``` mysql [doris_14555]>select * from table_9436528_3; +------+------+------+------+------------------------+--------------------+------+ | col1 | col2 | col3 | col5 | col4 | col6 | col7 | +------+------+------+------+------------------------+--------------------+------+ | -100 | 1 | -82 | 1 | 2024-02-16 04:37:37.00 | -1299962421.904282 | NULL | | -100 | 1 | 92 | 1 | 2024-02-16 04:37:37.00 | 23423423.0324234 | NULL | | -100 | 0 | -82 | 0 | 2023-11-11 10:49:43.00 | 840968969.872149 | NULL | ``` wrong result: ``` mysql [doris_14555]>select * from table_9436528_3 where col1 <= -100 and col2 in (true, false) and col3 = -82; +------+------+------+------+------------------------+--------------------+------+ | col1 | col2 | col3 | col5 | col4 | col6 | col7 | +------+------+------+------+------------------------+--------------------+------+ | -100 | 1 | -82 | 1 | 2024-02-16 04:37:37.00 | -1299962421.904282 | NULL | | -100 | 1 | 92 | 1 | 2024-02-16 04:37:37.00 | 23423423.0324234 | NULL | +------+------+------+------+------------------------+--------------------+------+ ``` ## Proposed changes Issue Number: close #xxx <!--Describe your changes.-->
dataroaring
pushed a commit
that referenced
this pull request
Sep 26, 2024
## Proposed changes ``` mysql [doris_14555]>select * from table_9436528_3; +------+------+------+------+------------------------+--------------------+------+ | col1 | col2 | col3 | col5 | col4 | col6 | col7 | +------+------+------+------+------------------------+--------------------+------+ | -100 | 1 | -82 | 1 | 2024-02-16 04:37:37.00 | -1299962421.904282 | NULL | | -100 | 1 | 92 | 1 | 2024-02-16 04:37:37.00 | 23423423.0324234 | NULL | | -100 | 0 | -82 | 0 | 2023-11-11 10:49:43.00 | 840968969.872149 | NULL | ``` wrong result: ``` mysql [doris_14555]>select * from table_9436528_3 where col1 <= -100 and col2 in (true, false) and col3 = -82; +------+------+------+------+------------------------+--------------------+------+ | col1 | col2 | col3 | col5 | col4 | col6 | col7 | +------+------+------+------+------------------------+--------------------+------+ | -100 | 1 | -82 | 1 | 2024-02-16 04:37:37.00 | -1299962421.904282 | NULL | | -100 | 1 | 92 | 1 | 2024-02-16 04:37:37.00 | 23423423.0324234 | NULL | +------+------+------+------+------------------------+--------------------+------+ ```
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Proposed changes
wrong result: