[fix](rf)Fix ColumnPredicate rf may not reset judge #58466

Mryange · 2025-11-27T10:47:01Z

What problem does this PR solve?

Previously, if always_true() returned true, judge would not be reset.
it was reset inside do_judge_selectivity.

        if (always_true()) {
            return size;
        }

        uint16_t new_size = _evaluate_inner(column, sel, size);
        if (_can_ignore()) {
            do_judge_selectivity(size - new_size, size);
        }
        update_filter_info(size - new_size, size);
        return new_size;

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

Thearas · 2025-11-27T10:47:07Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

Mryange · 2025-11-27T10:47:56Z

run buildall

be/src/olap/column_predicate.h

doris-robot · 2025-11-27T12:38:57Z

TPC-H: Total hot run time: 34311 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5acc0e8199cfd705ab4949775438899462c1e40d, data reload: false

------ Round 1 ----------------------------------
q1	17590	5095	4964	4964
q2	1997	316	217	217
q3	10267	1329	732	732
q4	10223	841	315	315
q5	7559	2572	2235	2235
q6	198	171	140	140
q7	977	814	669	669
q8	9375	1344	1022	1022
q9	7185	5442	5371	5371
q10	6832	2191	1761	1761
q11	525	313	282	282
q12	391	365	231	231
q13	17787	3671	3045	3045
q14	244	248	213	213
q15	598	517	512	512
q16	1038	1037	944	944
q17	608	737	520	520
q18	7530	7085	7044	7044
q19	1097	952	560	560
q20	342	339	227	227
q21	3556	3755	2361	2361
q22	1017	1017	946	946
Total cold run time: 106936 ms
Total hot run time: 34311 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4931	4957	4941	4941
q2	333	413	314	314
q3	2138	2665	2270	2270
q4	1305	1788	1294	1294
q5	4223	4430	4642	4430
q6	239	189	137	137
q7	2072	1963	1868	1868
q8	2694	2619	2517	2517
q9	7590	7527	7450	7450
q10	3081	3246	2833	2833
q11	610	511	492	492
q12	664	748	646	646
q13	3658	3962	3243	3243
q14	327	302	269	269
q15	550	512	514	512
q16	1058	1149	1078	1078
q17	1327	1407	1386	1386
q18	7914	7743	7414	7414
q19	793	764	790	764
q20	1978	1980	1788	1788
q21	4597	4229	4086	4086
q22	1053	1038	998	998
Total cold run time: 53135 ms
Total hot run time: 50730 ms

doris-robot · 2025-11-27T12:50:25Z

TPC-DS: Total hot run time: 184934 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5acc0e8199cfd705ab4949775438899462c1e40d, data reload: false

query1	1036	414	397	397
query2	6630	1544	1574	1544
query3	6736	227	224	224
query4	25390	23296	23003	23003
query5	4948	651	484	484
query6	341	258	223	223
query7	4660	522	322	322
query8	309	267	252	252
query9	8742	2638	2598	2598
query10	551	351	310	310
query11	15165	15188	14906	14906
query12	185	124	117	117
query13	1705	579	445	445
query14	11049	8925	8789	8789
query15	218	200	193	193
query16	7664	727	536	536
query17	1222	775	663	663
query18	2051	441	351	351
query19	226	216	188	188
query20	140	126	121	121
query21	223	149	118	118
query22	3942	3875	3945	3875
query23	32940	31805	32018	31805
query24	8460	2455	2433	2433
query25	607	526	468	468
query26	1248	275	171	171
query27	2738	482	358	358
query28	4315	2159	2159	2159
query29	781	635	520	520
query30	310	240	213	213
query31	815	719	639	639
query32	82	75	76	75
query33	614	381	337	337
query34	834	873	545	545
query35	833	835	746	746
query36	895	946	820	820
query37	127	119	91	91
query38	3304	3450	3253	3253
query39	1472	1408	1448	1408
query40	227	139	120	120
query41	66	64	60	60
query42	125	110	109	109
query43	437	466	422	422
query44	1299	755	753	753
query45	200	193	190	190
query46	867	1002	673	673
query47	1696	1715	1642	1642
query48	403	436	340	340
query49	793	507	412	412
query50	662	696	414	414
query51	3982	3945	3870	3870
query52	114	118	105	105
query53	241	271	196	196
query54	312	306	291	291
query55	94	93	94	93
query56	335	322	327	322
query57	1148	1162	1096	1096
query58	301	285	279	279
query59	2352	2418	2344	2344
query60	355	346	340	340
query61	169	165	165	165
query62	818	735	666	666
query63	234	196	202	196
query64	4593	1230	894	894
query65	4087	3975	4006	3975
query66	1108	441	350	350
query67	15012	14869	14788	14788
query68	5812	1013	622	622
query69	525	348	335	335
query70	1244	1184	1151	1151
query71	448	342	322	322
query72	5816	4969	4869	4869
query73	669	568	345	345
query74	8809	8848	8409	8409
query75	3335	3367	2837	2837
query76	3329	1138	728	728
query77	506	405	316	316
query78	9594	9593	8971	8971
query79	1493	880	588	588
query80	1693	604	518	518
query81	565	270	246	246
query82	437	163	136	136
query83	371	268	265	265
query84	263	123	96	96
query85	912	497	461	461
query86	388	295	291	291
query87	3468	3489	3428	3428
query88	3018	2321	2299	2299
query89	387	341	307	307
query90	1749	235	228	228
query91	180	174	141	141
query92	69	72	68	68
query93	1205	995	669	669
query94	763	473	380	380
query95	533	437	434	434
query96	528	569	295	295
query97	2911	3007	2890	2890
query98	241	217	217	217
query99	1321	1413	1267	1267
Total cold run time: 268120 ms
Total hot run time: 184934 ms

doris-robot · 2025-11-27T12:55:28Z

ClickBench: Total hot run time: 27.53 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5acc0e8199cfd705ab4949775438899462c1e40d, data reload: false

query1	0.06	0.05	0.04
query2	0.10	0.05	0.04
query3	0.26	0.09	0.09
query4	1.60	0.11	0.11
query5	0.27	0.27	0.27
query6	1.16	0.63	0.63
query7	0.03	0.03	0.03
query8	0.06	0.04	0.04
query9	0.59	0.52	0.51
query10	0.55	0.56	0.56
query11	0.16	0.11	0.10
query12	0.14	0.12	0.12
query13	0.64	0.60	0.60
query14	0.99	0.96	1.00
query15	0.82	0.79	0.82
query16	0.40	0.42	0.40
query17	1.04	1.04	1.06
query18	0.23	0.24	0.22
query19	1.95	1.78	1.76
query20	0.02	0.01	0.01
query21	15.44	0.27	0.14
query22	4.96	0.05	0.05
query23	16.13	0.27	0.11
query24	1.14	0.73	0.34
query25	0.09	0.08	0.05
query26	0.15	0.14	0.13
query27	0.07	0.06	0.05
query28	3.35	1.22	1.02
query29	12.59	3.94	3.26
query30	0.28	0.14	0.13
query31	2.82	0.60	0.40
query32	3.26	0.55	0.47
query33	3.05	3.08	3.17
query34	17.09	5.21	4.57
query35	4.58	4.54	4.55
query36	0.67	0.50	0.50
query37	0.11	0.08	0.07
query38	0.07	0.04	0.04
query39	0.05	0.03	0.04
query40	0.20	0.15	0.14
query41	0.08	0.03	0.03
query42	0.04	0.03	0.03
query43	0.05	0.03	0.04
Total cold run time: 97.34 s
Total hot run time: 27.53 s

be/src/olap/column_predicate.h

Mryange · 2025-11-27T16:19:00Z

run buildall

doris-robot · 2025-11-27T17:12:44Z

TPC-H: Total hot run time: 34502 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit de0785dd07180e60e0ff14215b1a20c5a075649c, data reload: false

------ Round 1 ----------------------------------
q1	17642	5042	4956	4956
q2	2053	315	223	223
q3	10246	1336	770	770
q4	10252	914	326	326
q5	7501	2421	2174	2174
q6	186	173	138	138
q7	938	790	657	657
q8	9349	1430	1044	1044
q9	7105	5273	5395	5273
q10	6852	2214	1807	1807
q11	517	311	282	282
q12	336	375	237	237
q13	17774	3693	3043	3043
q14	232	239	227	227
q15	593	511	521	511
q16	1062	1024	972	972
q17	618	711	556	556
q18	8073	7090	7126	7090
q19	1097	961	602	602
q20	385	371	241	241
q21	3594	3494	2435	2435
q22	1058	972	938	938
Total cold run time: 107463 ms
Total hot run time: 34502 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5099	4956	4929	4929
q2	348	395	308	308
q3	2125	2694	2312	2312
q4	1304	1800	1323	1323
q5	4234	4686	4621	4621
q6	239	185	128	128
q7	2056	2010	1835	1835
q8	2647	2660	2557	2557
q9	7773	7530	7365	7365
q10	3084	3297	2870	2870
q11	586	537	503	503
q12	660	713	622	622
q13	3477	3968	3215	3215
q14	295	290	301	290
q15	573	523	533	523
q16	1125	1150	1066	1066
q17	1167	1512	1452	1452
q18	7823	7634	7579	7579
q19	845	826	826	826
q20	1943	2072	1906	1906
q21	4916	4212	4143	4143
q22	1095	1020	977	977
Total cold run time: 53414 ms
Total hot run time: 51350 ms

doris-robot · 2025-11-27T17:24:17Z

TPC-DS: Total hot run time: 185249 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit de0785dd07180e60e0ff14215b1a20c5a075649c, data reload: false

query1	1075	408	399	399
query2	6559	1595	1626	1595
query3	6742	236	225	225
query4	25700	23087	22940	22940
query5	4525	647	462	462
query6	348	253	233	233
query7	4657	507	299	299
query8	323	269	261	261
query9	8708	2663	2642	2642
query10	529	361	309	309
query11	15182	15151	14952	14952
query12	204	126	119	119
query13	1689	573	464	464
query14	11038	8902	8936	8902
query15	212	204	193	193
query16	7598	697	514	514
query17	1257	792	657	657
query18	2047	441	355	355
query19	221	209	185	185
query20	133	125	124	124
query21	233	138	119	119
query22	3961	4077	3781	3781
query23	33037	31925	32167	31925
query24	8498	2456	2433	2433
query25	648	570	507	507
query26	1256	297	167	167
query27	2717	500	372	372
query28	4371	2172	2165	2165
query29	837	659	563	563
query30	314	247	219	219
query31	826	700	634	634
query32	89	76	77	76
query33	615	390	399	390
query34	810	861	550	550
query35	801	819	761	761
query36	879	928	846	846
query37	125	116	88	88
query38	3354	3357	3240	3240
query39	1579	1431	1432	1431
query40	228	135	125	125
query41	64	62	62	62
query42	127	110	111	110
query43	448	448	433	433
query44	1257	762	751	751
query45	205	192	188	188
query46	885	1004	645	645
query47	1700	1719	1649	1649
query48	386	437	331	331
query49	771	508	406	406
query50	653	683	409	409
query51	3884	3920	3875	3875
query52	115	111	109	109
query53	236	253	196	196
query54	310	311	287	287
query55	95	95	95	95
query56	342	319	315	315
query57	1157	1170	1092	1092
query58	297	284	289	284
query59	2349	2424	2316	2316
query60	372	361	327	327
query61	166	155	165	155
query62	795	731	650	650
query63	235	197	195	195
query64	4574	1222	917	917
query65	4081	3941	3957	3941
query66	1136	443	344	344
query67	15371	14985	15036	14985
query68	8387	964	630	630
query69	530	354	309	309
query70	1302	1195	1102	1102
query71	488	340	312	312
query72	5778	4913	4908	4908
query73	710	576	344	344
query74	8915	8920	8560	8560
query75	4031	3314	2850	2850
query76	3795	1139	707	707
query77	834	398	319	319
query78	9509	9652	8843	8843
query79	2046	850	603	603
query80	654	585	510	510
query81	502	271	243	243
query82	430	165	134	134
query83	285	269	253	253
query84	251	119	100	100
query85	931	499	465	465
query86	348	286	300	286
query87	3495	3598	3397	3397
query88	3653	2316	2285	2285
query89	390	332	300	300
query90	1984	235	223	223
query91	190	180	148	148
query92	85	72	68	68
query93	1191	1000	662	662
query94	741	469	360	360
query95	522	424	404	404
query96	509	563	289	289
query97	2929	2992	2869	2869
query98	245	220	215	215
query99	1412	1397	1262	1262
Total cold run time: 273008 ms
Total hot run time: 185249 ms

doris-robot · 2025-11-27T17:29:22Z

ClickBench: Total hot run time: 27.63 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit de0785dd07180e60e0ff14215b1a20c5a075649c, data reload: false

query1	0.05	0.05	0.05
query2	0.11	0.05	0.05
query3	0.27	0.09	0.08
query4	1.62	0.12	0.12
query5	0.28	0.25	0.26
query6	1.14	0.66	0.63
query7	0.03	0.03	0.03
query8	0.06	0.04	0.05
query9	0.57	0.51	0.51
query10	0.55	0.55	0.56
query11	0.16	0.12	0.12
query12	0.15	0.11	0.12
query13	0.64	0.60	0.60
query14	1.00	0.99	0.99
query15	0.82	0.79	0.79
query16	0.39	0.43	0.40
query17	1.01	1.02	1.00
query18	0.24	0.22	0.22
query19	1.94	1.81	1.82
query20	0.02	0.01	0.02
query21	15.44	0.26	0.14
query22	4.64	0.05	0.05
query23	16.00	0.26	0.10
query24	2.28	0.58	0.63
query25	0.06	0.05	0.05
query26	0.16	0.13	0.13
query27	0.06	0.08	0.05
query28	4.85	1.22	1.03
query29	12.59	3.95	3.17
query30	0.28	0.14	0.12
query31	2.81	0.61	0.39
query32	3.24	0.54	0.50
query33	3.01	3.04	3.10
query34	16.74	5.27	4.52
query35	4.55	4.55	4.59
query36	0.66	0.50	0.50
query37	0.11	0.07	0.06
query38	0.07	0.05	0.03
query39	0.05	0.03	0.04
query40	0.16	0.14	0.14
query41	0.08	0.04	0.03
query42	0.04	0.02	0.03
query43	0.04	0.04	0.04
Total cold run time: 98.97 s
Total hot run time: 27.63 s

doris-robot · 2025-11-27T18:47:10Z

BE UT Coverage Report

Increment line coverage 57.14% (4/7) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.89% (18505/34987)
Line Coverage	38.42% (169475/441056)
Region Coverage	33.23% (131565/395972)
Branch Coverage	34.18% (56666/165771)

hello-stephen · 2025-11-27T20:04:45Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (7/7) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.66% (24582/34305)
Line Coverage	58.20% (256448/440662)
Region Coverage	53.26% (213446/400782)
Branch Coverage	54.78% (91293/166666)

BiteTheDDDDt

LGTM

HappenLee · 2025-12-01T03:10:52Z

be/src/olap/column_predicate.h

-    void do_judge_selectivity(uint64_t filter_rows, uint64_t input_rows) const {
-        if ((_judge_counter--) == 0) {
+    void try_reset_judge_selectivity() const {
+        if (_can_ignore() && ((_judge_counter--) == 0)) {


why reset judge selectivity need _can_ignore()

HappenLee

LGTM

github-actions · 2025-12-01T03:22:25Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-12-01T03:22:28Z

PR approved by anyone and no changes requested.

Previously, if always_true() returned true, judge would not be reset. it was reset inside do_judge_selectivity. ``` if (always_true()) { return size; } uint16_t new_size = _evaluate_inner(column, sel, size); if (_can_ignore()) { do_judge_selectivity(size - new_size, size); } update_filter_info(size - new_size, size); return new_size; ```

#58558) Cherry-picked from #58466 Co-authored-by: Mryange <[email protected]>

fix

5acc0e8

BiteTheDDDDt reviewed Nov 27, 2025

View reviewed changes

be/src/olap/column_predicate.h Show resolved Hide resolved

BiteTheDDDDt reviewed Nov 27, 2025

View reviewed changes

be/src/olap/column_predicate.h Outdated Show resolved Hide resolved

fix

de0785d

BiteTheDDDDt added the dev/4.0.x label Nov 28, 2025

BiteTheDDDDt reviewed Nov 28, 2025

View reviewed changes

HappenLee reviewed Dec 1, 2025

View reviewed changes

HappenLee approved these changes Dec 1, 2025

View reviewed changes

github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Dec 1, 2025

BiteTheDDDDt approved these changes Dec 1, 2025

View reviewed changes

BiteTheDDDDt merged commit f931a29 into apache:master Dec 1, 2025
29 of 33 checks passed

github-actions bot mentioned this pull request Dec 1, 2025

branch-4.0: [fix](rf)Fix ColumnPredicate rf may not reset judge #58466 #58558

Merged

yiguolei pushed a commit that referenced this pull request Dec 2, 2025

branch-4.0: [fix](rf)Fix ColumnPredicate rf may not reset judge #58466 (

daa1b6f

#58558) Cherry-picked from #58466 Co-authored-by: Mryange <[email protected]>

yiguolei added dev/4.0.2-merged and removed dev/4.0.x labels Dec 2, 2025

[fix](rf)Fix ColumnPredicate rf may not reset judge #58466

[fix](rf)Fix ColumnPredicate rf may not reset judge #58466

Conversation

Mryange commented Nov 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

Thearas commented Nov 27, 2025

Uh oh!

Mryange commented Nov 27, 2025

Uh oh!

Uh oh!

doris-robot commented Nov 27, 2025

Uh oh!

doris-robot commented Nov 27, 2025

Uh oh!

doris-robot commented Nov 27, 2025

Uh oh!

Uh oh!

Mryange commented Nov 27, 2025

Uh oh!

doris-robot commented Nov 27, 2025

Uh oh!

doris-robot commented Nov 27, 2025

Uh oh!

doris-robot commented Nov 27, 2025

Uh oh!

doris-robot commented Nov 27, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented Nov 27, 2025

BE Regression && UT Coverage Report

Uh oh!

BiteTheDDDDt left a comment

Choose a reason for hiding this comment

Uh oh!

HappenLee Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

HappenLee left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Dec 1, 2025

Uh oh!

github-actions bot commented Dec 1, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Mryange commented Nov 27, 2025 •

edited

Loading