[Fix](Query Stats) Add QueryStatsRecorder for column-level query and filter - Part1

[nsivarajan]

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Show Query Stats returns only 0, as it is broken . This PR is intended to make Query and Filter Stats to available when
below config is enabled.

  admin set frontend config ("enable_query_hit_stats" = "true");
  set enable_nereids_planner = true;

This is planned as Sequence of PR with incremental capture of more cases later (like Group by, order by, Window and more).

Release note

None

Check List (For Author)

• Test

  • Regression test
  • Unit Test
  • Manual test (add detailed scripts or steps below)
  • No need to test or manual test. Explain why:
    • This is a refactor/code format and no logic has been changed.
    • Previous test can cover this change.
    • No code files have been changed.
    • Other reason

• Behavior changed:

  • No.
  • Yes.

• Does this need documentation?

  • No.
  • Yes.

Check List (For Reviewer who merge this PR)

• Confirm the release note
• Confirm test cases
• Confirm document
• Add branch pick label

[hello-stephen]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?

[nsivarajan]

run buildall

[nsivarajan]

run buildall

[hello-stephen]

TPC-H: Total hot run time: 29525 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bca335380238e64d9cb7fd1630450bc2ac9a3620, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17634	3930	3887	3887
q2	q3	10712	893	624	624
q4	4688	470	339	339
q5	7601	1322	1125	1125
q6	208	166	136	136
q7	935	933	747	747
q8	9574	1397	1298	1298
q9	6631	5439	5334	5334
q10	6344	2065	1790	1790
q11	477	267	263	263
q12	697	417	292	292
q13	18187	3319	2683	2683
q14	296	288	266	266
q15	q16	909	875	787	787
q17	1009	1071	738	738
q18	6542	5738	5611	5611
q19	1291	1210	1086	1086
q20	517	391	259	259
q21	5004	2338	1934	1934
q22	462	407	326	326
Total cold run time: 99718 ms
Total hot run time: 29525 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4677	4559	4540	4540
q2	q3	4696	4823	4206	4206
q4	2115	2199	1389	1389
q5	4987	5002	5295	5002
q6	192	166	132	132
q7	2035	1812	1623	1623
q8	3402	3087	3119	3087
q9	8405	8323	8587	8323
q10	4467	4482	4237	4237
q11	581	400	390	390
q12	699	777	530	530
q13	3244	3592	2989	2989
q14	319	309	288	288
q15	q16	759	774	692	692
q17	1377	1261	1261	1261
q18	8003	7051	7170	7051
q19	1148	1179	1179	1179
q20	2237	2258	1951	1951
q21	6135	5335	4797	4797
q22	527	479	404	404
Total cold run time: 60005 ms
Total hot run time: 54071 ms

[hello-stephen]

TPC-DS: Total hot run time: 171741 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bca335380238e64d9cb7fd1630450bc2ac9a3620, data reload: false

query5	4331	638	515	515
query6	327	224	204	204
query7	4220	581	319	319
query8	329	227	212	212
query9	8835	4042	3970	3970
query10	439	327	296	296
query11	5747	2381	2228	2228
query12	187	129	124	124
query13	1275	606	419	419
query14	6574	5382	5060	5060
query14_1	4396	4390	4358	4358
query15	215	202	185	185
query16	1073	474	447	447
query17	1163	756	610	610
query18	2731	482	347	347
query19	216	200	173	173
query20	140	130	132	130
query21	210	139	118	118
query22	13538	13672	13397	13397
query23	17235	16401	16123	16123
query23_1	16203	16342	16216	16216
query24	7442	1766	1386	1386
query24_1	1351	1379	1385	1379
query25	553	496	425	425
query26	1083	310	168	168
query27	2748	616	339	339
query28	4338	1997	1961	1961
query29	971	619	516	516
query30	301	235	200	200
query31	1127	1063	970	970
query32	88	76	75	75
query33	555	350	298	298
query34	1196	1118	645	645
query35	752	789	682	682
query36	1324	1378	1192	1192
query37	152	107	91	91
query38	3220	3138	3087	3087
query39	933	917	890	890
query39_1	875	873	870	870
query40	233	155	136	136
query41	66	61	63	61
query42	112	107	110	107
query43	332	329	293	293
query44	
query45	211	200	193	193
query46	1099	1173	728	728
query47	2323	2295	2168	2168
query48	403	416	289	289
query49	656	533	422	422
query50	707	299	220	220
query51	4377	4361	4219	4219
query52	103	106	95	95
query53	252	286	213	213
query54	317	267	256	256
query55	93	90	85	85
query56	306	311	301	301
query57	1413	1387	1293	1293
query58	301	282	274	274
query59	1548	1642	1375	1375
query60	362	340	324	324
query61	156	152	146	146
query62	697	625	570	570
query63	248	198	199	198
query64	2084	802	676	676
query65	
query66	1647	515	391	391
query67	29978	29841	29648	29648
query68	
query69	442	353	307	307
query70	1000	994	973	973
query71	306	274	269	269
query72	2905	2697	2504	2504
query73	846	754	434	434
query74	5097	4915	4747	4747
query75	2820	2688	2362	2362
query76	2288	1160	763	763
query77	423	437	359	359
query78	13107	13022	12429	12429
query79	1504	959	754	754
query80	665	603	485	485
query81	450	275	242	242
query82	1294	165	123	123
query83	352	276	247	247
query84	257	146	110	110
query85	850	517	435	435
query86	390	333	309	309
query87	3416	3353	3228	3228
query88	3543	2692	2678	2678
query89	449	390	337	337
query90	1900	179	176	176
query91	179	161	136	136
query92	75	76	73	73
query93	952	973	569	569
query94	551	354	289	289
query95	680	383	349	349
query96	1017	766	341	341
query97	2715	2689	2589	2589
query98	239	233	229	229
query99	1097	1113	986	986
Total cold run time: 252885 ms
Total hot run time: 171741 ms

[nsivarajan]

run buildall

[hello-stephen]

TPC-H: Total hot run time: 29507 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 93d2aaae398f9bbc8839b2486f422994e67ca6e7, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17637	3861	3821	3821
q2	q3	10723	868	614	614
q4	4659	465	352	352
q5	7455	1351	1132	1132
q6	186	167	143	143
q7	901	931	763	763
q8	9321	1357	1280	1280
q9	5624	5358	5330	5330
q10	6259	2089	1778	1778
q11	466	265	258	258
q12	630	417	290	290
q13	18109	3261	2706	2706
q14	286	281	268	268
q15	q16	868	858	794	794
q17	904	964	849	849
q18	6488	5693	5533	5533
q19	1151	1255	1030	1030
q20	509	412	263	263
q21	4700	2481	1973	1973
q22	482	436	330	330
Total cold run time: 97358 ms
Total hot run time: 29507 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4725	4689	4717	4689
q2	q3	4625	4775	4231	4231
q4	2207	2223	1417	1417
q5	5001	4969	5268	4969
q6	189	172	135	135
q7	2080	1801	1631	1631
q8	3321	3142	3122	3122
q9	8414	8344	8407	8344
q10	4488	4523	4228	4228
q11	613	406	382	382
q12	688	744	517	517
q13	3288	3602	2925	2925
q14	304	307	273	273
q15	q16	779	847	773	773
q17	1347	1324	1268	1268
q18	7963	7001	7053	7001
q19	1159	1171	1129	1129
q20	2229	2241	1962	1962
q21	6136	5463	4854	4854
q22	546	511	414	414
Total cold run time: 60102 ms
Total hot run time: 54264 ms

[hello-stephen]

TPC-DS: Total hot run time: 170937 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 93d2aaae398f9bbc8839b2486f422994e67ca6e7, data reload: false

query5	4328	661	518	518
query6	329	213	196	196
query7	4221	542	305	305
query8	320	235	219	219
query9	8806	4004	3999	3999
query10	456	342	294	294
query11	5828	2410	2206	2206
query12	179	133	134	133
query13	1348	596	433	433
query14	6087	5411	5047	5047
query14_1	4406	4413	4389	4389
query15	219	204	185	185
query16	1044	420	447	420
query17	1158	784	635	635
query18	2548	495	372	372
query19	220	212	172	172
query20	138	137	136	136
query21	217	139	119	119
query22	13571	13563	13330	13330
query23	17369	16273	16583	16273
query23_1	16369	16433	16265	16265
query24	7612	1813	1362	1362
query24_1	1398	1323	1370	1323
query25	558	484	431	431
query26	1300	314	167	167
query27	2736	613	331	331
query28	4414	1953	1943	1943
query29	981	612	548	548
query30	290	237	196	196
query31	1096	1063	934	934
query32	84	75	75	75
query33	539	361	290	290
query34	1174	1128	613	613
query35	761	783	670	670
query36	1290	1354	1113	1113
query37	147	100	86	86
query38	3218	3126	3079	3079
query39	922	918	915	915
query39_1	895	870	896	870
query40	236	157	135	135
query41	62	59	60	59
query42	108	106	104	104
query43	326	326	286	286
query44	
query45	204	204	191	191
query46	1084	1269	736	736
query47	2312	2302	2147	2147
query48	398	401	272	272
query49	642	530	439	439
query50	693	296	217	217
query51	4247	4319	4236	4236
query52	107	103	92	92
query53	248	279	201	201
query54	309	270	252	252
query55	92	88	82	82
query56	303	307	296	296
query57	1418	1379	1322	1322
query58	297	277	262	262
query59	1582	1638	1417	1417
query60	349	343	328	328
query61	167	171	154	154
query62	664	615	560	560
query63	243	198	209	198
query64	2430	836	668	668
query65	
query66	1715	521	398	398
query67	30053	30049	29934	29934
query68	
query69	459	337	298	298
query70	1015	1002	996	996
query71	303	284	275	275
query72	2930	2724	2493	2493
query73	820	759	421	421
query74	5072	4950	4691	4691
query75	2760	2648	2328	2328
query76	2284	1160	735	735
query77	415	439	358	358
query78	12945	13008	12253	12253
query79	1502	991	744	744
query80	680	569	488	488
query81	453	278	237	237
query82	1325	170	124	124
query83	363	279	247	247
query84	253	140	107	107
query85	848	524	443	443
query86	401	333	326	326
query87	3411	3360	3208	3208
query88	3502	2644	2640	2640
query89	436	384	338	338
query90	1879	171	177	171
query91	175	175	140	140
query92	80	72	74	72
query93	959	973	556	556
query94	540	359	302	302
query95	664	362	342	342
query96	1105	740	342	342
query97	2722	2683	2545	2545
query98	251	233	228	228
query99	1106	1140	971	971
Total cold run time: 253360 ms
Total hot run time: 170937 ms

[nsivarajan]

run buildall

[hello-stephen]

TPC-H: Total hot run time: 29331 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d4deb957bccbdd0be6a99db74147d017fee7e777, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17604	3867	3875	3867
q2	q3	10703	888	594	594
q4	4657	457	340	340
q5	7452	1323	1124	1124
q6	182	164	137	137
q7	936	960	741	741
q8	9458	1342	1288	1288
q9	5769	5317	5291	5291
q10	6300	2079	1793	1793
q11	474	268	250	250
q12	688	424	297	297
q13	18222	3351	2751	2751
q14	295	280	260	260
q15	q16	903	859	782	782
q17	1032	983	755	755
q18	6371	5612	5538	5538
q19	1234	1250	1096	1096
q20	504	402	257	257
q21	4571	2256	1873	1873
q22	416	362	297	297
Total cold run time: 97771 ms
Total hot run time: 29331 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4172	4047	4126	4047
q2	q3	4609	4730	4166	4166
q4	2076	2157	1389	1389
q5	4940	4889	5161	4889
q6	186	159	129	129
q7	2027	1777	1686	1686
q8	3509	3203	3177	3177
q9	8474	8476	8394	8394
q10	4470	4523	4235	4235
q11	607	419	400	400
q12	698	745	531	531
q13	3158	3563	2906	2906
q14	303	314	272	272
q15	q16	756	804	679	679
q17	1324	1286	1271	1271
q18	8227	7047	7045	7045
q19	1148	1160	1179	1160
q20	2221	2232	1949	1949
q21	6148	5384	4953	4953
q22	563	527	421	421
Total cold run time: 59616 ms
Total hot run time: 53699 ms

[hello-stephen]

TPC-DS: Total hot run time: 171025 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d4deb957bccbdd0be6a99db74147d017fee7e777, data reload: false

query5	4338	658	515	515
query6	318	220	201	201
query7	4305	527	300	300
query8	321	231	222	222
query9	8854	4031	4050	4031
query10	475	337	290	290
query11	5988	2347	2137	2137
query12	181	132	121	121
query13	1352	580	434	434
query14	6852	5405	5090	5090
query14_1	4410	4391	4389	4389
query15	209	204	186	186
query16	1003	473	437	437
query17	1250	773	641	641
query18	2782	509	355	355
query19	230	212	170	170
query20	141	141	127	127
query21	222	143	118	118
query22	13615	13897	14396	13897
query23	17402	16435	16194	16194
query23_1	16419	16265	16134	16134
query24	7989	1831	1472	1472
query24_1	1388	1374	1365	1365
query25	563	471	418	418
query26	1291	314	166	166
query27	2685	614	335	335
query28	4389	1944	1955	1944
query29	1013	613	509	509
query30	302	231	195	195
query31	1131	1084	942	942
query32	85	76	69	69
query33	554	344	283	283
query34	1171	1119	619	619
query35	753	761	676	676
query36	1338	1359	1174	1174
query37	149	97	86	86
query38	3190	3155	3059	3059
query39	914	903	869	869
query39_1	865	850	869	850
query40	237	152	131	131
query41	61	60	59	59
query42	114	106	105	105
query43	336	325	275	275
query44	
query45	204	195	188	188
query46	1060	1236	741	741
query47	2287	2289	2180	2180
query48	408	426	295	295
query49	627	528	435	435
query50	725	307	227	227
query51	4336	4348	4297	4297
query52	106	106	95	95
query53	260	289	203	203
query54	316	276	257	257
query55	92	96	87	87
query56	314	301	296	296
query57	1424	1384	1305	1305
query58	307	268	269	268
query59	1522	1639	1403	1403
query60	345	327	312	312
query61	156	163	156	156
query62	667	604	562	562
query63	246	205	204	204
query64	2313	825	691	691
query65	
query66	1687	499	388	388
query67	30126	29975	29938	29938
query68	
query69	455	344	306	306
query70	1017	1045	1002	1002
query71	330	282	275	275
query72	3065	2928	2604	2604
query73	844	792	434	434
query74	5070	4876	4749	4749
query75	2782	2667	2320	2320
query76	2314	1153	771	771
query77	416	413	359	359
query78	12986	12995	12247	12247
query79	1580	975	756	756
query80	1375	589	498	498
query81	518	280	244	244
query82	933	163	119	119
query83	363	282	251	251
query84	264	139	108	108
query85	898	501	434	434
query86	460	333	332	332
query87	3418	3358	3205	3205
query88	3589	2705	2656	2656
query89	452	383	333	333
query90	1910	174	178	174
query91	178	172	144	144
query92	80	76	73	73
query93	1171	967	565	565
query94	720	327	292	292
query95	677	468	344	344
query96	1077	735	358	358
query97	2689	2692	2558	2558
query98	251	234	226	226
query99	1122	1098	968	968
Total cold run time: 256605 ms
Total hot run time: 171025 ms

[hello-stephen]

FE UT Coverage Report

Increment line coverage 14.71% (15/102) 🎉
Increment coverage report
Complete coverage report

[hello-stephen]

TPC-H: Total hot run time: 29382 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d4deb957bccbdd0be6a99db74147d017fee7e777, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17751	3812	3790	3790
q2	q3	10711	857	614	614
q4	4663	454	342	342
q5	7449	1322	1132	1132
q6	193	170	137	137
q7	896	936	738	738
q8	9295	1385	1252	1252
q9	5592	5374	5319	5319
q10	6303	2068	1824	1824
q11	471	263	250	250
q12	640	415	292	292
q13	18129	3347	2750	2750
q14	284	278	258	258
q15	q16	895	864	779	779
q17	924	969	694	694
q18	6404	5638	5551	5551
q19	1189	1217	1122	1122
q20	498	384	259	259
q21	4768	2314	1963	1963
q22	458	384	316	316
Total cold run time: 97513 ms
Total hot run time: 29382 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4711	4569	4516	4516
q2	q3	4632	4779	4195	4195
q4	2117	2170	1413	1413
q5	4954	5018	5373	5018
q6	190	172	131	131
q7	2077	1796	1630	1630
q8	3343	3102	3065	3065
q9	8339	8651	8301	8301
q10	4457	4483	4215	4215
q11	667	407	401	401
q12	687	746	510	510
q13	3258	3482	3001	3001
q14	303	316	300	300
q15	q16	770	773	686	686
q17	1321	1303	1246	1246
q18	7926	7130	7194	7130
q19	1141	1162	1143	1143
q20	2184	2174	1927	1927
q21	5993	5281	4744	4744
q22	516	474	400	400
Total cold run time: 59586 ms
Total hot run time: 53972 ms

[hello-stephen]

TPC-DS: Total hot run time: 170173 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d4deb957bccbdd0be6a99db74147d017fee7e777, data reload: false

query5	4318	667	522	522
query6	329	226	209	209
query7	4218	553	319	319
query8	326	224	239	224
query9	8821	4015	3998	3998
query10	433	364	305	305
query11	5801	2388	2267	2267
query12	178	130	130	130
query13	1333	602	421	421
query14	6536	5399	5054	5054
query14_1	4413	4365	4387	4365
query15	216	203	181	181
query16	1009	479	408	408
query17	1149	764	638	638
query18	2741	495	368	368
query19	225	217	183	183
query20	143	131	134	131
query21	222	143	126	126
query22	13653	13590	13552	13552
query23	17232	16298	15966	15966
query23_1	16184	16156	16182	16156
query24	7422	1782	1354	1354
query24_1	1364	1358	1383	1358
query25	535	494	441	441
query26	1303	327	179	179
query27	2681	616	340	340
query28	4405	1949	1944	1944
query29	965	624	516	516
query30	303	234	200	200
query31	1122	1062	926	926
query32	87	74	72	72
query33	526	350	294	294
query34	1189	1106	640	640
query35	759	775	667	667
query36	1359	1363	1190	1190
query37	153	103	90	90
query38	3217	3128	3041	3041
query39	925	929	882	882
query39_1	888	902	866	866
query40	236	152	140	140
query41	68	61	62	61
query42	113	112	109	109
query43	322	324	293	293
query44	
query45	215	199	197	197
query46	1067	1179	735	735
query47	2273	2313	2156	2156
query48	398	409	295	295
query49	661	522	432	432
query50	710	280	214	214
query51	4269	4342	4202	4202
query52	105	108	96	96
query53	259	280	208	208
query54	305	269	247	247
query55	91	92	84	84
query56	306	294	297	294
query57	1394	1392	1334	1334
query58	291	271	256	256
query59	1554	1677	1414	1414
query60	352	328	315	315
query61	161	157	155	155
query62	657	630	562	562
query63	245	208	203	203
query64	2354	819	677	677
query65	
query66	1699	510	402	402
query67	29990	29918	29894	29894
query68	
query69	461	345	305	305
query70	1068	1043	1038	1038
query71	300	279	267	267
query72	2961	2749	2409	2409
query73	860	757	455	455
query74	5099	4899	4742	4742
query75	2782	2680	2317	2317
query76	2310	1110	735	735
query77	414	430	349	349
query78	12857	12868	12374	12374
query79	1448	1016	761	761
query80	689	608	484	484
query81	453	280	244	244
query82	1328	162	124	124
query83	364	284	244	244
query84	267	138	115	115
query85	869	547	441	441
query86	377	371	334	334
query87	3464	3338	3185	3185
query88	3534	2695	2633	2633
query89	447	379	338	338
query90	1892	176	177	176
query91	176	169	139	139
query92	81	80	72	72
query93	969	961	552	552
query94	576	344	297	297
query95	637	451	346	346
query96	1048	749	348	348
query97	2715	2671	2573	2573
query98	240	227	231	227
query99	1107	1096	966	966
Total cold run time: 253415 ms
Total hot run time: 170173 ms