Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Test](parquet-reader) Add fix_length_dict_decoder_test and fix bugs by these tests. #49027

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

kaka11chen
Copy link
Contributor

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@kaka11chen kaka11chen force-pushed the fix_length_dict_decoder_test branch from 487af2e to a9a1064 Compare March 13, 2025 07:56
@kaka11chen kaka11chen force-pushed the fix_length_dict_decoder_test branch from a9a1064 to 3bcdebf Compare March 13, 2025 13:20
@kaka11chen
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32817 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3bcdebf54bb4f12c3ecb0705c90799865bf5e1e6, data reload: false

------ Round 1 ----------------------------------
q1	17592	5261	5064	5064
q2	2043	289	157	157
q3	10445	1273	748	748
q4	10240	1002	567	567
q5	7731	2410	2322	2322
q6	186	164	130	130
q7	919	742	600	600
q8	9308	1277	1089	1089
q9	4921	4877	4917	4877
q10	6822	2318	1885	1885
q11	496	273	255	255
q12	360	358	217	217
q13	17941	3865	3179	3179
q14	225	233	208	208
q15	531	482	475	475
q16	632	627	583	583
q17	605	875	354	354
q18	6827	6483	6431	6431
q19	1923	960	568	568
q20	317	332	195	195
q21	2737	2117	1933	1933
q22	1081	1061	980	980
Total cold run time: 103882 ms
Total hot run time: 32817 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5184	5113	5146	5113
q2	233	331	228	228
q3	2122	2645	2266	2266
q4	1420	1816	1362	1362
q5	4221	4083	4151	4083
q6	208	165	124	124
q7	1907	1923	1750	1750
q8	2589	2585	2515	2515
q9	7233	7173	7192	7173
q10	3018	3235	2774	2774
q11	587	508	480	480
q12	698	743	580	580
q13	3469	3892	3240	3240
q14	282	304	276	276
q15	512	481	477	477
q16	649	672	666	666
q17	1139	1580	1373	1373
q18	7637	7546	7514	7514
q19	809	856	928	856
q20	1977	2017	1892	1892
q21	5455	4987	4750	4750
q22	1106	1065	1031	1031
Total cold run time: 52455 ms
Total hot run time: 50523 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191804 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3bcdebf54bb4f12c3ecb0705c90799865bf5e1e6, data reload: false

query1	1393	1006	988	988
query2	6233	1934	1932	1932
query3	11122	4653	4527	4527
query4	25461	23322	23343	23322
query5	4474	667	471	471
query6	300	199	185	185
query7	3983	500	295	295
query8	292	243	222	222
query9	8496	2604	2601	2601
query10	471	312	243	243
query11	15547	15119	14866	14866
query12	161	104	109	104
query13	1553	508	378	378
query14	9535	6248	6181	6181
query15	210	183	172	172
query16	7567	632	467	467
query17	1166	724	589	589
query18	1939	410	313	313
query19	194	193	158	158
query20	128	122	119	119
query21	209	122	113	113
query22	4411	4765	4321	4321
query23	34310	33461	33363	33363
query24	7784	2458	2371	2371
query25	510	507	408	408
query26	1171	273	156	156
query27	2264	489	339	339
query28	4318	2453	2421	2421
query29	715	565	438	438
query30	273	217	203	203
query31	931	883	854	854
query32	74	65	65	65
query33	546	359	287	287
query34	826	867	509	509
query35	815	839	754	754
query36	977	1008	902	902
query37	124	109	82	82
query38	4286	4253	4213	4213
query39	1512	1421	1396	1396
query40	223	119	118	118
query41	62	52	66	52
query42	130	110	106	106
query43	505	514	499	499
query44	1313	792	790	790
query45	186	172	164	164
query46	873	1029	644	644
query47	1860	1915	1862	1862
query48	381	419	328	328
query49	756	514	436	436
query50	699	761	412	412
query51	4279	4298	4310	4298
query52	114	104	95	95
query53	239	269	195	195
query54	524	516	421	421
query55	85	82	84	82
query56	276	291	262	262
query57	1162	1158	1090	1090
query58	264	254	251	251
query59	2990	2973	2765	2765
query60	302	281	274	274
query61	147	165	147	147
query62	793	754	674	674
query63	226	191	191	191
query64	4391	1055	674	674
query65	4536	4507	4506	4506
query66	982	415	302	302
query67	16098	15575	15345	15345
query68	8510	883	498	498
query69	486	288	264	264
query70	1194	1134	1112	1112
query71	462	287	261	261
query72	5655	3496	3622	3496
query73	761	686	351	351
query74	9566	9123	8847	8847
query75	3756	3122	2686	2686
query76	3648	1167	752	752
query77	784	357	337	337
query78	10040	10254	9347	9347
query79	2630	817	593	593
query80	637	501	431	431
query81	502	256	224	224
query82	711	123	92	92
query83	170	170	150	150
query84	240	97	72	72
query85	793	351	315	315
query86	388	303	304	303
query87	4459	4410	4335	4335
query88	3671	2244	2220	2220
query89	414	325	285	285
query90	1785	210	207	207
query91	137	134	107	107
query92	75	59	55	55
query93	1780	1030	572	572
query94	650	415	293	293
query95	352	263	249	249
query96	485	559	271	271
query97	3357	3362	3290	3290
query98	216	207	195	195
query99	1571	1366	1283	1283
Total cold run time: 280562 ms
Total hot run time: 191804 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.72 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3bcdebf54bb4f12c3ecb0705c90799865bf5e1e6, data reload: false

query1	0.04	0.03	0.04
query2	0.07	0.03	0.04
query3	0.23	0.07	0.06
query4	1.63	0.11	0.11
query5	0.57	0.55	0.54
query6	1.19	0.71	0.72
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.60	0.50	0.51
query10	0.57	0.61	0.57
query11	0.16	0.11	0.11
query12	0.14	0.11	0.11
query13	0.62	0.61	0.60
query14	2.66	2.85	2.70
query15	0.90	0.83	0.84
query16	0.38	0.37	0.37
query17	1.00	0.98	1.01
query18	0.20	0.20	0.20
query19	1.90	1.80	2.00
query20	0.01	0.01	0.02
query21	15.36	0.91	0.53
query22	0.76	1.15	0.63
query23	15.00	1.38	0.59
query24	6.47	2.08	0.81
query25	0.52	0.39	0.10
query26	0.54	0.16	0.13
query27	0.05	0.05	0.05
query28	9.71	0.87	0.43
query29	12.56	4.02	3.32
query30	0.25	0.09	0.07
query31	2.84	0.59	0.38
query32	3.22	0.54	0.46
query33	2.99	3.09	3.02
query34	15.79	5.16	4.54
query35	4.59	4.55	4.53
query36	0.65	0.49	0.48
query37	0.09	0.06	0.06
query38	0.05	0.05	0.04
query39	0.03	0.03	0.03
query40	0.17	0.14	0.14
query41	0.08	0.03	0.02
query42	0.03	0.03	0.02
query43	0.03	0.03	0.03
Total cold run time: 104.71 s
Total hot run time: 30.72 s

@kaka11chen
Copy link
Contributor Author

run beut

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants