Skip to content

[fix](load) Use atomic operations for _try_close flag and remove unused _close_wait#61593

Open
zclllyybb wants to merge 1 commit intoapache:masterfrom
zclllyybb:fix_try_close_atomic
Open

[fix](load) Use atomic operations for _try_close flag and remove unused _close_wait#61593
zclllyybb wants to merge 1 commit intoapache:masterfrom
zclllyybb:fix_try_close_atomic

Conversation

@zclllyybb
Copy link
Contributor

  • Change _try_close from bool to std::atomic<bool> with acquire/release memory ordering
  • Use memory_order_acquire when reading in _send_batch_process() (bthread)
  • Use memory_order_release when writing in _do_try_close() (pthread)
  • Remove unused _close_wait field

This fixes a potential race condition where _try_close is written by pthread and read by bthread without proper synchronization.

It's ok on x86 so no test added.

…move unused _close_wait

- Change `_try_close` from `bool` to `std::atomic<bool>` with acquire/release memory ordering
- Use `memory_order_acquire` when reading in `_send_batch_process()` (bthread)
- Use `memory_order_release` when writing in `_do_try_close()` (pthread)
- Remove unused `_close_wait` field

This fixes a potential race condition where `_try_close` is written by pthread and read by bthread without proper synchronization.
@zclllyybb
Copy link
Contributor Author

run buildall

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@doris-robot
Copy link

TPC-H: Total hot run time: 26821 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 49082df2c5a276328c44138237eaf24c5dd18e55, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17627	4525	4289	4289
q2	q3	10653	773	525	525
q4	4679	360	243	243
q5	7556	1211	1015	1015
q6	178	175	147	147
q7	793	857	664	664
q8	9312	1496	1395	1395
q9	4945	4746	4659	4659
q10	6321	1917	1647	1647
q11	469	266	254	254
q12	753	587	480	480
q13	18039	2947	2178	2178
q14	230	229	206	206
q15	q16	757	721	653	653
q17	716	833	451	451
q18	5862	5291	5293	5291
q19	1258	993	633	633
q20	538	493	376	376
q21	4483	1845	1416	1416
q22	498	405	299	299
Total cold run time: 95667 ms
Total hot run time: 26821 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4794	4571	4690	4571
q2	q3	3870	4383	3894	3894
q4	859	1203	801	801
q5	4093	4380	4370	4370
q6	190	184	144	144
q7	1886	1626	1537	1537
q8	2490	2758	2649	2649
q9	7632	7397	7383	7383
q10	3750	3989	3597	3597
q11	522	459	450	450
q12	534	597	463	463
q13	2678	3308	2398	2398
q14	302	306	286	286
q15	q16	738	765	726	726
q17	1226	1412	1389	1389
q18	7093	6741	6623	6623
q19	956	941	952	941
q20	2141	2131	2019	2019
q21	3951	3550	3352	3352
q22	462	436	389	389
Total cold run time: 50167 ms
Total hot run time: 47982 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 168617 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 49082df2c5a276328c44138237eaf24c5dd18e55, data reload: false

query5	4343	629	498	498
query6	343	222	200	200
query7	4223	467	261	261
query8	336	245	235	235
query9	8682	2715	2715	2715
query10	524	428	339	339
query11	6976	5083	4885	4885
query12	184	133	137	133
query13	1313	445	331	331
query14	5738	3717	3406	3406
query14_1	2822	2791	2782	2782
query15	200	189	177	177
query16	954	451	363	363
query17	878	715	591	591
query18	2452	441	340	340
query19	209	211	186	186
query20	136	134	126	126
query21	214	142	113	113
query22	13255	14102	14656	14102
query23	16496	16014	15626	15626
query23_1	15648	15634	15495	15495
query24	7131	1621	1233	1233
query24_1	1211	1231	1242	1231
query25	579	504	455	455
query26	1264	272	160	160
query27	2841	490	315	315
query28	4513	1853	1836	1836
query29	872	600	505	505
query30	300	232	193	193
query31	999	961	864	864
query32	84	73	72	72
query33	516	358	308	308
query34	910	863	545	545
query35	638	692	615	615
query36	1104	1106	990	990
query37	138	100	87	87
query38	2943	2927	2880	2880
query39	867	829	802	802
query39_1	799	808	799	799
query40	244	163	144	144
query41	70	66	64	64
query42	262	259	258	258
query43	255	249	228	228
query44	
query45	203	193	189	189
query46	876	988	609	609
query47	2162	2153	2048	2048
query48	315	323	229	229
query49	635	468	392	392
query50	689	282	225	225
query51	4060	4056	4019	4019
query52	264	264	252	252
query53	298	344	292	292
query54	311	276	272	272
query55	90	85	85	85
query56	314	320	322	320
query57	1936	1915	1705	1705
query58	282	281	272	272
query59	2782	2966	2742	2742
query60	354	340	323	323
query61	161	161	172	161
query62	627	599	548	548
query63	328	278	283	278
query64	5082	1314	1016	1016
query65	
query66	1458	465	358	358
query67	24194	24332	24341	24332
query68	
query69	407	322	291	291
query70	935	953	936	936
query71	343	314	300	300
query72	2802	2743	2463	2463
query73	532	551	323	323
query74	9595	9536	9410	9410
query75	2847	2756	2457	2457
query76	2273	1048	686	686
query77	356	357	311	311
query78	11030	11174	10529	10529
query79	1137	772	562	562
query80	719	640	567	567
query81	494	269	231	231
query82	1361	152	128	128
query83	385	258	252	252
query84	293	117	104	104
query85	869	508	455	455
query86	371	300	308	300
query87	3152	3085	3047	3047
query88	3519	2639	2637	2637
query89	431	372	339	339
query90	1990	195	180	180
query91	177	169	145	145
query92	84	72	73	72
query93	888	879	499	499
query94	467	332	302	302
query95	588	410	325	325
query96	651	530	224	224
query97	2464	2522	2416	2416
query98	233	219	215	215
query99	1024	1006	916	916
Total cold run time: 248869 ms
Total hot run time: 168617 ms

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants