Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(performance) : to improve the performance of Rapid #363

Open
ShannonBase opened this issue Mar 4, 2025 · 4 comments
Open

feat(performance) : to improve the performance of Rapid #363

ShannonBase opened this issue Mar 4, 2025 · 4 comments
Assignees
Labels
enhancement New feature or request performance about the performance issuess

Comments

@ShannonBase
Copy link
Contributor

ShannonBase commented Mar 4, 2025

Summary
To improve the performance of query. The perf report data is as following: (From InnoDB engine)

+   99.46%     0.00%  ib_rpd_pop-0     libc.so.6            [.] clone3                                                                     ◆
+   99.46%     0.00%  ib_rpd_pop-0     libc.so.6            [.] start_thread                                                               ▒
+   99.46%     0.00%  ib_rpd_pop-0     libstdc++.so.6.0.33  [.] 0x00007ba2bfaecdb4                                                         ▒
+   99.46%     0.00%  ib_rpd_pop-0     mysqld               [.] std::thread::_State_impl<std::thread::_Invoker<std::tuple<Detached_thread, ▒
+   99.46%     0.00%  ib_rpd_pop-0     mysqld               [.] std::thread::_Invoker<std::tuple<Detached_thread, void (*)(log_t*), log_t*>▒
+   99.46%     0.00%  ib_rpd_pop-0     mysqld               [.] void std::thread::_Invoker<std::tuple<Detached_thread, void (*)(log_t*), lo▒
+   99.46%     0.00%  ib_rpd_pop-0     mysqld               [.] std::__invoke_result<Detached_thread, void (*)(log_t*), log_t*>::type std::▒
+   99.46%     0.00%  ib_rpd_pop-0     mysqld               [.] void std::__invoke_impl<void, Detached_thread, void (*)(log_t*), log_t*>(st▒
+   99.46%     0.00%  ib_rpd_pop-0     mysqld               [.] void Detached_thread::operator()<void (*)(log_t*), log_t*>(void (*&&)(log_t▒
+   99.46%     0.00%  ib_rpd_pop-0     mysqld               [.] std::invoke_result<void (*)(log_t*), log_t*>::type std::invoke<void (*)(log▒
+   99.46%     0.00%  ib_rpd_pop-0     mysqld               [.] std::__invoke_result<void (*)(log_t*), log_t*>::type std::__invoke<void (*)▒
+   99.46%     0.00%  ib_rpd_pop-0     mysqld               [.] void std::__invoke_impl<void, void (*)(log_t*), log_t*>(std::__invoke_other▒
+   99.46%     0.00%  ib_rpd_pop-0     mysqld               [.] ShannonBase::Populate::parse_log_func_main(log_t*)                         ▒
+   99.28%    96.24%  ib_rpd_pop-0     mysqld               [.] Wait_stats os_event_wait_for<ShannonBase::Populate::parse_log_func_main(log▒
+    1.80%     0.79%  ib_rpd_pop-0     mysqld               [.] ShannonBase::Populate::parse_log_func_main(log_t*)::{lambda(bool)#1}::opera▒
+    1.43%     1.43%  ib_rpd_pop-0     mysqld               [.] unlikely(bool)                                                             ▒
+    0.59%     0.59%  ib_rpd_pop-0     mysqld               [.] std::__is_constant_evaluated()                                             ▒
     0.41%     0.00%  ib_rpd_pop-0     [kernel.kallsyms]    [k] 0xffffffff8fc00f4b                                                         
@ShannonBase
Copy link
Contributor Author

From the perf report, the bottleneck is in ib_rpd_pop-0 thread.

@ShannonBase ShannonBase self-assigned this Mar 4, 2025
@ShannonBase ShannonBase added enhancement New feature or request performance about the performance issuess labels Mar 4, 2025
@ShannonBase
Copy link
Contributor Author

ShannonBase commented Mar 4, 2025

If we disable rpd-pop-0 thread, the perf report is (From InnoDB)

+   28.15%    28.15%  ib_log_files_g   [kernel.kallsyms]    [k] task_mm_cid_work                                                           ▒
+   28.15%     0.00%  ib_log_files_g   mysqld               [.] pthread_mutex_unlock@plt                                                   ▒
+   28.15%     0.00%  ib_log_files_g   [kernel.kallsyms]    [k] asm_sysvec_apic_timer_interrupt                                            ▒
+   28.15%     0.00%  ib_log_files_g   [kernel.kallsyms]    [k] sysvec_apic_timer_interrupt                                                ▒
+   28.15%     0.00%  ib_log_files_g   [kernel.kallsyms]    [k] irqentry_exit                                                              ▒
+   28.15%     0.00%  ib_log_files_g   [kernel.kallsyms]    [k] irqentry_exit_to_user_mode                                                 ▒
+   28.15%     0.00%  ib_log_files_g   [kernel.kallsyms]    [k] task_work_run                                                              ▒
+   25.57%    25.57%  connection       mysqld               [.] Field_longlong::val_int() const                                            ▒
+   25.57%     0.00%  connection       mysqld               [.] dd::Entity_object_table_impl::restore_object_from_record(dd::Open_dictionar▒
+   25.57%     0.00%  connection       mysqld               [.] dd::Table_impl::restore_children(dd::Open_dictionary_tables_ctx*)          ▒
+   25.57%     0.00%  connection       mysqld               [.] dd::Abstract_table_impl::restore_children(dd::Open_dictionary_tables_ctx*) ▒
+   25.57%     0.00%  connection       mysqld               [.] bool dd::Collection<dd::Column*>::restore_items<dd::Abstract_table_impl>(dd▒
+   25.57%     0.00%  connection       mysqld               [.] bool dd::Collection<dd::Column*>::restore_items<dd::Abstract_table_impl, bo▒
+   25.57%     0.00%  connection       mysqld               [.] dd::Column_impl::restore_attributes(dd::Raw_record const&)                 ▒
+   25.57%     0.00%  connection       mysqld               [.] dd::Raw_record::read_ref_id(int) const                                     ▒
+   25.57%     0.00%  connection       mysqld               [.] dd::Raw_record::read_int(int) const                                        ▒
+   10.22%    10.22%  connection       mysqld               [.] innobase_index_lookup(INNOBASE_SHARE*, unsigned int)                       ▒
+   10.22%     0.00%  connection       mysqld               [.] dd::Raw_table::find_record(dd::Object_key const&, std::unique_ptr<dd::Raw_r▒
+   10.22%     0.00%  connection       mysqld               [.] handler::ha_index_init(unsigned int, bool)                                 ▒
+   10.22%     0.00%  connection       mysqld               [.] ha_innobase::index_init(unsigned int, bool)                                ▒
+   10.22%     0.00%  connection       mysqld               [.] ha_innobase::change_active_index(unsigned int)                             ▒
+   10.22%     0.00%  connection       mysqld               [.] ha_innobase::innobase_get_index(unsigned int)                              ▒
+    2.45%     0.00%  ib_src_main      libc.so.6            [.] clone3                                                                     ▒
+    2.45%     0.00%  ib_src_main      libc.so.6            [.] start_thread                                                               

@ShannonBase
Copy link
Contributor Author

perf report data with using rapid engine.

+   99.34%     0.00%  ib_rpd_pop-0     libc.so.6            [.] clone3                                                                     ◆
+   99.34%     0.00%  ib_rpd_pop-0     libc.so.6            [.] start_thread                                                               ▒
+   99.34%     0.00%  ib_rpd_pop-0     libstdc++.so.6.0.33  [.] 0x0000783c454ecdb4                                                         ▒
+   99.34%     0.00%  ib_rpd_pop-0     mysqld               [.] std::thread::_State_impl<std::thread::_Invoker<std::tuple<Detached_thread, ▒
+   99.34%     0.00%  ib_rpd_pop-0     mysqld               [.] std::thread::_Invoker<std::tuple<Detached_thread, void (*)(log_t*), log_t*>▒
+   99.34%     0.00%  ib_rpd_pop-0     mysqld               [.] void std::thread::_Invoker<std::tuple<Detached_thread, void (*)(log_t*), lo▒
+   99.34%     0.00%  ib_rpd_pop-0     mysqld               [.] std::__invoke_result<Detached_thread, void (*)(log_t*), log_t*>::type std::▒
+   99.34%     0.00%  ib_rpd_pop-0     mysqld               [.] void std::__invoke_impl<void, Detached_thread, void (*)(log_t*), log_t*>(st▒
+   99.34%     0.00%  ib_rpd_pop-0     mysqld               [.] void Detached_thread::operator()<void (*)(log_t*), log_t*>(void (*&&)(log_t▒
+   99.34%     0.00%  ib_rpd_pop-0     mysqld               [.] std::invoke_result<void (*)(log_t*), log_t*>::type std::invoke<void (*)(log▒
+   99.34%     0.00%  ib_rpd_pop-0     mysqld               [.] std::__invoke_result<void (*)(log_t*), log_t*>::type std::__invoke<void (*)▒
+   99.34%     0.00%  ib_rpd_pop-0     mysqld               [.] void std::__invoke_impl<void, void (*)(log_t*), log_t*>(std::__invoke_other▒
+   99.34%    95.80%  ib_rpd_pop-0     mysqld               [.] Wait_stats os_event_wait_for<ShannonBase::Populate::parse_log_func_main(log▒
+   99.34%     0.00%  ib_rpd_pop-0     mysqld               [.] ShannonBase::Populate::parse_log_func_main(log_t*)                         ▒
+    1.96%     0.27%  ib_rpd_pop-0     mysqld               [.] ShannonBase::Populate::parse_log_func_main(log_t*)::{lambda(bool)#1}::opera▒
+    1.59%     1.59%  ib_rpd_pop-0     mysqld               [.] unlikely(bool)                                                             ▒
+    1.40%     1.40%  ib_rpd_pop-0     mysqld               [.] std::operator&(std::memory_order, std::__memory_order_modifier)            ▒
+    0.53%     0.00%  connection       libc.so.6            [.] clone3                                                                     ▒
+    0.53%     0.00%  connection       libc.so.6            [.] start_thread                                                               ▒
+    0.53%     0.00%  connection       mysqld               [.] pfs_spawn_thread                                                           ▒
+    0.53%     0.00%  connection       mysqld               [.] handle_connection                                                          ▒
+    0.53%     0.00%  connection       mysqld               [.] do_command(THD*)                                                           ▒
+    0.53%     0.00%  connection       mysqld               [.] dispatch_command(THD*, COM_DATA const*, enum_server_command)               ▒
     0.27%     0.27%  ib_rpd_pop-0     mysqld               [.] std::__is_constant_evaluated()                                             ▒
     0.27%     0.00%  connection       mysqld               [.] dispatch_sql_command(THD*, Parser_state*)                                  

@ShannonBase
Copy link
Contributor Author

It seems that the root cause leads performance deteriorate as

[.] Wait_stats os_event_wait_for<ShannonBase::Populate::parse_log_func_main(log

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request performance about the performance issuess
Projects
None yet
Development

No branches or pull requests

1 participant