|
1 | 1 | # Thread pool
|
2 | 2 |
|
3 |
| -Thread pooling can improve performance and scalability for MySQL databases. This technique reuses a fixed number of threads to handle multiple client connections and execute statements. It reduces the overhead of creating and destroying threads and avoids the contention and context switching that can occur when there are too many threads. |
| 3 | +## Introduction |
4 | 4 |
|
5 |
| -If you have fewer than 20,000 connections, using the thread pool does not provide significant benefits. It’s better to keep thread pooling disabled and use the default method. |
| 5 | +Thread pooling improves performance and scalability for MySQL-compatible databases by |
| 6 | +reusing a fixed number of pre-created threads to manage multiple client |
| 7 | +sessions. This design reduces resource overhead, lowers contention, and avoids |
| 8 | +context switching bottlenecks during high concurrency. |
6 | 9 |
|
7 |
| -The default method, called one-thread-per-connection, creates a new thread for each client that connects to the MySQL server. This thread manages all queries and responses for that connection until it’s closed. This approach works well for a moderate number of connections, but it can become inefficient as the number of connections increases. |
| 10 | +The default MySQL method creates one thread per client connection. This approach |
| 11 | +works efficiently under moderate connection loads. As the number of active |
| 12 | +connections increases, especially beyond 20,000, system overhead becomes |
| 13 | +significant and throughput decreases. |
8 | 14 |
|
9 |
| -MySQL supports thread pooling through the thread pool plugin, which replaces the default one-thread-per-connection model. When a statement arrives, the thread group either begins executing it immediately or queues it for later execution in a round-robin fashion. The high-priority queue consists of several thread groups, each managing client connections. Each thread group has a listener thread that listens for incoming statements from the connections assigned to the group. The thread pool exposes several system variables that can be used to configure its operation, such as thread_pool_size, thread_pool_algorithm, thread_pool_stall_limit, and others. |
| 15 | +Percona Server for MySQL includes an integrated thread pool that replaces the |
| 16 | +default model. The thread pool manages connections more efficiently by queuing |
| 17 | +work and reusing threads, especially in OLTP environments with many short-lived |
| 18 | +queries. |
10 | 19 |
|
11 |
| -The thread pool plugin consists of several thread groups, each of which manages a set of client connections. As connections are established, the thread pool assigns them to thread groups using the round-robin method. This method assigns threads fairly and efficiently. Here's how it works: |
| 20 | +## Version-specific notes |
12 | 21 |
|
13 |
| -1. The thread pool starts with a set number of thread groups. |
| 22 | +Thread pool support in Percona Server for MySQL evolved across releases. Review |
| 23 | +the following version details to avoid configuration errors and ensure |
| 24 | +compatibility. |
14 | 25 |
|
15 |
| -2. When a new task arrives, the pool needs to assign it to a group. |
| 26 | + |
| 27 | +### Changed in 8.0.14 |
16 | 28 |
|
17 |
| -3. It does this by going through the groups in order, one by one. |
| 29 | +Beginning with 8.0.14, the upstream implementation of `admin_port` replaced the |
| 30 | +previous mechanism that used `extra_port` and `extra_max_connections`. |
18 | 31 |
|
19 |
| -4. Let's say you have four thread groups. The assignment would work like this: |
20 |
| - - Task 1 goes to Group 1 |
21 |
| - - Task 2 goes to Group 2 |
22 |
| - - Task 3 goes to Group 3 |
23 |
| - - Task 4 goes to Group 4 |
24 |
| - - Task 5 goes back to Group 1 |
25 |
| - |
26 |
| -5. This pattern continues, always moving to the next group and starting over when it reaches the end. |
| 32 | + |
| 33 | +**What changed:** |
27 | 34 |
|
28 |
| -6. Each group handles its assigned tasks using its available threads. |
| 35 | +* `extra_port` and `extra_max_connections` are removed. |
29 | 36 |
|
30 |
| -This round-robin approach spreads work evenly across all groups. It prevents any single group from getting overloaded while others sit idle. This method helps maintain balanced performance across the system. |
| 37 | +* These variables are no longer recognized and cause server startup failure. |
31 | 38 |
|
32 |
| -MySQL executes statements using one thread per client connection. When the number of connections increases past a specific point, performance degrades. |
33 |
| -This feature introduces a dynamic thread pool, which enables the server to maintain top performance even with a large number of client connections. The server decreases the number of threads using the thread pool and reduces the context switching and hot lock contentions. The thread pool is most effective with `OLTP` workloads (relatively short CPU-bound queries). |
| 39 | + |
| 40 | +**Migration steps:** |
34 | 41 |
|
35 |
| -Set the thread pool variable [thread_handling](#thread_handling) to `pool-of-threads` by adding the following line to `my.cnf`: |
| 42 | +* Remove all references to `extra_port` and `extra_max_connections` from the |
| 43 | + configuration file before upgrading to 8.0.14 or later. |
36 | 44 |
|
37 |
| -```text |
38 |
| -thread_handling=pool-of-threads |
| 45 | +* Use the `admin_port` variable, as supported by upstream MySQL, to configure |
| 46 | + administrative access if needed. |
| 47 | + |
| 48 | + |
| 49 | +### Implemented in 8.0.12-1 |
| 50 | + |
| 51 | +Percona Server for MySQL 8.0.12-1 introduced native support for the thread pool. |
| 52 | +This version ported the feature from Percona Server 5.7. |
| 53 | + |
| 54 | +## Core concepts and usage examples |
| 55 | + |
| 56 | +### Fixed pool of threads |
| 57 | + |
| 58 | +The server initializes a defined number of threads at startup and reuses them |
| 59 | +to process incoming queries. |
| 60 | + |
| 61 | + |
| 62 | +```ini |
| 63 | +[mysqld] |
| 64 | +thread_handling = pool-of-threads |
39 | 65 | ```
|
| 66 | +This setting activates the thread pool model. |
| 67 | + |
| 68 | +### How the thread pool works |
| 69 | + |
| 70 | +When a client connects, the server assigns the connection to a thread group using a round-robin method. Each group contains a listener thread and worker threads. The listener monitors active connections and queues incoming statements. |
| 71 | + |
| 72 | +Each new query is routed to either the high-priority queue or the low-priority queue within the group. A query enters the high-priority queue if the connection has an open transaction and available high-priority tickets. All other queries are sent to the low-priority queue. |
| 73 | + |
| 74 | +Worker threads scan the high-priority queue first, then process queries from the low-priority queue when no high-priority items remain. After finishing a query, the thread becomes idle and waits for the next task. This reuse avoids excessive thread creation and maintains steady performance under load. |
| 75 | + |
| 76 | +The pool adapts to workload shifts by redistributing queued tasks and prioritizing available threads. The result is better system responsiveness with fewer total threads. |
40 | 77 |
|
41 |
| -Although the default values for the thread pool should provide good performance, additional tuning should be performed with the dynamic system variables. The goal is to minimize the number of open transactions on the server. Short-running transactions commit faster and deallocate server resources and locks. |
| 78 | +<img src="../_static/thread-pool-diagram.png" alt="Thread pool diagram" width="250" /> |
42 | 79 |
|
43 |
| -Due to the following differences, this implementation is not compatible with upstream: |
| 80 | +### Thread groups |
44 | 81 |
|
45 |
| -* Built into the server, upstream implements the thread pool as a plugin |
| 82 | +The thread pool organizes threads into groups. Each group contains a listener thread, a set of worker threads, and handles a portion of the client connections. |
46 | 83 |
|
47 |
| -* Does not minimize the number of concurrent transactions |
48 | 84 |
|
49 |
| -Priority Queue: |
| 85 | +```ini |
| 86 | +[mysqld] |
| 87 | +thread_pool_size = 8 |
| 88 | +``` |
| 89 | + |
| 90 | +This configuration spreads work across eight thread groups. |
| 91 | + |
| 92 | +### Priority queues |
| 93 | + |
| 94 | +Each thread group maintains a high-priority queue and a low-priority queue. New queries are added to one of the two based on ticket availability and transaction state. The thread pool always checks the high-priority queue first. |
| 95 | + |
| 96 | +```mysql |
| 97 | +mysql> SHOW STATUS LIKE 'Threadpool%'; |
| 98 | +``` |
50 | 99 |
|
51 |
| -A queue that assigns a priority to each data element and processes them according to their priority. The data element with the highest priority is served first, regardless of its order in the queue. A priority queue can be implemented using an array, a linked list, a heap, or a binary search tree. It can also be ascending or descending, meaning that the highest priority is either the smallest or the largest value. |
| 100 | +This command displays queue lengths and thread statistics. |
52 | 101 |
|
53 |
| -## Version specific information |
| 102 | +### High-priority ticketing |
54 | 103 |
|
55 |
| -Starting with 8.0.14, Percona Server for MySQL uses the upstream implementation of the [`admin_port`](https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_admin_port). The variables [extra_port](#extra_port) and [extra_max_connections](#extra_max_connections) are removed and not supported. Remove the `extra_port` and `extra_max_connections` variables from your configuration file before upgrading to 8.0.14 or higher. In 8.0.14 or higher, the variables cause a boot error, and the server refuses to start. |
| 104 | +Each connection begins with a set number of high-priority tickets. These tickets allow prioritization for a limited number of queries. One ticket is used every time the thread pool promotes a query to the high-priority queue. |
56 | 105 |
|
57 |
| -Implemented in 8.0.12-1: We ported the `Thread Pool` feature from Percona Server for MySQL 5.7. |
| 106 | +#### Static configuration |
58 | 107 |
|
59 |
| -## Priority connection scheduling |
| 108 | +```ini |
| 109 | +[mysqld] |
| 110 | +thread_pool_high_prio_tickets = 5 |
| 111 | +``` |
60 | 112 |
|
61 |
| -The thread pool limits the number of concurrently running queries. The number of open transactions may remain high. Connections with already-started transactions are added to the end of the queue. A high number of open transactions has implications for the currently running queries. The [thread_pool_high_prio_tickets](#thread_pool_high_prio_tickets) variable controls the high-priority queue policy and assigns tickets to each new connection. |
| 113 | +#### Dynamic configuration |
62 | 114 |
|
63 |
| -The thread pool adds the connection to the high-priority queue and decrements the ticket if the connection has the following attributes: |
| 115 | +```mysql |
| 116 | +mysql> mysql> SET GLOBAL thread_pool_high_prio_tickets = 5; |
| 117 | +``` |
64 | 118 |
|
65 |
| -* Has an open transaction |
| 119 | +Either of these configurations assign five tickets to each new connection. |
66 | 120 |
|
67 |
| -* Has a non-zero number of high-priority tickets |
68 | 121 |
|
69 |
| -Otherwise, the variable adds the connection to the low-priority queue with the initial value. |
| 122 | +### High-priority modes |
| 123 | + |
| 124 | +The `thread_pool_high_prio_mode` setting controls how the thread pool schedules queries using tickets. |
| 125 | + |
| 126 | +```mysql |
| 127 | +mysql> SET GLOBAL thread_pool_high_prio_mode = 'transactions'; |
| 128 | +``` |
| 129 | + |
| 130 | +This setting limits prioritization to queries within open transactions. |
| 131 | + |
| 132 | +### Configuration `thread_pool_high_prio_mode` |
| 133 | + |
| 134 | +To set the variable persistently, add the following to your `my.cnf` file: |
| 135 | + |
| 136 | +```ini |
| 137 | +[mysqld] |
| 138 | +thread_pool_high_prio_mode = transactions |
| 139 | +``` |
| 140 | + |
| 141 | +You can also change this setting at runtime without restarting: |
| 142 | + |
| 143 | +``` |
| 144 | +mysql> SET GLOBAL thread_pool_high_prio_mode = 'transactions'; |
| 145 | +``` |
| 146 | + |
| 147 | +This change takes effect immediately for new connections only. Existing client sessions continue to operate under the previously active mode. To ensure all clients reflect the new behavior, either: |
| 148 | + |
| 149 | +* Restart the server, or |
| 150 | + |
| 151 | +* Have connected clients disconnect and reconnect manually. |
| 152 | + |
| 153 | +This flexibility allows scheduling behavior to be fine-tuned without disrupting active workloads. |
| 154 | + |
| 155 | +### Adaptive scheduling |
| 156 | + |
| 157 | +The server assigns new connections to thread groups using a round-robin method. This approach balances workload evenly and prevents any group from becoming a bottleneck. |
| 158 | + |
| 159 | + |
| 160 | +Connection 1 goes to Group 1 |
| 161 | + |
| 162 | +Connection 2 goes to Group 2 |
| 163 | + |
| 164 | +Connection 3 goes to Group 3 |
| 165 | + |
| 166 | +Connection 4 goes to Group 4 |
| 167 | + |
| 168 | +Connection 5 returns to Group 1 |
| 169 | + |
| 170 | + |
| 171 | + |
| 172 | +### Configuration and monitoring |
| 173 | + |
| 174 | +Administrators can fine-tune thread pool behavior by adjusting dynamic and static variables. |
| 175 | + |
| 176 | + |
| 177 | +#### Static configuration |
| 178 | + |
| 179 | +```ini |
| 180 | +[mysqld] |
| 181 | +thread_pool_stall_limit = 100 |
| 182 | +``` |
| 183 | + |
| 184 | +#### Dynamic configuration |
| 185 | + |
| 186 | +```mysql |
| 187 | +mysql> SET GLOBAL thread_pool_stall_limit = 100; |
| 188 | +``` |
| 189 | + |
| 190 | +This value, in milliseconds, limits how long a task can stall before being redistributed. As with other thread pool settings, existing sessions retain their previous values. |
| 191 | + |
| 192 | +#### Check the status |
| 193 | + |
| 194 | +```mysql |
| 195 | +mysql> SHOW ENGINE THREAD_POOL STATUS; |
| 196 | +``` |
| 197 | + |
| 198 | +This command returns information about thread usage and group activity. |
| 199 | + |
| 200 | +## When to use the thread pool |
| 201 | + |
| 202 | +Choosing between the thread pool and the default threading model depends on |
| 203 | +system size, workload patterns, and performance goals. Use the guidelines below |
| 204 | +to decide which approach fits best. |
| 205 | + |
| 206 | +### Use the thread pool when: |
| 207 | + |
| 208 | +* The system experiences hundreds or thousands of concurrent connections. |
| 209 | + |
| 210 | +* Most queries are short, frequent, and CPU-bound (typical of OLTP workloads). |
| 211 | + |
| 212 | +* The server suffers from context switching, thread contention, or memory |
| 213 | + overhead due to excessive threads. |
| 214 | + |
| 215 | +* Transactions should be prioritized, and precise query scheduling is required. |
| 216 | + |
| 217 | +* Thread count must remain predictable and resource usage needs tighter control. |
| 218 | + |
| 219 | + |
| 220 | +### Use the default model when: |
| 221 | + |
| 222 | +* The deployment serves fewer than 500 concurrent clients. |
| 223 | + |
| 224 | +* Queries are long-running or resource-heavy and better suited to isolated |
| 225 | + execution. |
| 226 | + |
| 227 | +* Thread overhead is not a problem, and the simplicity of one-thread-per- |
| 228 | + connection is acceptable. |
| 229 | + |
| 230 | +* There is no pressing need to control concurrency or prioritize active |
| 231 | + transactions. |
| 232 | + |
| 233 | + |
| 234 | +Thread pooling offers greater control and efficiency, but only in workloads that |
| 235 | +justify that complexity. Measure concurrency patterns and CPU behavior before |
| 236 | +enabling the feature. |
| 237 | + |
| 238 | +## Thread Pool Configuration Reference Sheet |
| 239 | + |
| 240 | +A quick overview of system variables available for configuring the thread pool in Percona Server for MySQL. |
| 241 | + |
| 242 | +### General Variables |
| 243 | + |
| 244 | +| Variable name | Default | Scope | Dynamic | Config file | Description | |
| 245 | +|--------------------------|-----------------------------|--------|---------|--------------|-------------------------------------------------------| |
| 246 | +| `thread_handling` | `one-thread-per-connection` | Global | No | Yes | Chooses the thread model. Set to `pool-of-threads` to enable thread pooling. | |
| 247 | +| `thread_pool_size` | `16` | Global | No | Yes | Defines the number of thread groups. | |
| 248 | +| `thread_pool_algorithm` | `first-in, first-out` | Global | Yes | Yes | Controls how queued queries are scheduled (`first-in, first-out` or `lowest_priority`). | |
| 249 | + |
| 250 | +### Priority Handling |
| 251 | + |
| 252 | +| Variable name | Default | Scope | Dynamic | Config file | Description | |
| 253 | +|----------------------------------|----------------|--------|---------|--------------|------------------------------------------------------------| |
| 254 | +| `thread_pool_high_prio_tickets` | `4294967295` | Global | Yes | Yes | High-priority executions per connection. | |
| 255 | +| `thread_pool_high_prio_mode` | `transactions` | Global | Yes | Yes | Determines when to assign high priority (`transactions`, `statements`, or `none`). | |
| 256 | + |
| 257 | +### Thread Management |
| 258 | + |
| 259 | +| Variable name | Default | Scope | Dynamic | Config file | Description | |
| 260 | +|---------------------------|---------|--------|---------|--------------|-----------------------------------------------------| |
| 261 | +| `thread_pool_min_threads` | `4` | Global | No | Yes | Minimum number of threads per group. | |
| 262 | +| `thread_pool_max_threads` | `1000` | Global | No | Yes | Maximum number of threads per group. | |
| 263 | +| `thread_pool_stall_limit` | `6` | Global | Yes | Yes | Time in milliseconds before retrying a stalled task. | |
| 264 | + |
| 265 | +### Observability Commands |
| 266 | + |
| 267 | +Use these SQL statements to view runtime metrics: |
| 268 | + |
| 269 | +```sql |
| 270 | +SHOW STATUS LIKE 'Threadpool%'; |
| 271 | +SHOW ENGINE THREAD_POOL STATUS; |
| 272 | +``` |
| 273 | + |
| 274 | +Note: For dynamic variables, changes apply only to new connections. Existing sessions retain their original settings until reconnect. |
| 275 | + |
70 | 276 |
|
71 |
| -Each time, the thread pool checks the high-priority queue for the next connection. When the high-priority queue is empty, the thread pool picks connections from the low-priority queue. The default behavior is to put events from already started transactions into the high-priority queue. |
72 | 277 |
|
73 |
| -If the value equals `0`, all connections are put into the low-priority queue. If the value exceeds zero, each connection could be put into a high-priority queue. |
74 | 278 |
|
75 |
| -The [thread_pool_high_prio_mode](#thread_pool_high_prio_mode) variable prioritizes all statements for a connection or assigns connections to the low-priority queue. To implement this new [thread_pool_high_prio_mode](#thread_pool_high_prio_mode) variable |
76 | 279 |
|
77 |
| -## Low-priority queue throttling |
78 | 280 |
|
79 |
| -One case that can limit thread pool performance and even lead to deadlocks under high concurrency is when thread groups are oversubscribed due to active threads reaching the oversubscribe limit. Still, all/most worker threads are waiting on locks currently held by a transaction from another connection that is not currently in the thread pool. |
80 | 281 |
|
81 |
| -In this case, the oversubscribe limit does not account for those threads in the pool that marked themselves inactive. As a result, the number of threads (both active and waiting) in the pool grows until it hits the [`thread_pool_max_threads`](#thread_pool_max_threads) value. If the connection executing the transaction holding the lock has managed to enter the thread pool by then, we get a large (depending on the [`thread_pool_max_threads`](#thread_pool_max_threads) value) number of concurrently running threads and, thus, suboptimal performance. Otherwise, we get a deadlock as no more threads can be created to process those transaction(s) and release the lock(s). |
82 | 282 |
|
83 |
| -Such situations are prevented by throttling the low-priority queue when the total number of worker threads (both active and waiting ones) reaches the oversubscribe limit. If there are too many worker threads, do not start new transactions; create new threads until queued events from the already-started transactions are processed. |
84 | 283 |
|
85 |
| -## Handling long network waits |
86 | 284 |
|
87 |
| -Specific workloads (large result sets, BLOBs, slow clients) can wait longer on network I/O (socket reads and writes). Whenever the server waits, this should be communicated to the thread pool so it can start a new query by either waking a waiting thread or sometimes creating a new one. This implementation has been ported from *MariaDB* patch MDEV-156. |
88 | 285 |
|
89 | 286 | ## System variables
|
90 | 287 |
|
@@ -121,8 +318,28 @@ This variable defines how the server handles threads for connections from the cl
|
121 | 318 |
|
122 | 319 | This variable can limit the time an idle thread should wait before exiting.
|
123 | 320 |
|
| 321 | + |
124 | 322 | ### `thread_pool_high_prio_mode`
|
125 | 323 |
|
| 324 | +| Option | Description | |
| 325 | +| -------------- | --------------------------------------------------------------------------- | |
| 326 | +| Command-line: | Yes | |
| 327 | +| Config file: | Yes | |
| 328 | +| Scope: | Global | |
| 329 | +| Dynamic: | Yes | |
| 330 | +| Data type: | Enumeration (`transactions`, `statements`, `none`) | |
| 331 | +| Default value: | `transactions` | |
| 332 | + |
| 333 | +Controls how the thread pool schedules high-priority work. When set to |
| 334 | +`transactions`, only statements within an open transaction are eligible for |
| 335 | +high-priority execution. The `statements` option prioritizes all statements |
| 336 | +from connections that still have unused high-priority tickets. Setting this |
| 337 | +variable to `none` disables high-priority queueing entirely. |
| 338 | + |
| 339 | +Changes take effect immediately for new connections. Existing sessions must |
| 340 | +reconnect to adopt the new behavior. |
| 341 | + |
| 342 | + |
126 | 343 | This variable provides more fine-grained control over high-priority scheduling globally or per connection.
|
127 | 344 |
|
128 | 345 | The following values are allowed:
|
|
0 commit comments