Skip to content

Commit aaf96c2

Browse files
committed
PS-10071 [DOCS] - update Threadpool to add implementation 8.0
new file: docs/_static/thread-pool-diagram.png modified: docs/threadpool.md
1 parent ab05d24 commit aaf96c2

File tree

2 files changed

+208
-47
lines changed

2 files changed

+208
-47
lines changed

docs/_static/thread-pool-diagram.png

782 KB
Loading

docs/threadpool.md

Lines changed: 208 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -1,90 +1,231 @@
11
# Thread pool
22

3-
Thread pooling can improve performance and scalability for MySQL databases. This technique reuses a fixed number of threads to handle multiple client connections and execute statements. It reduces the overhead of creating and destroying threads and avoids the contention and context switching that can occur when there are too many threads.
3+
## Introduction
44

5-
If you have fewer than 20,000 connections, using the thread pool does not provide significant benefits. It’s better to keep thread pooling disabled and use the default method.
5+
Thread pooling improves performance and scalability for MySQL-compatible databases by
6+
reusing a fixed number of pre-created threads to manage multiple client
7+
sessions. This design reduces resource overhead, lowers contention, and avoids
8+
context switching bottlenecks during high concurrency.
69

7-
The default method, called one-thread-per-connection, creates a new thread for each client that connects to the MySQL server. This thread manages all queries and responses for that connection until it’s closed. This approach works well for a moderate number of connections, but it can become inefficient as the number of connections increases.
10+
The default MySQL method creates one thread per client connection. This approach
11+
works efficiently under moderate connection loads. As the number of active
12+
connections increases, especially beyond 20,000, system overhead becomes
13+
significant and throughput decreases.
814

9-
MySQL supports thread pooling through the thread pool plugin, which replaces the default one-thread-per-connection model. When a statement arrives, the thread group either begins executing it immediately or queues it for later execution in a round-robin fashion. The high-priority queue consists of several thread groups, each managing client connections. Each thread group has a listener thread that listens for incoming statements from the connections assigned to the group. The thread pool exposes several system variables that can be used to configure its operation, such as thread_pool_size, thread_pool_algorithm, thread_pool_stall_limit, and others.
15+
Percona Server for MySQL includes an integrated thread pool that replaces the
16+
default model. The thread pool manages connections more efficiently by queuing
17+
work and reusing threads, especially in OLTP environments with many short-lived
18+
queries.
1019

11-
The thread pool plugin consists of several thread groups, each of which manages a set of client connections. As connections are established, the thread pool assigns them to thread groups using the round-robin method. This method assigns threads fairly and efficiently. Here's how it works:
20+
## Version-specific notes
1221

13-
1. The thread pool starts with a set number of thread groups.
22+
Thread pool support in Percona Server for MySQL evolved across releases. Review
23+
the following version details to avoid configuration errors and ensure
24+
compatibility.
1425

15-
2. When a new task arrives, the pool needs to assign it to a group.
26+
27+
### Changed in 8.0.14
1628

17-
3. It does this by going through the groups in order, one by one.
29+
Beginning with 8.0.14, the upstream implementation of `admin_port` replaced the
30+
previous mechanism that used `extra_port` and `extra_max_connections`.
1831

19-
4. Let's say you have four thread groups. The assignment would work like this:
20-
- Task 1 goes to Group 1
21-
- Task 2 goes to Group 2
22-
- Task 3 goes to Group 3
23-
- Task 4 goes to Group 4
24-
- Task 5 goes back to Group 1
25-
26-
5. This pattern continues, always moving to the next group and starting over when it reaches the end.
32+
33+
**What changed:**
2734

28-
6. Each group handles its assigned tasks using its available threads.
35+
* `extra_port` and `extra_max_connections` are removed.
2936

30-
This round-robin approach spreads work evenly across all groups. It prevents any single group from getting overloaded while others sit idle. This method helps maintain balanced performance across the system.
37+
* These variables are no longer recognized and cause server startup failure.
3138

32-
MySQL executes statements using one thread per client connection. When the number of connections increases past a specific point, performance degrades.
33-
This feature introduces a dynamic thread pool, which enables the server to maintain top performance even with a large number of client connections. The server decreases the number of threads using the thread pool and reduces the context switching and hot lock contentions. The thread pool is most effective with `OLTP` workloads (relatively short CPU-bound queries).
39+
40+
**Migration steps:**
3441

35-
Set the thread pool variable [thread_handling](#thread_handling) to `pool-of-threads` by adding the following line to `my.cnf`:
42+
* Remove all references to `extra_port` and `extra_max_connections` from the
43+
configuration file before upgrading to 8.0.14 or later.
3644

37-
```text
38-
thread_handling=pool-of-threads
45+
* Use the `admin_port` variable, as supported by upstream MySQL, to configure
46+
administrative access if needed.
47+
48+
49+
### Implemented in 8.0.12-1
50+
51+
Percona Server for MySQL 8.0.12-1 introduced native support for the thread pool.
52+
This version ported the feature from Percona Server 5.7.
53+
54+
## Core concepts and usage examples
55+
56+
### Fixed pool of threads
57+
58+
The server initializes a defined number of threads at startup and reuses them
59+
to process incoming queries.
60+
61+
62+
```ini
63+
[mysqld]
64+
thread_handling = pool-of-threads
65+
```
66+
This setting activates the thread pool model.
67+
68+
### How the thread pool works
69+
70+
When a client connects, the server assigns the connection to a thread group using a round-robin method. Each group contains a listener thread and worker threads. The listener monitors active connections and queues incoming statements.
71+
72+
Each new query is routed to either the high-priority queue or the low-priority queue within the group. A query enters the high-priority queue if the connection has an open transaction and available high-priority tickets. All other queries are sent to the low-priority queue.
73+
74+
Worker threads scan the high-priority queue first, then process queries from the low-priority queue when no high-priority items remain. After finishing a query, the thread becomes idle and waits for the next task. This reuse avoids excessive thread creation and maintains steady performance under load.
75+
76+
The pool adapts to workload shifts by redistributing queued tasks and prioritizing available threads. The result is better system responsiveness with fewer total threads.
77+
78+
### Thread groups
79+
80+
The thread pool organizes threads into groups. Each group contains a listener thread, a set of worker threads, and handles a portion of the client connections.
81+
82+
83+
```ini
84+
[mysqld]
85+
thread_pool_size = 8
86+
```
87+
88+
This configuration spreads work across eight thread groups.
89+
90+
### Priority queues
91+
92+
Each thread group maintains a high-priority queue and a low-priority queue. New queries are added to one of the two based on ticket availability and transaction state. The thread pool always checks the high-priority queue first.
93+
94+
```mysql
95+
mysql> SHOW STATUS LIKE 'Threadpool%';
96+
```
97+
98+
This command displays queue lengths and thread statistics.
99+
100+
### High-priority ticketing
101+
102+
Each connection begins with a set number of high-priority tickets. These tickets allow prioritization for a limited number of queries. One ticket is used every time the thread pool promotes a query to the high-priority queue.
103+
104+
```ini
105+
[mysqld]
106+
thread_pool_high_prio_tickets = 5
107+
```
108+
109+
This assigns five tickets to each new connection.
110+
111+
112+
### High-priority modes
113+
114+
The `thread_pool_high_prio_mode` setting controls how the thread pool schedules queries using tickets.
115+
116+
```mysql
117+
mysql> SET GLOBAL thread_pool_high_prio_mode = 'transactions';
118+
```
119+
120+
This setting limits prioritization to queries within open transactions.
121+
122+
123+
### Adaptive scheduling
124+
125+
The server assigns new connections to thread groups using a round-robin method. This approach balances workload evenly and prevents any group from becoming a bottleneck.
126+
127+
128+
Connection 1 goes to Group 1
129+
130+
Connection 2 goes to Group 2
131+
132+
Connection 3 goes to Group 3
133+
134+
Connection 4 goes to Group 4
135+
136+
Connection 5 returns to Group 1
137+
138+
<img src="../_static/thread-pool-diagram.png" alt="Thread pool diagram" width="250" />
139+
140+
### Configuration and monitoring
141+
142+
Administrators can fine-tune thread pool behavior by adjusting dynamic and static variables.
143+
144+
```ini
145+
[mysqld]
146+
thread_pool_stall_limit = 100
39147
```
40148

41-
Although the default values for the thread pool should provide good performance, additional tuning should be performed with the dynamic system variables. The goal is to minimize the number of open transactions on the server. Short-running transactions commit faster and deallocate server resources and locks.
149+
This value, in milliseconds, limits how long a task can stall before being redistributed.
42150

43-
Due to the following differences, this implementation is not compatible with upstream:
151+
```mysql
152+
mysql> SHOW ENGINE THREAD_POOL STATUS;
153+
```
44154

45-
* Built into the server, upstream implements the thread pool as a plugin
155+
This command returns information about thread usage and group activity.
46156

47-
* Does not minimize the number of concurrent transactions
157+
## When to use the thread pool
48158

49-
Priority Queue:
159+
Choosing between the thread pool and the default threading model depends on
160+
system size, workload patterns, and performance goals. Use the guidelines below
161+
to decide which approach fits best.
50162

51-
A queue that assigns a priority to each data element and processes them according to their priority. The data element with the highest priority is served first, regardless of its order in the queue. A priority queue can be implemented using an array, a linked list, a heap, or a binary search tree. It can also be ascending or descending, meaning that the highest priority is either the smallest or the largest value.
163+
### Use the thread pool when:
52164

53-
## Version specific information
165+
* The system experiences hundreds or thousands of concurrent connections.
54166

55-
Starting with 8.0.14, Percona Server for MySQL uses the upstream implementation of the [`admin_port`](https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_admin_port). The variables [extra_port](#extra_port) and [extra_max_connections](#extra_max_connections) are removed and not supported. Remove the `extra_port` and `extra_max_connections` variables from your configuration file before upgrading to 8.0.14 or higher. In 8.0.14 or higher, the variables cause a boot error, and the server refuses to start.
167+
* Most queries are short, frequent, and CPU-bound (typical of OLTP workloads).
56168

57-
Implemented in 8.0.12-1: We ported the `Thread Pool` feature from Percona Server for MySQL 5.7.
169+
* The server suffers from context switching, thread contention, or memory
170+
overhead due to excessive threads.
58171

59-
## Priority connection scheduling
172+
* Transactions should be prioritized, and precise query scheduling is required.
60173

61-
The thread pool limits the number of concurrently running queries. The number of open transactions may remain high. Connections with already-started transactions are added to the end of the queue. A high number of open transactions has implications for the currently running queries. The [thread_pool_high_prio_tickets](#thread_pool_high_prio_tickets) variable controls the high-priority queue policy and assigns tickets to each new connection.
174+
* Thread count must remain predictable and resource usage needs tighter control.
62175

63-
The thread pool adds the connection to the high-priority queue and decrements the ticket if the connection has the following attributes:
176+
177+
### Use the default model when:
64178

65-
* Has an open transaction
179+
* The deployment serves fewer than 500 concurrent clients.
66180

67-
* Has a non-zero number of high-priority tickets
181+
* Queries are long-running or resource-heavy and better suited to isolated
182+
execution.
68183

69-
Otherwise, the variable adds the connection to the low-priority queue with the initial value.
184+
* Thread overhead is not a problem, and the simplicity of one-thread-per-
185+
connection is acceptable.
70186

71-
Each time, the thread pool checks the high-priority queue for the next connection. When the high-priority queue is empty, the thread pool picks connections from the low-priority queue. The default behavior is to put events from already started transactions into the high-priority queue.
187+
* There is no pressing need to control concurrency or prioritize active
188+
transactions.
189+
190+
191+
Thread pooling offers greater control and efficiency, but only in workloads that
192+
justify that complexity. Measure concurrency patterns and CPU behavior before
193+
enabling the feature.
194+
```
72195
73-
If the value equals `0`, all connections are put into the low-priority queue. If the value exceeds zero, each connection could be put into a high-priority queue.
74196
75-
The [thread_pool_high_prio_mode](#thread_pool_high_prio_mode) variable prioritizes all statements for a connection or assigns connections to the low-priority queue. To implement this new [thread_pool_high_prio_mode](#thread_pool_high_prio_mode) variable
76197
77-
## Low-priority queue throttling
78198
79-
One case that can limit thread pool performance and even lead to deadlocks under high concurrency is when thread groups are oversubscribed due to active threads reaching the oversubscribe limit. Still, all/most worker threads are waiting on locks currently held by a transaction from another connection that is not currently in the thread pool.
80199
81-
In this case, the oversubscribe limit does not account for those threads in the pool that marked themselves inactive. As a result, the number of threads (both active and waiting) in the pool grows until it hits the [`thread_pool_max_threads`](#thread_pool_max_threads) value. If the connection executing the transaction holding the lock has managed to enter the thread pool by then, we get a large (depending on the [`thread_pool_max_threads`](#thread_pool_max_threads) value) number of concurrently running threads and, thus, suboptimal performance. Otherwise, we get a deadlock as no more threads can be created to process those transaction(s) and release the lock(s).
82200
83-
Such situations are prevented by throttling the low-priority queue when the total number of worker threads (both active and waiting ones) reaches the oversubscribe limit. If there are too many worker threads, do not start new transactions; create new threads until queued events from the already-started transactions are processed.
201+
### Static configuration
202+
203+
To set the variable persistently, add the following to your `my.cnf` file:
204+
205+
```ini
206+
[mysqld]
207+
thread_pool_high_prio_mode = transactions
208+
```
209+
210+
### Dynamic configuration
211+
212+
You can also change this setting at runtime without restarting:
213+
214+
```
215+
mysql> SET GLOBAL thread_pool_high_prio_mode = 'transactions';
216+
```
217+
218+
This change takes effect immediately for new connections only. Existing client sessions continue to operate under the previously active mode. To ensure all clients reflect the new behavior, either:
219+
220+
* Restart the server, or
221+
222+
* Have connected clients disconnect and reconnect manually.
223+
224+
This dynamic adjustment enables fine-tuning thread scheduling behavior in production systems without downtime.
225+
226+
84227

85-
## Handling long network waits
86228

87-
Specific workloads (large result sets, BLOBs, slow clients) can wait longer on network I/O (socket reads and writes). Whenever the server waits, this should be communicated to the thread pool so it can start a new query by either waking a waiting thread or sometimes creating a new one. This implementation has been ported from *MariaDB* patch MDEV-156.
88229

89230
## System variables
90231

@@ -121,8 +262,28 @@ This variable defines how the server handles threads for connections from the cl
121262

122263
This variable can limit the time an idle thread should wait before exiting.
123264

265+
124266
### `thread_pool_high_prio_mode`
125267

268+
| Option | Description |
269+
| -------------- | --------------------------------------------------------------------------- |
270+
| Command-line: | Yes |
271+
| Config file: | Yes |
272+
| Scope: | Global |
273+
| Dynamic: | Yes |
274+
| Data type: | Enumeration (`transactions`, `statements`, `none`) |
275+
| Default value: | `transactions` |
276+
277+
Controls how the thread pool schedules high-priority work. When set to
278+
`transactions`, only statements within an open transaction are eligible for
279+
high-priority execution. The `statements` option prioritizes all statements
280+
from connections that still have unused high-priority tickets. Setting this
281+
variable to `none` disables high-priority queueing entirely.
282+
283+
Changes take effect immediately for new connections. Existing sessions must
284+
reconnect to adopt the new behavior.
285+
286+
126287
This variable provides more fine-grained control over high-priority scheduling globally or per connection.
127288

128289
The following values are allowed:

0 commit comments

Comments
 (0)