Skip to content

Commit 5b9e40b

Browse files
committed
PS-10071 [DOCS] - update Threadpool to add implementation 8.0
new file: docs/_static/thread-pool-diagram.png modified: docs/threadpool.md
1 parent ab05d24 commit 5b9e40b

File tree

2 files changed

+264
-47
lines changed

2 files changed

+264
-47
lines changed

docs/_static/thread-pool-diagram.png

782 KB
Loading

docs/threadpool.md

Lines changed: 264 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -1,90 +1,287 @@
11
# Thread pool
22

3-
Thread pooling can improve performance and scalability for MySQL databases. This technique reuses a fixed number of threads to handle multiple client connections and execute statements. It reduces the overhead of creating and destroying threads and avoids the contention and context switching that can occur when there are too many threads.
3+
## Introduction
44

5-
If you have fewer than 20,000 connections, using the thread pool does not provide significant benefits. It’s better to keep thread pooling disabled and use the default method.
5+
Thread pooling improves performance and scalability for MySQL-compatible databases by
6+
reusing a fixed number of pre-created threads to manage multiple client
7+
sessions. This design reduces resource overhead, lowers contention, and avoids
8+
context switching bottlenecks during high concurrency.
69

7-
The default method, called one-thread-per-connection, creates a new thread for each client that connects to the MySQL server. This thread manages all queries and responses for that connection until it’s closed. This approach works well for a moderate number of connections, but it can become inefficient as the number of connections increases.
10+
The default MySQL method creates one thread per client connection. This approach
11+
works efficiently under moderate connection loads. As the number of active
12+
connections increases, especially beyond 20,000, system overhead becomes
13+
significant and throughput decreases.
814

9-
MySQL supports thread pooling through the thread pool plugin, which replaces the default one-thread-per-connection model. When a statement arrives, the thread group either begins executing it immediately or queues it for later execution in a round-robin fashion. The high-priority queue consists of several thread groups, each managing client connections. Each thread group has a listener thread that listens for incoming statements from the connections assigned to the group. The thread pool exposes several system variables that can be used to configure its operation, such as thread_pool_size, thread_pool_algorithm, thread_pool_stall_limit, and others.
15+
Percona Server for MySQL includes an integrated thread pool that replaces the
16+
default model. The thread pool manages connections more efficiently by queuing
17+
work and reusing threads, especially in OLTP environments with many short-lived
18+
queries.
1019

11-
The thread pool plugin consists of several thread groups, each of which manages a set of client connections. As connections are established, the thread pool assigns them to thread groups using the round-robin method. This method assigns threads fairly and efficiently. Here's how it works:
20+
## Version-specific notes
1221

13-
1. The thread pool starts with a set number of thread groups.
22+
Thread pool support in Percona Server for MySQL evolved across releases. Review
23+
the following version details to avoid configuration errors and ensure
24+
compatibility.
1425

15-
2. When a new task arrives, the pool needs to assign it to a group.
26+
27+
### Changed in 8.0.14
1628

17-
3. It does this by going through the groups in order, one by one.
29+
Beginning with 8.0.14, the upstream implementation of `admin_port` replaced the
30+
previous mechanism that used `extra_port` and `extra_max_connections`.
1831

19-
4. Let's say you have four thread groups. The assignment would work like this:
20-
- Task 1 goes to Group 1
21-
- Task 2 goes to Group 2
22-
- Task 3 goes to Group 3
23-
- Task 4 goes to Group 4
24-
- Task 5 goes back to Group 1
25-
26-
5. This pattern continues, always moving to the next group and starting over when it reaches the end.
32+
33+
**What changed:**
2734

28-
6. Each group handles its assigned tasks using its available threads.
35+
* `extra_port` and `extra_max_connections` are removed.
2936

30-
This round-robin approach spreads work evenly across all groups. It prevents any single group from getting overloaded while others sit idle. This method helps maintain balanced performance across the system.
37+
* These variables are no longer recognized and cause server startup failure.
3138

32-
MySQL executes statements using one thread per client connection. When the number of connections increases past a specific point, performance degrades.
33-
This feature introduces a dynamic thread pool, which enables the server to maintain top performance even with a large number of client connections. The server decreases the number of threads using the thread pool and reduces the context switching and hot lock contentions. The thread pool is most effective with `OLTP` workloads (relatively short CPU-bound queries).
39+
40+
**Migration steps:**
3441

35-
Set the thread pool variable [thread_handling](#thread_handling) to `pool-of-threads` by adding the following line to `my.cnf`:
42+
* Remove all references to `extra_port` and `extra_max_connections` from the
43+
configuration file before upgrading to 8.0.14 or later.
3644

37-
```text
38-
thread_handling=pool-of-threads
45+
* Use the `admin_port` variable, as supported by upstream MySQL, to configure
46+
administrative access if needed.
47+
48+
49+
### Implemented in 8.0.12-1
50+
51+
Percona Server for MySQL 8.0.12-1 introduced native support for the thread pool.
52+
This version ported the feature from Percona Server 5.7.
53+
54+
## Core concepts and usage examples
55+
56+
### Fixed pool of threads
57+
58+
The server initializes a defined number of threads at startup and reuses them
59+
to process incoming queries.
60+
61+
62+
```ini
63+
[mysqld]
64+
thread_handling = pool-of-threads
3965
```
66+
This setting activates the thread pool model.
67+
68+
### How the thread pool works
69+
70+
When a client connects, the server assigns the connection to a thread group using a round-robin method. Each group contains a listener thread and worker threads. The listener monitors active connections and queues incoming statements.
71+
72+
Each new query is routed to either the high-priority queue or the low-priority queue within the group. A query enters the high-priority queue if the connection has an open transaction and available high-priority tickets. All other queries are sent to the low-priority queue.
73+
74+
Worker threads scan the high-priority queue first, then process queries from the low-priority queue when no high-priority items remain. After finishing a query, the thread becomes idle and waits for the next task. This reuse avoids excessive thread creation and maintains steady performance under load.
75+
76+
The pool adapts to workload shifts by redistributing queued tasks and prioritizing available threads. The result is better system responsiveness with fewer total threads.
4077

41-
Although the default values for the thread pool should provide good performance, additional tuning should be performed with the dynamic system variables. The goal is to minimize the number of open transactions on the server. Short-running transactions commit faster and deallocate server resources and locks.
78+
<img src="../_static/thread-pool-diagram.png" alt="Thread pool diagram" width="250" />
4279

43-
Due to the following differences, this implementation is not compatible with upstream:
80+
### Thread groups
4481

45-
* Built into the server, upstream implements the thread pool as a plugin
82+
The thread pool organizes threads into groups. Each group contains a listener thread, a set of worker threads, and handles a portion of the client connections.
4683

47-
* Does not minimize the number of concurrent transactions
4884

49-
Priority Queue:
85+
```ini
86+
[mysqld]
87+
thread_pool_size = 8
88+
```
89+
90+
This configuration spreads work across eight thread groups.
91+
92+
### Priority queues
93+
94+
Each thread group maintains a high-priority queue and a low-priority queue. New queries are added to one of the two based on ticket availability and transaction state. The thread pool always checks the high-priority queue first.
95+
96+
```mysql
97+
mysql> SHOW STATUS LIKE 'Threadpool%';
98+
```
5099

51-
A queue that assigns a priority to each data element and processes them according to their priority. The data element with the highest priority is served first, regardless of its order in the queue. A priority queue can be implemented using an array, a linked list, a heap, or a binary search tree. It can also be ascending or descending, meaning that the highest priority is either the smallest or the largest value.
100+
This command displays queue lengths and thread statistics.
52101

53-
## Version specific information
102+
### High-priority ticketing
54103

55-
Starting with 8.0.14, Percona Server for MySQL uses the upstream implementation of the [`admin_port`](https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_admin_port). The variables [extra_port](#extra_port) and [extra_max_connections](#extra_max_connections) are removed and not supported. Remove the `extra_port` and `extra_max_connections` variables from your configuration file before upgrading to 8.0.14 or higher. In 8.0.14 or higher, the variables cause a boot error, and the server refuses to start.
104+
Each connection begins with a set number of high-priority tickets. These tickets allow prioritization for a limited number of queries. One ticket is used every time the thread pool promotes a query to the high-priority queue.
56105

57-
Implemented in 8.0.12-1: We ported the `Thread Pool` feature from Percona Server for MySQL 5.7.
106+
#### Static configuration
58107

59-
## Priority connection scheduling
108+
```ini
109+
[mysqld]
110+
thread_pool_high_prio_tickets = 5
111+
```
60112

61-
The thread pool limits the number of concurrently running queries. The number of open transactions may remain high. Connections with already-started transactions are added to the end of the queue. A high number of open transactions has implications for the currently running queries. The [thread_pool_high_prio_tickets](#thread_pool_high_prio_tickets) variable controls the high-priority queue policy and assigns tickets to each new connection.
113+
#### Dynamic configuration
62114

63-
The thread pool adds the connection to the high-priority queue and decrements the ticket if the connection has the following attributes:
115+
```mysql
116+
mysql> mysql> SET GLOBAL thread_pool_high_prio_tickets = 5;
117+
```
64118

65-
* Has an open transaction
119+
Either of these configurations assign five tickets to each new connection.
66120

67-
* Has a non-zero number of high-priority tickets
68121

69-
Otherwise, the variable adds the connection to the low-priority queue with the initial value.
122+
### High-priority modes
123+
124+
The `thread_pool_high_prio_mode` setting controls how the thread pool schedules queries using tickets.
125+
126+
```mysql
127+
mysql> SET GLOBAL thread_pool_high_prio_mode = 'transactions';
128+
```
129+
130+
This setting limits prioritization to queries within open transactions.
131+
132+
### Configuration `thread_pool_high_prio_mode`
133+
134+
To set the variable persistently, add the following to your `my.cnf` file:
135+
136+
```ini
137+
[mysqld]
138+
thread_pool_high_prio_mode = transactions
139+
```
140+
141+
You can also change this setting at runtime without restarting:
142+
143+
```
144+
mysql> SET GLOBAL thread_pool_high_prio_mode = 'transactions';
145+
```
146+
147+
This change takes effect immediately for new connections only. Existing client sessions continue to operate under the previously active mode. To ensure all clients reflect the new behavior, either:
148+
149+
* Restart the server, or
150+
151+
* Have connected clients disconnect and reconnect manually.
152+
153+
This flexibility allows scheduling behavior to be fine-tuned without disrupting active workloads.
154+
155+
### Adaptive scheduling
156+
157+
The server assigns new connections to thread groups using a round-robin method. This approach balances workload evenly and prevents any group from becoming a bottleneck.
158+
159+
160+
Connection 1 goes to Group 1
161+
162+
Connection 2 goes to Group 2
163+
164+
Connection 3 goes to Group 3
165+
166+
Connection 4 goes to Group 4
167+
168+
Connection 5 returns to Group 1
169+
170+
171+
172+
### Configuration and monitoring
173+
174+
Administrators can fine-tune thread pool behavior by adjusting dynamic and static variables.
175+
176+
177+
#### Static configuration
178+
179+
```ini
180+
[mysqld]
181+
thread_pool_stall_limit = 100
182+
```
183+
184+
#### Dynamic configuration
185+
186+
```mysql
187+
mysql> SET GLOBAL thread_pool_stall_limit = 100;
188+
```
189+
190+
This value, in milliseconds, limits how long a task can stall before being redistributed. As with other thread pool settings, existing sessions retain their previous values.
191+
192+
#### Check the status
193+
194+
```mysql
195+
mysql> SHOW ENGINE THREAD_POOL STATUS;
196+
```
197+
198+
This command returns information about thread usage and group activity.
199+
200+
## When to use the thread pool
201+
202+
Choosing between the thread pool and the default threading model depends on
203+
system size, workload patterns, and performance goals. Use the guidelines below
204+
to decide which approach fits best.
205+
206+
### Use the thread pool when:
207+
208+
* The system experiences hundreds or thousands of concurrent connections.
209+
210+
* Most queries are short, frequent, and CPU-bound (typical of OLTP workloads).
211+
212+
* The server suffers from context switching, thread contention, or memory
213+
overhead due to excessive threads.
214+
215+
* Transactions should be prioritized, and precise query scheduling is required.
216+
217+
* Thread count must remain predictable and resource usage needs tighter control.
218+
219+
220+
### Use the default model when:
221+
222+
* The deployment serves fewer than 500 concurrent clients.
223+
224+
* Queries are long-running or resource-heavy and better suited to isolated
225+
execution.
226+
227+
* Thread overhead is not a problem, and the simplicity of one-thread-per-
228+
connection is acceptable.
229+
230+
* There is no pressing need to control concurrency or prioritize active
231+
transactions.
232+
233+
234+
Thread pooling offers greater control and efficiency, but only in workloads that
235+
justify that complexity. Measure concurrency patterns and CPU behavior before
236+
enabling the feature.
237+
238+
## Thread Pool Configuration Reference Sheet
239+
240+
A quick overview of system variables available for configuring the thread pool in Percona Server for MySQL.
241+
242+
### General Variables
243+
244+
| Variable name | Default | Scope | Dynamic | Config file | Description |
245+
|--------------------------|-----------------------------|--------|---------|--------------|-------------------------------------------------------|
246+
| `thread_handling` | `one-thread-per-connection` | Global | No | Yes | Chooses the thread model. Set to `pool-of-threads` to enable thread pooling. |
247+
| `thread_pool_size` | `16` | Global | No | Yes | Defines the number of thread groups. |
248+
| `thread_pool_algorithm` | `first-in, first-out` | Global | Yes | Yes | Controls how queued queries are scheduled (`first-in, first-out` or `lowest_priority`). |
249+
250+
### Priority Handling
251+
252+
| Variable name | Default | Scope | Dynamic | Config file | Description |
253+
|----------------------------------|----------------|--------|---------|--------------|------------------------------------------------------------|
254+
| `thread_pool_high_prio_tickets` | `4294967295` | Global | Yes | Yes | High-priority executions per connection. |
255+
| `thread_pool_high_prio_mode` | `transactions` | Global | Yes | Yes | Determines when to assign high priority (`transactions`, `statements`, or `none`). |
256+
257+
### Thread Management
258+
259+
| Variable name | Default | Scope | Dynamic | Config file | Description |
260+
|---------------------------|---------|--------|---------|--------------|-----------------------------------------------------|
261+
| `thread_pool_min_threads` | `4` | Global | No | Yes | Minimum number of threads per group. |
262+
| `thread_pool_max_threads` | `1000` | Global | No | Yes | Maximum number of threads per group. |
263+
| `thread_pool_stall_limit` | `6` | Global | Yes | Yes | Time in milliseconds before retrying a stalled task. |
264+
265+
### Observability Commands
266+
267+
Use these SQL statements to view runtime metrics:
268+
269+
```sql
270+
SHOW STATUS LIKE 'Threadpool%';
271+
SHOW ENGINE THREAD_POOL STATUS;
272+
```
273+
274+
Note: For dynamic variables, changes apply only to new connections. Existing sessions retain their original settings until reconnect.
275+
70276

71-
Each time, the thread pool checks the high-priority queue for the next connection. When the high-priority queue is empty, the thread pool picks connections from the low-priority queue. The default behavior is to put events from already started transactions into the high-priority queue.
72277

73-
If the value equals `0`, all connections are put into the low-priority queue. If the value exceeds zero, each connection could be put into a high-priority queue.
74278

75-
The [thread_pool_high_prio_mode](#thread_pool_high_prio_mode) variable prioritizes all statements for a connection or assigns connections to the low-priority queue. To implement this new [thread_pool_high_prio_mode](#thread_pool_high_prio_mode) variable
76279

77-
## Low-priority queue throttling
78280

79-
One case that can limit thread pool performance and even lead to deadlocks under high concurrency is when thread groups are oversubscribed due to active threads reaching the oversubscribe limit. Still, all/most worker threads are waiting on locks currently held by a transaction from another connection that is not currently in the thread pool.
80281

81-
In this case, the oversubscribe limit does not account for those threads in the pool that marked themselves inactive. As a result, the number of threads (both active and waiting) in the pool grows until it hits the [`thread_pool_max_threads`](#thread_pool_max_threads) value. If the connection executing the transaction holding the lock has managed to enter the thread pool by then, we get a large (depending on the [`thread_pool_max_threads`](#thread_pool_max_threads) value) number of concurrently running threads and, thus, suboptimal performance. Otherwise, we get a deadlock as no more threads can be created to process those transaction(s) and release the lock(s).
82282

83-
Such situations are prevented by throttling the low-priority queue when the total number of worker threads (both active and waiting ones) reaches the oversubscribe limit. If there are too many worker threads, do not start new transactions; create new threads until queued events from the already-started transactions are processed.
84283

85-
## Handling long network waits
86284

87-
Specific workloads (large result sets, BLOBs, slow clients) can wait longer on network I/O (socket reads and writes). Whenever the server waits, this should be communicated to the thread pool so it can start a new query by either waking a waiting thread or sometimes creating a new one. This implementation has been ported from *MariaDB* patch MDEV-156.
88285

89286
## System variables
90287

@@ -121,8 +318,28 @@ This variable defines how the server handles threads for connections from the cl
121318

122319
This variable can limit the time an idle thread should wait before exiting.
123320

321+
124322
### `thread_pool_high_prio_mode`
125323

324+
| Option | Description |
325+
| -------------- | --------------------------------------------------------------------------- |
326+
| Command-line: | Yes |
327+
| Config file: | Yes |
328+
| Scope: | Global |
329+
| Dynamic: | Yes |
330+
| Data type: | Enumeration (`transactions`, `statements`, `none`) |
331+
| Default value: | `transactions` |
332+
333+
Controls how the thread pool schedules high-priority work. When set to
334+
`transactions`, only statements within an open transaction are eligible for
335+
high-priority execution. The `statements` option prioritizes all statements
336+
from connections that still have unused high-priority tickets. Setting this
337+
variable to `none` disables high-priority queueing entirely.
338+
339+
Changes take effect immediately for new connections. Existing sessions must
340+
reconnect to adopt the new behavior.
341+
342+
126343
This variable provides more fine-grained control over high-priority scheduling globally or per connection.
127344

128345
The following values are allowed:

0 commit comments

Comments
 (0)