IGNITE-28859 Add doc section about LRT diagnostics and termination

docs/_docs/code-snippets/java/src/main/java/org/apache/ignite/snippets/PerformingTransactions.java

@@ -174,8 +174,13 @@ void timeout() {
         // Create a Transaction configuration
         TransactionConfiguration txCfg = new TransactionConfiguration();
 
+        // tag::pme[]
         // Set the timeout to 20 seconds
         txCfg.setTxTimeoutOnPartitionMapExchange(20000);
+        // end::pme[]
+
+        // Set the timeout to 5 minutes
+        txCfg.setDefaultTxTimeout(300000);
 
         cfg.setTransactionConfiguration(txCfg);
 

docs/_docs/code-snippets/xml/transactions.xml

@@ -31,8 +31,16 @@
         <!-- tag::configuration[] -->
         <property name="transactionConfiguration">
             <bean class="org.apache.ignite.configuration.TransactionConfiguration">
+                <!-- tag::timeout[] -->
+                <!--Set isolation level to READ_COMMITTED-->
+                <property  name="defaultTxIsolation"  value="READ_COMMITTED" />
+                <!--Set the timeout to 5 minutes-->
+                <property  name="defaultTxTimeout"  value="300000" />
                 <!--Set the timeout to 20 seconds-->
+                <!-- end::timeout[] -->
+                <!-- tag::pme[] -->
                 <property name="TxTimeoutOnPartitionMapExchange" value="20000"/>
+                <!-- end::pme[] -->
             </bean>
         </property>
 

docs/_docs/key-value-api/transactions.adoc

@@ -204,7 +204,26 @@ Only during the commit phase, in case of any conflict, a `TransactionOptimisticE
 
 IMPORTANT: If you are not using PESSIMISTIC REPEATABLE_READ or SERIALIZABLE transactions or OPTIMISTIC SERIALIZABLE transactions, then it is possible to see a partial transaction state. This means that if one transaction updates objects A and B, then another transaction may see the new value for A and the old value for B.
 
+== Transactions Statuses
+A transaction, from its initiation to its completion via commit or rollback, goes through several statuses as required by the two-phase commit protocol (2-phase commit, 2PC):
+[cols="15%,10%,75%"]
+|===
+| Transaction Status | 2PC Phase | Description
+| `ACTIVE` | — | The user created a transaction using the `txStart(...)` method; the `commit`/`rollback` methods have not been called
+| `PREPARING` | `PREPARE` | The user called the `commit` method on the transaction; the 2PC protocol has started the preparation phase
+| `PREPARED` | `PREPARE` | The 2PC protocol has completed the preparation phase (nodes participating in the transaction have sent notifications of readiness to commit)
+| `COMMITTING` | `COMMIT` | The 2PC protocol has started the final commit phase and sent the commit command to the nodes
+| `COMMITTED` | `COMMIT` | The 2PC protocol has received responses from all nodes; the transaction commit is complete, and its context is being cleared
+| `MARKED_ROLLBACK` | `PREPARE`/`COMMIT` (\*) | The user called the `rollback` method on the transaction; the 2PC protocol has started the rollback procedure
+| `ROLLING_BACK` | `PREPARE`/`COMMIT` (\*) | The 2PC protocol has sent rollback requests to the nodes participating in the transaction
+| `ROLLED_BACK` | `PREPARE`/`COMMIT` (\*) | The 2PC protocol has received responses from all nodes; the rollback is complete, and the transaction context is being cleared
+| `SUSPENDED` | — | The transaction is suspended. This can occur in two cases: upon explicit user request by calling the `suspend` method on the `Transaction` object, or when starting a transaction using a thin client. In this case, the transaction is immediately moved to the `SUSPENDED` status after creation and exits it upon receiving subsequent requests from the client
+| `UNKNOWN` | — | An error occurred during rollback or transaction processing; the state is unknown
+|===
 
+=== Transaction Lifecycle
+* Transaction with a successful commit: `ACTIVE` → `PREPARING` → `PREPARED` → `COMMITTING` → `COMMITTED`.
+* Transaction that rolled back due to one of the nodes being unready to commit: `ACTIVE` → `PREPARING` → `ROLLING_BACK` → `ROLLED_BACK`.
 
 == Deadlock Detection
 
@@ -322,6 +341,92 @@ A transaction might fail with the following exceptions:
 
 
 == Long Running Transactions Termination
+The `tx` command of the `control.sh` utility allows you to search for transactions based on various criteria (such as ID, duration, or label) and roll back transactions. Since transactions in Apache Ignite are generally distributed (multiple nodes are involved in processing a single transaction), this command may display the same transaction as present on multiple nodes simultaneously.
+
+Rolling back a transaction using the `tx` command is only possible for the `ACTIVE`, `PREPARING`, and `PREPARED` statuses. A transaction in statuses related to the `rollback` procedure is already in the process of rolling back. Transactions in the `COMMITTING` and `COMMITTED` statuses cannot be rolled back due to 2PC protocol restrictions.
+
+* First termination stage — cancelling transactions using `tx kill`
+
+To cancel transactions, use the `control.sh` utility to minimize the impact on the cluster:
+# Execute the `control.sh --tx --min-duration <transaction_timeout_in_ms>` command in the cluster. Example output:
+----
+Command [TX] started
+Arguments: --tx --min-duration 30
+--------------------------------------------------------------------------------
+Matching transactions:
+TcpDiscoveryNode [id=<id>, addrs=[<address>], order=1, ver=2.18.0#YYYYMMDD-sha1:00000000, isClient=false, consistentId=gridCommandHandlerTest0]
+    Tx: [xid=<id>, label=null, state=ACTIVE, startTime=YYYY-MM-DD 15:15:16.852, duration=3 sec, isolation=REPEATABLE_READ, concurrency=PESSIMISTIC, topVer=AffinityTopologyVersion [topVer=3, minorTopVer=2], timeout=0 sec, size=100, dhtNodes=[<id>, <id>], nearXid=<id>, parentNodeIds=[<id>]]
+    Tx: [xid=<id>, label=null, state=ACTIVE, startTime=YYYY-MM-DD 15:15:16.852, duration=3 sec, isolation=REPEATABLE_READ, concurrency=PESSIMISTIC, topVer=AffinityTopologyVersion [topVer=3, minorTopVer=2], timeout=0 sec, size=100, dhtNodes=[<id>, <id>], nearXid=<id>, parentNodeIds=[<id>]]
+Command [TX] finished with code: 0
+----
+# Extract the `xid` from all transactions across all nodes and remove duplicate `xid` values. Duplicates occur because the same transaction can exist on multiple nodes — in this case, its `xid` will appear in the command output multiple times.
+# For each `xid`, run the `tx kill` command to roll back that transaction. Example call: `control.sh --tx --xid <id> --kill`.
+
+* Second mitigation stage — restarting nodes that block transaction completion
+Check if the first stage was sufficient to handle LRT. To do this, follow these steps:
+# After completing the first stage, wait for a period equal to the configured transaction timeout.
+# Run the `control.sh --tx --min-duration <transaction_timeout>` command.
+
+If the list is empty, all LRTs have been successfully canceled, and the LRT impact should be resolved.
+If the list is not empty, the command output will contain information about nodes and the transactions running on them:
+----
+Command [TX] started
+Arguments: --tx
+--------------------------------------------------------------------------------
+Matching transactions:
+TcpDiscoveryNode [id=<id>, addrs=[<address>], order=1, ver=2.18.0#YYYYMMDD-sha1:00000000, isClient=false, consistentId=gridCommandHandlerTest0]
+    Tx: [xid=<id>, label=null, state=ACTIVE, startTime=YYYY-MM-DD 16:58:45.348, duration=3 sec, isolation=REPEATABLE_READ, concurrency=PESSIMISTIC, topVer=AffinityTopologyVersion [topVer=3, minorTopVer=2], timeout=0 sec, size=100, dhtNodes=[<id>, <id>], nearXid=<id>, parentNodeIds=[<id>]]
+    Tx: [xid=<id>, label=label1, state=ACTIVE, startTime=YYYY-MM-DD 16:58:48.451, duration=0 sec, isolation=READ_COMMITTED, concurrency=PESSIMISTIC, topVer=AffinityTopologyVersion [topVer=3, minorTopVer=2], timeout=2147483 sec, size=111, dhtNodes=[<id>, <id>], nearXid=<id>, parentNodeIds=[<id>]]
+TcpDiscoveryNode [id=<id>, addrs=[<address>], order=2, ver=2.18.0#YYYYMMDD-sha1:00000000, isClient=false, consistentId=gridCommandHandlerTest1]
+    Tx: [xid=<id>, label=null, state=ACTIVE, startTime=YYYY-MM-DD 16:58:45.348, duration=3 sec, isolation=REPEATABLE_READ, concurrency=PESSIMISTIC, topVer=AffinityTopologyVersion [topVer=3, minorTopVer=2], timeout=0 sec, size=1, dhtNodes=[<id>], nearXid=<id>, parentNodeIds=[<id>]]
+TcpDiscoveryNode [id=<id>, addrs=[<address>], order=3, ver=2.18.0#YYYYMMDD-sha1:00000000, isClient=true, consistentId=client]
+    Tx: [xid=<id>, label=label2, state=PREPARING, startTime=YYYY-MM-DD 16:58:45.348, duration=3 sec, isolation=READ_COMMITTED, concurrency=OPTIMISTIC, topVer=AffinityTopologyVersion [topVer=3, minorTopVer=2], timeout=0 sec, size=11, dhtNodes=[<id>, <id>], nearXid=<id>, parentNodeIds=[<id>]]
+Command [TX] finished with code: 0
+----
+To find and restart nodes that are blocking transaction completion:
+# For each transaction, extract the value of the `dhtNodes` field.
+# For each ID in `dhtNodes`, obtain detailed node information using the `control.sh --system-view NODES` command.
+# In the command output, find the node whose `nodeId` starts with the string from `dhtNodes`.
+# Restart this node to unblock the transaction.
+
+== Causes of LRT
+- High system load, which leads to slower transaction processing. Indicators:
+  * log messages about the start of page eviction for in-memory data regions or page replacement for persistent data regions;
+  * growth of queues in the striped pool;
+  * increased checkpoint duration for clusters with persistence enabled;
+  * long GC pauses.
+- Unstable network operation. Indicators: log messages about connection loss between nodes, socket closures, or network timeouts triggering.
+- Resource-intensive operations (for example, deleting a cache that is part of a cache group). Indicators: queue growth on individual threads of the system pool and striped pool, while other threads and overall node utilization may remain low.
+- Internal Apache Ignite bugs that can lead to deadlocks. Indicators: thread dumps (`thread dump`) show threads waiting to acquire a read lock (`readLock`) or write lock (`writeLock`) during checkpoint creation; over time, the thread state does not change (stack trace remains unchanged).
+
+== Transactions Timeout
+
+=== Overriding default transaction timeout
+- `TransactionConfiguration#setDefaultTxTimeout` — default transaction timeout (used when user code starts a transaction without specifying a timeout). If a transaction execution exceeds this time, the cluster attempts to roll it back. If the transaction state does not allow a rollback, the timeout will have no effect.
+
+Example:
+
+[tabs]
+--
+tab:XML[]
+[source,xml]
+----
+include::code-snippets/xml/transactions.xml[tags=ignite-config;configuration;timeout;!pme;!cache;!discovery, indent=0]
+----
+tab:Java[]
+[source,java]
+----
+include::{javaFile}[tag=timeout;!pme;indent=0]
+----
+tab:C#/.NET[]
+[source,csharp]
+----
+include::code-snippets/dotnet/PerformingTransactions.cs[tag=pmeTimeout,indent=0]
+----
+tab:C++[unsupported]
+--
+
+=== Timeout On Partition Map Exchange
 
 Some cluster events trigger partition map exchange process and data rebalancing within an Ignite cluster to ensure even data distribution cluster-wide. An example of one such event is the cluster-topology-change event that takes place whenever a new node joins the cluster or an existing one leaves it. Plus, every time a new cache or SQL table is created, the partition map exchange gets triggered.
 
@@ -330,8 +435,7 @@ When the partition map exchange starts, Ignite acquires a global lock at a parti
 Use the `TransactionConfiguration.setTxTimeoutOnPartitionMapExchange(...)` method to set the maximum time allowed for your long-running transactions to block the partition map exchange.
 Once the timeout fires, all incomplete transactions are rolled back letting the partition map exchange proceed.
 
-This example shows how to configure the timeout:
-
+Example:
 
 [tabs]
 --