New pool implementation

design/connection-pool/connection-pool-design-doc.md

@@ -0,0 +1,87 @@
+# Design Document: `ChannelDbConnectionPool`
+## Problem Statement
+The current connection pool implementation is slow to open new connections and does not follow async best practices.
+
+Connection opening is serialized, causing delays when multiple new connections are required simultaneously. This is done using a semaphore to rate limit connection creation. When multiple new connections are requested, they queue up waiting for the semaphore. Once acquired, the thread opens the connection and releases the semaphore, allowing the next thread to proceed. This approach was initially designed to prevent overwhelming the server but can lead to significant delays, especially in high-latency environments.
+
+Async requests are also serialized through an additional queue. When an async request is made, it is added to a queue, and a reader thread processes these requests one by one. This method was chosen due to the lack of async APIs in native SNI, resulting in synchronous handling of async requests on a dedicated thread.
+
+
+## Design Goals
+- Enable parallel connection opening.
+- Minimize thread contention and synchronization overhead.
+- Follow async best practices to reduce managed threadpool pressure and enable other components of the driver to modernize their async code.
+
+## Overview
+The core of the design is the Channel data structure from the [System.Threading.Channels library](https://learn.microsoft.com/en-us/dotnet/core/extensions/channels) (available to .NET Framework as a nuget package) (Also see Stephen Toub's intro [here](https://devblogs.microsoft.com/dotnet/an-introduction-to-system-threading-channels/)). Channels are thread-safe, async-first queues that fit well for the connection pooling use case.
+
+A single channel holds the idle connections managed by the pool. A channel reader reads idle connections out of the channel to vend them to SqlConnections. A channel writer writes connections back to the channel when they are returned to the pool.
+
+Pool maintenance operations (warmup, pruning) are handled asynchronously as Tasks.
+
+Transaction-enlisted connections are stored in a separate dictionary data structure, in the same manner as the WaitHandleDbConnectionPool implementation.
+
+This design is based on the [PoolingDataSource](https://github.com/npgsql/npgsql/blob/main/src/Npgsql/PoolingDataSource.cs) class from the npgsql driver. The npgsql implemenation is proven to be reliable and performant in real production workloads. 
+
+### Why the Channel Data Structure is a Good Fit
+
+1. **Thread-Safety**:
+   Channels are designed to facilitate thread-safe communication between producers (e.g., threads returning connections to the pool) and consumers (e.g., threads requesting connections). This eliminates the need for complex locking mechanisms, reducing the risk of race conditions and deadlocks.
+
+2. **Built-In Request Queueing**:  
+   Channels provide a succinct API to wait asynchronously if no connections are available at the time of the request.
+
+3. **Asynchronous Support**:  
+   Channels provide a robust async API surface, simplifying the async paths for the connection pool.
+
+4. **Performant**:  
+   Channels are fast and avoid extra allocations, making them suitable for high throughput applications: https://devblogs.microsoft.com/dotnet/an-introduction-to-system-threading-channels/#performance
+
+## Workflows
+1. **Warmup**:
+New connections are written to the tail of the idle channel by an async task.
+
+2. **Acquire Connection**:
+Idle connections are acquired from the head of the idle channel.
+
+3. **Release Connection**:
+Connections that are released to the pool are added to the tail of the idle channel.
+
+4. **Pruning**:
+Connections are pruned from the head of the idle channel.
+
+![Diagram showing user interactions and subprocesses interacting with the idle connection channel](<SqlClient Connection Pool (11).png>)
+
+
+## Performance Benchmarks
+Note: All graphed results use managed SNI. See full results below for native SNI.
+
+The channel based implementation shows significant performance improvements across frameworks and operating systems. In particular, interacting with a warm pool is much faster.
+
+![Chart showing mean time to open 100 local connections with cold pool](MeanTimeLocal.png)
+![Chart showing mean time to open 100 local connections with warm pool](MeanTimeLocalWarm-1.png)
+![Chart showing mean time to open 10 azure connections with warm pool](MeanTimeAzureWarm-1.png)
+
+
+When interacting with a cold pool and connecting to an Azure database, performance is equivalent to the legacy implementation *provided enough threads are made available in the managed threadpool*. This requirement highlights a bottleneck present further down the stack when acquiring federated auth tokens.
+
+![Chart showing mean time to open 10 azure connections with cold pool](MeanTimeAzureCold-1.png)
+
+
+### Windows - .NET 8.0
+![Performance results Windows - .NET 8.0](Core8Local.png)
+
+### Windows - .NET Framework 4.8.1
+![Performance results Windows - .NET Framework 4.8.1](Framework481Local.png)
+
+### Linux - net8.0
+![Performance results Linux - net8.0](Core8LocalLinux.png)
+
+### Windows - net8.0 - AzureSQL - Default Azure Credential
+![Performance results Windows - net8.0 - AzureSQL - Default Azure Credential](Core8AzureDefault.png)
+
+### Windows - .NET Framework 4.8.1 - AzureSQL - Default Azure Credential
+![Performance results Windows - .NET Framework 4.8.1 - AzureSQL - Default Azure Credential](Framework481AzureDefault.png)
+
+### Windows - NET 8.0 - Azure SQL - AccessToken
+![Performance results Windows - NET 8.0 - Azure SQL - AccessToken](Core8AzureAccessToken.png)

design/connection-pool/connection-pooling-primer.md

@@ -0,0 +1,18 @@
+# Primer on Connection Pooling
+
+## Overview
+Connection pooling is a technique used to manage database connections efficiently. It involves maintaining a pool of connections that can be reused, reducing the overhead of opening and closing connections frequently.
+
+## Key Concepts
+- **Connection Pool Initialization**: A connection pool is created for each unique connection string. The pool maintains a collection of connections that are already logged in and ready to use.
+- **Connection Reuse and Creation**: Connections are reused from the pool if available; otherwise, new connections are created. Connection strings are used to differentiate pools, and any change in the connection string results in a new pool.
+- **Connection String Sensitivity**: Connection pooling is not sensitive to whitespace in connection strings. Different authentication methods for the same user result in separate pools.
+- **Pool Management**: Pools are managed per process. A pool manager oversees all pools and determines which pool to use based on the connection string.
+- **Session Settings**: SQL Server provides a procedure (SP reset connection) to reset session settings when reusing a connection. SP reset connection is triggered every time a connection is reused from the pool.
+- **Handling Transactions**: Connections involved in transactions are handled separately and may be reset while preserving the transaction state.
+- **Connection Liveness**: Connection liveness is checked when pulling connections from the pool. Dead connections are discarded, and new ones are created if necessary.
+- **Connection Pruning**: Idle connections above the minimum threshold are closed periodically to manage resources. Pruning helps reclaim leaked connections and maintain the pool size within the defined limits.
+- **Warm-Up Process**: On application startup, the pool warms up to the minimum pool size by creating connections in the background.
+- **Handling Broken Connections**: Broken connections are detected and handled by creating new connections if the session cannot be recovered.
+- **Concurrency and Async Handling**: Connection creation should happen on separate threads to avoid queuing and improve performance. 
+- **Security Considerations**: Pools are separated based on user authentication to prevent unauthorized access. 

src/Microsoft.Data.SqlClient/netcore/src/Microsoft.Data.SqlClient.csproj

@@ -95,6 +95,9 @@
     <Compile Include="$(CommonSourceRoot)\Microsoft\Data\ProviderBase\DbConnectionFactory.cs">
       <Link>Microsoft\Data\ProviderBase\DbConnectionFactory.cs</Link>
     </Compile>
+    <Compile Include="$(CommonSourceRoot)Microsoft\Data\SqlClient\ConnectionPool\ChannelDbConnectionPool.cs">
+      <Link>Microsoft\Data\SqlClient\ConnectionPool\ChannelDbConnectionPool.cs</Link>
+    </Compile>
     <Compile Include="$(CommonSourceRoot)Microsoft\Data\SqlClient\ConnectionPool\DbConnectionPool.cs">
       <Link>Microsoft\Data\SqlClient\ConnectionPool\DbConnectionPool.cs</Link>
     </Compile>
@@ -137,6 +140,24 @@
     <Compile Include="$(CommonSourceRoot)Microsoft\Data\SqlClient\ConnectionPool\SqlConnectionPoolProviderInfo.cs">
       <Link>Microsoft\Data\SqlClient\ConnectionPool\SqlConnectionPoolProviderInfo.cs</Link>
     </Compile>
+    <Compile Include="$(CommonSourceRoot)Microsoft\Data\SqlClient\ConnectionPool\TransactedConnectionPool.cs">
+      <Link>Microsoft\Data\SqlClient\ConnectionPool\TransactedConnectionPool.cs</Link>
+    </Compile>
+    <Compile Include="$(CommonSourceRoot)Microsoft\Data\SqlClient\RateLimiter\AsyncFlagFunc.cs">
+      <Link>Microsoft\Data\SqlClient\RateLimiter\AsyncFlagFunc.cs</Link>
+    </Compile>
+    <Compile Include="$(CommonSourceRoot)Microsoft\Data\SqlClient\RateLimiter\BlockingPeriodRateLimiter.cs">
+      <Link>Microsoft\Data\SqlClient\RateLimiter\BlockingPeriodRateLimiter.cs</Link>
+    </Compile>
+    <Compile Include="$(CommonSourceRoot)Microsoft\Data\SqlClient\RateLimiter\ConcurrencyRateLimiter.cs">
+      <Link>Microsoft\Data\SqlClient\RateLimiter\ConcurrencyRateLimiter.cs</Link>
+    </Compile>
+    <Compile Include="$(CommonSourceRoot)Microsoft\Data\SqlClient\RateLimiter\PassthroughRateLimiter.cs">
+      <Link>Microsoft\Data\SqlClient\RateLimiter\PassthroughRateLimiter.cs</Link>
+    </Compile>
+    <Compile Include="$(CommonSourceRoot)Microsoft\Data\SqlClient\RateLimiter\RateLimiterBase.cs">
+      <Link>Microsoft\Data\SqlClient\RateLimiter\RateLimiterBase.cs</Link>
+    </Compile>
     <Compile Include="$(CommonSourceRoot)Microsoft\Data\ProviderBase\DbMetaDataFactory.cs">
       <Link>Microsoft\Data\ProviderBase\DbMetaDataFactory.cs</Link>
     </Compile>

src/Microsoft.Data.SqlClient/netcore/src/Microsoft/Data/SqlClient/SqlConnection.cs

@@ -1439,8 +1439,8 @@ public override void Open()
             Open(SqlConnectionOverrides.None);
         }
 
-        private bool TryOpenWithRetry(TaskCompletionSource<DbConnectionInternal> retry, SqlConnectionOverrides overrides)
-            => RetryLogicProvider.Execute(this, () => TryOpen(retry, overrides));
+        private bool TryOpenWithRetry(TaskCompletionSource<DbConnectionInternal> taskCompletionSource, SqlConnectionOverrides overrides)
+            => RetryLogicProvider.Execute(this, () => TryOpen(taskCompletionSource, overrides));
 
         /// <include file='../../../../../../../doc/snippets/Microsoft.Data.SqlClient/SqlConnection.xml' path='docs/members[@name="SqlConnection"]/OpenWithOverrides/*' />
         public void Open(SqlConnectionOverrides overrides)
@@ -1949,7 +1949,7 @@ private void PrepareStatisticsForNewConnection()
             }
         }
 
-        private bool TryOpen(TaskCompletionSource<DbConnectionInternal> retry, SqlConnectionOverrides overrides = SqlConnectionOverrides.None)
+        private bool TryOpen(TaskCompletionSource<DbConnectionInternal> taskCompletionSource, SqlConnectionOverrides overrides = SqlConnectionOverrides.None)
         {
             SqlConnectionString connectionOptions = (SqlConnectionString)ConnectionOptions;
 
@@ -2011,14 +2011,14 @@ private bool TryOpen(TaskCompletionSource<DbConnectionInternal> retry, SqlConnec
 
             if (ForceNewConnection)
             {
-                if (!InnerConnection.TryReplaceConnection(this, ConnectionFactory, retry, UserConnectionOptions))
+                if (!InnerConnection.TryReplaceConnection(this, ConnectionFactory, taskCompletionSource, UserConnectionOptions))
                 {
                     return false;
                 }
             }
             else
             {
-                if (!InnerConnection.TryOpenConnection(this, ConnectionFactory, retry, UserConnectionOptions))
+                if (!InnerConnection.TryOpenConnection(this, ConnectionFactory, taskCompletionSource, UserConnectionOptions))
                 {
                     return false;
                 }

src/Microsoft.Data.SqlClient/netcore/src/Microsoft/Data/SqlClient/SqlConnectionFactory.cs

@@ -86,7 +86,7 @@ override protected DbConnectionInternal CreateConnection(DbConnectionOptions opt
                 redirectedUserInstance = true;
                 string instanceName;
 
-                if (pool == null || (pool != null && pool.Count <= 0))
+                if (pool == null || (pool != null && ((SqlConnectionPoolProviderInfo)pool.ProviderInfo).InstanceName == null))
                 { // Non-pooled or pooled and no connections in the pool.
                     SqlInternalConnectionTds sseConnection = null;
                     try

src/Microsoft.Data.SqlClient/netfx/src/Microsoft.Data.SqlClient.csproj

@@ -279,6 +279,9 @@
     <Compile Include="$(CommonSourceRoot)Microsoft\Data\ProviderBase\DbConnectionInternal.cs">
       <Link>Microsoft\Data\ProviderBase\DbConnectionInternal.cs</Link>
     </Compile>
+    <Compile Include="$(CommonSourceRoot)Microsoft\Data\SqlClient\ConnectionPool\ChannelDbConnectionPool.cs">
+      <Link>Microsoft\Data\SqlClient\ConnectionPool\ChannelDbConnectionPool.cs</Link>
+    </Compile>
     <Compile Include="$(CommonSourceRoot)Microsoft\Data\SqlClient\ConnectionPool\DbConnectionPool.cs">
       <Link>Microsoft\Data\SqlClient\ConnectionPool\DbConnectionPool.cs</Link>
     </Compile>
@@ -318,12 +321,30 @@
     <Compile Include="$(CommonSourceRoot)Microsoft\Data\SqlClient\ConnectionPool\SqlConnectionPoolGroupProviderInfo.cs">
       <Link>Microsoft\Data\SqlClient\ConnectionPool\SqlConnectionPoolGroupProviderInfo.cs</Link>
     </Compile>
+    <Compile Include="$(CommonSourceRoot)Microsoft\Data\SqlClient\ConnectionPool\TransactedConnectionPool.cs">
+      <Link>Microsoft\Data\SqlClient\ConnectionPool\TransactedConnectionPool.cs</Link>
+    </Compile>
     <Compile Include="$(CommonSourceRoot)Microsoft\Data\SqlClient\ConnectionPool\SqlConnectionPoolKey.cs">
       <Link>Microsoft\Data\SqlClient\SqlConnectionPoolKey.cs</Link>
     </Compile>
     <Compile Include="$(CommonSourceRoot)Microsoft\Data\SqlClient\ConnectionPool\SqlConnectionPoolProviderInfo.cs">
       <Link>Microsoft\Data\SqlClient\ConnectionPool\SqlConnectionPoolProviderInfo.cs</Link>
     </Compile>
+    <Compile Include="$(CommonSourceRoot)Microsoft\Data\SqlClient\RateLimiter\AsyncFlagFunc.cs">
+      <Link>Microsoft\Data\SqlClient\RateLimiter\AsyncFlagFunc.cs</Link>
+    </Compile>
+    <Compile Include="$(CommonSourceRoot)Microsoft\Data\SqlClient\RateLimiter\BlockingPeriodRateLimiter.cs">
+      <Link>Microsoft\Data\SqlClient\RateLimiter\BlockingPeriodRateLimiter.cs</Link>
+    </Compile>
+    <Compile Include="$(CommonSourceRoot)Microsoft\Data\SqlClient\RateLimiter\ConcurrencyRateLimiter.cs">
+      <Link>Microsoft\Data\SqlClient\RateLimiter\ConcurrencyRateLimiter.cs</Link>
+    </Compile>
+    <Compile Include="$(CommonSourceRoot)Microsoft\Data\SqlClient\RateLimiter\PassthroughRateLimiter.cs">
+      <Link>Microsoft\Data\SqlClient\RateLimiter\PassthroughRateLimiter.cs</Link>
+    </Compile>
+    <Compile Include="$(CommonSourceRoot)Microsoft\Data\SqlClient\RateLimiter\RateLimiterBase.cs">
+      <Link>Microsoft\Data\SqlClient\RateLimiter\RateLimiterBase.cs</Link>
+    </Compile>
     <Compile Include="$(CommonSourceRoot)Microsoft\Data\SqlClient\Diagnostics\SqlClientMetrics.cs">
       <Link>Microsoft\Data\SqlClient\Diagnostics\SqlClientMetrics.cs</Link>
     </Compile>
@@ -984,6 +1005,7 @@
     <PackageReference Include="System.Buffers" Version="$(SystemBuffersVersion)" />
     <PackageReference Include="System.Security.Cryptography.Pkcs" Version="$(SystemSecurityCryptographyPkcsVersion)" />
     <PackageReference Include="System.Text.Json" Version="$(SystemTextJsonVersion)" />
+    <PackageReference Include="System.Threading.Channels" Version="$(SystemThreadingChannelsVersion)" />
   </ItemGroup>
   <Import Project="$(CommonSourceRoot)tools\targets\GenerateResourceStringsSource.targets" />
   <Import Project="$(NetFxSource)tools\targets\GenerateThisAssemblyCs.targets" />

src/Microsoft.Data.SqlClient/netfx/src/Microsoft/Data/SqlClient/SqlConnectionFactory.cs

@@ -93,7 +93,7 @@ override protected DbConnectionInternal CreateConnection(DbConnectionOptions opt
                     redirectedUserInstance = true;
                     string instanceName;
 
-                    if (pool == null || (pool != null && pool.Count <= 0))
+                    if (pool == null || (pool != null && ((SqlConnectionPoolProviderInfo)pool.ProviderInfo).InstanceName == null))
                     { // Non-pooled or pooled and no connections in the pool.
 
                         SqlInternalConnectionTds sseConnection = null;

src/Microsoft.Data.SqlClient/src/Microsoft.Data.SqlClient.csproj

@@ -6,7 +6,7 @@
 	</PropertyGroup>
 
   <ItemGroup>
-    <Reference Include="System.Configuration" Condition="'$(TargetFramework)' == 'net462'"/>
+    <Reference Include="System.Configuration" Condition="'$(TargetFramework)' == 'net462'" />
 	  <Reference Include="System.Transactions" Condition="'$(TargetFramework)' == 'net462'" />
 	</ItemGroup>
 	<ItemGroup>

src/Microsoft.Data.SqlClient/src/Microsoft/Data/ProviderBase/DbConnectionInternal.cs

@@ -95,6 +95,7 @@ internal DbConnectionInternal(ConnectionState state, bool hidePassword, bool all
         }
 
         #region Properties
+        internal DateTime OpenTimestamp => _createTime;
 
         internal bool AllowSetConnectionString { get; }
 
@@ -464,8 +465,8 @@ internal virtual void CloseConnection(DbConnection owningObject, DbConnectionFac
                         // into the pool.
                         if (connectionPool is not null)
                         {
-                            // PutObject calls Deactivate for us...
-                            connectionPool.PutObject(this, owningObject);
+                            // ReturnInternalConnection calls Deactivate for us...
+                            connectionPool.ReturnInternalConnection(this, owningObject);
 
                             // NOTE: Before we leave the PutObject call, another thread may have
                             // already popped the connection from the pool, so don't expect to be
@@ -814,7 +815,7 @@ internal void SetInStasis()
         internal virtual bool TryOpenConnection(
             DbConnection outerConnection,
             DbConnectionFactory connectionFactory,
-            TaskCompletionSource<DbConnectionInternal> retry,
+            TaskCompletionSource<DbConnectionInternal> taskCompletionSource,
             DbConnectionOptions userOptions)
         {
             throw ADP.ConnectionAlreadyOpen(State);
@@ -823,7 +824,7 @@ internal virtual bool TryOpenConnection(
         internal virtual bool TryReplaceConnection(
             DbConnection outerConnection,
             DbConnectionFactory connectionFactory,
-            TaskCompletionSource<DbConnectionInternal> retry,
+            TaskCompletionSource<DbConnectionInternal> taskCompletionSource,
             DbConnectionOptions userOptions)
         {
             throw ADP.MethodNotImplemented();