Issues with Session Handling in Selenium Grid Tests and Configuration

[tathagatk22]

Problem Statement:

I am encountering an issue while running Selenium Grid tests using Chrome in a Kubernetes setup. The test suite includes a long-running test (Thread.Sleep(600000) for 10 minutes). Below is the relevant configuration and code, followed by the error encountered.

Helm values.yaml (Selenium Grid Configuration):

edgeNode:
  enabled: false
firefoxNode:
  enabled: false
chromeNode:
  terminationGracePeriodSeconds: 600
  enabled: true
  replicas: 1
  resources:
    limits:
      cpu: "1"
      memory: "1Gi"
    requests:
      cpu: "0.5"
      memory: "500Mi"
global:
  seleniumGrid:
    defaultNodeStartupProbe: httpGet
ingress:
  enabled: false
ingress-nginx:
  enabled: false
tls:
  enabled: false
  ingress:
    enabled: false
autoscaling:
  enableWithExistingKEDA: true
  enabled: false
  scaledJobOptions:
    failedJobsHistoryLimit: 10
    rolloutStrategy: gradual
    pollingInterval: 10
    scalingStrategy:
      strategy: default
    successfulJobsHistoryLimit: 3
    failedJobsHistoryLimit: 10
  scaledOptions:
    maxReplicaCount: 40
    minReplicaCount: 0
  scalingType: job
  terminationGracePeriodSeconds: 600

C# Test Code (for Selenium Grid):

using Microsoft.VisualStudio.TestTools.UnitTesting;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Remote;

[assembly: Parallelize(Workers = 30, Scope = ExecutionScope.ClassLevel)]
namespace UITests
{
    [TestClass]
    public class SeleniumGridTests
    {
        private IWebDriver driver;
        private string seleniumGridUrl = "https://user:[email protected]/test3/wd/hub";

        [TestInitialize]
        public void Setup()
        {
            var options = new ChromeOptions();
            options.AddArgument("--start-maximized");

            driver = new RemoteWebDriver(new Uri(seleniumGridUrl), options);
            driver.Manage().Timeouts().ImplicitWait = TimeSpan.FromMinutes(10);
        }

        [TestMethod]
        public void TestGoogleTitle8()
        {
            driver.Navigate().GoToUrl("https://www.google.com");
            Assert.IsTrue(driver.Title.Contains("Google"), "Title does not contain 'Google'");
            Thread.Sleep(600000);  // 10 minutes
        }

        [TestMethod]
        public void TestExamplePageHeading9()
        {
            driver.Navigate().GoToUrl("https://www.example.com");
            IWebElement heading = driver.FindElement(By.TagName("h1"));
            Assert.AreEqual("Example Domain", heading.Text, "Heading text does not match.");
        }

        [TestCleanup]
        public void Teardown()
        {
            driver.Quit();
        }
    }
}

.runsettings

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
  <MSTest>
    <Parallelize>
      <Workers>30</Workers>
      <Scope>MethodLevel</Scope>
    </Parallelize>
  </MSTest>
</RunSettings>

Command used to execute:

dotnet test -s .\SeleniumGridTests\.runsettings

Error Output:

Failed TestGoogleTitle8 [10 m 8 s]
Error Message:
Test method UITests.SeleniumGridTests.TestGoogleTitle8 threw exception:
OpenQA.Selenium.WebDriverException: Unable to find session with ID: dea75b34d16d2413423d3af005b5b025
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1075-azure', java.version: '17.0.11'
Driver info: driver.version: unknown
Stack Trace:
      at OpenQA.Selenium.WebDriver.UnpackAndThrowOnError(Response errorResponse, String commandToExecute)
   at OpenQA.Selenium.WebDriver.ExecuteAsync(String driverCommandToExecute, Dictionary`2 parameters)
   at OpenQA.Selenium.Navigator.GoToUrlAsync(String url)
   at OpenQA.Selenium.Navigator.<>c__DisplayClass6_0.<<GoToUrl>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at OpenQA.Selenium.Navigator.GoToUrl(String url)
   at UITests.SeleniumGridTests.TestGoogleTitle8() in C:\Users\Tathagat.Khanorkar\Downloads\SeleniumGridTests\SeleniumGridTests\UnitTest1.cs:line 35idTests\UnitTest1.cs:line 35
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
   
Failed!  - Failed:     1, Passed:     1, Skipped:     0, Total:     2, Duration: 10 m 8 s - SeleniumGridTests.dll (net8.0)   

Pod Logs:

Questions/Issues:

• Session Loss: The long Thread.Sleep(600000) causes a loss of session after 10 minutes.

Request for Assistance:

• Session Handling: How can I handle long-running tests (up to 10 minutes) without losing the WebDriver session?
• Kubernetes Configuration: Are there any specific recommendations for resource limits or timeouts (such as terminationGracePeriodSeconds) to ensure stability for long-duration tests?
• Best Practices: Any suggestions for improving test reliability in this Selenium Grid + Kubernetes setup?

[VietND96]

Hi, as your description, within step Thread.Sleep(600,000) for 10 minutes will cause the session timeout (since it will reach idle time without action to the session). By default, the session timeout is 300 seconds (5 mins). You can proactive to adjust this via Node environment variables.
In Helm config, it is chromeNode.extraEnvironmentVariables. And the env var name is SE_NODE_SESSION_TIMEOUT, the input in unit seconds.

[tathagatk22]

Dotnet Test Code

        public void Setup()
        {
            // Define desired capabilities for the browser
            var options = new ChromeOptions();
            // Example: Set additional options if needed (headless mode, window size, etc.)
            options.AddArgument("--start-maximized");


            // Initialize RemoteWebDriver with the Selenium Grid URL and browser options
            driver = new RemoteWebDriver(new Uri(seleniumGridUrl), options);
            driver.Manage().Timeouts().ImplicitWait = TimeSpan.FromMinutes(10);
        }

        [TestMethod]
        public void TestGoogleTitle()
        {
            driver.Navigate().GoToUrl("https://www.google.com");
            Assert.IsTrue(driver.Title.Contains("Google"), "Title does not contain 'Google'");
        }

        [TestMethod]
        public void TestExamplePageHeading()
        {
            driver.Navigate().GoToUrl("https://www.example.com");
            Thread.Sleep(600000);
            IWebElement heading = driver.FindElement(By.TagName("h1"));
            Assert.AreEqual("Example Domain", heading.Text, "Heading text does not match.");
        }
        
        [TestCleanup]
        public void Teardown()
        {
            driver.Quit();
        }

Values.yaml

edgeNode:
  enabled: false
firefoxNode:
  enabled: false
chromeNode:
  extraEnvironmentVariables:
  - name: SE_NODE_SESSION_TIMEOUT
    value: "600"
  enabled: true
  replicas: 1
  resources:
    limits:
      cpu: "1"
      memory: "1Gi"
    requests:
      cpu: "0.5"
      memory: "500Mi"
global:
  seleniumGrid:
    defaultNodeStartupProbe: httpGet
ingress:
  enabled: false
ingress-nginx:
  enabled: false
tls:
  enabled: false
  ingress:
    enabled: false
autoscaling:
  annotations:
    helm.sh/hook: post-install,post-upgrade,post-rollback
    helm.sh/hook-weight: "1"
  deregisterLifecycle:
    preStop:
      exec:
        command:
        - bash
        - -c
        - |
          curl -X POST 127.0.0.1:5555/se/grid/node/drain --header 'X-REGISTRATION-SECRET;' && \
          while curl 127.0.0.1:5555/status; do sleep 1; done;
  enableWithExistingKEDA: true
  enabled: false
  scaledJobOptions:
    failedJobsHistoryLimit: 10
    rolloutStrategy: gradual
    pollingInterval: 10
    scalingStrategy:
      strategy: default
    successfulJobsHistoryLimit: 3
    failedJobsHistoryLimit: 10
  scaledOptions:
    maxReplicaCount: 40
    minReplicaCount: 3
  scalingType: job
  terminationGracePeriodSeconds: 60

Error Msg

PS C:\Users\Tathagat.Khanorkar\Downloads\SeleniumGridTests (1)\SeleniumGridTests> dotnet test -s .\SeleniumGridTests\.runsettings
  Determining projects to restore...
  All projects are up-to-date for restore.
  SeleniumGridTests -> C:\Users\Tathagat.Khanorkar\Downloads\SeleniumGridTests (1)\SeleniumGridTests\SeleniumGridTests\bin\Debug\net8.0\SeleniumGridTests.dll
Test run for C:\Users\Tathagat.Khanorkar\Downloads\SeleniumGridTests %281%29\SeleniumGridTests\SeleniumGridTests\bin\Debug\net8.0\SeleniumGridTests.dll (.NETCoreApp,Version=v8.0)
VSTest version 17.11.1 (x64)

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
  Failed TestExamplePageHeading [10 m 1 s]
  Error Message:
   Test method UITests.SeleniumGridTests.TestExamplePageHeading threw exception:
OpenQA.Selenium.WebDriverException: Cannot find session with id: 13893d2a6e4d8e2b9d47cd4f1949aac4
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1075-azure', java.version: '17.0.11'
Driver info: driver.version: unknown
  Stack Trace:
      at OpenQA.Selenium.WebDriver.UnpackAndThrowOnError(Response errorResponse, String commandToExecute)
   at OpenQA.Selenium.WebDriver.ExecuteAsync(String driverCommandToExecute, Dictionary`2 parameters)
   at OpenQA.Selenium.WebDriver.Execute(String driverCommandToExecute, Dictionary`2 parameters)
   at OpenQA.Selenium.WebDriver.FindElement(String mechanism, String value)
   at OpenQA.Selenium.By.<.ctor>b__11_0(ISearchContext context)
   at OpenQA.Selenium.By.FindElement(ISearchContext context)
   at OpenQA.Selenium.WebDriver.FindElement(By by)
   at UITests.SeleniumGridTests.TestExamplePageHeading() in C:\Users\Tathagat.Khanorkar\Downloads\SeleniumGridTests (1)\SeleniumGridTests\SeleniumGridTests\UnitTest1.cs:line 43
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)


Failed!  - Failed:     1, Passed:     1, Skipped:     0, Total:     2, Duration: 10 m 1 s - SeleniumGridTests.dll (net8.0)

Pod Logs

2024-12-17 16:44:33,737 INFO Included extra file "/etc/supervisor/conf.d/chrome-cleanup.conf" during parsing
2024-12-17 16:44:33,737 INFO Included extra file "/etc/supervisor/conf.d/selenium.conf" during parsing
2024-12-17 16:44:33,739 INFO RPC interface 'supervisor' initialized
2024-12-17 16:44:33,739 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2024-12-17 16:44:33,739 INFO supervisord started with pid 8
2024-12-17 16:44:34,741 INFO spawned: 'xvfb' with pid 9
2024-12-17 16:44:34,743 INFO spawned: 'vnc' with pid 10
2024-12-17 16:44:34,744 INFO spawned: 'novnc' with pid 11
2024-12-17 16:44:34,745 INFO spawned: 'selenium-node' with pid 12
2024-12-17 16:44:34,749 INFO success: selenium-node entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
Appending Selenium options: --session-timeout 600
Appending Selenium options: --register-period 60
Appending Selenium options: --register-cycle 5
Appending Selenium options: --heartbeat-period 30
Appending Selenium options: --log-level INFO
Generating Selenium Config
Setting up SE_NODE_HOST...
Tracing is disabled
Selenium Grid Node configuration: 
[events]
publish = "tcp://selenium-job-selenium-hub.selenium-job:4442"
subscribe = "tcp://selenium-job-selenium-hub.selenium-job:4443"

[server]
port = "5555"
[node]
grid-url = "http://seleniumuser:[email protected]:4444"
session-timeout = "600"
override-max-sessions = false
detect-drivers = false
drain-after-session-count = 1
max-sessions = 1

[[node.driver-configuration]]
display-name = "chrome"
stereotype = '{"browserName": "chrome", "browserVersion": "126.0", "platformName": "Linux", "goog:chromeOptions": {"binary": "/usr/bin/google-chrome"}}'
max-sessions = 1
    
Starting Selenium Grid Node...
16:44:35.566 INFO [LoggingOptions.configureLogEncoding] - Using the system default encoding
16:44:35.570 INFO [OpenTelemetryTracer.createTracer] - Using OpenTelemetry for tracing
2024-12-17 16:44:35,763 INFO success: xvfb entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-12-17 16:44:35,763 INFO success: vnc entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-12-17 16:44:35,763 INFO success: novnc entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
16:44:35.763 INFO [UnboundZmqEventBus.<init>] - Connecting to tcp://selenium-job-selenium-hub.selenium-job:4442 and tcp://selenium-job-selenium-hub.selenium-job:4443
16:44:35.853 INFO [UnboundZmqEventBus.<init>] - Sockets created
16:44:36.854 INFO [UnboundZmqEventBus.<init>] - Event bus ready
16:44:36.957 INFO [NodeServer.createHandlers] - Reporting self as: http://10.84.0.211:5555
16:44:36.968 INFO [NodeOptions.getSessionFactories] - Detected 1 available processors
16:44:36.997 INFO [NodeOptions.report] - Adding chrome for {"browserName": "chrome","browserVersion": "126.0","goog:chromeOptions": {"binary": "\u002fusr\u002fbin\u002fgoogle-chrome"},"platformName": "linux","se:noVncPort": 7900,"se:vncEnabled": true} 1 times
16:44:37.059 INFO [Node.<init>] - Binding additional locator mechanisms: relative
16:44:37.263 INFO [NodeServer$1.start] - Starting registration process for Node http://10.84.0.211:5555
16:44:37.264 INFO [NodeServer.execute] - Started Selenium node 4.22.0 (revision c5f3146703): http://10.84.0.211:5555
16:44:37.271 INFO [NodeServer$1.lambda$start$1] - Sending registration event...
16:44:37.400 INFO [NodeServer.lambda$createHandlers$2] - Node has been added
17:27:37.266 INFO [LocalNode.checkSessionCount] - Draining Node, configured sessions value (1) has been reached.
17:27:37.269 INFO [LocalNode.newSession] - Session created by the Node. Id: 13893d2a6e4d8e2b9d47cd4f1949aac4, Caps: Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 126.0.6478.114, chrome: {chromedriverVersion: 126.0.6478.63 (df799988fdc9..., userDataDir: /tmp/.org.chromium.Chromium...}, fedcm:accounts: true, goog:chromeOptions: {debuggerAddress: localhost:34245}, networkConnectionEnabled: false, pageLoadStrategy: normal, platformName: linux, proxy: Proxy(), se:bidiEnabled: false, se:cdp: ws://seleniumuser:admin@sel..., se:cdpVersion: 126.0.6478.114, se:vnc: ws://seleniumuser:admin@sel..., se:vncEnabled: true, se:vncLocalAddress: ws://10.84.0.211:7900, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:extension:minPinLength: true, webauthn:extension:prf: true, webauthn:virtualAuthenticators: true}
17:38:07.078 INFO [LocalNode.stopTimedOutSession] - Session id 13893d2a6e4d8e2b9d47cd4f1949aac4 timed out, stopping...
17:38:07.167 INFO [SessionSlot.stop] - Stopping session 13893d2a6e4d8e2b9d47cd4f1949aac4
17:38:07.167 INFO [LocalNode.stopTimedOutSession] - Node draining complete!
17:38:08.169 INFO [NodeServer.lambda$createHandlers$3] - Shutting down
2024-12-17 17:38:08,506 INFO exited: selenium-node (exit status 0; expected)
2024-12-17 17:38:08,506 WARN received SIGINT indicating exit request
2024-12-17 17:38:08,506 INFO waiting for xvfb, vnc, novnc to die
2024-12-17 17:38:11,510 INFO stopped: novnc (terminated by SIGTERM)
2024-12-17 17:38:11,511 INFO waiting for xvfb, vnc to die
2024-12-17 17:38:12,512 INFO stopped: vnc (terminated by SIGTERM)
2024-12-17 17:38:13,514 INFO stopped: xvfb (terminated by SIGTERM)

[tathagatk22]

@VietND96 I tried the configuration you asked, It failed

[tathagatk22]

Sometimes this error comes

Error Message:

   Initialization method UITests.SeleniumGridTests.Setup threw exception. OpenQA.Selenium.WebDriverException: The HTTP request to the remote WebDriver server for URL https://seleniumuser:[email protected]/test3/wd/hub/session timed out after 60 seconds. ---> System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 60 seconds elapsing. ---> System.TimeoutException: The operation was canceled. ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled. ---> System.IO.IOException: Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request.. ---> System.Net.Sockets.SocketException: The I/O operation has been aborted because of either a thread exit or an application request..

TestCleanup method UITests.SeleniumGridTests.Teardown threw exception. System.NullReferenceException: Object reference not set to an instance of an object..

  Stack Trace:

      at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)

   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)

   at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](CancellationToken cancellationToken, Int32 estimatedSize)

   at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)

   at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](Memory`1 buffer, CancellationToken cancellationToken)

   at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)

   at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)