Skip to content

update tests README #7951

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 25, 2025
Merged

update tests README #7951

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
9c5c6ce
update tests readMe
vepatel Jun 24, 2025
cdd6a4c
use make target for venv
vepatel Jun 24, 2025
82744a2
Merge branch 'main' into docs/update-tests-readme
vepatel Jun 25, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
127 changes: 75 additions & 52 deletions tests/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,18 +1,15 @@
# Tests

The project includes automated tests for testing the Ingress Controller in a Kubernetes cluster. The tests are written
in Python3 and use the pytest framework.
The project includes automated tests for testing the NGINX Ingress Controller in a Kubernetes cluster. The tests are written in Python 3 and use the pytest framework with additional tools like Playwright for browser automation.

Below you will find the instructions on how to run the tests against a Minikube and kind clusters. However, you are not
limited to those options and can use other types of Kubernetes clusters. See the [Configuring the
Tests](#configuring-the-tests) section to find out about various configuration options.
This documentation covers how to run tests against Minikube and Kind clusters, though the tests can be run against any Kubernetes cluster. See the [Configuring the Tests](#configuring-the-tests) section for various configuration options.

## Running Tests in Minikube

### Prerequisites

- Minikube.
- Python3 or Docker.
- [Minikube](https://minikube.sigs.k8s.io/docs/)
- Python 3.10+ or Docker

#### Step 1 - Create a Minikube Cluster

Expand All @@ -22,39 +19,29 @@ minikube start

#### Step 2 - Run the Tests

**Note**: if you have the Ingress Controller deployed in the cluster, please uninstall it first, making sure to remove
its namespace and RBAC resources.
**Note**: If you have the Ingress Controller already deployed in the cluster, please uninstall it first, ensuring you remove its namespace and RBAC resources.

Run the tests:
Run the tests using one of the following methods:

- Use local Python3 installation:
- **Use Python3 virtual environment (recommended):**

```bash
cd tests
pip3 install -r requirements.txt
python3 -m pytest --node-ip=$(minikube ip)
```

- Use Python3 virtual environment:

Create and activate ```virtualenv```
Create and activate a virtual environment:

```bash
$ cd tests
$ python3 -m venv ~/venv
$ source ~/venv/bin/activate
(venv) $
cd tests
make setup-venv
pytest --node-ip=$(minikube ip)
```

Install dependencies and run tests
- **Use local Python3 installation:**

```bash
(venv) $ cd tests
(venv) $ pip3 install -r requirements.txt
(venv) $ python3 -m pytest --node-ip=$(minikube ip)
cd tests
pip install -r requirements.txt
pytest --node-ip=$(minikube ip)
```

- Use Docker:
- **Use Docker:**

```bash
cd tests
Expand All @@ -69,24 +56,25 @@ below to learn how to configure the tests including the image and the type of NG

### Prerequisites

- [Kind](https://kind.sigs.k8s.io/).
- Docker.
- [Kind](https://kind.sigs.k8s.io/)
- Docker

**Note**: all commands in steps below are executed from the ```tests``` directory
**Note**: All commands in the steps below are executed from the `tests` directory.

List available make commands
List available make commands:

```bash
$ make

help Show available make targets
build Run build
run-tests Run tests
run-tests-in-kind Run tests in Kind
create-kind-cluster Create Kind cluster
delete-kind-cluster Delete Kind cluster
make help
```

This will show you all available targets including:

- `build` - Build the test container image
- `run-tests` - Run tests in Docker
- `run-tests-in-kind` - Run tests in Kind cluster
- `create-kind-cluster` - Create a Kind cluster
- `delete-kind-cluster` - Delete a Kind cluster

#### Step 1 - Create a Kind Cluster

```bash
Expand All @@ -95,8 +83,7 @@ make create-kind-cluster

#### Step 2 - Run the Tests

**Note**: if you have the Ingress Controller deployed in the cluster, please uninstall it first, making sure to remove
its namespace and RBAC resources.
**Note**: If you have the Ingress Controller already deployed in the cluster, please uninstall it first, ensuring you remove its namespace and RBAC resources.

Run the tests in Docker:

Expand All @@ -105,13 +92,27 @@ make build
make run-tests-in-kind
```

The tests will use the Ingress Controller for NGINX with the default *nginx/nginx-ingress:edge* image. See the section
below to learn how to configure the tests including the image and the type of NGINX -- NGINX or NGINX Plus.
The tests will use the NGINX Ingress Controller with the default `nginx/nginx-ingress:edge` image. See the [Configuring the Tests](#configuring-the-tests) section to learn how to configure different images and NGINX types (OSS vs Plus).

## Additional Make Targets

The test suite includes several additional make targets for cluster management and cleanup:


- `make create-mini-cluster` - Create a Minikube K8S cluster
- `make delete-mini-cluster` - Delete a Minikube K8S cluster
- `make run-tests-in-minikube` - Run tests in Minikube
- `make mini-image-load` - Load the image into the Minikube K8S cluster
- `make image-load` - Load the image into the Kind K8S cluster
- `make mini-image-load` - Load the image into the Minikube K8S cluster
- `make setup-venv` - Create Python virtual environment with all dependencies
- `make clean-venv` - Remove Python virtual environment
- `make run-local-tests` - Run tests using local Python environment
- `make test-lint` - Run Python linting tools (isort, black)

## Configuring the Tests

The table below shows various configuration options for the tests. If you use Python3 to run the tests, use the
command-line arguments. If you use Docker, use the [Makefile](Makefile) variables.
The table below shows various configuration options for the tests. If you use Python 3 to run the tests locally, use the command-line arguments. If you use Docker, use the [Makefile](Makefile) variables.

| Command-line Argument | Makefile Variable | Description | Default |
| :----------------------- | :------------ | :------------ | :----------------------- |
Expand All @@ -126,20 +127,42 @@ command-line arguments. If you use Docker, use the [Makefile](Makefile) variable
| `N/A` | `KUBE_CONFIG_FOLDER`, not supported by `run-tests-in-kind` target. | A path to a folder with a kubeconfig file. | `~/.kube/` |
| `--show-ic-logs` | `SHOW_IC_LOGS` | A flag to control accumulating IC logs in stdout. | `no` |
| `--skip-fixture-teardown` | `N/A` | A flag to skip test fixture teardown for debugging. | `no` |
| `--plus-jwt` | `PLUS_JWT` | JWT token for NGINX Plus image authentication. | `""` |
| `N/A` | `PYTEST_ARGS` | Any additional pytest command-line arguments (i.e `-m "smoke"`) | `""` |

If you would like to use an IDE (such as PyCharm) to run the tests, use the [pytest.ini](pytest.ini) file to set the
command-line arguments.
If you would like to use an IDE (such as PyCharm) to run the tests, use the [pyproject.toml](../pyproject.toml) file to view pytest configuration and markers.

Tests are marked with custom markers. The markers allow to logically split all the tests into smaller groups. The full
list can be found in the [pytest.ini](pytest.ini) file or via command line:
Tests are marked with custom markers that allow you to logically split all tests into smaller groups. The full list can be found in the [pyproject.toml](../pyproject.toml) file or via command line:

```bash
python3 -m pytest --markers
pytest --markers
```

## Test Containers

The source code for the tests containers used in some tests, for example the
[transport-server-tcp-load-balance](./data/transport-server-tcp-load-balance/standard/service_deployment.yaml) is
located at [kic-test-containers](https://github.com/nginx/kic-test-containers).

## Test Structure

The test suite is organized as follows:

- `suite/` - Main test files organized by functionality
- `data/` - Test data and configuration files
- `ci-files/` - Continuous integration configuration
- `conftest.py` - pytest configuration and fixtures
- `requirements.txt` - Python dependencies with hash verification
- `Makefile` - Build and test automation

### Test Dependencies

The tests require several Python packages including:

- `pytest` - Testing framework
- `kubernetes` - Kubernetes Python client
- `requests` - HTTP library
- `playwright` - Browser automation for UI tests etc.

You can find rest of the dependencies in `requirements.txt`
All dependencies are pinned with hashes.
Loading