Docs: Update README.md for validation #1258
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
KarolinaPomian
requested review from
Sakoram,
DawidWesierski4 and
Falron98
as code owners
Sakoram
reviewed
Sep 5, 2025
tests/validation/README.md
Outdated
Comment on lines
12
to
44
| tests/validation/ | ||
| ├── common/ # Shared utilities for tests | ||
| │ ├── ffmpeg_handler/ # FFmpeg integration utilities | ||
| │ ├── integrity/ # Data integrity verification tools | ||
| │ └── nicctl.py # Network interface control | ||
| ├── configs/ # Test configuration files | ||
| │ ├── test_config.yaml # Test environment settings | ||
| │ └── topology_config.yaml # Network topology configuration | ||
| ├── create_pcap_file/ # Tools for packet capture file creation | ||
| ├── mtl_engine/ # Core test framework components | ||
| │ ├── execute.py # Test execution management | ||
| │ ├── RxTxApp.py # RX/TX application interface | ||
| │ ├── GstreamerApp.py # GStreamer integration | ||
| │ ├── ffmpeg_app.py # FFmpeg integration | ||
| │ ├── csv_report.py # Test result reporting | ||
| │ └── ramdisk.py # RAM disk management | ||
| ├── tests/ # Test modules | ||
| │ ├── single/ # Single-flow test scenarios | ||
| │ │ ├── dma/ # DMA tests | ||
| │ │ ├── ffmpeg/ # FFmpeg integration tests | ||
| │ │ ├── gstreamer/ # GStreamer integration tests | ||
| │ │ ├── kernel_socket/ # Kernel socket tests | ||
| │ │ ├── performance/ # Performance benchmarking | ||
| │ │ ├── ptp/ # Precision Time Protocol tests | ||
| │ │ ├── st20p/ # ST2110-20 video tests | ||
| │ │ ├── st22p/ # ST2110-22 compressed video tests | ||
| │ │ ├── st30p/ # ST2110-30 audio tests | ||
| │ │ └── st41/ # ST2110-40 ancillary data tests | ||
| │ ├── dual/ # Dual-flow test scenarios | ||
| │ └── invalid/ # Error handling and negative test cases | ||
| ├── conftest.py # pytest configuration and fixtures | ||
| ├── pytest.ini # pytest settings | ||
| └── requirements.txt # Python dependencies |
There was a problem hiding this comment.
This will be difficult to maintain and does not provide any new information beyond what is already in the file name. I would remove it or leave description just to common files with extended description.
tests/validation/README.md
Outdated
Comment on lines
49
to
54
| ### Prerequisites | ||
|
|
||
| - Python 3.9 or higher | ||
| - Media Transport Library built and installed | ||
| - Network interfaces configured for testing | ||
| - Sufficient permissions for network management |
There was a problem hiding this comment.
- It is also necessary to have the files used as input data. We currently have them on NFS.
- What does it mean to have "Network interfaces configured for testing"? I guess you need to have to have everything done form MTL's run.md, but VF are created automatically.
- by "Sufficient permissions for network management" you mean you have to be root user? Is there any other option?
There was a problem hiding this comment.
- you need to install ffmpeg and gstreamer plugins to run some of the tests.
|
|
||
| ## Running Tests | ||
|
|
||
| ### Basic Test Execution |
There was a problem hiding this comment.
please add info that you also can run given testcase with specific parameters like : pytest --topology_config=configs/topology_config.yaml --test_config=configs/test_config.yaml "tests/single/st20p/fps/test_fps.py::test_fps[|fps = p60|-ParkJoy_1080p]"
Sakoram
reviewed
Sep 8, 2025
| - **name**: Host identifier | ||
| - **instantiate**: Whether to instantiate the host | ||
| - **role**: Host role (e.g., sut for System Under Test) | ||
| - **network_interfaces**: List of network interfaces |
There was a problem hiding this comment.
can I specify device differently? I think there was an option to provide system name. How to do it?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request adds comprehensive documentation to the Media Transport Library validation test suite by introducing new README.md files across key directories. These documents provide detailed overviews of the test framework's structure, configuration, utilities, engine components, and test modules, significantly improving onboarding and usability for developers and testers.
Documentation Additions:
Top-level test suite documentation:
tests/validation/README.mdwith an overview of the validation framework, directory structure, setup instructions, test execution examples, configuration guidance, troubleshooting tips, and licensing information.Common utilities documentation:
tests/validation/common/README.mddescribing shared utilities for network control, data integrity checks, FFmpeg integration, and usage examples.Configuration documentation:
tests/validation/configs/README.mdexplaining the structure and customization oftest_config.yamlandtopology_config.yaml, including environment-specific overrides and programmatic usage.Test engine documentation:
tests/validation/mtl_engine/README.mddetailing the core engine modules, such as process management, application interfaces, reporting, integrity checking, RAM disk management, and extension points.Test modules documentation:
tests/validation/tests/README.mdoutlining the organization of test categories, test types, execution instructions, and guidelines for adding new tests.