Add RestSend framework, enums, and shared unit test infrastructure by allenrobel · Pull Request #185 · CiscoDevNet/ansible-nd

allenrobel · 2026-03-02T18:41:56Z

Summary

Adds the RestSend HTTP request framework, enums.py, and the shared unit test infrastructure for the ND collection. This PR is one of two companion PRs targeting nd42_integration:

This PR nd42_rest_send — RestSend framework, enums.py, and all shared test infrastructure (merge first)
Add Log class and logging config #184 nd42_logging — Log class, logging config, and test_log.py

Note: Imports of Log (log.py) will show as broken until nd42_logging (#184) is merged into nd42_integration. The unit tests in this PR also have a dependency on log.py (via common_utils) and will not run until both branches are merged.

TODO

2026-02-03 - Add unit tests for sender_file.py as well.

Plugin files added

File	Description
`plugins/module_utils/enums.py`	`HttpVerbEnum` and `OperationType` enums — required by RestSend, Results, ResponseHandler, and Sender
`plugins/module_utils/rest_send.py`	`RestSend` — orchestrates HTTP requests via a pluggable `Sender`, with retries, timeout, and unit-test mode
`plugins/module_utils/results.py`	`Results` — tracks changed/failed state and diff output across a task's lifetime
`plugins/module_utils/protocol_response_handler.py`	`ResponseHandlerProtocol` — structural protocol for response handler implementations
`plugins/module_utils/response_handler_nd.py`	`ResponseHandler` — ND-specific response handler; normalises HTTP status codes and extracts error messages
`plugins/module_utils/protocol_sender.py`	`SenderProtocol` — structural protocol for sender implementations
`plugins/module_utils/sender_nd.py`	`Sender` — ND-specific sender using Ansible's `httpapi` connection plugin
`plugins/module_utils/pydantic_compat.py`	Pydantic v1/v2 compatibility shim used by `Results`
`plugins/module_utils/protocol_response_validation.py`	`ResponseValidationStrategy` protocol for version-specific ND API response validation
`plugins/module_utils/response_validation_nd_v1.py`	ND API v1 implementation of `ResponseValidationStrategy`

Shared unit test infrastructure added

File	Description
`plugins/module_utils/enums.py`	(see above)
`tests/unit/__init__.py`	Package marker
`tests/unit/module_utils/__init__.py`	Package marker
`tests/unit/module_utils/common_utils.py`	`does_not_raise()` and other shared test helpers
`tests/unit/module_utils/fixtures/load_fixture.py`	JSON fixture loader
`tests/unit/module_utils/mock_ansible_module.py`	Lightweight `AnsibleModule` mock
`tests/unit/module_utils/response_generator.py`	Generator-based mock response sequencer
`tests/unit/module_utils/sender_file.py`	File-based `Sender` for replaying canned HTTP responses in tests

Unit tests added

File	Tests
`tests/unit/module_utils/test_rest_send.py`	`RestSend` class
`tests/unit/module_utils/test_response_handler_nd.py`	`ResponseHandler` class
`tests/unit/module_utils/test_sender_nd.py`	`Sender` class
`tests/unit/module_utils/fixtures/fixture_data/test_rest_send.json`	Canned HTTP responses for `RestSend` tests

Merge order

Merge this PR → nd42_integration (provides enums.py and shared test infrastructure needed by nd42_logging)
Merge nd42_logging (Add Log class and logging config #184) → nd42_integration

Test plan

After merging both PRs into nd42_integration: python -m pytest tests/unit/module_utils/ passes cleanly
tox -e linters passes (black, isort, pylint, mypy)
ansible-test sanity --docker passes

🤖 Generated with Claude Code

Plugin files: - plugins/module_utils/rest_send.py: RestSend class for HTTP request handling - plugins/module_utils/results.py: Results class for tracking task state - plugins/module_utils/protocol_response_handler.py: ResponseHandlerProtocol - plugins/module_utils/response_handler_nd.py: ND-specific ResponseHandler - plugins/module_utils/protocol_sender.py: SenderProtocol - plugins/module_utils/sender_nd.py: ND-specific Sender (httpapi connection) - plugins/module_utils/pydantic_compat.py: Pydantic v1/v2 compatibility shim - plugins/module_utils/protocol_response_validation.py: ResponseValidationStrategy protocol - plugins/module_utils/response_validation_nd_v1.py: ND v1 API response validation Unit tests: - tests/unit/module_utils/test_rest_send.py - tests/unit/module_utils/test_response_handler_nd.py - tests/unit/module_utils/test_sender_nd.py - tests/unit/module_utils/fixtures/fixture_data/test_rest_send.json Note: imports of Log and enums (log.py, enums.py) are intentionally broken in this branch. Both will resolve once nd42_logging is merged into nd42_integration. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

plugins/module_utils/rest/protocols/response_handler.py

plugins/module_utils/common/pydantic_compat.py

plugins/module_utils/rest/rest_send.py

plugins/module_utils/rest/results.py

Moved from nd42_logging to this branch so that enums.py (a required dependency of the RestSend framework) is co-located with the code that uses it. Plugin files: - plugins/module_utils/enums.py: HttpVerbEnum and OperationType enums Shared unit test infrastructure: - tests/unit/__init__.py - tests/unit/module_utils/__init__.py - tests/unit/module_utils/common_utils.py: does_not_raise() and shared helpers - tests/unit/module_utils/fixtures/load_fixture.py: JSON fixture loader - tests/unit/module_utils/mock_ansible_module.py: lightweight AnsibleModule mock - tests/unit/module_utils/response_generator.py: generator-based mock response sequencer - tests/unit/module_utils/sender_file.py: file-based Sender for replaying canned responses Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

plugins/module_utils/nd_v2.py demonstrates how to wire together the RestSend framework (RestSend, Sender, ResponseHandler) and Smart Endpoints in a module context. Included as a working example for teams building new modules against nd42_integration. Note: imports Log from nd42_logging and endpoint classes from nd42_smart_endpoints; will have broken imports until all three branches are merged into nd42_integration. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

plugins/module_utils/enums.py

plugins/module_utils/nd_v2.py

tests/unit/module_utils/sender_file.py

tests/unit/module_utils/test_response_handler_nd.py

plugins/module_utils/common/pydantic_compat.py

plugins/module_utils/response_handler_nd.py

plugins/module_utils/rest/response_handler_nd.py

plugins/module_utils/rest/response_strategies/nd_v1_strategy.py

ResponseHandler now delegates status code checks and error message extraction to an injected ResponseValidationStrategy (defaulting to NdV1Strategy) instead of hardcoding the logic. This removes the duplicate error extraction code and the three hardcoded class constants, and exposes a `validation_strategy` property for future v2 support. Unit tests updated to reflect the removed class constants and to cover the new `validation_strategy` property. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

plugins/module_utils/nd_v2.py

akinross · 2026-03-04T15:04:29Z

plugins/module_utils/rest/results.py

+        self.log.debug(msg)
+
+        # Early exit for read-only operations
+        if self._current.check_mode or self._current.operation_type.is_read_only() or self._current.state == "query":


Comment from previous PR: #180 (comment)

why would both self._current.operation_type.is_read_only() or self._current.state == "query" be checked? is the smart enum pattern with the is_read_only() not suffficient?

another thing which I am wondering is why check_mode here and not already before the function call, since checked_mode or arguably query would never result in a change. so following your focus on single responsibility did you consider splitting the functionality into ("should changes be possible?") and a detector ("did changes happen?")?

Answer:

Yes, this is one of the things that I wanted to change but didn't have time to adequately test.

And, yes, agree that we can optimize this logic. As some history, the enum was added fairly recently in the DCNM collection and was intended to replace any other inputs to conditionals (like _current_state) so that (ideally) we have a single thing to check.

Open item: Do we track this somewhere? Or do we look at doing this in the restsend PR?

I'll address this now. Should be done by EOD.

Addressed removing unneeded or self._current.state == "query" with commit: 9ee6b7a

Personally, I don't see the value in moving check_mode out of this conditional. If you feel strongly about it, please open an issue with proposed changes. Too, we may end up not even using Results so, in the interest of prioritizing the other TODOs in the Log PR and Smart Endpoints PR, I'll defer this to you.

Don't feel strongly about it, was just an observation and a question regarding separation of concern and if we should. Separation of concerned was raised, together with other valid principles thus noticed this and a discussion could be held regarding were we want to draw the line of concern.

plugins/module_utils/rest/rest_send.py

plugins/module_utils/rest/results.py

plugins/module_utils/rest/rest_send.py

Move files into logical subdirectories: - common/pydantic_compat.py - rest/response_handler_nd.py, rest_send.py, results.py, sender_nd.py - rest/protocols/response_handler.py, response_validation.py, sender.py - rest/response_strategies/nd_v1_strategy.py Update all imports in moved files, nd_v2.py, and unit tests. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

1. Claude forgot to add the linted files to the last commit. 2. Likewise for the updated docstrings 3. Addressing Akini’s request to remove RestSend.implements property

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Includes linter configurations (pylint, isort, black) and uv dependency lockfile for reproducible environments. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…ire_pydantic() Two changes based on code review feedback: 1. Consolidate duplicate `if TYPE_CHECKING:` blocks. `HAS_PYDANTIC = True` and `PYDANTIC_IMPORT_ERROR = None` were previously set in a second `if TYPE_CHECKING:` block at the bottom of the module. They now live inside the original `if TYPE_CHECKING:` block alongside the real pydantic imports, removing the redundant second block. 2. Add `require_pydantic(module)` helper function. Ansible modules that depend on pydantic need to fail with a clear error message when pydantic is not installed. The standard Ansible pattern is: if not HAS_PYDANTIC: module.fail_json(msg=missing_required_lib("pydantic"), exception=PYDANTIC_IMPORT_ERROR) However, placing this multi-line boilerplate in every module was rejected in favor of a single-call pattern. The alternatives considered were: - Raising an exception at import time: not viable because `fail_json` requires an AnsibleModule instance, which doesn't exist yet when module-level imports run. - Importing AnsibleModule inside pydantic_compat.py and instantiating it: not viable because AnsibleModule.__init__ requires argument_spec and performs sys.argv parsing specific to each calling module. The compromise is `require_pydantic(module)`, called once in `main()` immediately after AnsibleModule is instantiated: def main(): module = AnsibleModule(argument_spec=...) require_pydantic(module) Implementation notes: - `missing_required_lib` is imported lazily inside the function body rather than at module level, avoiding an unnecessary import on the happy path (pydantic installed) and during sanity tests. - The function is defined outside the TYPE_CHECKING/else block so it is always available regardless of environment. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Operation type and _current.check_mode are sufficient to determine whether the operation was read only.

We can discuss adding these to the integration branch later.

1. We need unnecessary-ellipsis due to differences in Python version behaviors. 2. For unit tests, _ is OK for var name

We removed the RestSend.implements property in an earlier commit. This commit updates the unit tests for RestSend to remove asserts related to this property.

The too-many-positional-arguments message ID was introduced in pylint 3.3 and is not recognized by the older pylint version used in ansible-test sanity. The existing too-many-arguments directive provides equivalent suppression in older versions. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

typing.Protocol and typing.runtime_checkable were added in Python 3.8. Use try/except to fall back to typing_extensions on Python 3.7. TODO (open issues): - Verify typing_extensions is listed in collection requirements - Verify mypy is satisfied with the type: ignore[assignment] suppression on the typing_extensions fallback import Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

typing_extensions is not available in the ansible-test sanity environment, so the previous two-level try/except was insufficient. Add a third fallback that defines minimal Protocol and runtime_checkable stubs so the module imports cleanly on Python 3.7 without any external dependencies. The stubs lose isinstance() checking on Python 3.7, but the collection requires Python >= 3.11 at runtime so this only needs to satisfy the sanity import test. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Adding for Ansible sanity tests.

Wrap the from __future__ import boilerplate in # isort: off / # isort: on block markers so isort cannot remove the parentheses or merge in the annotations import. Keeps from __future__ import annotations on a separate line within the protected block. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Black reformats `from __future__ import (absolute_import, ...)` by removing the parentheses. Add `# fmt: off` / `# fmt: on` inside the existing `# isort: off` / `# isort: on` blocks to prevent this. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

allenrobel requested review from akinross, anvitha-jain, gmicol, lhercot, sajagana, samiib and shrsr as code owners March 2, 2026 18:41

mikewiebe reviewed Mar 2, 2026

View reviewed changes

plugins/module_utils/rest/protocols/response_handler.py Show resolved Hide resolved

mikewiebe reviewed Mar 2, 2026

View reviewed changes

plugins/module_utils/common/pydantic_compat.py Show resolved Hide resolved

mikewiebe reviewed Mar 2, 2026

View reviewed changes

plugins/module_utils/rest/rest_send.py Show resolved Hide resolved

mikewiebe reviewed Mar 2, 2026

View reviewed changes

plugins/module_utils/rest/results.py Show resolved Hide resolved

allenrobel mentioned this pull request Mar 2, 2026

Add Log class and logging config #184

Open

3 tasks

allenrobel changed the title ~~Add RestSend framework: RestSend, Results, ResponseHandler, Sender~~ Add RestSend framework, enums, and shared unit test infrastructure Mar 2, 2026

allenrobel mentioned this pull request Mar 3, 2026

Add Smart Endpoints framework for ND API v1 #186

Open

3 tasks

akinross reviewed Mar 3, 2026

View reviewed changes

plugins/module_utils/enums.py Show resolved Hide resolved

plugins/module_utils/nd_v2.py Show resolved Hide resolved

tests/unit/module_utils/sender_file.py Show resolved Hide resolved

tests/unit/module_utils/test_response_handler_nd.py Show resolved Hide resolved

mikewiebe reviewed Mar 3, 2026

View reviewed changes

plugins/module_utils/common/pydantic_compat.py Show resolved Hide resolved

mikewiebe reviewed Mar 3, 2026

View reviewed changes

plugins/module_utils/response_handler_nd.py Outdated Show resolved Hide resolved

mikewiebe reviewed Mar 3, 2026

View reviewed changes

plugins/module_utils/rest/response_handler_nd.py Show resolved Hide resolved

mikewiebe reviewed Mar 3, 2026

View reviewed changes

plugins/module_utils/rest/response_strategies/nd_v1_strategy.py Show resolved Hide resolved

allenrobel and others added 5 commits March 3, 2026 09:27

Add Usage section

d8d6fa9

Remove references to ND 3.x and NDFC

146b369

Remove unused import: inspect

b413b10

Update docstrings to indicate optionality of register_task_result

6647191

lhercot requested changes Mar 4, 2026

View reviewed changes

plugins/module_utils/nd_v2.py Outdated Show resolved Hide resolved

akinross reviewed Mar 4, 2026

View reviewed changes

allenrobel mentioned this pull request Mar 4, 2026

CR: Unit tests for tests/module_utils/sender_file.py (DCNE-689) #187

Open

allenrobel mentioned this pull request Mar 4, 2026

CR: Should ResponseHandler convert response to Pydantic model? (DCNE-690) #188

Open

allenrobel and others added 17 commits March 4, 2026 12:42

Lint, update docstrings, remove implements property

f2b6711

1. Claude forgot to add the linted files to the last commit. 2. Likewise for the updated docstrings 3. Addressing Akini’s request to remove RestSend.implements property

Move NDErrorData and NDModuleError to common/exceptions.py

3c3a9bf

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add pyproject.toml and uv.lock from rest_send_integration

0ff98fb

Includes linter configurations (pylint, isort, black) and uv dependency lockfile for reproducible environments. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Remove self._current.state from conditional

9ee6b7a

Operation type and _current.check_mode are sufficient to determine whether the operation was read only.

Remove pyproject.toml and uv.lock

9b91c2c

We can discuss adding these to the integration branch later.

Add pylint suppresession directives

e513963

1. We need unnecessary-ellipsis due to differences in Python version behaviors. 2. For unit tests, _ is OK for var name

Update RestSend unit tests (implements property)

96fa949

We removed the RestSend.implements property in an earlier commit. This commit updates the unit tests for RestSend to remove asserts related to this property.

Add tests/sanity/config.yml

9499c13

Adding for Ansible sanity tests.

Appease linters

d7c9f36

Move config.yml under tests/

8f6b51f

Conversation

allenrobel commented Mar 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

TODO

Plugin files added

Shared unit test infrastructure added

Unit tests added

Merge order

Test plan

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

akinross Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

allenrobel Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

allenrobel Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

akinross Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

allenrobel commented Mar 2, 2026 •

edited

Loading

allenrobel Mar 5, 2026 •

edited

Loading