Refresh Python testing docs #8102
Open
+77
−238
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
eleanorjboyd
reviewed
Apr 2, 2025
docs/python/testing.md
Outdated
| `python.testing.autoTestDiscoverOnSaveEnabled` is set to `true` by default, meaning that test discovery is also performed automatically whenever you add, delete, or update any Python file in the workspace. To disable this feature, set the value to `false`. You can refine for which files auto test discovery occurs by specifying a glob pattern in the `python.testing.autoTestDiscoverOnSavePattern` setting. Its default value is `**/*.py`. | ||
|
|
||
| You can [configure the settings](/docs/configure/settings.md) in the Settings editor or directly in the `settings.json` file. You need to reload the window for the test discovery setting to take effect. | ||
| > **Tip**: `python.testing.autoTestDiscoverOnSaveEnabled` is set to `true` by default, meaning that test discovery is also performed automatically whenever you add, delete, or update any Python file in the workspace. To disable this feature, set the value to `false`, which can be done either in the Settings editor or in the `settings.json` file as described in the VS Code [Settings](/docs/editor/settings.md) documentation. You will need to reload the window for this setting to take effect. |
There was a problem hiding this comment.
think we should still include this section: python.testing.autoTestDiscoverOnSavePattern setting. Its default value is **/*.py.
docs/python/testing.md
Outdated
|
|
||
|  | ||
|
|
||
| If discovery fails (for example, the test framework isn't installed or you have a syntax error in your test file), you'll see an error message displayed in the Test Explorer. The Python extension utilizes error tolerant discovery, meaning if there is an error during discovery that effects some tests but not all, you will see a node of tests and an error node in the Test Explorer. You can check the **Python** output panel to see the entire error message (use the **View** > **Output** menu command to show the **Output** panel, then select **Python** from the dropdown on the right side). |
There was a problem hiding this comment.
there should also be a link to the logs in the error node you see
docs/python/testing.md
Outdated
|
|
||
| If discovery fails (for example, the test framework isn't installed or you have a syntax error in your test file), you'll see an error message displayed in the Test Explorer. The Python extension utilizes error tolerant discovery, meaning if there is an error during discovery that effects some tests but not all, you will see a node of tests and an error node in the Test Explorer. You can check the **Python** output panel to see the entire error message (use the **View** > **Output** menu command to show the **Output** panel, then select **Python** from the dropdown on the right side). | ||
|
|
||
| > **Tip**: Unittests can be discovered by pytest, but not vice versa. In the case where testing is configured using unittest but tests are written in pytest, there will be an error message in the Test Explorer. |
There was a problem hiding this comment.
there will not be an error message it will just not find the tests. So if you have pytests but you select unittests then we wont be able to find any tests in the workspace
There was a problem hiding this comment.
I see this in the Test Explorer....is this different or unexpected?
cwebster-99
commented
Apr 2, 2025
cwebster-99
commented
Apr 2, 2025
cwebster-99
commented
Apr 2, 2025
cwebster-99
commented
Apr 2, 2025
cwebster-99
commented
Apr 3, 2025
eleanorjboyd
previously approved these changes
Apr 3, 2025
|
@ntrogh would you mind giving these changes a quick pass? We are cutting quite a bit of content. |
ntrogh
reviewed
Apr 4, 2025
There was a problem hiding this comment.
@cwebster-99 Added some suggestions.
Some general comments about the article (didn't review all sections in detail now):
- Review language for use of future tense and passive voice
- Review the screenshots and use size them consistently. Also, for smaller cut-outs, make them smaller because they look massive compared to full-screen screenshots. Choose 2 or max 3 widths and stick to them.
- You can use the GH styling for alerts (tips, note)
- Happy to do a full review for language and structure in the future - article feels text-heavy right now.
docs/python/testing.md
Outdated
| The combined results of all the tests is your test report, which tells you whether the function (the unit), is behaving as expected across all test cases. That is, when a unit passes all of its tests, you can be confident that it's functioning properly. (The practice of **test-driven development** is where you actually write the tests first, then write the code to pass increasingly more tests until all of them pass.) | ||
|
|
||
| Because unit tests are small, isolated pieces of code (in unit testing you avoid external dependencies and use mock data or otherwise simulated inputs), they're quick and inexpensive to run. This characteristic means that you can run unit tests early and often. Developers typically run unit tests even before committing code to a repository; gated check-in systems can also run unit tests before merging a commit. Many continuous integration systems also run unit tests after every build. Running the unit test early and often means that you quickly catch **regressions,** which are unexpected changes in the behavior of code that previously passed all its unit tests. Because the test failure can easily be traced to a particular code change, it's easy to find and remedy the cause of the failure, which is undoubtedly better than discovering a problem much later in the process! | ||
|
|
||
| For a general background on unit testing, read [Unit testing](https://wikipedia.org/wiki/Unit_testing) on Wikipedia. For useful unit test examples, you can review [https://github.com/gwtw/py-sorting](https://github.com/gwtw/py-sorting), a repository with tests for different sorting algorithms. |
There was a problem hiding this comment.
Can we include a reference to the general testing doc for VS Code (/docs/debugtest/testing.md)?
docs/python/testing.md
Outdated
| @@ -12,144 +12,27 @@ MetaSocialImage: images/tutorial/python-social.png | |||
|
|
|||
| The [Python extension](https://marketplace.visualstudio.com/items?itemName=ms-python.python) supports testing with Python's built-in [unittest](https://docs.python.org/3/library/unittest.html) framework and [pytest](https://docs.pytest.org/). | |||
There was a problem hiding this comment.
Provide a high-level description of what the extension brings: it builds on the built-in testing features in VS Code and provides test discovery, test coverage, and running and debugging tests.
docs/python/testing.md
Outdated
| @@ -252,6 +105,8 @@ Once the coverage run is complete, lines will be highlighted in the editor for l | |||
|
|
|||
| For finer grain control of your coverage run when using pytest, you can edit the `python.testing.pytestArgs` setting to include your specifications. When the pytest argument `--cov` exists in `python.testing.pytestArgs`, the Python extension will make no additional edits to coverage args, to allow your customizations to take effect. If there is no `--cov` argument found, the extension will add `--cov=.` to the pytest args prior to run to enable coverage at the workspace root. | |||
|
|
|||
| > **Note**: Django does not currently support running tests with coverage. | |||
There was a problem hiding this comment.
You can use stylized notes as well
> [!NOTE]
> put the note here
cwebster-99
commented
Apr 4, 2025
cwebster-99
commented
Apr 4, 2025
cwebster-99
commented
Apr 4, 2025
Co-authored-by: Nick Trogh <[email protected]>
|
@ntrogh Thanks for the tips! Incorporated the feedback and made some more cuts. The only thing I wasn't sure about was how to ensure consistent sizing across all of the images. |
|
@cwebster-99 I'll review on Monday and will check and fix any image issues. |
cwebster-99
commented
Apr 4, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Cutting background information from Python testing docs and updating based on new testing functionality.