refactor(tests): hatch-ify and correct imports in tests

[dhdaines]

PR Goal?

Standardize the directory laout of tests

Fixes?

Fixes: #281

Feedback sought?

Is this easy enough to understand and does it work for you? Should we go all the way and use pytest markers to distinguish the test suites, at which point we can also use pytest-cov to run the tests

Priority?

Happy holidays!

Tests added?

Rien ne se perd, rien ne se crée, tout se transforme!

How to test?

hatch test
python run_tests.py

Confidence?

Might want to look it over a bit. Definitely verify the checklist in #281.

Version change?

Nope, just affects tests.

[semanticdiff-com]

Review changes with  

Changed Files

File	Status
run_tests.py	88% smaller
.github/workflows/tests.yml	67% smaller
.github/workflows/matrix-tests.yml	54% smaller
tests/test_align_cli.py	44% smaller
tests/test_api.py	34% smaller
.gitignore	Unsupported file format
README.md	Unsupported file format
pyproject.toml	Unsupported file format
tests/test_anchors.py	0% smaller
tests/test_audio.py	0% smaller
tests/test_dna_text.py	0% smaller
tests/test_force_align.py	0% smaller
tests/test_g2p_cli.py	0% smaller
tests/test_make_xml_cli.py	0% smaller
tests/test_misc.py	0% smaller
tests/test_package_urls.py	0% smaller
tests/test_silence.py	0% smaller
tests/test_smil.py	0% smaller
tests/test_tokenize_cli.py	0% smaller
tests/test_web_api.py	0% smaller

[github-actions]

CLI load time: 0:00.04
Pull Request HEAD: add1bbd00b38b16c409383eb6e13eb88df3540f9
No imports take more than 0.1 s.

[joanise]

This is a nice improvement, for sure, but it's loosing one thing I like: cd tests; ./test_misc.py (e.g.) no longer works. Playing around with things to understand what happens made me realize that can be fixed by replacing from .basic_test_case import BasicTestCase with from tests.basic_test_case import BasicTestCase, which means pip install -e . in Studio has been installing tests (formerly test) in the Pythonpath all along.

Yikes! I thought editable pip installs put your project's main source directory into the Pythonpath, not every directory in your source tree. I guess it's every directory in your source with a __init__.py, isn't it?

That means if we have interralated projects each with its own tests in <root>/tests (like we do here) rather than <root>/<projectname>/tests (like we do in g2p) the tests dirs of the projects would conflict with each other.

Now, none of this is related to the changes you're proposing, but it makes me think that maybe readalongs/tests would indeed be a preferrable place for the test suite here.

BTW, in EveryVoice we explicitly exclude the tests from our wheels and sdists with this bit of configuration:
https://github.com/EveryVoiceTTS/EveryVoice/blob/main/pyproject.toml#L163-L184

[tool.hatch.build.targets.wheel]
include = ["/everyvoice"]
exclude = [
  ...
  "tests/",
]

(and similar for tool.hatch.build.targets.sdist) so moving tests under readalongs/ would not have to mean including it in our wheels. Mind you, with the 49M CMU model already in our wheels, the 4M of test data would not represent that much of an increase, but I'd still leave it out and it's easy to do so.

Something else: I'll want to add this warning filter to pyproject.toml to silence that warning we're already addressed:

[tool.pytest]
filterwarnings = [
    "ignore:^'audioop' is deprecated and slated for removal in Python 3.13:DeprecationWarning",
]

There's something I don't understand: uv pip install hatch; hatch test works, but uvx hatch test does not.

C:...\uv\cache\archive-v0-9SkXSIFElkdvl2dDniCy\Scripts\python.exe: No module named pip

I don't understand why I get this error. If actually installing hatch is required, I would add hatch to our dev dependencies so it's ready to use. Just pytest also works, with identical output, so it could be either pytest or hatch we install.

Anyway, lots of notes here, but the output of hatch test and pytest is much nicer than our current output, and I'll be happy to refine this PR and then merge -- but probably after the holidays, in January. Thanks for getting this going!

[dhdaines]

This is a nice improvement, for sure, but it's loosing one thing I like: cd tests; ./test_misc.py (e.g.) no longer works.

Thanks! You can just run hatch test tests/test_misc.py (or pytest tests/test_misc.py from the dev virtualenv, which does the same thing)

Happy holidays and new year!

[joanise]

I took the liberty to apply some further refactorings to the test suites, in particular enabling doing cd tests; ./test_misc.py which I prefer using, otherwise this is great, thanks for figuring how to make pytest work. Good to merge now.

[joanise]

Cool, hatch test -p runs tests on 16 workers in parallel and runs all the tests in 35-45s instead of 1:45! Nice.