Modernize project

[surister]

This pr attempts to modernize the project by:

• Making use of pyproject.toml moving away from setup.py
• Use modern build tool like hatchling moving away from buildout
• Make it possible to run tests with just pytest moving away from zope and unittests
• Simplify development setup to work with uv and any other pyproject.toml compatible tool
• Simplify versioning by using hatchling dynamic versioning moving away from setup.py shenanigans
• Simplifying tests by moving away from the custom Testlayer to testcontainers

---

Changes

• Migrate all metadata from setup.py to pyproject.toml
• Make project installable with uv and other venv tools.
• Make project buildable with both uv build and python -m build
• Migrate test_cursor to pytest
• Migrate test_connection to pytest
• Migrate test_exception to pytest
• Migrate test_http to pytest
• Migrate test_cursor to pytest
• Migrate doctests to pytest
• Migrate testlayer to testcontainers

Aditional changes

• Every tests has a docstring explaining what's expected of the test.
• test files like test_http are split into smaller tests files, in this case, there are tests that test the http.Client and the serialization of data, the new scheme will phase test_http into test_serialization and test_client. This is just to give an idea of what the change is like, might actually differ later.

Notes to remember:

• serializers testing have been revamped, now we only test the serialization call chain and then test the serialize

[amotl]

Hi. Thanks for modernizing. We appreciate using testcontainers instead of testlayers and specifically to get rid of Zope-related packages from previous decades where maintenance seems to be fading. (see also GH-723, GH-725, GH-736)

However, is it possible to split project infrastructure / metadata updates vs. those test modernizations which will have a much larger impact to the testing codebase, and also possibly migrate test cases progressively? We otherwise fear the patch will become too big to review reasonably, a problem we've currently encountered elsewhere.

testcontainers/python-unittest demonstrates a way to use the modern testcontainers way of testing together with traditional Python unittest, intended to support this and other repositories on their modernization paths. Maybe this can help a bit on this occasion?

CrateDBTestAdapter is effectively just a little wrapper using the CrateDBContainer class that is implementing the testcontainers specification, so it can be slotted into unittest layer concepts, at least as an intermediate workaround adapter for supporting a progressive migration, when possible.

NB: In particular, zope-testing does not seem to cause problems (GH-738), so we could possibly afford to keep it for a while, while still migrating away from setup.py?

[surister]

However, is it possible to split project infrastructure / metadata updates vs. those test modernizations which will have a much larger impact to the testing codebase, and also possibly migrate test cases progressively? We otherwise fear the patch will become too big to review reasonably, a problem we've currently encountered elsewhere.

Yes I can split the PR into -> metadata & tests. I put them together because reviewing a setup.py -> pyproject.toml migration is trivial for us, since we are all well accustomed to a pyproject.toml definition. So it felt that it wouldn't make the review easier.

testcontainers/python-unittest demonstrates a way to use the modern testcontainers way of testing together with traditional Python unittest, intended to support this and other repositories on their modernization paths. Maybe this can help a bit on this occasion?

My goal is to move to the current python standard which is to use pytest, moving the Testlayer -> unittest testcontainers -> pytest testcontainers is not something I want to do, since moving to pytest is very straight forward

NB: In particular, zope-testing does not seem to cause problems (#738), so we could possibly afford to keep it for a while, while still migrating away from setup.py?

I kind of find it annoying, with just pytest every test need is pretty much solved, I don't think there is any need for it nowadays, migrating away from zope-testing is part of the unittest -> pytest story.

I understand that the review could be a bit painful but I think it's one of those that happen every few years that you just have to do it, I think that we are all very used to pytest and as you can see in the ones that I migrated, the resulting code is very light and shorter than the unitests', so I'd ask for a little bit of help soldiering here 🙏

[amotl]

Hi. Thanks a stack for bringing in pytest.

[...] as you can see in the test cases that I migrated, the resulting code is very light and shorter than the unitests', so I'd ask for a little bit of help soldiering here.

I very much appreciate the transition to pytest, I am certainly supporting it. However, I don't know if I will be able to provide hands-on help due to other obligations, asking for your understanding.

My fears the patch would become too large have not been too substantial so far: I think it provides a reasonable diff size to review properly.

Most probably, I am looking at those more delicate future(?) changes which have not been tackled yet, and which might increase diff size / review complexity:

• Migrate doctests to pytest
• Migrate testlayer to testcontainers

I think all reviewers will like if those can be tackled separately. So, let me quickly ask a question about the ingredients/state of the patch wrt. those matters: Are integration- and doctests currently still functional by using traditional unittest test layers / will they still be executed by the traditional zope testrunner, as I see not much code might have been removed in this area?

My goal is to move to the current python standard which is to use pytest, moving the Testlayer -> unittest testcontainers -> pytest testcontainers is not something I want to do, since moving to pytest is very straight forward.

If this means corresponding modernizations will be tackled using a separate patch, I think everyone will be happy.

NB: Maybe change the PR title to something like "Migrate unit tests to pytest. Switch from setuptools to hatchling.", if this better matches up with its ingredients? Apologies if I am missing important details yet: I just quickly skimmed the patch to provide a few fragments of early feedback. Feel free to reach out directly if you think I am misunderstanding some details.

NB: Even if, or just because "Switch from setuptools to hatchling" might be so trivial, submitting this as a separate patch will always be preferred / way to go, not just from the reviewing perspective, but also from aspects around repository maintenance: If this change is isolated and brought in on behalf of a single commit, any obstacles around it (git revert, when applicable) can be processed much better in the aftermath.

[surister]

Most probably, I am looking at those more delicate future(?) changes which have not been tackled yet, and which might increase diff size / review complexity:

Going to leave those for later and wrap this up

[surister]

From the last review, only new tests were added to increase coverage.

[amotl]

I think the code is still compatible down to Python 3.6?

[surister]

We have two users running 2.7, so perhaps even lower than 3.6

[amotl]

Hm. I don't think the current version of the driver is compatible with Python 2 any longer.