test(sigcheck): check function signature parity across backends #10008
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is a fork of some of the utilities in Scott Sanderson's `python-interface` project (https://github.com/ssanderson/python-interface). While the upstream is no longer maintained, the goal of that project aligns quite well with some of the issues we face with maintaining consistent interfaces across backends. I'm grabbing a few of these utilities and ripping out Python 2 support, then I'm going to work on setting up tests for making sure that backend methods match the signatures in the parent `BaseBackend` class.
Should be `implementation, interface_def`, not the other way around
gforsyth
changed the title
cpcloud
approved these changes
Sep 5, 2024
| "to_parquet_dir", | ||
| marks=pytest.mark.notyet(["pyspark"]), | ||
| ), | ||
| "truncate_table": pytest.param( |
There was a problem hiding this comment.
So, basically the goal is to, over time, remove as many elements of this dict as possible?
There was a problem hiding this comment.
Ideally, we'd remove all of them. There's a more generic "skip for all backends" list for things like do_connect, but I think we should be able to remove all of the entries in this dict if we standardize positional keyword names, and enforce keyword-only arguments
There was a problem hiding this comment.
Also, once this is merged in, it will also prevent us from adding new methods with differing argument names (assuming those methods are defined in the base classes, which they should be)
13 tasks
ncclementi
pushed a commit
to ncclementi/ibis
that referenced
this pull request
Sep 24, 2024
…-project#10008) Opening this in favor of ibis-project#9383 -- that PR also included all of the breaking changes to unify function signatures and it was too much at once. This PR adds only the signature checking mechanism, plus the requisite xfails to lay out which inconsistencies are currently in Ibis. ## Motivation We want to ensure that, for a given backend, that the argument names, plus usage of positional, positional-only, keyword, and keyword-only arguments match, so that there is API consistency when moving between backends. I've grabbed a few small parts of some of the utilities in Scott Sanderson's `python-interface` project (https://github.com/ssanderson/python-interface). While the upstream is no longer maintained, the goal of that project aligns quite well with some of the issues we face with maintaining consistent interfaces across backends. Note that while the upstream project focused on _runtime_ enforcement of these signatures matching, here it is only run in the test suite. ## Rough procedure Any method that doesn't match can be skipped entirely (this is useful for things like `do_connect`, which cannot reasonably be assumed to match) or individually (by specifying a `pytest.param` and marking the failing backends). Then we scrape across the common parent classes and add any methods that are NOT currently specified in the pre-existing xfailed ones. It's a bit of a nuisance, but it's done, and ideally the manual listing of the inconsistent methods goes away as we unify things. I've opted for not checking that type annotations match, because that seems... unreasonable. This would satisfy ibis-project#9125 once all of the xfail markers are removed, e.g., it checks that all keyword and positional arguments are standardized.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Opening this in favor of #9383 -- that PR also included all of the breaking changes to unify function signatures and it was too much at once. This PR adds only the signature checking mechanism, plus the requisite xfails to lay out which inconsistencies are currently in Ibis.
Motivation
We want to ensure that, for a given backend, that the argument names, plus usage of positional, positional-only, keyword, and keyword-only arguments match, so that there is API consistency when moving between backends.
I've grabbed a few small parts of some of the utilities in Scott Sanderson's
python-interfaceproject (https://github.com/ssanderson/python-interface).While the upstream is no longer maintained, the goal of that project
aligns quite well with some of the issues we face with maintaining
consistent interfaces across backends.
Note that while the upstream project focused on runtime enforcement of these signatures matching, here it is only run in the test suite.
Rough procedure
Any method that doesn't match can be skipped entirely (this is useful for things like
do_connect, which cannot reasonably be assumed to match) or individually (by specifying apytest.paramand marking the failing backends).Then we scrape across the common parent classes and add any methods that are NOT currently specified in the pre-existing xfailed ones. It's a bit of a nuisance, but it's done, and ideally the manual listing of the inconsistent methods goes away as we unify things.
I've opted for not checking that type annotations match, because that seems... unreasonable.
This would satisfy #9125 once all of the xfail markers are removed, e.g., it checks that all keyword and positional arguments are standardized.