Add Index.validate_dataarray_coord

[benbovy]

• Tests added
• User visible changes (including notable bug fixes) are documented in whats-new.rst
• New functions/methods are listed in api.rst

Same goal than #10116 using the alternative approach suggested in #10116 (comment) where the propagation (validation) of coordinates in a DataArray is delegated to their index (if any).

I find this approach cleaner than #10116. Here is the same notebook example adapted for this PR.

Index API alternatives

1. Index.validate_dataarray_coord(self, name: Hashable, var: Variable, dims: set[Hashable]) -> None

   • check one index coordinate at a time, thus called multiple times for a multi-coordinate index

2. Index.validate_dataarray_coords(self, variables: dict[Hashable, Variable], dims: set[Hashable]) -> None

   • check all coordinates of an index at once
   • perhaps could make things easier for developers of custom Xarray indexes (perhaps not?), but it would make Xarray internals more complicated with possible significant impact on performance (we'd have to rely on Indexes.group_by_index while in option 1 we simply need to iterate over _variables, _coords and/or _indexes).

3. Index.validate_dataarray_coords(self, variables: dict[Hashable, Variable], dims: set[Hashable]) -> Coordinates

   • Same than option 2 but allow returning a new set of coordinate variables and their indexes.
   • This has much more flexibility (e.g., convert a custom 2D spatial index to a 1D default PandasIndex when the result has only one of the two spatial dimensions), but I find it also complicated and prone to confusion.

Option 1, the simplest and working well with IntervalIndex, is currently implemented in this PR.

cc @shoyer @dcherian

[shoyer]

Some quick thoughts

[benbovy]

Thanks @shoyer for taking a look!

This is now ready for (another round of) review.

Compared to #10116 this PR still represents a major data model change for DataArray (in the sense that it allows overriding strict enforcement of the DataArray model in certain cases), but I think that the risk is mitigated since it is here done explicitly via Index.validate_dataarray_coord and the default behavior is to keep conforming to the DataArray model. Limiting an index to (in)validate its coordinates may also be good to keep things simple and predictable for end-users.

@dcherian regarding your #10116 (comment), in the example notebook the DataArray reprs shown without any list of dimensions for the Coordinates section do not strike me much. That said, the Coordinates repr is still basic and might benefit from such addition (it doesn't have any html repr BTW).

[dcherian]

Nice!

[dcherian]

That said, the Coordinates repr is still basic and might benefit from such addition (it doesn't have any html repr BTW).

Yeah I think it could be improved, but certainly not a blocker here.

IMO we can merge. I'm not a huge fan of the method name validate_dataarray_coord but I couldn't come up with anything better

[benbovy]

Thanks @dcherian and @shoyer. I added "plan to merge" but I'm open to revisit anything here.

[shoyer]

Thanks @benbovy for this change, and apologies for the slow review.

My one big question is if validate_dataarray_coord could be replaced by a method that returns a boolean rather than potentially raising an error. It's not clear to me what the advantages of raising an error are.

The fall-back logic of checking coordinate dimensions could also potentially be implemented on the base Index class, which would allow for customizing it if desired.

[benbovy]

@shoyer I like your suggestion and find it cleaner.

In 3e55af0 I replaced the validate_dataarray_coord with should_add_coord_in_dataarray (couldn't find a better name, suggestions welcome!).

I also removed the fall-back in Dataset._construct_dataarray. The logic of checking coordinate dimensions is implemented in the Index base class (in Index.should_add_coord_in_dataarray).

Is that what you had in mind?

[dcherian]

Yeah this method's a hard one to name. The only thing that comes to mind is should_propagate_variable_with_dataarray :/