Remove use of __root__ in Pydantic models

[pederhan]

Pydantic V2 removes support for __root__ in models. The Pre-release blog post provides 2 different ways to create models with "root" fields.

The easiest replacement seems to be AnalyzedType. This uses the new Pydantic V2 type that allows for creating arbitrary types from generic annotations.

Another, more complicated approach, if custom behavior is required, is to provide custom model validators and serializers, as demonstrated in the Pydantic test suite. This seems to provide more flexibility, but adds a lot of boilerplate code and it's unclear if we actually need this flexibility.

[pederhan]

I think the name AnalyzedType was changed at some point, even though the blog post still points to that name. Will update once I can be bothered to dig up the new name.

[pederhan]

With #57 , we can use the newest version of datamodel-code-generator to generate Pydantic V2 models. That uses the new RootModel which serializes to JSON correctly, but requires us to manually add the methods for supporting iteration.

https://docs.pydantic.dev/latest/usage/models/#rootmodel-and-custom-root-types

Furthermore, we need to ensure we don't break current user apps that directly access the __root__ attribute.

All this can probably be done with some patching at runtime? Unless we can specify our own RootModel to use in datamodel-code-generator...?

[pederhan]

datamodel-codegen --collapse-root-models is NOT the answer we are looking for at least, because that will definitely break stuff.