[Feature] Retries on Pydantic Validation Errors

[fmmoret]

What feature would you like to see?

Maybe I'm wrong but I thought I recalled the old version of DSPY (pre-litellm / pre-adapter era) doing retries for pydantic validation failures, but I'm not seeing any retrying in the codepaths I'm looking at in the stack trace (predict.py, json_adapter.py)
Is there a way built into the new version of DSPY to control retries? Or do I need to impl it into my own modules now?

E.g.:

pydantic_core._pydantic_core.ValidationError: 1 validation error for SomeSignature
some_list
  List should have at most 3 items after validation, not 4 [type=too_long, input_value=[1,2,3,4], input_type=list]
    For further information visit https://errors.pydantic.dev/2.9/v/too_long

I would expect this to feed back into the LLM to let it correct its outputs.

Would you like to contribute?

• Yes, I'd like to help implement this.
• No, I just want to request it.

Additional Context

No response

[bnsh]

I agree.. So, here's what I see: When they switched to using LiteLLM: there's this:

dspy/dspy/clients/lm.py

Line 65 in 84b9b88

num_retries: The number of times to retry a request if it fails transiently due to

Which seems to do something different from what I remember dspy.TypedPredictor doing..

Once upon a time: at least at commit hash b32b2ab:

When it would get a TypeError: it would

1. Capture the error
2. Make an example
3. Send that back to the LLM

I agree that this was useful behaviour...

In looking into it tho, I found dspy.predict.retry.py... Which is totally commented out. I was all set to work on adding back the retrying in, till I saw that.. Can someone explain what the plan for retrying Pydantic validation failures?