Skip to content

Conversation

@vblagoje
Copy link
Member

@vblagoje vblagoje commented Oct 9, 2025

Why

Introduces a resilient chat-generator wrapper so production pipelines can survive provider hiccups while keeping latency predictable and responses flowing.

What

  • Added FallbackChatGenerator component that cascades across multiple chat generators with detailed success/failure metadata.
  • Registered the component in chat.__init__ and documented behavior with docstrings.
  • Created FallbackChatGenerator unit tests covering sync/async execution and error propagation.
  • Reno note highlighting why the fallback matters for reliability-focused deployments.

How can it be used

from haystack.components.generators.chat import FallbackChatGenerator, OpenAIChatGenerator

primary = OpenAIChatGenerator(model="gpt-5")
backup = OpenAIChatGenerator(model="gpt-4o-mini")

generator = FallbackChatGenerator(generators=[primary, backup])
result = generator.run(messages=[ChatMessage.from_user("Plan the rollout")])
print(result["meta"]["successful_generator_class"])  # e.g. OpenAIChatGenerator

How did you test it

  • pytest test/components/generators/chat/test_fallback.py
  • Numerous manual examples and experiments

Notes for the reviewer

Focus on the fallback loop and metadata contract (failed_generators, successful_generator_*); downstream consumers might rely on those keys for telemetry.

@github-actions github-actions bot added topic:tests type:documentation Improvements on the docs labels Oct 9, 2025
@coveralls
Copy link
Collaborator

coveralls commented Oct 9, 2025

Pull Request Test Coverage Report for Build 18592817487

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 57 unchanged lines in 11 files lost coverage.
  • Overall coverage increased (+0.1%) to 92.2%

Files with Coverage Reduction New Missed Lines %
components/embedders/sentence_transformers_text_embedder.py 1 96.43%
core/pipeline/pipeline.py 1 92.44%
components/converters/csv.py 4 95.45%
components/generators/azure.py 4 91.84%
components/generators/chat/azure.py 4 93.55%
components/generators/openai.py 4 94.2%
components/agents/agent.py 5 96.77%
core/pipeline/breakpoint.py 5 89.89%
components/generators/chat/openai.py 6 96.76%
core/errors.py 7 89.06%
Totals Coverage Status
Change from base Build 18340248289: 0.1%
Covered Lines: 13346
Relevant Lines: 14475

💛 - Coveralls

@vblagoje vblagoje marked this pull request as ready for review October 10, 2025 08:08
@vblagoje vblagoje requested a review from a team as a code owner October 10, 2025 08:08
@vblagoje vblagoje requested review from Amnah199 and removed request for a team October 10, 2025 08:08
- Any other exception
"""

def __init__(self, generators: list[ChatGenerator]):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for consistency with our other components that accept a ChatGenerator let's call this chat_generators

Suggested change
def __init__(self, generators: list[ChatGenerator]):
def __init__(self, chat_generators: list[ChatGenerator]):

@sjrl
Copy link
Contributor

sjrl commented Oct 10, 2025

@vblagoje looking very clean! I just had a few comments, I'll let @Amnah199 finish the review :)

@vblagoje
Copy link
Member Author

Thank you @sjrl I'll fix these now!

Copy link
Contributor

@Amnah199 Amnah199 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks very good overall. Some minor comments, plus maybe we need to update pydocs, generators_api.yml to add the new chat generator.

Comment on lines +118 to +124
def run(
self,
messages: list[ChatMessage],
generation_kwargs: Union[dict[str, Any], None] = None,
tools: Union[list[Tool], Toolset, None] = None,
streaming_callback: Union[StreamingCallbackT, None] = None,
) -> dict[str, Any]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets explain these parameters and return instance in docstrings.

@Amnah199
Copy link
Contributor

@vblagoje lets just add consistent docstrings for run methods etc. Otherwise looks good to merge!

Copy link
Contributor

@Amnah199 Amnah199 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Only docstrings of to_dict and from_dict can be proved. Buts its your call.

@vblagoje vblagoje merged commit 90edcda into main Oct 17, 2025
21 checks passed
@vblagoje vblagoje deleted the fb_chat branch October 17, 2025 13:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

topic:tests type:documentation Improvements on the docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants