feat: Add GitHub integration with agent_prompts and github_components

[julian-risch]

Related Issues

• Related to Add GitHub integration with agent_prompts and github_components #1650

• Related to Move Agent from experimental to main haystack-experimental#250
  @sjrl brought up that one way to keep example files from the experimental Agent around is an integration chore: Remove Agent after Haystack 2.12 release haystack-experimental#263 (comment)

Proposed Changes:

• Move github_components from experimental to a new integration
• Move agent_prompts from experimental to a new integration
• Add tools that wrap the new components

The idea is to enable users to run the example notebook (or a version with updated imports) after having installed this new integration.

How did you test it?

New unit tests and I ran all usage examples successfully with a test repo.

I have tested it with the updated notebook. I'll update the cookbook PR once the integration is released: deepset-ai/haystack-cookbook#183

Notes for the reviewer

• I suggest we rename github_token parameter to api_key for consistency with many other integrations.
• While we could find a way to set up integration tests, I would rather leave them out of this PR.
• GithubRepositoryViewer has a branch parameter in the run method, which could also be named ref to make more clear it can also be a tag or commit hash. I prefer keeping the parameter name branch.
• Some components have github_token: Optional[Secret] = None, because they can work without any token while others use Secret.from_env_var("GITHUB_TOKEN"). I suggest we use Secret.from_env_var("GITHUB_TOKEN", strict=False) where we currently have None as the default.
• The internal implementation of the components differs in how they use _get_headers or _get_request_headers or define headers inline. We could refactor that.

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• I added unit tests and updated the docstrings
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.

[sjrl]

Oh interesting I wasn't thinking to inherit from ComponentTool but do something like

@component
class GitHubFileEditor:
    ...

GitHubFileEditorTool = ComponentTool(GitHubFileEditor(), ...)

and then people could import the pre-made GitHubFileEditorTool but I can see how this version would be more customizable.

[sjrl]

If we go this route of inheriting from ComponentTool, we won't be able to use this to_dict method I think. At least we use different sede methods for Tools with this dict structure

{"type": generate_qualified_class_name(type(self)), "data": serialized}

so we'd probably need to follow that as well right?

Since when deserializing this in a pipeline I believe we will eventually call deserialize_tools_or_toolset_inplace. Do these methods work in that case?

[sjrl]

@julian-risch okay interesting that this works since you added the pipeline serialization test.

I still wonder to be consistent if we should not rename init_parameters to data in the serialized dict since that appears to be the pattern we use in our other tools and expect in deserialize_tools_or_toolset_inplace

[julian-risch]

@sjrl I added a test called test_pipeline_serialization, added _get_request_headers to all components and here is the example notebook with updated code up until the GitHub token is required. I commented out "message": {"source": "documents", "handler": message_handler}, because it didn't work for me and need to ask @mathislucka for advice.

https://colab.research.google.com/drive/1ktlwQ-CDLGDs2uZXvzgG8XspfjPidYqZ?usp=sharing

@sjrl If GitHubFileEditorTool looks good to you, I will add tools for all other components and probably update the directory structure a bit.

[sjrl]

@sjrl I added a test called test_pipeline_serialization, added _get_request_headers to all components and here is the example notebook with updated code up until the GitHub token is required. I commented out "message": {"source": "documents", "handler": message_handler}, because it didn't work for me and need to ask @mathislucka for advice.

@julian-risch This is related to the change we made to tools to have a new variable called outputs_to_string. So the google colab code should be updated to

    ...
    outputs_to_state={
        #"message": {"source": "documents", "handler": message_handler}, TODO
        "documents": {"source": "documents"},
    },
    outputs_to_string={"source": "documents", "handler": message_handler}
    ...

[julian-risch]

@sjrl Finally ready for another review! We're using "data" instead of "init_parameters" in serialization now and all newly implemented tools expose outputs_to_string, inputs_from_state, and outputs_to_state as init parameters.
I tested this by running the updated notebook. The Agent forked the repo and committed to a branch.

What do you think about the parameter name github_token? In almost every other place, we use api_key, so for consistency with the many other integrations, we could rename github_token to api_key or leave it as is.