Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add 'json_schema' alias to GrammarType.Json #2982

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

aW3st
Copy link
Contributor

@aW3st aW3st commented Jan 31, 2025

Adds a serde alias to GrammarType.Json such that {"type": "json_schema", "value": json_schema} deserializes correctly.

This brings the OpenAI-compatible API closer to OpenAI's official spec and makes the server more interoperable with other OpenAI-compatible servers.

Eventually it would be nice to be able to use beta.chat.completions.parse method on OpenAI's python client.

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline,
    Pull Request section?
  • Was this discussed/approved via a Github issue or the forum? Please add a link
    to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the
    documentation guidelines, and
    here are tips on formatting docstrings.
  • Did you write any new necessary tests?

Who can review?

@OlivierDehaene OR @Narsil

aW3st added 2 commits January 30, 2025 14:03

Verified

This commit was signed with the committer’s verified signature.
hubertdeng123 Hubert Deng
@drbh drbh mentioned this pull request Feb 20, 2025
@Narsil
Copy link
Collaborator

Narsil commented Mar 10, 2025

Hi @aW3st,

Thanks a lot for the PR. We're unlikely to adhere to anything beta from OpenAI, or even everything that OpenAI supports.
The OpenAI adaptation layer exists for simple, convenient ubiquitous tool support from client libraries (most of which have a OpenAI compatibility layer with various degrees of support).

We intend to support those very common use cases and be as easy as possible to be consumed. However, the intent of TGI is to support the many various different open source models, which sometimes operate very differently than OpenAI's. Therefore adhering to exact OpenAI spec can become quite difficult, especially for the most advanced use cases.

That being said json_schema should probably be supported. However I don't think json_schema is equivalent to the current json, we should probably upgrade outlines which now has support for both.

Pinging @drbh in case I said anything wrong here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants