Skip to content

Conversation

@hudson-ai
Copy link
Collaborator

Example notebook using guidance to add constraints to GPT-OSS.

GPT-OSS is rather idiosyncratic and currently doesn't fit incredibly well into some of guidance's existing API, so this notebook uses lower-level llguidance grammars alongside openai-harmony and sglang as an inference backend.

@nking-1
Copy link
Collaborator

nking-1 commented Aug 22, 2025

This is very helpful. I would suggest adding some introductory prose about what the notebook covers (harmony format examples, tool use, lark grammar) as well as requirements (active GPT-OSS deployment). I thought it was pretty easy to follow along otherwise.

A few other ideas:

  • Add an example of running the model locally with llamacpp if possible.
  • Add a few sentences explaining that the ll_grammar() function exports lark format and that it's useful here because the OpenAI client directly supports sending it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants