| meta | content | tags | dates | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
chat-completions-api |
|
Structured outputs allow users to get consistent, machine-readable JSON format responses from language models. JSON, as a widely-used format, enables seamless integration with a variety of platforms and applications. Its interoperability is crucial for developers aiming to incorporate AI functionality into their current systems with minimal adjustments.
By specifying a response format when using the Chat Completions API, you can ensure that responses are returned in a JSON structure. There are two main modes for generating JSON: Object Mode (schemaless) and Schema Mode (deterministic, structured output).
There are several ways to interact with language models:
- The Scaleway console provides a complete playground, aiming to test models, adapt parameters, and observe how these changes affect the output in real-time.
- Via the Chat API
- A Scaleway account logged into the console
- Owner status or IAM permissions allowing you to perform actions in the intended Organization
- A valid API key for API authentication
- Python 3.7+ installed on your system
-
Structured outputs (schema mode):
- Type
{"type": "json_schema"} - This mode enforces a strict schema format, where the output adheres to the predefined structure.
- Supports complex types and validation mechanisms as per the JSON schema specification, including nested schemas composition (
anyOf,allOf,oneOfetc),$ref,alltypes, and regular expressions.
- Type
-
JSON mode (Legacy method):
- Type:
{"type": "json_object"} - This mode is non-deterministic and allows the model to output a JSON object without strict validation.
- Useful for flexible outputs when you expect the model to infer a reasonable structure based on your prompt.
- JSON mode is older and has been used by developers since early API implementations, but lacks reliability in response formats.
- Type:
The following Python examples demonstrate how to use Structured outputs to generate structured responses.
We are using the base code below to send our LLM a voice note transcript to structure:
import json
from openai import OpenAI
from pydantic import BaseModel, Field
# Set your preferred model
MODEL = "llama-3.1-8b-instruct"
# Set your API key
API_KEY = "<SCW_API_KEY>"
client = OpenAI(
base_url="https://api.scaleway.ai/v1",
api_key=API_KEY,
)
# Define the schema for the output using Pydantic
class VoiceNote(BaseModel):
title: str = Field(description="A title for the voice note")
summary: str = Field(description="A short one sentence summary of the voice note.")
actionItems: list[str] = Field(description="A list of action items from the voice note")
# Transcript to use for the output
TRANSCRIPT = (
"Good evening! It's 6:30 PM, and I'm just getting home from work. I have a few things to do "
"before I can relax. First, I'll need to water the plants in the garden since they've been in the sun all day. "
"Then, I'll start preparing dinner. I think a simple pasta dish with some garlic bread should be good. "
"While that's cooking, I'll catch up on a couple of phone calls I missed earlier."
)Using Pydantic, users can define the schema as a Python class and enforce the model to return results adhering to this schema.
extract = client.chat.completions.create(
messages=[
{
"role": "system",
"content": "The following is a voice message transcript. Only answer in JSON using '{' as the first character.",
},
{
"role": "user",
"content": TRANSCRIPT,
},
],
model=MODEL,
response_format={
"type": "json_schema",
"json_schema": {
"name": "VoiceNote",
"schema": VoiceNote.model_json_schema(),
}
},
)
output = json.loads(extract.choices[0].message.content)
print(json.dumps(output, indent=2))Output example:
{
"title": "To-Do List",
"summary": "Returning from work, need to complete tasks before relaxing",
"actionItems": [
"Water garden",
"Prepare dinner: pasta dish with garlic bread",
"Catch up on missed phone calls"
]
}Alternatively, users can manually define the JSON schema inline when calling the model.
extract = client.chat.completions.create(
messages=[
{
"role": "system",
"content": "The following is a voice message transcript. Only answer in JSON using '{' as the first character.",
},
{
"role": "user",
"content": TRANSCRIPT,
},
],
model=MODEL,
response_format={
"type": "json_schema",
"json_schema": {
"name": "VoiceNote",
"schema": {
"type": "object",
"properties": {
"title": {"type": "string"},
"summary": {"type": "string"},
"actionItems": {
"type": "array",
"items": {"type": "string"}
}
},
"additionalProperties": False,
"required": ["title", "summary", "actionItems"]
}
}
}
)
output = json.loads(extract.choices[0].message.content)
print(json.dumps(output, indent=2))Output example:
{
"title": "Evening Routine",
"actionItems": [
"Water the plants",
"Cook dinner (pasta and garlic bread)",
"Make phone calls"
],
"summary": "Made a list of tasks to accomplish before relaxing tonight"
}In JSON mode, you can prompt the model to output a JSON object without enforcing a strict schema.
extract = client.chat.completions.create(
messages=[
{
"role": "system",
"content": "The following is a voice message transcript. Only answer in JSON using '{' as the first character.",
},
{
"role": "user",
"content": TRANSCRIPT,
},
],
model=MODEL,
response_format={
"type": "json_object",
},
)
output = json.loads(extract.choices[0].message.content)
print(json.dumps(output, indent=2))Output example:
{
"current_time": "6:30 PM",
"tasks": [
{
"task": "water the plants in the garden",
"priority": "high"
},
{
"task": "prepare dinner (pasta with garlic bread)",
"priority": "high"
},
{
"task": "catch up on phone calls",
"priority": "medium"
}
]
}Using structured outputs with LLMs can significantly improve their reliability, especially to implement agentic use cases.
- Structured outputs provide strict adherence to a predefined schema, ensuring consistency.
- JSON mode (Legacy Method) is flexible but less predictable.
We recommend using structured outputs (json_schema) for most use cases.