Skip to content

Conversation

@relyt0925
Copy link

@relyt0925 relyt0925 commented Oct 21, 2025

This conforms chat completion new parameters to match the api docs and include the stream option
https://platform.openai.com/docs/api-reference/chat/create

@relyt0925 relyt0925 requested a review from a team as a code owner October 21, 2025 03:13
@relyt0925
Copy link
Author

We are also similar to others trying to create a base fully compliant open-ai "API proxy". Currently to mirror the full openai chat completion spec we have to embed the ChatCompletionNewParams struct in another struct and add the stream option to be able to get all key json data from the request. Would be nice if the struct itself would just contain all the api parameters with the proper json tags but not sure if there's a reason why that is not that way.

// CreateInferenceRequest
type CreateInferenceRequest struct {
	openai.ChatCompletionNewParams
	// need to just add a stream detection option to parameters to conform to spec
	Stream bool `json:"stream,omitempty"`
}

func (r *CreateInferenceRequest) MarshalJSON() (data []byte, err error) {
	data, err = r.ChatCompletionNewParams.MarshalJSON()
	if err != nil {
		return data, err
	}
	var parseStream map[string]interface{}
	err = json.Unmarshal(data, &parseStream)
	if err != nil {
		return data, err
	}
	if r.Stream {
		parseStream["stream"] = r.Stream
	}
	return json.Marshal(parseStream)
}

func (r *CreateInferenceRequest) UnmarshalJSON(data []byte) error {
	bufferClone := bytes.Clone(data)
	err := r.ChatCompletionNewParams.UnmarshalJSON(data)
	if err != nil {
		return err
	}
	var parseStream map[string]interface{}
	err = json.Unmarshal(bufferClone, &parseStream)
	if err != nil {
		return err
	}
	streamValue, ok := parseStream["stream"]
	if !ok {
		return nil
	}
	streamBoolVal, typeConvSuccessful := streamValue.(bool)
	if !typeConvSuccessful {
		return nil
	}
	r.Stream = streamBoolVal
	return nil
}

@Csrayz
Copy link

Csrayz commented Oct 29, 2025

The use case is the same as in #247 During unmarshalling, it was found that the stream parameter cannot be deserialized. Can this PR be merged?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants