Skip to content

truncate parameter ignored for OpenAI chat_completions endpoint #1765

@calycekr

Description

@calycekr

Bug description

The truncate parameter in the ChatUI configuration is not being applied when using the OpenAI chat_completions endpoint.

Root Cause

The issue arises because the chat_completions endpoint does not utilize the buildPrompt function where the truncate parameter is handled. The logic for truncation is solely within buildPrompt and is therefore bypassed entirely when processing chat_completions requests. This means there's no truncation mechanism applied to the chat history before it's sent to vllm-openai or OpenAI.

#1654

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions