-
Notifications
You must be signed in to change notification settings - Fork 6k
Closed
Labels
Description
chat_box.ai_say("正在思考...")
text = ""
started = False
client = openai.Client(base_url=f"{api_address()}/chat", api_key="NONE")
extra_body = dict(
temperature=temperature,
prompt_name=prompt_name,
)
stream = True
params = dict(
messages=messages,
model=llm_model,
stream=stream, # TODO:xinference qwen-vl-chat 流式输出会出错,后续看更新
extra_body=extra_body,
)
if stream:
try:
for d in client.chat.completions.create(**params):
# import rich
# rich.print(d)
# print(d)
message_id = d.message_id
metadata = {
"message_id": message_id,
}
# clear initial message
if not started:
chat_box.update_msg("", streaming=False)
这里的prompt_name是是来自prompt_settings.yaml中,大模型返回的信息,与模版没关系,这是什么问题