Skip to content

Commit e4c1e5e

Browse files
chg0901chg0901
authored
update daddy-like bf in readme with gradio web demo (#240)
2 parents 91ec1b5 + 03c0668 commit e4c1e5e

File tree

5 files changed

+184
-3
lines changed

5 files changed

+184
-3
lines changed

‎README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -98,6 +98,8 @@
9898
</table>
9999

100100
## 🎇最近更新
101+
102+
- 【2024.05.08】EmoLLM**爹系男友阅览体验版**上线 [1. **百度AppBuilder**](https://appbuilder.baidu.com/s/4cLyw) [2. **OpenXLab**](https://openxlab.org.cn/apps/detail/chg0901/EmoLLM3.0_Gradio_Llama3-8B-Instruct3.0), 欢迎点赞收藏
101103
- 【2024.05.07】[增量预训练指南](xtuner_config/pt/README.md)
102104
- 【2024.05.04】基于LLaMA3_8b_instruct的[EmoLLM3.0 OpenXLab Demo](https://st-app-center-006861-9746-jlroxvg.openxlab.space/)上线（[重启链接](https://openxlab.org.cn/apps/detail/chg0901/EmoLLM-Llama3-8B-Instruct3.0)）, [**LLAMA3微调指南**](xtuner_config/README_llama3_8b_instruct_qlora_alpaca_e3_M.md)**更新**，在[**OpenXLab**](https://openxlab.org.cn/models/detail/chg0901/EmoLLM-Llama3-8B-Instruct3.0)和[**ModelScope**](https://modelscope.cn/models/chg0901/EmoLLM-Llama3-8B-Instruct3.0/summary)平台发布**LLaMA3_8b_instruct-8B QLoRA微调模型 EmoLLM3.0权重**
103105
- 【2024.04.20】[LLAMA3微调指南](xtuner_config/README_llama3_8b_instruct_qlora_alpaca_e3_M.md)及基于[LLaMA3_8b_instruct的艾薇](https://openxlab.org.cn/models/detail/ajupyter/EmoLLM-LLaMA3_8b_instruct_aiwei)开源

‎README_EN.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -101,7 +101,9 @@ The Model aims to fully understand and promote the mental health of individuals,
101101
</table>
102102

103103
## Recent Updates
104-
- [2024.05.07][Incremental Pre-training Guide](xtuner_config/pt/README.md)
104+
105+
- [2024.05.08] EmoLLM**Daddy-like BF V0.1** is public now in [1. **Baidu AppBuilder**](https://appbuilder.baidu.com/s/4cLyw) and [2. **OpenXLab**](https://openxlab.org.cn/apps/detail/chg0901/EmoLLM3.0_Gradio_Llama3-8B-Instruct3.0), welcome to like and add it to your collections!
106+
- [2024.05.07] [Incremental Pre-training Guide](xtuner_config/pt/README.md)
105107
- [2024.05.04] [EmoLLM3.0 OpenXLab Demo](https://st-app-center-006861-9746-jlroxvg.openxlab.space/) based on LLaMA3_8b_instruct is available now ([restart link]((https://openxlab.org.cn/apps/detail/chg0901/EmoLLM-Llama3-8B-Instruct3.0))), [LLAMA3 fine-tuning guide](xtuner_config/README_llama3_8b_instruct_qlora_alpaca_e3_M.md) is updated, LLaMA3_8b_instruct-8B QLoRA fine-tuning model EmoLLM3.0 weights are released on [**OpenXLab**](https://openxlab.org.cn/models/detail/chg0901/EmoLLM-Llama3-8B-Instruct3.0) and [**ModelScope**](https://modelscope.cn/models/chg0901/EmoLLM-Llama3-8B-Instruct3.0/summary) platforms
106108
- [2024.04.20] [LLAMA3 fine-tuning guide](xtuner_config/README_llama3_8b_instruct_qlora_alpaca_e3_M.md) and based on [LLaMA3_8b_instruct's aiwei](https://openxlab.org.cn/models/detail/ajupyter/EmoLLM-LLaMA3_8b_instruct_aiwei) open source
107109
- [2023.04.14] Added [Quick Start](docs/quick_start_EN.md) and Nanny level tutorial [BabyEmoLLM](Baby_EmoLLM.ipynb)

‎app_Llama3_Gradio.py

Lines changed: 177 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,177 @@
1+
import gradio as gr
2+
import os
3+
import torch
4+
from transformers import GemmaTokenizer, AutoModelForCausalLM
5+
from transformers import AutoModelForCausalLM, AutoTokenizer, TextIteratorStreamer
6+
from threading import Thread
7+
8+
9+
DESCRIPTION = '''
10+
<div>
11+
<h1 style="text-align: center;">EmoLLM Llama3 心理咨询室 V4.0</h1>
12+
13+
<p align="center">
14+
<a href="https://github.com/SmartFlowAI/EmoLLM/">
15+
<img src="https://st-app-center-006861-9746-jlroxvg.openxlab.space/media/cda6c1a05dc8ba5b19ad3e7a24920fdf3750c917751202385a6dbc51.png" alt="Logo" width="20%">
16+
</a>
17+
</p>
18+
19+
<div align="center">
20+
21+
<!-- PROJECT SHIELDS -->
22+
[![OpenXLab_Model][OpenXLab_Model-image]][OpenXLab_Model-url]
23+
24+
<h2 style="text-align: center;"> EmoLLM是一系列能够支持 理解用户-支持用户-帮助用户 心理健康辅导链路的 心理健康大模型 ，欢迎大家star~⭐⭐</h2>
25+
<p>https://github.com/SmartFlowAI/EmoLLM</p>
26+
</div>
27+
28+
</div>
29+
30+
[OpenXLab_Model-image]: https://cdn-static.openxlab.org.cn/header/openxlab_models.svg
31+
[OpenXLab_Model-url]: https://openxlab.org.cn/models/detail/chg0901/EmoLLM-Llama3-8B-Instruct3.0
32+
33+
'''
34+
35+
LICENSE = """
36+
<p align="center"> Built with Meta Llama 3 </>
37+
"""
38+
39+
PLACEHOLDER = """
40+
<div style="padding: 30px; text-align: center; display: flex; flex-direction: column; align-items: center;">
41+
42+
</div>
43+
"""
44+
45+
46+
css = """
47+
h1 {
48+
text-align: center;
49+
display: block;
50+
}
51+
<!--
52+
#duplicate-button {
53+
margin: auto;
54+
color: white;
55+
background: #1565c0;
56+
border-radius: 100vh;
57+
}
58+
-->
59+
"""
60+
61+
# download internlm2 to the base_path directory using git tool
62+
base_path = './EmoLLM-Llama3-8B-Instruct3.0'
63+
os.system(f'git clone https://code.openxlab.org.cn/chg0901/EmoLLM-Llama3-8B-Instruct3.0.git {base_path}')
64+
os.system(f'cd {base_path} && git lfs pull')
65+
66+
67+
# Load the tokenizer and model
68+
tokenizer = AutoTokenizer.from_pretrained(base_path,trust_remote_code=True)
69+
model = AutoModelForCausalLM.from_pretrained(base_path,trust_remote_code=True, device_map="auto", torch_dtype=torch.float16).eval() # to("cuda:0")
70+
terminators = [
71+
tokenizer.eos_token_id,
72+
tokenizer.convert_tokens_to_ids("<|eot_id|>")
73+
]
74+
75+
def chat_llama3_8b(message: str,
76+
history: list,
77+
temperature: float,
78+
max_new_tokens: int,
79+
top_p: float
80+
) -> str:
81+
"""
82+
Generate a streaming response using the llama3-8b model.
83+
Args:
84+
message (str): The input message.
85+
history (list): The conversation history used by ChatInterface.
86+
temperature (float): The temperature for generating the response.
87+
max_new_tokens (int): The maximum number of new tokens to generate.
88+
Returns:
89+
str: The generated response.
90+
"""
91+
conversation = []
92+
93+
for user, assistant in history:
94+
conversation.extend([{"role": "user", "content": user}, {"role": "assistant", "content": assistant}])
95+
conversation.append({"role": "user", "content": message})
96+
97+
input_ids = tokenizer.apply_chat_template(conversation, return_tensors="pt").to(model.device)
98+
99+
streamer = TextIteratorStreamer(tokenizer, timeout=10.0, skip_prompt=True, skip_special_tokens=True)
100+
101+
generate_kwargs = dict(
102+
input_ids= input_ids,
103+
streamer=streamer,
104+
max_new_tokens=max_new_tokens,
105+
do_sample=True,
106+
temperature=temperature,
107+
top_p = top_p,
108+
eos_token_id=terminators,
109+
)
110+
# This will enforce greedy generation (do_sample=False) when the temperature is passed 0, avoiding the crash.
111+
if temperature == 0:
112+
generate_kwargs['do_sample'] = False
113+
114+
t = Thread(target=model.generate, kwargs=generate_kwargs)
115+
t.start()
116+
117+
outputs = []
118+
for text in streamer:
119+
outputs.append(text)
120+
yield "".join(outputs)
121+
122+
123+
124+
# Gradio block
125+
chatbot=gr.Chatbot(height=450, placeholder=PLACEHOLDER, label='EmoLLM Chat')
126+
127+
with gr.Blocks(fill_height=True, css=css) as demo:
128+
129+
gr.Markdown(DESCRIPTION)
130+
# gr.DuplicateButton(value="Duplicate Space for private use", elem_id="duplicate-button")
131+
gr.ChatInterface(
132+
fn=chat_llama3_8b,
133+
chatbot=chatbot,
134+
fill_height=True,
135+
additional_inputs_accordion=gr.Accordion(label="⚙️ Parameters", open=False, render=False),
136+
additional_inputs=[
137+
gr.Slider(minimum=0,
138+
maximum=1,
139+
step=0.1,
140+
value=0.95,
141+
label="Temperature",
142+
render=False),
143+
gr.Slider(minimum=128,
144+
maximum=4096,
145+
step=1,
146+
value=4096,
147+
label="Max new tokens",
148+
render=False ),
149+
gr.Slider(minimum=0.0,
150+
maximum=1,
151+
step=0.01,
152+
value=0.8,
153+
label="Top P",
154+
render=False ),
155+
# gr.Slider(minimum=128,
156+
# maximum=4096,
157+
# step=1,
158+
# value=512,
159+
# label="Max new tokens",
160+
# render=False ),
161+
],
162+
examples=[
163+
['请介绍你自己。'],
164+
['我觉得我在学校的学习压力好大啊，虽然我真的很喜欢我的专业，但最近总是担心自己无法达到自己的期望，这让我有点焦虑。'],
165+
['我最近总觉得自己在感情上陷入了困境，我喜欢上了我的朋友，但又害怕表达出来会破坏我们现在的关系...'],
166+
['我感觉自己像是被困在一个无尽的循环中。每天醒来都感到身体沉重，对日常活动提不起兴趣，工作、锻炼甚至是我曾经喜欢的事物都让我觉得厌倦'],
167+
['最近工作压力特别大，还有一些家庭矛盾']
168+
],
169+
cache_examples=False,
170+
)
171+
172+
gr.Markdown(LICENSE)
173+
174+
if __name__ == "__main__":
175+
demo.launch()
176+
177+

‎app_web_demo-Llama3.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -207,7 +207,7 @@ def prepare_generation_config():
207207
def combine_history(prompt):
208208
messages = st.session_state.messages
209209
meta_instruction = (
210-
"你是心理健康助手EmoLLM, 由EmoLLM团队打造, 是一个研究过无数具有心理健康问题的病人与心理健康医生对话的心理专家, 在心理方面拥有广博的知识储备和丰富的研究咨询经验。你旨在通过专业心理咨询, 协助来访者完成心理诊断。请充分利用专业心理学知识与咨询技术, 一步步帮助来访者解决心理问题。\n\n"
210+
"你是心理健康助手EmoLLM, 由EmoLLM团队打造。\n\n"
211211
)
212212
total_prompt =f"<|start_header_id|>system<|end_header_id|>\n\n{meta_instruction}<|eot_id|>\n\n"
213213
for message in messages:

‎web_demo-Llama3.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -205,7 +205,7 @@ def prepare_generation_config():
205205
def combine_history(prompt):
206206
messages = st.session_state.messages
207207
meta_instruction = (
208-
"你是心理健康助手EmoLLM, 由EmoLLM团队打造, 是一个研究过无数具有心理健康问题的病人与心理健康医生对话的心理专家, 在心理方面拥有广博的知识储备和丰富的研究咨询经验。你旨在通过专业心理咨询, 协助来访者完成心理诊断。请充分利用专业心理学知识与咨询技术, 一步步帮助来访者解决心理问题。\n\n"
208+
"你是心理健康助手EmoLLM, 由EmoLLM团队打造\n\n"
209209
)
210210
total_prompt =f"<|start_header_id|>system<|end_header_id|>\n\n{meta_instruction}<|eot_id|>\n\n"
211211
for message in messages:

0 commit comments

Comments
 (0)