|
1 | 1 | # XiaoZhi AI Chatbot
|
2 | 2 |
|
3 |
| -([中文](README.md) | English | [日本語](README_ja.md)) |
| 3 | +([中文](README.md) | English | [日本語](README_ja.md)) |
4 | 4 |
|
5 | 5 | This is Terrence's first hardware project.
|
6 | 6 |
|
7 |
| -👉 [Build your AI chat companion with ESP32+SenseVoice+Qwen72B! [bilibili]](https://www.bilibili.com/video/BV11msTenEH3/?share_source=copy_web&vd_source=ee1aafe19d6e60cf22e60a93881faeba) |
| 7 | +👉 [Build your AI chat companion with ESP32+SenseVoice+Qwen72B!【bilibili】](https://www.bilibili.com/video/BV11msTenEH3/) |
8 | 8 |
|
9 |
| -👉 [DIY Your AI Companion - Beginner's Tutorial [bilibili]](https://www.bilibili.com/video/BV1XnmFYLEJN/) |
| 9 | +👉 [Equipping XiaoZhi with DeepSeek's smart brain【bilibili】](https://www.bilibili.com/video/BV1GQP6eNEFG/) |
| 10 | + |
| 11 | +👉 [Build your own AI companion, a beginner's guide【bilibili】](https://www.bilibili.com/video/BV1XnmFYLEJN/) |
10 | 12 |
|
11 | 13 | ## Project Purpose
|
12 | 14 |
|
13 |
| -This project is developed based on Espressif's ESP-IDF. |
| 15 | +This is an open-source project released under the MIT license, allowing anyone to use it freely, including for commercial purposes. |
14 | 16 |
|
15 |
| -This is an open-source project primarily for educational purposes. Through this project, we aim to help more people get started with AI hardware development and understand how to integrate rapidly evolving large language models into actual hardware devices. Whether you're a student interested in AI or a developer looking to explore new technologies, this project offers valuable learning experiences. |
| 17 | +Through this project, we aim to help more people get started with AI hardware development and understand how to implement rapidly evolving large language models in actual hardware devices. Whether you're a student interested in AI or a developer exploring new technologies, this project offers valuable learning experiences. |
16 | 18 |
|
17 |
| -Everyone is welcome to participate in the project's development and improvement. If you have any ideas or suggestions, please feel free to raise an Issue or join our chat group. |
| 19 | +Everyone is welcome to participate in the project's development and improvement. If you have any ideas or suggestions, please feel free to raise an Issue or join the chat group. |
18 | 20 |
|
19 | 21 | Learning & Discussion QQ Group: 946599635
|
20 | 22 |
|
21 | 23 | ## Implemented Features
|
22 | 24 |
|
23 | 25 | - Wi-Fi / ML307 Cat.1 4G
|
24 |
| -- BOOT button wake-up and interrupt, supporting both click and long-press triggers |
| 26 | +- BOOT button wake-up and interruption, supporting both click and long-press triggers |
25 | 27 | - Offline voice wake-up [ESP-SR](https://github.com/espressif/esp-sr)
|
26 | 28 | - Streaming voice dialogue (WebSocket or UDP protocol)
|
27 | 29 | - Support for 5 languages: Mandarin, Cantonese, English, Japanese, Korean [SenseVoice](https://github.com/FunAudioLLM/SenseVoice)
|
28 | 30 | - Voice print recognition to identify who's calling AI's name [3D Speaker](https://github.com/modelscope/3D-Speaker)
|
29 |
| -- Large model TTS (Volcengine or CosyVoice) |
30 |
| -- Large Language Model (Qwen2.5 72B or Doubao API) |
| 31 | +- Large model TTS (Volcano Engine or CosyVoice) |
| 32 | +- Large Language Models (Qwen, DeepSeek, Doubao) |
31 | 33 | - Configurable prompts and voice tones (custom characters)
|
32 |
| -- Short-term memory with self-summary after each conversation round |
| 34 | +- Short-term memory, self-summarizing after each conversation round |
33 | 35 | - OLED / LCD display showing signal strength or conversation content
|
34 |
| -- Support for displaying emoji images on LCD |
| 36 | +- Support for LCD image expressions |
| 37 | +- Multi-language support (Chinese, English) |
35 | 38 |
|
36 | 39 | ## Hardware Section
|
37 | 40 |
|
38 |
| -### Breadboard Practice |
| 41 | +### Breadboard DIY Practice |
39 | 42 |
|
40 |
| -For detailed tutorial, see the Feishu document: |
| 43 | +See the Feishu document tutorial: |
41 | 44 |
|
42 | 45 | 👉 [XiaoZhi AI Chatbot Encyclopedia](https://ccnphfhqs21z.feishu.cn/wiki/F5krwD16viZoF0kKkvDcrZNYnhb?from=from_copylink)
|
43 | 46 |
|
44 |
| -Breadboard setup shown below: |
| 47 | +Breadboard demonstration: |
45 | 48 |
|
46 |
| - |
| 49 | + |
47 | 50 |
|
48 |
| -### Supported Open-Source Hardware |
| 51 | +### Supported Open Source Hardware |
49 | 52 |
|
50 | 53 | - <a href="https://oshwhub.com/li-chuang-kai-fa-ban/li-chuang-shi-zhan-pai-esp32-s3-kai-fa-ban" target="_blank" title="LiChuang ESP32-S3 Development Board">LiChuang ESP32-S3 Development Board</a>
|
51 | 54 | - <a href="https://github.com/espressif/esp-box" target="_blank" title="Espressif ESP32-S3-BOX3">Espressif ESP32-S3-BOX3</a>
|
52 | 55 | - <a href="https://docs.m5stack.com/zh_CN/core/CoreS3" target="_blank" title="M5Stack CoreS3">M5Stack CoreS3</a>
|
53 | 56 | - <a href="https://docs.m5stack.com/en/atom/Atomic%20Echo%20Base" target="_blank" title="AtomS3R + Echo Base">AtomS3R + Echo Base</a>
|
54 | 57 | - <a href="https://docs.m5stack.com/en/core/ATOM%20Matrix" target="_blank" title="AtomMatrix + Echo Base">AtomMatrix + Echo Base</a>
|
55 |
| -- <a href="https://gf.bilibili.com/item/detail/1108782064" target="_blank" title="MagiClick 2.4">MagiClick 2.4</a> |
| 58 | +- <a href="https://gf.bilibili.com/item/detail/1108782064" target="_blank" title="Magic Button 2.4">Magic Button 2.4</a> |
56 | 59 | - <a href="https://www.waveshare.net/shop/ESP32-S3-Touch-AMOLED-1.8.htm" target="_blank" title="Waveshare ESP32-S3-Touch-AMOLED-1.8">Waveshare ESP32-S3-Touch-AMOLED-1.8</a>
|
57 | 60 | - <a href="https://github.com/Xinyuan-LilyGO/T-Circle-S3" target="_blank" title="LILYGO T-Circle-S3">LILYGO T-Circle-S3</a>
|
58 |
| -- <a href="https://oshwhub.com/tenclass01/xmini_c3" target="_blank" title="Xmini C3">Xmini C3</a> |
59 |
| -- <a href="https://oshwhub.com/movecall/moji-xiaozhi-ai-derivative-editi" target="_blank" title="Movecall Moji ESP32S3">Movecall Moji ESP32S3</a> |
| 61 | +- <a href="https://oshwhub.com/tenclass01/xmini_c3" target="_blank" title="XiaGe Mini C3">XiaGe Mini C3</a> |
| 62 | +- <a href="https://oshwhub.com/movecall/moji-xiaozhi-ai-derivative-editi" target="_blank" title="Movecall Moji ESP32S3">Moji XiaoZhi AI Derivative Version</a> |
60 | 63 |
|
61 | 64 | <div style="display: flex; justify-content: space-between;">
|
62 | 65 | <a href="docs/v1/lichuang-s3.jpg" target="_blank" title="LiChuang ESP32-S3 Development Board">
|
@@ -95,21 +98,30 @@ Breadboard setup shown below:
|
95 | 98 |
|
96 | 99 | ### Flashing Without Development Environment
|
97 | 100 |
|
98 |
| -For beginners, it's recommended to first try flashing the firmware without setting up a development environment. The firmware uses a test server provided by the author, currently available for free use (not for commercial purposes). |
| 101 | +For beginners, it's recommended to first use the firmware that can be flashed without setting up a development environment. |
| 102 | + |
| 103 | +The firmware connects to the official [xiaozhi.me](https://xiaozhi.me) server by default. Currently, personal users can register an account to use the Qwen real-time model for free. |
99 | 104 |
|
100 |
| -👉 [Flash Firmware Guide (No IDF Environment Required)](https://ccnphfhqs21z.feishu.cn/wiki/Zpz4wXBtdimBrLk25WdcXzxcnNS) |
| 105 | +👉 [Flash Firmware Guide (No IDF Environment)](https://ccnphfhqs21z.feishu.cn/wiki/Zpz4wXBtdimBrLk25WdcXzxcnNS) |
101 | 106 |
|
102 | 107 | ### Development Environment
|
103 | 108 |
|
104 | 109 | - Cursor or VSCode
|
105 | 110 | - Install ESP-IDF plugin, select SDK version 5.3 or above
|
106 | 111 | - Linux is preferred over Windows for faster compilation and fewer driver issues
|
| 112 | +- Use Google C++ code style, ensure compliance when submitting code |
| 113 | + |
| 114 | +## AI Agent Configuration |
| 115 | + |
| 116 | +If you already have a XiaoZhi AI chatbot device, you can configure it through the [xiaozhi.me](https://xiaozhi.me) console. |
| 117 | + |
| 118 | +👉 [Backend Operation Tutorial (Old Interface)](https://www.bilibili.com/video/BV1jUCUY2EKM/) |
107 | 119 |
|
108 |
| -## AI Character Configuration |
| 120 | +## Technical Principles and Private Deployment |
109 | 121 |
|
110 |
| -If you already have a XiaoZhi AI chatbot, please refer to 👉 [Backend Operation Video Tutorial](https://www.bilibili.com/video/BV1jUCUY2EKM/) |
| 122 | +👉 [Detailed WebSocket Communication Protocol Documentation](docs/websocket.md) |
111 | 123 |
|
112 |
| -For detailed usage instructions and test server notes, please refer to 👉 [XiaoZhi Test Server Help Guide](https://xiaozhi.me/help). |
| 124 | +For server deployment on personal computers, refer to another MIT-licensed project [xiaozhi-esp32-server](https://github.com/xinnan-tech/xiaozhi-esp32-server) |
113 | 125 |
|
114 | 126 | ## Star History
|
115 | 127 |
|
|
0 commit comments