Skip to content

Conversation

@yinmin2020
Copy link

What type of PR is this?
docs(website): add blog post about semantic routing with VLLM and Milvus

What this PR does / why we need it:
This PR adds a new blog post that introduces how to build a semantic routing layer using VLLM and Milvus. The article covers:

  • The motivation behind semantic routing
  • Architecture overview with mixture of models
  • Step-by-step guide to set up the system using Docker Compose
  • How semantic caching can improve performance and reduce costs
  • Integration with Milvus for vector storage and retrieval

This content helps users understand advanced use cases of the project and demonstrates practical integration with popular LLM and vector database tools.

Which issue(s) this PR fixes:
None

Release Notes: No

@yinmin2020 yinmin2020 requested a review from Xunzhuo as a code owner October 31, 2025 02:09
@netlify
Copy link

netlify bot commented Oct 31, 2025

Deploy Preview for vllm-semantic-router failed.

Name Link
🔨 Latest commit e77c27d
🔍 Latest deploy log https://app.netlify.com/projects/vllm-semantic-router/deploys/6904ac7573e15800071af78b

@github-actions
Copy link

github-actions bot commented Oct 31, 2025

👥 vLLM Semantic Team Notification

The following members have been identified for the changed files in this PR and have been automatically assigned:

📁 website

Owners: @Xunzhuo, @rootfs, @yuluo-yx
Files changed:

  • website/blog/2025-10-30-milvus.md
  • website/static/img/docker-compose-up.png
  • website/static/img/docker-compose.png
  • website/static/img/download-models.png
  • website/static/img/mixture-of-models.png
  • website/static/img/semantic-caching-challenge.png
  • website/static/img/semantic-routing-layer.png
  • website/static/img/traditional-caching.png

vLLM

🎉 Thanks for your contributions!

This comment was automatically generated based on the OWNER files in the repository.

@rootfs
Copy link
Collaborator

rootfs commented Oct 31, 2025

@yinmin2020 thank you for the contribution! can you sign the DCO and fix the CI issues?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants