Skip to content

Add links to new TPU recipes #84

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
Closed

Add links to new TPU recipes #84

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b09d9ef
Add a TPU recipes section, and link to qwen recipes
karan Oct 7, 2025
bc48999
Move TPU recipes to be within the Guides section
karan Oct 7, 2025
6af9c5d
Add llama recipes' links
karan Oct 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@ This repo intends to host community maintained common recipes to run vLLM answer
### Llama
- [Llama4-Scout](Llama/Llama4-Scout.md)
- [Llama3.3-70B](Llama/Llama3.3-70B.md)
- [Llama3.1-70B on TPU (Trillium)](https://github.com/AI-Hypercomputer/tpu-recipes/tree/main/inference/trillium/vLLM/Llama3.1)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm you can add them inside Llama/Llama3.3-70B.md ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Llama3.3-70B.md file targets GPUs. The second file's title indicates its distinct focus on TPU Trillium support.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Llama3.3-70B.md file should target at this model. TPU support should be listed in this file.

We don't want the flat directory to explode into {model} x {hardware} matrix.

Copy link
Author

@karan karan Oct 14, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TPU recipes are maintained in a separate repo and it's a lot of easier to keep them up to date there (the repo has other frameworks and hardware supported there too).

@simon-mo thoughts in @youkaichao's proposal of moving TPU links within the model file?

- [Llama3.1-8B on TPU (Trillium)](https://github.com/AI-Hypercomputer/tpu-recipes/tree/main/inference/trillium/vLLM/Llama3.1)

### OpenAI <img src="https://avatars.githubusercontent.com/u/14957082?v=4" alt="OpenAI" width="16" height="16" style="vertical-align:middle;">
- [gpt-oss](OpenAI/GPT-OSS.md)
Expand All @@ -35,7 +37,10 @@ This repo intends to host community maintained common recipes to run vLLM answer
- [Qwen3-VL](Qwen/Qwen3-VL.md)
- [Qwen3-Next](Qwen/Qwen3-Next.md)
- [Qwen3-Coder-480B-A35B](Qwen/Qwen3-Coder-480B-A35B.md)
- [Qwen3-32B on TPU (Trillium)](https://github.com/AI-Hypercomputer/tpu-recipes/tree/main/inference/trillium/vLLM/Qwen3)
- [Qwen3-4B on TPU (Trillium)](https://github.com/AI-Hypercomputer/tpu-recipes/tree/main/inference/trillium/vLLM/Qwen3)
- [Qwen2.5-VL](Qwen/Qwen2.5-VL.md)
- [Qwen2.5-VL on TPU (Trillium)](https://github.com/AI-Hypercomputer/tpu-recipes/tree/main/inference/trillium/vLLM/Qwen2.5-VL)

### Seed <img src="https://avatars.githubusercontent.com/u/4158466?s=200&v=4" alt="Seed" width="16" height="16" style="vertical-align:middle;">
- [Seed-OSS-36B](Seed/Seed-OSS-36B.md)
Expand Down