[Model] Upstream Deepseek-OCR model

[Isotr0py]

Purpose

• Fix [New Model]: deepseek-ai/DeepSeek-OCR upstream support #27236
• See if we can reuse deepseek-vl2's implementation and clip's implementation as much as possible

Test Plan

Test Result

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[mergify]

Documentation preview: https://vllm--27247.org.readthedocs.build/en/27247/

[Isotr0py]

The model should work now, but we really need a full clean for commented debug codes from upstream codebase everywhere. 😅

[ywang96]

The model should work now, but we really need a full clean for commented debug codes from upstream codebase everywhere. 😅

I'll clean it up - thanks for your work!

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[whx-sjtu]

Why can't we use DeepseekV2ForCausalLM in non-mla scenario? Because DeepseekV2ForCausalLM also supports non-mla:

vllm/vllm/model_executor/models/deepseek_v2.py

Line 1001 in 09a7e6f

if model_config.use_mla:

Currently DeepseekForCausalLM is hard coded to use triton version of fused_experts and fused_topk with all_reduce path of MoE implementation, witch fails to utilize SharedFusedMoE witch supports CustomOp. So if there are reasons that DeepseekForCausalLM must be used, I suggest upgrade it to reuse SharedFusedMoE too.

[ywang96]

I believe this was needed back then when our DSv2 implementation didn't support non-mla (you can see the same code in dsvl2)

vllm/vllm/model_executor/models/deepseek_vl2.py

Lines 403 to 409 in ab3e800

	if self.text_config.topk_method == "noaux_tc":
	architectures = ["DeepseekV3ForCausalLM"]
	elif not self.text_config.use_mla:
	architectures = ["DeepseekForCausalLM"]
	else:
	architectures = ["DeepseekV2ForCausalLM"]

but you're right this should be no longer needed.

[Ucas-HaoranWei]

Great work! I have tested it.