[chat-template] Unify tests and clean up 🧼

[zucchini-nlp]

What does this PR do?

As per title, we added the video loading logic in rush for SmolVLM. But the API is a bit over-complicated, so the decision was to clean up later. As the first step of clean up, we will

• Remove sampling_fn from public kwargs, I don't think anyone uses it so we will not deprecate
• Squash typed dict manipulations into a smaller helper
• Put video loading for each model in model's processor. Until the PR for adding separate API for video processors is merged, we'll keep it private under processor. In the future, each video processor can have its own logic to load videos, following model-specific treatment
• Clean up tests to load only slow image/video processors. As we are moving to fast-only image processors, we need a way to run tests without relying only on torch. For torch inputs we already have test_chat_template_return_dict_torch
• Unify similar test under one and parameterize

[github-actions]

Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the Ready for review button (at the bottom of the PR page). This will assign reviewers and trigger CI.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[zucchini-nlp]

ready for review :)

[Rocketknight1]

Merging the model-specific tests into a common test makes a lot of sense! One comment, but otherwise this LGTM

[Rocketknight1]

Making sure I understand this: When we have a proper VideoProcessorBase API, there won't be model-specific _load_video_for_model methods anymore?

[zucchini-nlp]

There will be model-specific method, in each video processor. Currently we have models loading videos with different sampling (SmolVLM) or even resizing (Qwen-Omni), even before the video is processed

As a second option, we can consider merging all these steps into self.preprocess(videos) as long as it doesn't slow it down. Haven't thought about it, but since you mentioned it that sounds interesting. I'll give it a try after API initial design is merged (under review still)