fix qwen2 vl crash in continous batching

[sywangyi]

@OlivierDehaene OR @Narsil

File "/usr/src/server/text_generation_server/interceptor.py", line 24, in intercept
return await response
File "/opt/conda/lib/python3.11/site-packages/opentelemetry/instrumentation/grpc/_aio_server.py", line 120, in _unary_interceptor
raise error
File "/opt/conda/lib/python3.11/site-packages/opentelemetry/instrumentation/grpc/_aio_server.py", line 111, in _unary_interceptor
return await behavior(request_or_iterator, context)
File "/usr/src/server/text_generation_server/server.py", line 212, in Decode
batch = self.model.batch_type.concatenate(batches)
File "/opt/conda/lib/python3.11/contextlib.py", line 81, in inner
return func(*args, **kwds)
File "/usr/src/server/text_generation_server/models/vlm_causal_lm.py", line 201, in concatenate
batch = super(VlmCausalLMBatch, cls).concatenate(batches)
File "/opt/conda/lib/python3.11/contextlib.py", line 81, in inner
return func(*args, **kwds)
File "/usr/src/server/text_generation_server/models/flash_causal_lm.py", line 838, in concatenate
position_ids[start_index:end_index] = batch.position_ids
RuntimeError: The expanded size of the tensor (1) must match the existing size (3) at non-singleton dimension 0. Target sizes: [1]. Tensor sizes: [3]
2025-02-10T09:58:20.846762Z ERROR batch{batch_size=2}:decode:decode{size=2}:decode{size=2}: text_generation_router_v3::client: backends/v3/src/client/mod.rs:45: Server error: The expanded size of the tensor (1) must match the existing size (3) at non-singleton dimension 0. Target sizes: [1]. Tensor sizes: [3]

[sywangyi]

@drbh please help review

[sywangyi]

see https://github.com/huggingface/text-generation-inference/blob/main/server/text_generation_server/models/custom_modeling/qwen2_vl.py#L499 for position id handling

[sywangyi]

easy to reproduce：

server:
text-generation-launcher --model-id=Qwen/Qwen2-VL-7B-Instruct

client:
curl -N 0.0.0.0:80/generate_stream
-X POST
-d '{"inputs":"What is in the picture?\n\n","parameters":{"max_new_tokens":100, "seed": 42, "do_sample":true}}'
-H 'Content-Type: application/json' &

curl -N 0.0.0.0:80/generate_stream
-X POST
-d '{"inputs":"What is in the picture?\n\n","parameters":{"max_new_tokens":100, "seed": 42, "do_sample":true}}'
-H 'Content-Type: application/json' &

[drbh]

lgtm, thanks for the fix!