Models with Phi3Config fail due to missing attention_bias

[manikawnth]

System Info

• transformers version: 4.41.1
• Platform: Linux-5.15.0-1055-aws-x86_64-with-glibc2.35
• Python version: 3.10.14
• Huggingface_hub version: 0.23.0
• Safetensors version: 0.4.3
• Accelerate version: 0.30.1
• Accelerate config: not found
• PyTorch version (GPU?): 2.3.0 (True)
• Tensorflow version (GPU?): not installed (NA)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Using GPU in script?: yes
• Using distributed or parallel set-up in script?: no

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

Running the model using huggingface text-generation-inference server will cause the issue

AttributeError: 'Phi3Config' object has no attribute 'attention_bias' 

Reference issue:
aws/sagemaker-huggingface-inference-toolkit#123

Expected behavior

tgi server should successfully run the Phi3 model

[amyeroberts]

cc @ArthurZucker @younesbelkada

[younesbelkada]

Hi @manikawnth !
See my comment on the PR, IMO the fix should go on TGI directly, what do you think?

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.