[Feature] create opea's own model downloader image

[lianhao]

Priority

Undecided

OS type

Ubuntu

Hardware type

Xeon-GNR

Running nodes

Single Node

Description

Currently, the helm chart is using upstream image huggingface/downloader:0.17.3 to download models. This image is buggy and quite old, and according to HF community feedback, they don't plan to maintain it any more.

We need to create our own image to download the model, and leverage the latest HF_HUB_ENABLE_HF_TRANSFER for fast model download.

[eero-t]

Something I did not notice earlier: https://huggingface.co/docs/huggingface_hub/en/package_reference/environment_variables#hfhubenablehftransfer

hf_transfer lacks several user-friendly features such as resumable downloads and proxies

I think this needs to wait until it has at least proxy support...

[lianhao]

no resumable/proxy support is clearly a no-go.

[eero-t]

Btw. Downloader Job that could be run before OPEA applications to download all specified models to given hostPath or PVC would be useful though, as it would:

• makes it sure that app startup does not fail to disk being full (models filling disk would have happened earlier)
• allow apps' PVCs to use readOnlyMany mode (AFAIK supported by all cloud provides, unlike the writeMany mode)
• (token) secret and additional privileges would be needed only for single Job
• better support offline (no internet / private cloud) use-cases