[AQUA] GPU Shape Recommendation

[elizjo]

Wrote an additional POST API and aqua command for recommending GPU shapes for a particular model

POST /gpu-shape-recommendation{    "model": "meta-llama/Llama-4-Scout-17B-16E",    "max_model_len": 4096}

ads aqua recommend which_gpu --model  "meta-llama/Llama-4-Scout-17B-16E" --max_model_len 4096

Returns

{
    "recommendations": [
        {
            "batch_size": 1,
            "max_seq_len": 65536,
            "precision": "bfloat16",
            "gb_used_by_model": 14.729,
            "shape_reports": [
                {
                    "shape": "VM.GPU2.1",
                    "gpu_reports": [
                        {
                            "gpu_count": 1,
                            "gpu_memory_in_gb": 16,
                            "limiting_factor": "The selected model configuration is close to GPU Memory Limit (14.7GB used / 15.2GB allowed).\\nBoth model weights and KV cache are significant contributors to memory use. (7.5GB KV cache, 7.2GB weights).\\n.\\n\\nTo reduce model size:\\n1. Consider using a model with fewer parameters. \\n2. Use the same model with float16, int8 quantization, which is smaller than the current quantization/ weight size: bfloat16."
                        }
                    ]
                },
    ]
}

TODO:
logic for extracting the config.json from the huggingface model needs to be improved
need to write a whole unit test module + conduct testing (business logic works, but API is untested)

[github-actions]

📌 Cov diff with main:

📌 Overall coverage: