Consider making AIServiceBackend an abstraction for Backends for AI Services

[arkodg]

Description:

Describe the desired behavior, what scenario it enables and how it
would be used.

Today https://aigateway.envoyproxy.io/docs/api/#aiservicebackendspec references two resources

• A BackendRef resource
• A BackendSecurityPolicyRef

For the egress case where clients are connecting to external model services, it will help if these references are converted into absolute values/fields are embedded instead of references, since the persona responsible for managing the AIServiceBackend is the same.

This would make it more inline with AIGatewayRoute which is an abstraction on top of HTTPRoute for routing AI traffic

Relates to #603

[optional Relevant Links:]

Any extra documentation required to understand the issue.

[Xunzhuo]

+1 for this, would love to make API simplify.

the ideal approach I think:

1. GatewayClass -> Gateway -> AIGatewayRoute -> AIServiceBackend

the current approach is:

1. GatewayClass -> Gateway -> AIGatewayRoute -> AIServiceBackend + n * Backend + n * BackendSecurityPolicy + n * BackendTLSPolicy

the user-facing API grows large, and this will bring some issues:

1. the UX is bad and people need to understand a lot of resources we define.
2. the system logics becomes more complex.
3. the frontend engineer feels bad about it, need a lof of works in console building

[yuzisun]

The proposal here is to embed the Backend and BackendSecurityPolicy type in the AIServiceBackend right?

[arkodg]

yeah @yuzisun, also Service prefix can be removed, since its redundant (a Service is a Backend), and can be reduced to AIBackend (even AIGatewayRoute could be simplified to AIRoute, since a Route always attaches to a Gateway/Listener parentRef)

[Xunzhuo]

Agreed with @arkodg

GatewayClass -> Gateway -> AIGatewayRoute -> AIServiceBackend

To:

GatewayClass -> Gateway -> AIRoute -> AIBackend

[yuzisun]

Let’s create a poll with the community ?

[Xunzhuo]

sure @yuzisun

the current approach is

GatewayClass -> Gateway -> AIGatewayRoute -> AIServiceBackend + n * Backend + n * BackendSecurityPolicy + n * BackendTLSPolicy

the proposed approach

GatewayClass -> Gateway -> AIRoute -> AIBackend

API Changes

1. AIRoute: renamed from AIGatewayRoute
2. AIBackend: renamed from AIServiceBackend
3. Merge Backend & BackendSecurityPolicy & BackendTLSPolicy to AIBackend

[missBerg]

For AI GW we could combine them, my hope had been that we could have promoted the notion that the resource BackendSecurityPolicy could eventually become a resource we could promote to Envoy Gateway. Especially when we can add also a generic Upstream OAuth integration. Before we had AI, I encountered numerous instances where I needed to integrate with upstream systems and auth via Gateway.

So my question would be, for BackendSecurityPolicy maybe we need to memorialize what the design is in it's pure form so we could look to upstream that resource into EG?