[Design] Usage Metrics Exporting

[missBerg]

Need for standardized metrics to monitor and manage usage.

The design doc should include:

• Motivation
• Feature Definition
• Control Plane API
• Technical Implementation Proposal

This issue was created from conversation during Dec 5th Community meeting
https://docs.google.com/document/d/10e1sfsF-3G3Du5nBHGmLjXw5GVMqqCvFDqp_O65B0_w/edit?tab=t.0#bookmark=id.dz9gpy397ymu

[codefromthecrypt]

probably there's a tie in to the metrics here, where server for ingress and client for egress https://github.com/open-telemetry/semantic-conventions/blob/main/docs/gen-ai/gen-ai-metrics.md

If it makes sense to hop on the GenAI call or one of us onto yours, lemme or someone else know! We are in the #otel-genai-instrumentation Cloud Native slack

[Steffen911]

@missBerg Are there also plans to support OTel Spans using the ai-gateway, ideally following the OTel conventions for those: https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-spans/? That could allow centralized monitoring of not just metrics, but also prompts and completions by the model plus details on the request parameters.