[Prometheus] Add ray_services_ready_duration_seconds metric

[win5923]

Why are these changes needed?

Add ray_services_ready_duration_seconds metric to track The time between RayServices created to ready (created to RayServiceReady)

Manual test

$ k apply -f config/samples/ray-service.sample.yaml

$ echo $(( $(date -d "$(kubectl get rayservice rayservice-sample -o jsonpath='{.status.conditions[?(@.type=="Ready")].lastTransitionTime}')" +%s) - $(date -d "$(kubectl get rayservice rayservice-sample -o jsonpath='{.metadata.creationTimestamp}')" +%s) )) seconds
62 seconds

$ k apply -f config/samples/ray-service.sample.yaml -n test

$ echo $(( $(date -d "$(kubectl get rayservice rayservice-sample -n test -o jsonpath='{.status.conditions[?(@.type=="Ready")].lastTransitionTime}')" +%s) - $(date -d "$(kubectl get rayservice rayservice-sample -n test -o jsonpath='{.metadata.creationTimestamp}')" +%s) )) seconds
32 seconds

$ k apply -f config/samples/ray-service.sample-2.yaml

$ echo $(( $(date -d "$(kubectl get rayservice rayservice-sample-2 -o jsonpath='{.status.conditions[?(@.type=="Ready")].lastTransitionTime}')" +%s) - $(date -d "$(kubectl get rayservice rayservice-sample-2 -o jsonpath='{.metadata.creationTimestamp}')" +%s) )) seconds
32 seconds

Related issue number

Closes #3177

Checks

• I've made sure the tests are passing.
• Testing Strategy
  • Unit tests
  • Manual tests
  • This PR is not tested :(

[win5923]

I recorded the metric here because the following issue causes the controller runtime to reconcile again. This results in the same RayService metric being recorded twice.

{"level":"info","ts":"2025-04-01T12:12:55.567Z","logger":"controllers.RayService","msg":"Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes reqeueuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler","RayService":{"name":"rayservice-sample","namespace":"default"},"reconcileID":"88aadcdd-dfa4-49b8-b290-ffb5d8b06ba0"}
{"level":"error","ts":"2025-04-01T12:12:55.568Z","logger":"controllers.RayService","msg":"Reconciler error","RayService":{"name":"rayservice-sample","namespace":"default"},"reconcileID":"88aadcdd-dfa4-49b8-b290-ffb5d8b06ba0","error":"Operation cannot be fulfilled on rayservices.ray.io \"rayservice-sample\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/home/ubuntu/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/home/ubuntu/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/home/ubuntu/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:224"}

[kevin85421]

could you rebase with the master branch?