[Queue Time Histogram] Add Job Queue Time Lambda #6435
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Skipped Deployment
|
facebook-github-bot
added
the
CLA Signed
added, I copied some of the File handling code from ci-pct.py |
added, it will be stored in array field as runner_labels |
| arguments = parse_args() | ||
|
|
||
| # update environment variables for input parameters | ||
| os.environ["CLICKHOUSE_ENDPOINT"] = arguments.clickhouse_endpoint |
There was a problem hiding this comment.
It is a not so great engineering standard to hide the original value by overwriting it with cli arguments.
It makes more sense to default the CLI argparse from the environment os.environ. But you can do with a special Config class if you want to as well.
There was a problem hiding this comment.
kk, I can make it different!
| runner_labels["other"].add(machine_type) | ||
|
|
||
|
|
||
| def create_runner_labels( |
There was a problem hiding this comment.
Can this piece of code be reused? So we can avoid having to update in multiple places?
Just today Nikita is planning to move macos-m2-15 from apple ownership to ours. So we'll need to update the metrics and aggregations...
If we create many places to update we're setting ourselves to make mistakes
There was a problem hiding this comment.
lambda is a bit tricky to do it, i think I can do it in a BE pr, right now my focus is get this kick in
|
fixed some nits, moving to next pr |
Description
Add aws lambda to generate in-queue job histgram, steps:
the snapshot data we generate includes:
{ queue_s, repo, workflow_name , job_name, htm_url, machine_type, time, runner_labels}
the runner_labels includes the machine_type, and other categories such as linux, dynamic etc
Design Doc:doc
working result in s3 (Ran locally)
s3 link