You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We founded that the pre-build query does not work about calculating job latency per job. It seems like that Properties_spark_app_id_s, Job_ID_d, Submission_Time_d, Completion_Time_d these fields does not existed in SparkListenerEvent_CL this table.
let results=SparkListenerEvent_CL
| where Event_s contains "SparkListenerJobStart"
| extend metricsns=columnifexists("Properties_spark_metrics_namespace_s",Properties_spark_app_id_s)
| extend apptag=iif(isnotempty(metricsns),metricsns,Properties_spark_app_id_s)
| project Job_ID_d,apptag,Properties_spark_databricks_clusterUsageTags_clusterName_s,
Submission_Time_d,TimeGenerated
| order by TimeGenerated asc nulls last
| join kind= inner (
SparkListenerEvent_CL
| where Event_s contains "SparkListenerJobEnd"
| where Job_Result_Result_s contains "JobSucceeded"
| project Event_s,Job_ID_d,Completion_Time_d,TimeGenerated
) on Job_ID_d;
results
| extend slice=strcat(Properties_spark_databricks_clusterUsageTags_clusterName_s,"-",apptag)
| extend jobDuration=Completion_Time_d - Submission_Time_d
| summarize percentiles(jobDuration,10,30,50,90) by bin(TimeGenerated, 1m), slice
| order by TimeGenerated asc nulls last
Please to update related query and fix it. Thank you
The text was updated successfully, but these errors were encountered:
We founded that the pre-build query does not work about calculating job latency per job. It seems like that
Properties_spark_app_id_s
,Job_ID_d
,Submission_Time_d
,Completion_Time_d
these fields does not existed inSparkListenerEvent_CL
this table.Please to update related query and fix it. Thank you
The text was updated successfully, but these errors were encountered: