You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We use spark-monitoring in our Databricks workspaces, but in a slightly non-standard way:
Metrics are collected and pushed to Log Analytics, as per project design;
Logs, on the other side, follow a different route: we want them formatted in JSON, but they are written to a shared storage (due to our overall enterprise architecture) and then eventually exported to our logging backend.
For this reason, we only need the JSONLayout piece of spark-monitoring when it comes to logs. We can do this by setting
in log4j.properties, but we have an additional requirement on JSONLayout: it must be configurable to add a newline at the end of the serialised string because the software which parses our logs expects one logging even per line.
We added a OneLogEventPerLine property to JSONLayout in our private fork of the project, it's a simple change and it's working well. Would you be interested in porting this to your project?
The text was updated successfully, but these errors were encountered:
Hi all,
We use spark-monitoring in our Databricks workspaces, but in a slightly non-standard way:
For this reason, we only need the
JSONLayout
piece of spark-monitoring when it comes to logs. We can do this by settingin
log4j.properties
, but we have an additional requirement on JSONLayout: it must be configurable to add a newline at the end of the serialised string because the software which parses our logs expects one logging even per line.We added a
OneLogEventPerLine
property to JSONLayout in our private fork of the project, it's a simple change and it's working well. Would you be interested in porting this to your project?The text was updated successfully, but these errors were encountered: