Merge pull request #262 from jeremydvoss/env-var

Enable env var configuration, defaulting, validation

Author: jeremydvoss

CHANGELOG.md

@@ -68,6 +68,8 @@
     ([#247](https://github.com/microsoft/ApplicationInsights-Python/pull/247))
 - Updating documents for new namespace
     ([#249](https://github.com/microsoft/ApplicationInsights-Python/pull/249))
+- Configuration via env vars and argument validation.
+    ([#262](https://github.com/microsoft/ApplicationInsights-Python/pull/262))
 
 ## [1.0.0b8](https://github.com/microsoft/ApplicationInsights-Python/releases/tag/v1.0.0b8) - 2022-09-26
 

azure-monitor-opentelemetry/README.md

@@ -11,7 +11,7 @@ This distro automatically installs the following libraries:
 
 OpenTelemetry instrumentations allow automatic collection of requests sent from underlying instrumented libraries. The following is a list of OpenTelemetry instrumentations that come bundled in with the Azure monitor distro. If you would like to add support for another OpenTelemetry instrumentation, please submit a feature [request][distro_feature_request]. In the meantime, you can use the OpenTelemetry instrumentation manually via it's own APIs (i.e. `instrument()`) in your code. See [this][samples_manual] for an example.
 
-| Instrumentation                       | Supported library | Supported versions |
+| Instrumentation | Supported library | Supported versions |
 | ------------------------------------- | ----------------- | ------------------ |
 | [OpenTelemetry Django Instrumentation][ot_instrumentation_django] | [django][pypi_django] | [link][ot_instrumentation_django_version]
 | [OpenTelemetry FastApi Instrumentation][ot_instrumentation_fastapi] | [fastapi][pypi_fastapi] | [link][ot_instrumentation_fastapi_version]
@@ -52,14 +52,24 @@ pip install azure-monitor-opentelemetry --pre
 
 You can use `configure_azure_monitor` to set up instrumentation for your app to Azure Monitor. `configure_azure_monitor` supports the following optional arguments:
 
-* connection_string - The [connection string][connection_string_doc] for your Application Insights resource. The connection string will be automatically populated from the `APPLICATIONINSIGHTS_CONNECTION_STRING` environment variable if not explicitly passed in.
-* disable_logging - If set to `True`, disables collection and export of logging telemetry. Defaults to `False`.
-* disable_metrics - If set to `True`, disables collection and export of metric telemetry. Defaults to `False`.
-* disable_tracing - If set to `True`, disables collection and export of distributed tracing telemetry. Defaults to `False`.
-* exclude_instrumentations - By default, all supported [instrumentations](#officially-supported-instrumentations) are enabled to collect telemetry. Specify instrumentations you do not want to enable to collect telemetry by passing in a comma separated list of instrumented library names. e.g. `["requests", "flask"]`
-* instrumentation_config - Specifies a dictionary of kwargs that will be applied to instrumentation configuration. You can specify which instrumentation you want to configure by name in the key field and value as a dictionary representing `kwargs` for the corresponding instrumentation.
-    Refer to the `Supported Library` section [above](#officially-supported-instrumentations) for the list of suppoprted library names.
-
+| Parameter | Description | Environment Variable |
+|-------------------|----------------------------------------------------|----------------------|
+| `connection_string` | The [connection string][connection_string_doc] for your Application Insights resource. The connection string will be automatically populated from the `APPLICATIONINSIGHTS_CONNECTION_STRING` environment variable if not explicitly passed in. | `APPLICATIONINSIGHTS_CONNECTION_STRING` |
+| `exclude_instrumentations` | By default, all supported [instrumentations](#officially-supported-instrumentations) are enabled to collect telemetry. Specify instrumentations you do not want to enable to collect telemetry by passing in a comma separated list of instrumented library names. e.g. `["requests", "flask"]` | |
+| `resource` | Specifies the OpenTelemetry [resource][opentelemetry_spec_resource] associated with your application. See [this][ot_sdk_python_resource] for default behavior. | [OTEL_SERVICE_NAME][opentelemetry_spec_service_name], [OTEL_RESOURCE_ATTRIBUTES][opentelemetry_spec_resource_attributes] |
+| `disable_logging` | If set to `True`, disables collection and export of logging telemetry. Defaults to `False`. | |
+| `disable_metrics` | If set to `True`, disables collection and export of metric telemetry. Defaults to `False`. | |
+| `disable_tracing` | If set to `True`, disables collection and export of distributed tracing telemetry. Defaults to `False`. | |
+| `logging_level` | Specifies the [logging level][logging_level] of the logs you would like to collect for your logging pipeline. Defaults to 0 which is `logging.NOTSET`. | |
+| `logger_name` | Specifies the [logger name][logger_name_hierarchy_doc] under which logging will be instrumented. Defaults to "" which corresponds to the root logger. | |
+| `logging_export_interval_ms`| Specifies the logging export interval in milliseconds. Defaults to 5000. | `OTEL_BLRP_SCHEDULE_DELAY` |
+| `metric_readers` | Specifies the [metric readers][ot_metric_reader] that you would like to use for your metric pipeline. Accepts a list of [metric readers][ot_sdk_python_metric_reader]. | |
+| `views` | Specifies the list of [views][opentelemetry_spec_view] to configure for the metric pipeline. See [here][ot_sdk_python_view_examples] for example usage. | |
+| `sampling_ratio` | Specifies the ratio of distributed tracing telemetry to be [sampled][application_insights_sampling]. Accepted values are in the range [0,1]. Defaults to 1.0, meaning no telemetry is sampled out. | `OTEL_TRACES_SAMPLER_ARG` |
+| `tracing_export_interval_ms`| Specifies the distributed tracing export interval in milliseconds. Defaults to 5000. | `OTEL_BSP_SCHEDULE_DELAY` |
+| `instrumentation_config` | Specifies a dictionary of kwargs that will be applied to instrumentation configuration. You can specify which instrumentation you want to configure by name in the key field and value as a dictionary representing `kwargs` for the corresponding instrumentation. Refer to the `Supported Library` section [above](#officially-supported-instrumentations) for the list of supported library names. | |
+
+Example for use of `instrumentation_config`:
 ```python
 ...
 configure_azure_monitor(
@@ -78,15 +88,6 @@ configure_azure_monitor(
 
 Take a look at the specific [instrumenation][ot_instrumentations] documentation for available configurations.
 
-* resource - Specified the OpenTelemetry [resource][opentelemetry_spec_resource] associated with your application. See [this][ot_sdk_python_resource] for default behavior.
-* logging_level - Specifies the [logging level][logging_level] of the logs you would like to collect for your logging pipeline. Defaults to logging.NOTSET.
-* logger_name = Specifies the [logger name][logger_name_hierarchy_doc] under which logging will be instrumented. Defaults to "" which corresponds to the root logger.
-* logging_export_interval_ms - Specifies the logging export interval in milliseconds. Defaults to 5000.
-* metric_readers - Specifies the [metric readers][ot_metric_reader] that you would like to use for your metric pipeline. Accepts a list of [metric readers][ot_sdk_python_metric_reader].
-* views - Specifies the list of [views][opentelemetry_spec_view] to configure for the metric pipeline. See [here][ot_sdk_python_view_examples] for example usage.
-* sampling_ratio - Specifies the ratio of distributed tracing telemetry to be [sampled][application_insights_sampling]. Accepted values are in the range [0,1]. Defaults to 1.0, meaning no telemetry is sampled out.
-* tracing_export_interval_ms - Specifies the distributed tracing export interval in milliseconds. Defaults to 5000.
-
 #### Azure monitor OpenTelemetry Exporter configurations
 
 You can pass Azure monitor OpenTelemetry exporter configuration parameters directly into `configure_azure_monitor`. See additional [configuration related to exporting here][exporter_configuration_docs].
@@ -143,6 +144,8 @@ Samples are available [here][samples] to demonstrate how to utilize the above co
 [ot_instrumentation_urllib3]: https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation/opentelemetry-instrumentation-urllib3
 [ot_instrumentation_urllib3_version]: https://github.com/open-telemetry/opentelemetry-python-contrib/blob/main/instrumentation/opentelemetry-instrumentation-urllib3/src/opentelemetry/instrumentation/urllib3/package.py#L16
 [opentelemetry_spec_resource]: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/resource/sdk.md#resource-sdk
+[opentelemetry_spec_resource_attributes]: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/resource/sdk.md#specifying-resource-information-via-an-environment-variable
+[opentelemetry_spec_service_name]: https://github.com/open-telemetry/opentelemetry-specification/tree/main/specification/resource/semantic_conventions#semantic-attributes-with-sdk-provided-default-value
 [opentelemetry_spec_view]: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/sdk.md#view
 [pip]: https://pypi.org/project/pip/
 [pypi_django]: https://pypi.org/project/Django/

azure-monitor-opentelemetry/azure/monitor/opentelemetry/_configure.py

@@ -3,9 +3,24 @@
 # Licensed under the MIT License. See License in the project root for
 # license information.
 # --------------------------------------------------------------------------
-from logging import NOTSET, getLogger
+from logging import getLogger
 from typing import Any, Dict
 
+from azure.monitor.opentelemetry._constants import (
+    DISABLE_LOGGING_ARG,
+    DISABLE_METRICS_ARG,
+    DISABLE_TRACING_ARG,
+    EXCLUDE_INSTRUMENTATIONS_ARG,
+    INSTRUMENTATION_CONFIG_ARG,
+    LOGGER_NAME_ARG,
+    LOGGING_EXPORT_INTERVAL_MS_ARG,
+    LOGGING_LEVEL_ARG,
+    METRIC_READERS_ARG,
+    RESOURCE_ARG,
+    SAMPLING_RATIO_ARG,
+    TRACING_EXPORT_INTERVAL_MS_ARG,
+    VIEWS_ARG,
+)
 from azure.monitor.opentelemetry._types import ConfigurationValue
 from azure.monitor.opentelemetry.exporter import (
     ApplicationInsightsSampler,
@@ -53,7 +68,7 @@ def configure_azure_monitor(**kwargs) -> None:
     end user to configure OpenTelemetry and Azure monitor components. The
     configuration can be done via arguments passed to this function.
     :keyword str connection_string: Connection string for your Application Insights resource.
-    :keyword Sequence[str] connection_string: Specifies the libraries with instrumentations to be enabled.
+    :keyword Sequence[str] exclude_instrumentations: Specifies instrumentations you want to disable.
     :keyword Resource resource: Specified the OpenTelemetry [resource][opentelemetry_spec_resource] associated with your application.
     :keyword bool disable_logging: If set to `True`, disables collection and export of logging telemetry. Defaults to `False`.
     :keyword bool disable_metrics: If set to `True`, disables collection and export of metric telemetry. Defaults to `False`.
@@ -75,9 +90,9 @@ def configure_azure_monitor(**kwargs) -> None:
 
     configurations = _get_configurations(**kwargs)
 
-    disable_tracing = configurations.get("disable_tracing", False)
-    disable_logging = configurations.get("disable_logging", False)
-    disable_metrics = configurations.get("disable_metrics", False)
+    disable_tracing = configurations[DISABLE_TRACING_ARG]
+    disable_logging = configurations[DISABLE_LOGGING_ARG]
+    disable_metrics = configurations[DISABLE_METRICS_ARG]
 
     resource = None
     if not disable_logging or not disable_tracing or not disable_metrics:
@@ -102,16 +117,14 @@ def configure_azure_monitor(**kwargs) -> None:
 
 
 def _get_resource(configurations: Dict[str, ConfigurationValue]) -> Resource:
-    return configurations.get("resource", Resource.create())
+    return configurations.get(RESOURCE_ARG, Resource.create())
 
 
 def _setup_tracing(
     resource: Resource, configurations: Dict[str, ConfigurationValue]
 ):
-    sampling_ratio = configurations.get("sampling_ratio", 1.0)
-    tracing_export_interval_ms = configurations.get(
-        "tracing_export_interval_ms", 5000
-    )
+    sampling_ratio = configurations[SAMPLING_RATIO_ARG]
+    tracing_export_interval_ms = configurations[TRACING_EXPORT_INTERVAL_MS_ARG]
     tracer_provider = TracerProvider(
         sampler=ApplicationInsightsSampler(sampling_ratio=sampling_ratio),
         resource=resource,
@@ -128,11 +141,9 @@ def _setup_tracing(
 def _setup_logging(
     resource: Resource, configurations: Dict[str, ConfigurationValue]
 ):
-    logger_name = configurations.get("logger_name", "")
-    logging_level = configurations.get("logging_level", NOTSET)
-    logging_export_interval_ms = configurations.get(
-        "logging_export_interval_ms", 5000
-    )
+    logger_name = configurations[LOGGER_NAME_ARG]
+    logging_level = configurations[LOGGING_LEVEL_ARG]
+    logging_export_interval_ms = configurations[LOGGING_EXPORT_INTERVAL_MS_ARG]
     logger_provider = LoggerProvider(resource=resource)
     set_logger_provider(logger_provider)
     log_exporter = AzureMonitorLogExporter(**configurations)
@@ -150,8 +161,8 @@ def _setup_logging(
 def _setup_metrics(
     resource: Resource, configurations: Dict[str, ConfigurationValue]
 ):
-    views = configurations.get("views", ())
-    metric_readers = configurations.get("metric_readers", [])
+    views = configurations[VIEWS_ARG]
+    metric_readers = configurations[METRIC_READERS_ARG]
     metric_exporter = AzureMonitorMetricExporter(**configurations)
     reader = PeriodicExportingMetricReader(metric_exporter)
     meter_provider = MeterProvider(
@@ -163,10 +174,8 @@ def _setup_metrics(
 
 
 def _setup_instrumentations(configurations: Dict[str, ConfigurationValue]):
-    exclude_instrumentations = configurations.get(
-        "exclude_instrumentations", []
-    )
-    instrumentation_configs = configurations.get("instrumentation_config", {})
+    exclude_instrumentations = configurations[EXCLUDE_INSTRUMENTATIONS_ARG]
+    instrumentation_configs = configurations[INSTRUMENTATION_CONFIG_ARG]
 
     # use pkg_resources for now until https://github.com/open-telemetry/opentelemetry-python/pull/3168 is merged
     for entry_point in iter_entry_points("opentelemetry_instrumentor"):

azure-monitor-opentelemetry/azure/monitor/opentelemetry/_constants.py

@@ -13,6 +13,25 @@
     ConnectionStringParser,
 )
 
+# --------------------Configuration------------------------------------------
+
+CONNECTION_STRING_ARG = "connection_string"
+EXCLUDE_INSTRUMENTATIONS_ARG = "exclude_instrumentations"
+RESOURCE_ARG = "resource"
+DISABLE_LOGGING_ARG = "disable_logging"
+DISABLE_METRICS_ARG = "disable_metrics"
+DISABLE_TRACING_ARG = "disable_tracing"
+# TODO: Consider Log Level to match env var
+LOGGING_LEVEL_ARG = "logging_level"
+LOGGER_NAME_ARG = "logger_name"
+LOGGING_EXPORT_INTERVAL_MS_ARG = "logging_export_interval_ms"
+METRIC_READERS_ARG = "metric_readers"
+VIEWS_ARG = "views"
+SAMPLING_RATIO_ARG = "sampling_ratio"
+TRACING_EXPORT_INTERVAL_MS_ARG = "tracing_export_interval_ms"
+INSTRUMENTATION_CONFIG_ARG = "instrumentation_config"
+
+
 # --------------------Diagnostic/status logging------------------------------
 
 _LOG_PATH_LINUX = "/var/log/applicationinsights"

azure-monitor-opentelemetry/azure/monitor/opentelemetry/util/configurations.py

@@ -4,9 +4,37 @@
 # license information.
 # --------------------------------------------------------------------------
 
+from logging import NOTSET, getLogger
+from os import environ
 from typing import Dict
 
+from azure.monitor.opentelemetry._constants import (
+    DISABLE_LOGGING_ARG,
+    DISABLE_METRICS_ARG,
+    DISABLE_TRACING_ARG,
+    EXCLUDE_INSTRUMENTATIONS_ARG,
+    INSTRUMENTATION_CONFIG_ARG,
+    LOGGER_NAME_ARG,
+    LOGGING_EXPORT_INTERVAL_MS_ARG,
+    LOGGING_LEVEL_ARG,
+    METRIC_READERS_ARG,
+    SAMPLING_RATIO_ARG,
+    TRACING_EXPORT_INTERVAL_MS_ARG,
+    VIEWS_ARG,
+)
 from azure.monitor.opentelemetry._types import ConfigurationValue
+from opentelemetry.sdk.environment_variables import OTEL_TRACES_SAMPLER_ARG
+
+_INVALID_FLOAT_MESSAGE = "Value of %s must be a float. Defaulting to %s: %s"
+
+
+# Speced out but unused by OTel SDK as of 1.15.0
+LOGGING_EXPORT_INTERVAL_MS_ENV_VAR = "OTEL_BLRP_SCHEDULE_DELAY"
+# TODO: remove when sampler uses env var instead
+SAMPLING_RATIO_ENV_VAR = OTEL_TRACES_SAMPLER_ARG
+
+
+_logger = getLogger(__name__)
 
 
 def _get_configurations(**kwargs) -> Dict[str, ConfigurationValue]:
@@ -15,7 +43,93 @@ def _get_configurations(**kwargs) -> Dict[str, ConfigurationValue]:
     for key, val in kwargs.items():
         configurations[key] = val
 
+    _default_exclude_instrumentations(configurations)
+    _default_disable_logging(configurations)
+    _default_disable_metrics(configurations)
+    _default_disable_tracing(configurations)
+    _default_logging_level(configurations)
+    _default_logger_name(configurations)
+    _default_logging_export_interval_ms(configurations)
+    _default_metric_readers(configurations)
+    _default_views(configurations)
+    _default_sampling_ratio(configurations)
+    _default_tracing_export_interval_ms(configurations)
+    _default_instrumentation_config(configurations)
+
+    # TODO: remove when validation added to BLRP
+    if configurations[LOGGING_EXPORT_INTERVAL_MS_ARG] <= 0:
+        raise ValueError(
+            "%s must be positive." % LOGGING_EXPORT_INTERVAL_MS_ARG
+        )
+
     return configurations
 
 
-# TODO: Add env var configuration
+def _default_exclude_instrumentations(configurations):
+    if EXCLUDE_INSTRUMENTATIONS_ARG not in configurations:
+        configurations[EXCLUDE_INSTRUMENTATIONS_ARG] = []
+
+
+def _default_disable_logging(configurations):
+    if DISABLE_LOGGING_ARG not in configurations:
+        configurations[DISABLE_LOGGING_ARG] = False
+
+
+def _default_disable_metrics(configurations):
+    if DISABLE_METRICS_ARG not in configurations:
+        configurations[DISABLE_METRICS_ARG] = False
+
+
+def _default_disable_tracing(configurations):
+    if DISABLE_TRACING_ARG not in configurations:
+        configurations[DISABLE_TRACING_ARG] = False
+
+
+def _default_logging_level(configurations):
+    if LOGGING_LEVEL_ARG not in configurations:
+        configurations[LOGGING_LEVEL_ARG] = NOTSET
+
+
+def _default_logger_name(configurations):
+    if LOGGER_NAME_ARG not in configurations:
+        configurations[LOGGER_NAME_ARG] = ""
+
+
+def _default_logging_export_interval_ms(configurations):
+    if LOGGING_EXPORT_INTERVAL_MS_ARG not in configurations:
+        configurations[LOGGING_EXPORT_INTERVAL_MS_ARG] = 5000
+
+
+def _default_metric_readers(configurations):
+    if METRIC_READERS_ARG not in configurations:
+        configurations[METRIC_READERS_ARG] = []
+
+
+def _default_views(configurations):
+    if VIEWS_ARG not in configurations:
+        configurations[VIEWS_ARG] = ()
+
+
+# TODO: remove when sampler uses env var instead
+def _default_sampling_ratio(configurations):
+    if SAMPLING_RATIO_ARG not in configurations:
+        default = 1.0
+        if SAMPLING_RATIO_ENV_VAR in environ:
+            try:
+                default = float(environ[SAMPLING_RATIO_ENV_VAR])
+            except ValueError as e:
+                _logger.error(
+                    _INVALID_FLOAT_MESSAGE
+                    % (SAMPLING_RATIO_ENV_VAR, default, e)
+                )
+        configurations[SAMPLING_RATIO_ARG] = default
+
+
+def _default_tracing_export_interval_ms(configurations):
+    if TRACING_EXPORT_INTERVAL_MS_ARG not in configurations:
+        configurations[TRACING_EXPORT_INTERVAL_MS_ARG] = None
+
+
+def _default_instrumentation_config(configurations):
+    if INSTRUMENTATION_CONFIG_ARG not in configurations:
+        configurations[INSTRUMENTATION_CONFIG_ARG] = {}

azure-monitor-opentelemetry/samples/metrics/views.py

@@ -8,7 +8,6 @@
 # Create a view matching the counter instrument `my.counter`
 # and configure the new name `my.counter.total` for the result metrics stream
 change_metric_name_view = View(
-    connection_string="<your-connection-string>",
     instrument_type=Counter,
     instrument_name="my.counter",
     name="my.counter.total",