[bug] Cannot configure OpenTelemetry based on data from DI-registered service

[perlun]

Package

OpenTelemetry

Package Version

Package Name	Version
OpenTelemetry.Api	1.10.0
OpenTelemetry	1.10.0

Runtime Version

net9.0

Description

Hi,

We are trying to configure the OpenTelemetry exporters based on "dynamic" settings coming from a dependency-injected service. Our code is in a private repo, but the overall approach is somewhat like this (using a Startup class, i.e. classical ASP.NET Core 5 approach):

public void ConfigureServices(IServiceCollection services) {
    services.AddControllers();

    IOpenTelemetryBuilder builder = services.AddOpenTelemetry();

    string serviceName = "serviceName";
    string serviceNamespace = "serviceNamespace";
    string serviceVersion = "12345";

    builder.ConfigureResource(resource => resource
            .AddService(serviceName, serviceNamespace, serviceVersion)
        )
        .WithTracing(tracing => {
            tracing.AddAspNetCoreInstrumentation(options => options.RecordException = true);
            tracing.AddSource(serviceName);

            // TODO: Enable/disable this dynamically based on data from a dependency-injected service
            if (true) {
                tracing.AddConsoleExporter();
            }

            // TODO: Enable/disable this, and use protocol/URL from dependency-injected service
            if (true) {
                tracing.AddOtlpExporter(exporter => {
                    exporter.Protocol = OtlpExportProtocol.HttpProtobuf;
                    exporter.Endpoint = new Uri(Path.Combine("https://some-endpoint/v1/traces"));
                });
            }
        })
        .WithMetrics(metrics => {
            metrics.AddAspNetCoreInstrumentation();

            // TODO: enable/disable like above
            if (true) {
                metrics.AddConsoleExporter();
            }

            // TODO: enable/disable like above
            if (true) {
                metrics.AddOtlpExporter(exporter => {
                    exporter.Protocol = OtlpExportProtocol.HttpProtobuf;
                    exporter.Endpoint = new Uri(Path.Combine("https://some-endpoint/v1/metrics"));
                });
            }
        })
        .WithLogging(logging => {
            logging.SetResourceBuilder(ResourceBuilder.CreateDefault().AddService(serviceName, serviceNamespace, serviceVersion));

            // TODO: enable/disable like above
            if (true) {
                logging.AddConsoleExporter();
            }

            // TODO: enable/disable like above
            if (true) {
                logging.AddOtlpExporter(exporter => {
                    exporter.Protocol = OtlpExportProtocol.HttpProtobuf;
                    exporter.Endpoint = new Uri(Path.Combine("https://some-endpoint/v1/logs"));
                });
            }
        });
}

Our problem is, as you might guess by now, that the required services are not yet available in the ConfigureServices phase.

They would be available during the Configure phase, at which point it is trival to inject anything you like from the IOC container (by just adding more parameters to the Configure method signature). The framework will do the rest for you:

// ISomeDependency has been registered during the ConfigureServices phase
public void Configure(IApplicationBuilder app, IWebHostEnvironment env, ISomeDependency someDependency) {
    app.UseHttpsRedirection();

    app.UseRouting();

    app.UseAuthorization();

    app.UseEndpoints(endpoints => {
        endpoints.MapControllers();
    });
}

However, it's not really possible to configure the OpenTelemetry things this later during the startup - or is it? At least when I've tried (perhaps a bit clumsily) to register the IOpenTelemetryBuilder and "do things" in the Configure step, nothing seemed to work. I believe this is perhaps because the TelemetryHostedService has already started when Configure gets called - I attached a debugger now and this does indeed seem to be the case.

I thought about reverse-engineering something like what AddOpenTelemetry does under the hood:

public static OpenTelemetryBuilder AddOpenTelemetry(this IServiceCollection services)
{
    Guard.ThrowIfNull(services);

    if (!services.Any((ServiceDescriptor d) => d.ServiceType == typeof(IHostedService) && d.ImplementationType == typeof(TelemetryHostedService)))
    {
        services.Insert(0, ServiceDescriptor.Singleton<IHostedService, TelemetryHostedService>());
    }

    return new(services);
}

...but a major problem here is that TelemetryHostedService is internal, as well as the OpenTelemetryBuilder constructor. I realize these kind of "hacks" goes against the "normal" way to use OpenTelemetry.NET. It would be great to hear some thoughts regarding this though. Are there any ways to configure (these aspects) of OpenTelemetry based on data not available until the IServiceProvider is available? 🤔

Steps to Reproduce

The Startup class referred to is available here, along with a MCVE (without any injected dependency), in case anyone wants to tinker around.

Expected Result

A way to configure OpenTelemetry exporters (enable/disable console/OTLP exporters dynamically/after services have been configured).

Actual Result

No available way to configure OpenTelemetry once the ASP.NET Core web application has started.

Additional Context

Semi-related problem: #3416