[dotnet] parametric app improvements

[lucaspimentel]

Motivation

Improve the developer experience of modifying, running, and troubleshooting the parametric app for .NET.

Changes

• reduce code repetition
  • refactor reflection code
  • refactor json serialization and deserialization, including validation and logging
• troubleshooting
  • enable more built-in ASP.NET Core logging
  • add more custom logging, in particular the request/response payloads
• reduce external dependencies
  • replace use of Newtonsoft.Json with the built-in System.Text.Json
• fix compiler warnings
  • add missing nullability annotations and null checks
  • make members private where possible

I will do the same with the OpenTelemetry test endpoints (ApmTestApiOtel.cs) in a separate PR and then remove the Newtonsoft.Json reference.

Note

While making these changes I found several unrelated bugs when using the .NET tracing library's manual API:

• setting Span.Service = null after the span is created causes an exception when the span is closed. Working on a fix here:
  [tracer] fix exception when closing a Span with null ServiceName dd-trace-dotnet#6748
• using SpanCreationSettings.Parent = SpanContext.None results in a 64-bit trace id even if 128-bit trace id generation is enabled (we generate 128 bits, but the upper 64 bits are truncated somewhere)
• passing a delegate to SpanContextExtractor that returns null causes an exception. Working on a fix here:
  [tracing] add null checks when iterating headers in propagators dd-trace-dotnet#6460

Workflow

1. ⚠️ Create your PR as draft ⚠️
2. Work on you PR until the CI passes (if something not related to your task is failing, you can ignore it)
3. Mark it as ready for review
   • Test logic is modified? -> Get a review from RFC owner. We're working on refining the codeowners file quickly.
   • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

• If PR title starts with [<language>], double-check that only <language> is impacted by the change
• No system-tests internal is modified. Otherwise, I have the approval from R&P team
• CI is green, or failing jobs are not related to this change (and you are 100% sure about this statement)
• A docker base image is modified?
  • the relevant build-XXX-image label is present
• A scenario is added (or removed)?
  • Get a review from R&P team

[cbeauchesne]

Thank you so much for improving this !!

[bouwkast]

Thank you!