[release/6.0] Correctly format chuncked response with Content-Length header

[MihaZupan]

Fixes #68581
Backport of #69016 to release/6.0

Customer Impact

This is fixing an edge-case bug in HttpClient where we currently violate RFC.
This is a regression in Core compared to Framework.

As per (RFC 2616 section 4.4 para 3):

If a message is received with both a Transfer-Encoding header field and a Content-Length header field, the latter MUST be ignored.

While the server should not respond with both headers, the spec is clear that the client must respect the latter one if it does happen.

Today's behavior (without this fix) means that the user of HttpClient will receive a response body including the chunk frames, which won't be removed. The only workaround is to detect this case and remove chunk framing manually (~600 LOC).

Testing

Added a targeted test case for the new behavior.
We have existing test coverage to ensure nothing else regressed.

Risk

A user could be relying on the current non-RFC-compliant 6.0 behavior. Such users will have to change their code.

The risk of unrelated regressions is low - the change is trivial (swapping the order of two else if branches).

[ghost]

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Fixes #68581
Backport of #69016 to release/6.0

Customer Impact

This is fixing an edge-case bug in HttpClient where we currently go against the RFC.
This is a regression in Core compared to Framework.

As per (RFC 2616 section 4.4 para 3):

If a message is received with both a Transfer-Encoding header field and a Content-Length header field, the latter MUST be ignored.

While the server should not respond with both headers, the spec is clear that the client must respect the latter one if it does happen.

In practice, the distinction means that the user of HttpClient will receive a malformed response body as the chunk framing will not be removed.

Without this change, the only workaround is to detect this case and remove chunk framing manually (~600 LOC).

Testing

Added a targeted test case for the new behavior.
We have existing test coverage to ensure nothing else regressed.

Risk

A user could be relying on the current non-RFC-compliant 6.0 behavior.

The risk of unrelated regressions is low - the change is trivial (swapping the order of two else if branches).

Author:	MihaZupan
Assignees:	-
Labels:	area-System.Net.Http, tenet-compatibility
Milestone:	-

[danmoseley]

approved for August -- missed July.