HTTP/2: Add automatic downgrade to HTTP/1.1 for Windows authentication

[Copilot]

Description

Session-based authentication (NTLM/Negotiate) requires persistent connections and fails over HTTP/2. When SocketsHttpHandler receives a 401 with these schemes on HTTP/2, it now automatically retries on HTTP/1.1 if the request is retryable.

Changes

Core retry mechanism:

• Added RequestRetryType.RetryOnSessionAuthenticationChallenge to signal auth-driven downgrades
• Http2Connection.SendAsync() detects session auth challenges (401 + NTLM/Negotiate) and checks version policy before attempting retry
• HttpConnectionPool.SendWithVersionDetectionAndRetryAsync() catches and retries on HTTP/1.1

Safety constraints:

• Only retries when ALL conditions are met:
  • Request has no content (request.Content == null)
  • Version policy allows downgrade (request.VersionPolicy == HttpVersionPolicy.RequestVersionOrLower)
• Returns original 401 response when any retry condition is not met (preserves backward compatibility)
• No race conditions or unsafe content access

Test coverage:

• Validates successful downgrade from HTTP/2 to HTTP/1.1 for GET requests without content
• Validates 401 behavior when version policy prevents downgrade (RequestVersionExact)
• Validates 401 behavior when request has content (POST requests)

Example

var handler = new SocketsHttpHandler { Credentials = credentials };
var client = new HttpClient(handler);

// Works: GET request without content + RequestVersionOrLower
var request = new HttpRequestMessage(HttpMethod.Get, "https://server/api");
request.Version = HttpVersion.Version20;
request.VersionPolicy = HttpVersionPolicy.RequestVersionOrLower;

// Previously: 401 Unauthorized (no retry)
// Now: Automatic retry on HTTP/1.1, successful auth
var response = await client.SendAsync(request);

// Not retried: POST request with content
var postRequest = new HttpRequestMessage(HttpMethod.Post, "https://server/api");
postRequest.Content = new StringContent("data");
postRequest.Version = HttpVersion.Version20;
postRequest.VersionPolicy = HttpVersionPolicy.RequestVersionOrLower;

// Returns: 401 Unauthorized (cannot safely retry with content)
var postResponse = await client.SendAsync(postRequest);

// Not retried: GET request with RequestVersionExact policy
var exactRequest = new HttpRequestMessage(HttpMethod.Get, "https://server/api");
exactRequest.Version = HttpVersion.Version20;
exactRequest.VersionPolicy = HttpVersionPolicy.RequestVersionExact;

// Returns: 401 Unauthorized (version policy doesn't allow downgrade)
var exactResponse = await client.SendAsync(exactRequest);

Limitations

This implementation only handles requests without content (e.g., GET, HEAD, DELETE without body) when the version policy allows downgrade. POST and PUT requests with content, or requests with RequestVersionExact policy, will receive a 401 response and must be handled by the application, as we cannot safely retry requests with content that may not be rewindable or when downgrade is not permitted.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>HTTP2: SocketsHttpHandler doesn't downgrade to HTTP/1.1 during Windows authentication</issue_title>
<issue_description>When SocketsHttpHandler connects to a server using HTTP/2, it is unable to downgrade to HTTP/1.1 when the server requests Windows authentication (Negotiate or NTLM). The net result is that 401 is returned to the client without even trying to reconnect (with HTTP/1.1) and send credentials.

// Repro will be posted later
```</issue_description>

<agent_instructions>Add logic that allows the SocketsHttpHandler connection pool to attempt to retry requests when the HTTP/2 request failed due to a Windows auth challenge, and the user-specified version & policy allow for the upgrade to happen.
Keep in mind that not every request will be possible to retry (e.g. a POST where we've tried sending the body and are unable to rewind).</agent_instructions>

## Comments on the Issue (you are @copilot in this section)

<comments>
<comment_new><author>@karelz</author><body>
@wfurt can you please chat with @davidsh about design / ideas here?</body></comment_new>
<comment_new><author>@karelz</author><body>
Given we scoped HTTP/2 to gRPC scenarios, this can wait for post-3.0.</body></comment_new>
<comment_new><author>@Tratcher</author><body>
I noticed this while doing some auth testing. IE, Chrome, and Edge all have this fallback behavior. Also, the downgrade isn't just for the auth handshake, all subsequent requests are made on the HTTP/1.1 connection so they share the cached auth context.

WinHttpHandler is a bit different. The first HTTP/2 request challenges, falls back to HTTP/1.1, authenticates, and completes successfully. However, the next request is still made over HTTP/2, the client attempts the fallback again, and then it re-authenticates on the already authenticated connection. That could be a major performance hole to fall into.</body></comment_new>
<comment_new><author>@halter73</author><body>
This came up again. This time from a SignalR client user. See https://github.com/dotnet/aspnetcore/issues/45371. Prior to .NET 7, the SignalR client forced HTTP/1.1 but we started allowing HTTP/2 in .NET 7. We're considering backporting a fix for SignalR to force HTTP/1.1 if the customer sets `HttpConnectionOptions.UseDefaultCredentials` to true.

@karelz Even though we have a fix for SignalR, we can't help but think HttpClient should have avoided this for us. Do you think we could do something similar in .NET 8 at the SocketsHttpHandler level when UseDefaultCredentials is set? Of course, the handler could be smarter than SignalR and wait for a challenge before downgrading,</body></comment_new>
<comment_new><author>@karelz</author><body>
@wfurt @ManickaP @CarnaViire any thoughts on this? (marking for re-triage given new scenario being impacted)</body></comment_new>
<comment_new><author>@wfurt</author><body>
There are (at least) two distance parts IMHO. 
1) get parity with WinHttp - The existing retry logic will be problematic as this needs new connection (and possibly check for version policy). This can probably be done by refactoring the connection pool logic.
2) Browsers do have global cache and therefore they can remember state. `HttpClientHandler` is stateless and each request is processed independently. To fix that, we would need whole new concept in `SocketsHttpHandler`.  That may be useful for other reasons but it would be fundamental change IMHO.

While we _should_ do something I would keep it in `Future` for now given complexity and no up votes e.g. no real customer pain. The workaround is somewhat easy -> just set version to 1.1 for problematic sites. </body></comment_new>
<comment_new><author>@karelz</author><body>
Agreed with @wfurt. It seems it is still rather rare with a complex solution, so leaving it in Future sounds like the right option now.
@halter73 let us know if you disagree.</body></comment_new>
</comments>

• Fixes HTTP2: SocketsHttpHandler doesn't downgrade to HTTP/1.1 during Windows authentication #1582

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[MihaZupan]

@copilot

[dotnet-policy-service]

Tagging subscribers to this area: @karelz, @dotnet/ncl
See info in area-owners.md if you want to be subscribed.