Is DigestAuthentication supported for proxy? Can you connect to HTTPS website this way?

[raffaem]

Does httpx support DigestAuthentication for proxies, and if so, can you connect to HTTPS websites through a proxy that requires DigestAuthentication?

requests-toolbet has a HTTPProxyDigestAuth class, but it does not allow you to connect to HTTPS through the proxy.

Apparently that feature won't be implemented in the foreseeable future in requests.

The only working implementation that I have found, and which allows you to connect to HTTPS websites through the proxy, is this one, which was provided in this StackOverflow post.

Does httpx support this feature, or will it support it?

[lovelydinosaur]

Does httpx support this feature

No - you're the first person to ask about digest authentication for proxies.

will it support it

It's not currently on the roadmap, beyond this initial query.

Out of interest - what's your motivation are? What proxy are you using, and why digest authentication?

[raffaem]

what's your motivation are?

To connect to a HTTPS website declared in the PAC file of the proxy using Python instead of the web browser

What proxy are you using

The proxy of my university

why digest authentication?

You have to ask the university why they chose DigestAuth instead of BasicAuth

[lovelydinosaur]

😀 Thanks - useful to get better idea of the complete context.

I had a quick look on my desktop (Mac OSX) and for the system controls for proxies it doesn't have any built-in way of using digest authentication at all. I think it's fairly unlikely we'd prioritise this, since it seems like a bit of a niche use-case.

Other useful questions just to get a well-rounded picture of this particular use-case in a real world setting...

• What browser and OS do you use?
• How do you configure those to access via the proxy?
• Is it an HTTP or an HTTPS proxy?

[raffaem]

What browser and OS do you use?

Browser: Firefox 95.0.2, OS: Fedora Workstation 35

How do you configure those to access via the proxy?

From Firefox you click on the hamburger menu -> Settings -> General tab -> Network settings -> Settings -> Automatic proxy configuration URL and you put the address of the PAC file there

Is it an HTTP or an HTTPS proxy?

The proxy itself is HTTP proxy

[lovelydinosaur]

From Firefox you click on the hamburger menu -> Settings -> General tab -> Network settings -> Settings -> Automatic proxy configuration URL and you put the address of the PAC file there

Gotcha thanks.

• How are you specifying the username/password? As part of the proxy URL or something else?
• How are you able to tell that the proxy requires digest auth? I don't think PAC files have any way of indicating that digest should be used?

[raffaem]

How are you specifying the username/password? As part of the proxy URL or something else?

Where?

Firefox asks me for username and password in a dialog when you connect to a website specified in the PAC file.

For the StackOverflow module, you specify it when you create the class.

How are you able to tell that the proxy requires digest auth?

Because of the Proxy-Authenticate: Digest here:

$ curl -v -x MYPROXYDOMAIN:8080 https://dev.elsevier.com/
*   Trying MYPROXYIP:8080...
* Connected to MYPROXYDOMAIN (MYPROXYIP) port 8080 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to dev.elsevier.com:443
> CONNECT dev.elsevier.com:443 HTTP/1.1
> Host: dev.elsevier.com:443
> User-Agent: curl/7.79.1
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 407 Proxy Authentication Required
< Server: squid
< Mime-Version: 1.0
< Date: Fri, 14 Jan 2022 13:06:15 GMT
< Content-Type: text/html;charset=utf-8
< Content-Length: 3279
< X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
< Vary: Accept-Language
< Content-Language: en
< Proxy-Authenticate: Digest realm="MYCODE", nonce="MYNONCE", qop="auth", stale=false
< X-Cache: MISS from MYPROXYDOMAIN
< Via: 1.1 MYPROXYDOMAIN (squid)
< Connection: keep-alive
< 
* Ignore 3279 bytes of response-body
* Received HTTP code 407 from proxy after CONNECT
* CONNECT phase completed!
* Closing connection 0
curl: (56) Received HTTP code 407 from proxy after CONNECT

I don't think PAC files have any way of indicating that digest should be used?

The proxy will answer with 407 and a Proxy-Authenticate: Digest

[lovelydinosaur]

Okay, that all makes sense, thanks.

If you did want to try to resolve this yourself, this is the point in httpcore where the proxy connect response comes back.

You'd need to adapt that to deal with the 407 Proxy-Authenticate: Digest case.

The digest auth implementation in the httpx codebase would be a good reference point for figuring out how to construct the correct auth header.

I could see us supporting this in the future perhaps, but it's not super high up on the list.

[raffaem]

Okay, that all makes sense, thanks.

If you did want to try to resolve this yourself, this is the point in httpcore where the proxy connect response comes back.

You'd need to adapt that to deal with the 407 Proxy-Authenticate: Digest case.

The digest auth implementation in the httpx codebase would be a good reference point for figuring out how to construct the correct auth header.

I could see us supporting this in the future perhaps, but it's not super high up on the list.

Yeah, I don't think I have the skills to do that.

It would take me much much longer to implement that than what it would take a skilled person.

[lovelydinosaur]

Okay, I've escalated this into an issue, so yeah - it'll get addressed at some point.
Looks valid and worthwhile to me. If anyone else is running into this at all it'd be helpful to also get their feedback on either this discussion, or on the issue, so we can assess how much need (or not) there is for it.

See #2033

[raffaem]

@tomchristie Thanks! :)