Proxy incorrectly changes 301 redirects to 200 responses when using chunked encoding

[czue]

I've identified what I believe is an issue where the proxy changes 301 redirects to 200 responses, but only when the response uses chunked encoding. This is maybe specifically in issue for Django applications, where the default APPEND_SLASH behavior results in this problem (at least when using gunicorn).

Here's how to reproduce in a Django application. Note you must serve the application with gunicorn or another server that automatically switches to chunked encoding in this circumstance):

urls.py:

    path("redirect_good/", views.redirect_good, name="redirect_good"),
    path("redirect_bad/", views.redirect_bad, name="redirect_bad"),
    path("redirect_result/", views.redirect_result, name="redirect_result"),

view.py

def redirect_good(request):
    return HttpResponsePermanentRedirect("/redirect_result/")


def redirect_bad(request):
    response = HttpResponsePermanentRedirect("/redirect_result/")
    response.streaming = True
    return response


def redirect_result(request):
    return HttpResponse("It worked!")

Basically, /redirect_good/ does a normal 301 redirect to /redirect_result/. /redirect_bad/ does the same but with chunked encoding. You can see from curl responses that the first one correctly returns a 301:

$ curl -v https://lifeweeks.app/redirect_good/
< HTTP/2 301 
< content-language: en
< content-type: text/html; charset=utf-8
< cross-origin-opener-policy: same-origin
< date: Fri, 21 Feb 2025 08:48:36 GMT
< location: /redirect_result/
< referrer-policy: same-origin
< server: gunicorn
< set-cookie: lifeweeks_language=en; Path=/
< vary: Accept-Language, Cookie
< x-content-type-options: nosniff
< x-frame-options: DENY
< content-length: 0

Which correctly resolves https://lifeweeks.app/redirect_good/

But the second returns a 200:

$ curl -v https://lifeweeks.app/redirect_bad/
< HTTP/2 200 
< content-language: en
< content-type: text/html; charset=utf-8
< cross-origin-opener-policy: same-origin
< date: Fri, 21 Feb 2025 08:48:43 GMT
< location: /redirect_result/
< referrer-policy: same-origin
< server: gunicorn
< set-cookie: lifeweeks_language=en; Path=/
< vary: Accept-Language, Cookie
< x-content-type-options: nosniff
< x-frame-options: DENY
< 

Which results in an empty page https://lifeweeks.app/redirect_bad/

[czue]

In case any other Django developers run into this, replacing CommonMiddleware with this one solves the broken APPEND_SLASH behavior.

class NoChunkedRedirectMiddleware(CommonMiddleware):
    """
    Forces redirects to not use chunked encoding (which gunicorn does by default)
    to workaround this bug: https://github.com/basecamp/kamal-proxy/discussions/115
    """

    def process_response(self, request, response):
        # Get the response from parent first (this creates redirects if needed)
        response = super().process_response(request, response)

        # Then fix chunked encoding if it's a redirect, except for test path
        if response.status_code == 301:
            response.streaming = False
            if "Transfer-Encoding" in response:
                del response["Transfer-Encoding"]
            if not response.content:
                response["Content-Length"] = "0"

        return response

[kevinmcconnell]

@czue thanks for reporting this. I've not been able to reproduce it by adding Transfer-Encoding: chunked -- I'm still seeing the expected redirect status code in that case. But there may be something else to it.

Could you share a full redirect response from your app that triggers this behavior? That is, not the response from the proxy, but the response your app generates directly. Thanks!

[czue]

Thanks for taking a look. These are the same traces for what's running behind kamal-proxy. I don't see any other notable differences, but I'm definitely not an expert in this space. Did you also remove the content-length?

Let me know if you need anything else!

Failing URL:

$ curl -v localhost:8001/redirect_bad/
* Host localhost:8001 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:8001...
* connect to ::1 port 8001 from ::1 port 54460 failed: Connection refused
*   Trying 127.0.0.1:8001...
* Connected to localhost (127.0.0.1) port 8001
> GET /redirect_bad/ HTTP/1.1
> Host: localhost:8001
> User-Agent: curl/8.5.0
> Accept: */*
> 
< HTTP/1.1 301 Moved Permanently
< Server: gunicorn
< Date: Thu, 27 Feb 2025 14:35:22 GMT
< Connection: keep-alive
< Transfer-Encoding: chunked
< Content-Type: text/html; charset=utf-8
< Location: /redirect_result/
< X-Frame-Options: DENY
< Vary: Accept-Language, Cookie
< Content-Language: en
< X-Content-Type-Options: nosniff
< Referrer-Policy: same-origin
< Cross-Origin-Opener-Policy: same-origin
< Set-Cookie: lifeweeks_language=en; Path=/
< 
* Connection #0 to host localhost left intact

Working URL:

$ curl -v localhost:8001/redirect_good/
* Host localhost:8001 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:8001...
* connect to ::1 port 8001 from ::1 port 50488 failed: Connection refused
*   Trying 127.0.0.1:8001...
* Connected to localhost (127.0.0.1) port 8001
> GET /redirect_good/ HTTP/1.1
> Host: localhost:8001
> User-Agent: curl/8.5.0
> Accept: */*
> 
< HTTP/1.1 301 Moved Permanently
< Server: gunicorn
< Date: Thu, 27 Feb 2025 14:35:27 GMT
< Connection: keep-alive
< Content-Type: text/html; charset=utf-8
< Location: /redirect_result/
< X-Frame-Options: DENY
< Content-Length: 0
< Vary: Accept-Language, Cookie
< Content-Language: en
< X-Content-Type-Options: nosniff
< Referrer-Policy: same-origin
< Cross-Origin-Opener-Policy: same-origin
< Set-Cookie: lifeweeks_language=en; Path=/
< 
* Connection #0 to host localhost left intact

[kevinmcconnell]

@czue I've still not been able to reproduce this. I've tried creating a Django app with the urls/views you shared above, and running it with gunicorn, but the redirects are coming through as expected:

Direct from app:

 $ curl -v http://localhost:8000/redirect_bad/
* Host localhost:8000 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:8000...
* connect to ::1 port 8000 from ::1 port 36166 failed: Connection refused
*   Trying 127.0.0.1:8000...
* Connected to localhost (127.0.0.1) port 8000
> GET /redirect_bad/ HTTP/1.1
> Host: localhost:8000
> User-Agent: curl/8.9.1
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 301 Moved Permanently
< Server: gunicorn
< Date: Fri, 28 Feb 2025 12:10:15 GMT
< Connection: close
< Transfer-Encoding: chunked
< Content-Type: text/html; charset=utf-8
< Location: /redirect_result/
< X-Frame-Options: DENY
< X-Content-Type-Options: nosniff
< Referrer-Policy: same-origin
< Cross-Origin-Opener-Policy: same-origin
<
* shutting down connection #0

Via Kamal Proxy:

 $ curl -v http://localhost:5000/redirect_bad/
* Host localhost:5000 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:5000...
* Connected to localhost (::1) port 5000
> GET /redirect_bad/ HTTP/1.1
> Host: localhost:5000
> User-Agent: curl/8.9.1
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 301 Moved Permanently
< Content-Type: text/html; charset=utf-8
< Cross-Origin-Opener-Policy: same-origin
< Date: Fri, 28 Feb 2025 12:10:41 GMT
< Location: /redirect_result/
< Referrer-Policy: same-origin
< Server: gunicorn
< X-Content-Type-Options: nosniff
< X-Frame-Options: DENY
< Transfer-Encoding: chunked
<
* Connection #0 to host localhost left intact

Same result when handling TLS/HTTP2 in the proxy as well.

Is there anything else in your setup that could be a factor here? Any other service running between the client and Kamal Proxy, for example?

[czue]

Wow! Weird. Nothing I'm aware of. Let me try to create a complete demo project that reproduces it and share it back here.

[kevinmcconnell]

That'd be great. Thanks!

[czue]

Arg. So I managed to create the project and reproduce the bug. Then, before sharing with you I thought I should check my kamal / proxy version. Turns out I was on an older one (2.2 / 8.0). After upgrading to the latest (2.5.3/8.4(?)) it's now working.

So sorry for wasting your time and thanks for taking a serious look at it!

If you ever do need to reproduce anything with Django in the future, maybe this repo will be helpful? https://github.com/czue/django-kamal-demo

[kevinmcconnell]

Aha! That's great though, I'm glad to hear it's no longer an issue!

Funnily enough I did try going back a couple of versions to check if it was an old bug, but I clearly didn't go back far enough.

Thanks for reporting this all the same, and for the clear steps to reproduce! That repo does look like it will be helpful 🙏