Skip to content

net/http/httputil: add WebSocket support to ReverseProxy #26937

Closed

Description

@bradfitz

Add WebSocket support to ReverseProxy.

I have code for this that I use myself in various projects, but I keep forgetting to add it to Go.

Activity

  1. added
    NeedsFixThe path to resolution is known, but the work has not been done.
    on Aug 12, 2018
  2. added this to the Go1.12 milestone on Aug 12, 2018
  3. self-assigned this
    on Aug 12, 2018
  4. nhooyr commented on Aug 12, 2018

    @nhooyr
    Contributor

    This would need to be off by default in case current clients rely on the lack of support to prevent websocket connections.

  5. bradfitz commented on Aug 12, 2018

    @bradfitz
    ContributorAuthor

    @nhooyr, that seems like a stretch. We'd document it in the release notes and say how to disable it if people wanted to. But if the two sides negotiate it and the Go user expressed their intent to wire up the two sides with a ReverseProxy, I don't think it's crazy to say we'd wire it up all the way.

  6. nhooyr commented on Aug 12, 2018

    @nhooyr
    Contributor

    @bradfitz Fair enough. Its extremely unlikely anyone would not want WebSockets support anyway.

  7. nhooyr commented on Aug 12, 2018

    @nhooyr
    Contributor

    @bradfitz Could you post the code for this, I've written something similar myself, want to make sure I'm doing it right.

  8. nhooyr commented on Aug 12, 2018

    @nhooyr
    Contributor

    We should also support arbitrary upgrades instead of just WebSockets.

  9. nhooyr commented on Aug 12, 2018

    @nhooyr
    Contributor

    @bradfitz In fact, if you post it, I'd love to work on the CL for this.

  10. bradfitz commented on Aug 12, 2018

    @bradfitz
    ContributorAuthor

    @nhooyr, I'll send it myself, as I already have the code. It won't save me any time to hand it to you to hand back to me. You can review it once it's on Gerrit, though.

  11. gopherbot commented on Aug 24, 2018

    @gopherbot
    Contributor

    Change https://golang.org/cl/131279 mentions this issue: net/http: make Transport return Writable Response.Body on WebSocket upgrade

  12. bradfitz commented on Aug 24, 2018

    @bradfitz
    ContributorAuthor

    I decided to implement this a different way, not using my existing code. Instead, I sent https://golang.org/cl/131279 to add Transport support for WebSockets, so the ReverseProxy code won't need to separately dial the backend.

  13. nhooyr commented on Sep 6, 2018

    @nhooyr
    Contributor

    Given its not going to use your existing code @bradfitz, can I work on the CL?

  14. dmitshur commented on Sep 7, 2018

    @dmitshur
    Member

    @bradfitz, I thought ReverseProxy already supported WebSocket connections as long as FlushInterval is set to a non-zero value. How does your proposed solution compare?

  15. 16 remaining items

  16. gopherbot commented on Sep 25, 2018

    @gopherbot
    Contributor

    Change https://golang.org/cl/137437 mentions this issue: net/http: make Transport send WebSocket upgrade requests over HTTP/1

  17. bradfitz commented on Oct 9, 2018

    @bradfitz
    ContributorAuthor

    @nhooyr, if we went with Hijacker for the client side, what would we do for HTTP/2 connections? It's not applicable to WebSockets today, as WS doesn't run on HTTP/2 yet, but I want to use the same mechanism for other stuff like CONNECT requests.

    Currently Hijacker only works on the server side and only for HTTP/1. We don't support HTTP/2 because Hijacker returns a net.Conn and in HTTP/2, the actual TCP/TLS connection is shared.

    So would we make our own net.Conn impl for the client side hijack? But if we did so, then we'd be in a situation where Hijacker works in 3 of the 4 cases (client HTTP/1 and HTTP/2, server HTTP/1, but not server HTTP/2), which seems like it should then also be fixed.

    What were you thinking?

    I agree the Response.Body being a ReadWriteCloser is also an API change, but I was thinking it'd only apply when the connection was already effectively taken over for life (Connection: upgrade and CONNECT), so it seemed harmless. Doing the ReadWriteCloser thing also doesn't preclude us from doing client-side Hijack (#28030) in the future if necessary. We could support both and people could choose the API that works best for them. Slightly redundant, but I'm not convinced we'd even get there before the HTTP packages are redesigned anyway.

  18. nhooyr commented on Oct 12, 2018

    @nhooyr
    Contributor

    So would we make our own net.Conn impl for the client side hijack? But if we did so, then we'd be in a situation where Hijacker works in 3 of the 4 cases (client HTTP/1 and HTTP/2, server HTTP/1, but not server HTTP/2), which seems like it should then also be fixed.

    For server HTTP/2 as you mentioned in #17227 (comment) its kind of useless to hijack. Though it'd be nice if we could close the stream and unlock any read/writes if a timeout of some sort gets triggered. It seems wrong to implement that separately from http.Hijacker though. More API = more user confusion and a harder to use package.

    In consideration of that, I'd be for a http.Hijacker based API where we implement net.Conn ourselves for all the http/2 cases. While it is more complicated, it handles every scenario and as a cherry on top, it is more performant as we can reuse the buffered read/writer.

    Another advantage is that when WebSockets do run on HTTP/2, existing code will have a very easy time adapting because everything will just work. They'll just need to check the pseudo protocol header described in https://tools.ietf.org/html/draft-ietf-httpbis-h2-websockets-00 which we would need to expose somehow.

    Though, would we have a buffered read/writer to return for the HTTP/2 stream?

  19. gopherbot commented on Oct 31, 2018

    @gopherbot
    Contributor

    Change https://golang.org/cl/146437 mentions this issue: net/http/httputil: make ReverseProxy automatically proxy WebSocket requests

  20. jrefior commented on Nov 2, 2018

    @jrefior

    I just ran into this problem today. Is there any workaround until a fix/feature is released?

  21. jrefior commented on Nov 3, 2018

    @jrefior

    I've gotten pretty far with writing my own reverse proxy based on your changes, but I'm basically stuck here right now:

        backConn, ok := res.Body.(io.ReadWriteCloser)
        if !ok {
            log.Error("internal error: 101 switching protocols response with non-writable body: %v (%T)",
     res.Body, res.Body)
            return
        }
    

    The type assertion fails. Output is:

    ERRO[0012] internal error: 101 switching protocols response with non-writable body: {} (http.noBody)

    Any suggestion for how to resolve?

    This is what res looks like in my log:

    Server response was &{Status:101 Switching Protocols StatusCode:101 Proto:HTTP/1.1 
    ProtoMajor:1 ProtoMinor:1 Header:map[Upgrade:[websocket] Connection:[Upgrade]
    Sec-Websocket-Accept:[B1fWETPTtNo0bAHFWkFn2lAm9iw=]] Body:{} ContentLength:0 
    TransferEncoding:[] Close:false Uncompressed:false Trailer:map[] Request:0xc000112100 
    TLS:<nil>}
    

    Ah, I think I figured out why I'm stuck here. The latest release of Go does not include this line in persistConn's readLoop function in transport.go in the net/http package:

    if resp.isProtocolSwitch() {
        resp.Body = newReadWriteCloserBody(pc.br, pc.conn)
    }
    

    See earlier change here. Any suggestion for a way to work around that?

    I also posted this as a StackOverflow question.

  22. bradfitz commented on Nov 5, 2018

    @bradfitz
    ContributorAuthor

    @jrefior, if you can't wait for Go 1.12, you can do something like https://gist.github.com/bradfitz/1d7bdf12278d4d713212ce6c74875dab

  23. jrefior commented on Nov 5, 2018

    @jrefior

    Wow, thank you @bradfitz ! For my purposes, I don't need TLS to the backend as traffic will stay within my network, so I can just replace tls.DialWithDialer with net.DialTimeout, right? Awesome, looks simpler than what I was building.

  24. bradfitz commented on Nov 5, 2018

    @bradfitz
    ContributorAuthor

    @jrefior, I'll let you play around with it and figure out what you need. I'm focused on Go 1.12. Let's move any pre-Go 1.12 discussion & updates out of this bug, though. I'd like to keep this focused on Go 1.12 stuff.

  25. added a commit that references this issue on Mar 1, 2019
  26. locked and limited conversation to collaborators on Nov 13, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    FrozenDueToAgeNeedsFixThe path to resolution is known, but the work has not been done.

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      ⌂ 换网址