Skip to content

panic on checkNegotiationNeeded #1494

Closed

Description

@tarrencev

Seems there is a case where Track is nil while executing checkNegotiationNeeded. I don't think this should be possible, since the Senders direction is Send* but it doesn't have a track set. I think all senders in Send* state should have a track defined.

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x15caf3c]
goroutine 4980 [running]:
github.com/pion/webrtc/v3.(*RTPSender).Track(0x0, 0x0)
	/Users/tvanas/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.0-beta.10/rtpsender.go:80 +0x3c
github.com/pion/webrtc/v3.(*PeerConnection).checkNegotiationNeeded(0xc00011eb60, 0x15dc500)
	/Users/tvanas/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.0-beta.10/peerconnection.go:359 +0x91a
github.com/pion/webrtc/v3.(*PeerConnection).negotiationNeededOp(0xc00011eb60)
	/Users/tvanas/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.0-beta.10/peerconnection.go:301 +0x90
github.com/pion/webrtc/v3.(*operations).start(0xc0009aa240)
	/Users/tvanas/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.0-beta.10/operations.go:84 +0x61
created by github.com/pion/webrtc/v3.(*operations).Enqueue
	/Users/tvanas/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.0-beta.10/operations.go:35 +0xcb
exit status 2

Activity

  1. tomsun commented on Nov 5, 2020

    @tomsun

    Have had the same thing happen, possibly a few times

    	2020-11-05T10:10:59.953+02:00	[2020-11-05 08:10:59.953] [INFO] [62][sfu.go][Handle] => Handling......
    	2020-11-05T10:11:00.805+02:00	[2020-11-05 08:11:00.805] [INFO] [62][sfu.go][Handle] => Handling......
    	2020-11-05T10:11:04.881+02:00	[2020-11-05 08:11:04.881] [INFO] [2041][server.go][ServeHTTP] => Closing peer
    	2020-11-05T10:11:04.881+02:00	[2020-11-05 08:11:04.881] [ERROR] [180][simplesender.go][func1] => Error closing sender: InvalidStateError: connection closed
    	2020-11-05T10:11:04.885+02:00	panic: runtime error: invalid memory address or nil pointer dereference
    	2020-11-05T10:11:04.885+02:00	[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x8c783c]
    	2020-11-05T10:11:04.885+02:00	goroutine 30185328 [running]:
    	2020-11-05T10:11:04.885+02:00	github.com/pion/webrtc/v3.(*RTPSender).Track(0x0, 0x0)
    	2020-11-05T10:11:04.885+02:00	/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.0-beta.10.0.20201025013753-76bc99210140/rtpsender.go:80 +0x3c
    	2020-11-05T10:11:04.885+02:00	github.com/pion/webrtc/v3.(*PeerConnection).checkNegotiationNeeded(0xc0390151e0, 0x10000c0a82e8200)
    	2020-11-05T10:11:04.885+02:00	/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.0-beta.10.0.20201025013753-76bc99210140/peerconnection.go:359 +0x91a
    	2020-11-05T10:11:04.885+02:00	github.com/pion/webrtc/v3.(*PeerConnection).negotiationNeededOp(0xc0390151e0)
    	2020-11-05T10:11:04.885+02:00	/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.0-beta.10.0.20201025013753-76bc99210140/peerconnection.go:301 +0x90
    	2020-11-05T10:11:04.885+02:00	github.com/pion/webrtc/v3.(*operations).start(0xc10de379b0)
    	2020-11-05T10:11:04.885+02:00	/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.0-beta.10.0.20201025013753-76bc99210140/operations.go:84 +0x61
    	2020-11-05T10:11:04.885+02:00	created by github.com/pion/webrtc/v3.(*operations).Enqueue
    	2020-11-05T10:11:04.885+02:00
    	/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.0-beta.10.0.20201025013753-76bc99210140/operations.go:35 +0xcb
    	/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.0-beta.10.0.20201025013753-76bc99210140/operations.go:35 +0xcb
    
  2. Sean-Der commented on Nov 6, 2020

    @Sean-Der
    Member

    Really sorry about that @tomsun!

    We are planning on making this feature only fire if you set OnNegotiationNeeded and that should help. There is a lot of complexity here we are still working on nailing down :)

    @OrlandoCo Did you get a chance to investigate this at all? Any idea how hard/easy it is? I can probably grab it tomorrow if you haven't start yet!

  3. OrlandoCo commented on Nov 6, 2020

    @OrlandoCo
    Contributor

    @Sean-Der Sorry I have been fighting with a nasty chrome bug for a week now, andI have not had time, hope we can solve it soon!

  4. tomsun commented on Nov 23, 2020

    @tomsun

    Same in beta 12:

    2020-11-23T00:42:37.752+02:00	[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x93aaec]
    2020-11-23T00:42:37.752+02:00	goroutine 23477494 [running]:
    2020-11-23T00:42:37.752+02:00	github.com/pion/webrtc/v3.(*RTPSender).Track(0x0, 0x0)
    2020-11-23T00:42:37.752+02:00	/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.0-beta.12/rtpsender.go:80 +0x3c
    2020-11-23T00:42:37.752+02:00	github.com/pion/webrtc/v3.(*PeerConnection).checkNegotiationNeeded(0xc02200ed00, 0x0)
    2020-11-23T00:42:37.752+02:00	/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.0-beta.12/peerconnection.go:356 +0x8e5
    2020-11-23T00:42:37.752+02:00	github.com/pion/webrtc/v3.(*PeerConnection).negotiationNeededOp(0xc02200ed00)
    2020-11-23T00:42:37.752+02:00	/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.0-beta.12/peerconnection.go:300 +0x90
    2020-11-23T00:42:37.752+02:00	github.com/pion/webrtc/v3.(*operations).start(0xc049bcd860)
    2020-11-23T00:42:37.753+02:00	/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.0-beta.12/operations.go:84 +0x61
    2020-11-23T00:42:37.753+02:00	created by github.com/pion/webrtc/v3.(*operations).Enqueue
    2020-11-23T00:42:37.753+02:00	/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.0-beta.12/operations.go:35 +0xcb
    
    

    Also a similar but different one:

    panic: runtime error: invalid memory address or nil pointer dereference
    [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x154d06b]
    
    goroutine 1190 [running]:
    github.com/pion/webrtc/v3.(*Track).Label(0x0, 0x0, 0x0)
    	/Users/a16/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.0-beta.12/track.go:73 +0x3b
    github.com/pion/webrtc/v3.(*Track).Msid(0x0, 0x0, 0x4)
    	/Users/a16/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.0-beta.12/track.go:87 +0x2f
    github.com/pion/webrtc/v3.(*PeerConnection).checkNegotiationNeeded(0xc000239ba0, 0x0)
    	/Users/a16/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.0-beta.12/peerconnection.go:356 +0x8f3
    github.com/pion/webrtc/v3.(*PeerConnection).negotiationNeededOp(0xc000239ba0)
    	/Users/a16/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.0-beta.12/peerconnection.go:300 +0x90
    github.com/pion/webrtc/v3.(*operations).start(0xc000afc960)
    	/Users/a16/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.0-beta.12/operations.go:84 +0x61
    created by github.com/pion/webrtc/v3.(*operations).Enqueue
    	/Users/a16/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.0-beta.12/operations.go:35 +0xcb
    exit status 2
    [Sentry] 2020/11/23 02:28:37 Buffer flushed successfully.
    
  5. Sean-Der commented on Nov 23, 2020

    @Sean-Der
    Member

    @tomsun sorry about that! I opened #1548

    We will get that merged today and tagged so you can stop seeing that :(

  6. Sean-Der commented on Nov 23, 2020

    @Sean-Der
    Member

    @tomsun I just tagged v3.0.0-beta.14, so you shouldn't hit this anymore!

    We are two PRs away from v3.0.0 as well. Will be working on lots of stabilization tasks as we get closer :)

    thanks for using Pion, really appreciate the patience with the bugs.

  7. tomsun commented on Nov 23, 2020

    @tomsun

    🙌 thanks @Sean-Der - awesome!

    (Mainly trying out https://github.com/pion/webrtc through https://github.com/pion/ion-sfu - maybe you're right that the sfu doesn't implement renegotiate itself..? Trying out your patch 00ec48a now 🙏 , on top of where we otherwise were, using some replace statements in relevant go.mod:s )

    (Also using pion/webrtc client-side, with peerConnection.OnNegotiationNeeded implemented. A crash in that client has a slightly better recovery path though, as it only has to recover itself. Haven't actually seen the above crashes in the client though (although the logs in that app are quite noisy))

  8. added 2 commits that reference this issue on Dec 5, 2020
    7ea9d7a
    d2ac79a
  9. tekig commented on Apr 3, 2021

    @tekig
    Member

    The problem came up when removing a track that was not thread safe that the PR closed. Now sender cannot become unexpectedly nil. I think the problem can be closed.

  10. Sean-Der commented on Apr 3, 2021

    @Sean-Der
    Member

    That's great news! Sorry that stuff got lost in the shuffle @tekig :(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      ⌂ 换网址