Skip to content

go 1.26 build panics on Windows 10 #11214

New issue
New issue
Closed
#11275
Closed
go 1.26 build panics on Windows 10#11214
#11275
Labels
kind/bugA bug in existing code (including security flaws)kind/maintenanceWork required to avoid breaking changes or harm to project's status quostatus/blockedUnable to be worked further until needs are mettopic/windowsWindows specific

Description

@lidel
lidel
opened on Feb 27, 2026

Found this while testing Windows package from

The daemon starts normally, gets to "Daemon is ready", WebUI at https://127.0.0.1:5001/webui works for a short while, then daemon crashes, RPC stops responding.

Some notes below, but need to look at this more closely + reproduce on a different machine.

Crash signature

Exception 0xc0000005 0x8 0x1ce5935180 0x1ce5935180
PC=0x1ce5935180
runtime: g 23791 gp=0x56c75f563c0: unknown pc 0x1ce5935180
  • 0xc0000005 = STATUS_ACCESS_VIOLATION
  • 0x8 = EXCEPTION_EXECUTE_FAULT (DEP violation: CPU attempted to execute non-executable memory)
  • The faulting PC (0x1ce5935180) is outside the binary's code range (legitimate addresses are in 0x7ff6e58xxxxx-0x7ff6e71xxxxx), indicating a corrupted return address on the stack

The crashing goroutine was created by go-yamux/v5.newSession and was doing normal TCP socket I/O through the Windows overlapped I/O path:

internal/poll.(*FD).execIO
  -> internal/syscall/windows.WSAGetOverlappedResult
    -> syscall.SyscallN

Suspecting Go regression on Windows

Go 1.25 introduced a major rework of Windows I/O completion port (IOCP) handling (go#19098). This rework has caused several regressions on Windows that are still being worked through in Go 1.25.x/1.26.x:

  • go#76391 - NewFile hangs on blocked handles (critical release-blocker, multiple CLs fixing execIO internals)
  • go#77142 - overlapped I/O interop issues

The crash happens in internal/poll.(*FD).execIO, which is the code path affected by these regressions. However, the specific bug causing this crash may not be one of the already-reported issues. It could be an unreported runtime bug, or potentially antivirus interference (go#40878 documented identical "unknown pc" crashes caused by AV software on Windows).

Suggested action

  • Ask someone to try to reproduce on Windows 10 or 11
    • If reproducible, file an upstream Go issue with the full crash log so the runtime team can assess whether this is a known or new bug in the IOCP rework
    • Consider building Windows release binaries with Go 1.24.x (last version before the IOCP rework) until the Go 1.25/1.26 Windows I/O issues are resolved

Environment

  • Kubo 0.40.0 (882b7d2a9)
  • Go 1.26.0
  • Windows 10, amd64

Activity

  1. lidel
    added
    kind/bugA bug in existing code (including security flaws)
    need/triageNeeds initial labeling and prioritization
    on Feb 27, 2026

  2. lidel commented on Feb 27, 2026

    @lidel
    lidel
    on Feb 27, 2026
    ContributorAuthor

    @aschmahmann are you able to reproduce by any chance?
    (run ipfs daemon on Windows, open http://127.0.0.1:5001/webui, wait a few minutes)

  3. aschmahmann commented on Feb 27, 2026

    @aschmahmann
    aschmahmann
    on Feb 27, 2026
    Contributor

    Was able to reproduce a crash fairly quickly when I initially tried.

    This is my error message from the first time I ran it, it's a little different from yours though:

    Details 
    ...
Daemon is ready
unexpected fault address 0x10012ec6b0
fatal error: fault
[signal 0xc0000005 code=0x0 addr=0x10012ec6b0 pc=0x7ff69eca76a6]

goroutine 59876 gp=0x10330263e5a0 m=16 mp=0x1032fa580808 [running]:
runtime.throw({0x7ff6a04a23a9?, 0x4?})
	runtime/panic.go:1229 +0x4d fp=0x1032fd031c68 sp=0x1032fd031c38 pc=0x7ff69e57b58d
runtime.sigpanic()
	runtime/signal_windows.go:398 +0xd0 fp=0x1032fd031cb0 sp=0x1032fd031c68 pc=0x7ff69e55b710
github.com/libp2p/go-libp2p-kad-dht/pb.(*Message).GetCloserPeers(...)
	github.com/libp2p/go-libp2p-kad-dht@v0.38.0/pb/dht.pb.go:225
github.com/libp2p/go-libp2p-kad-dht/pb.(*ProtocolMessenger).GetClosestPeers(0x1032fa77d090, {0x7ff6a083cf90?, 0x1032fba782a0?}, {0x103301351ec0, 0x26}, {0x103301351ec0, 0x26})
	github.com/libp2p/go-libp2p-kad-dht@v0.38.0/pb/protocol_messenger.go:168 +0x586 fp=0x1032fd031e80 sp=0x1032fd031cb0 pc=0x7ff69eca76a6
github.com/libp2p/go-libp2p-kad-dht.(*IpfsDHT).lookupCheck(0x1032fa71ea88, {0x7ff6a083cf90?, 0x1032fba782a0?}, {0x103301351ec0, 0x26})
	github.com/libp2p/go-libp2p-kad-dht@v0.38.0/dht.go:383 +0x49 fp=0x1032fd031ef8 sp=0x1032fd031e80 pc=0x7ff69ecd90c9
github.com/libp2p/go-libp2p-kad-dht.(*IpfsDHT).peerFound.func1()
	github.com/libp2p/go-libp2p-kad-dht@v0.38.0/dht.go:692 +0x8f fp=0x1032fd031fe0 sp=0x1032fd031ef8 pc=0x7ff69ecdb2af
runtime.goexit({})
	runtime/asm_amd64.s:1771 +0x1 fp=0x1032fd031fe8 sp=0x1032fd031fe0 pc=0x7ff69e583821
created by github.com/libp2p/go-libp2p-kad-dht.(*IpfsDHT).peerFound in goroutine 295
	github.com/libp2p/go-libp2p-kad-dht@v0.38.0/dht.go:687 +0x179
...

    Regarding Go versions:

    1. I wouldn't blame Go 1.25, the last release had that too and we haven't seen issues here
    2. We can't actually build this version of kubo with Go 1.25 because we updated to Go 1.26 here (feat: update to Go 1.26 #11189). Note: we'd have to go even further back to be able to build with Go 1.24.

    It seems like reverting 9ca1dbb shouldn't interfere with much, so I've locally reverted that commit on top of the 0.40.0 commit and build with Go 1.26.0 and I'm seeing the error. I'm now building and running with Go 1.25.7, so far it looks good. I'll keep it running overnight and push a PR with the reversion to master which can get cherry-picked into the patch release.

  4. lidel
    changed the title [-]0.40.0 panic on Windows 10[/-] [+]go 1.26 build panics on Windows 10[/+] on Feb 27, 2026

  5. lidel commented on Feb 27, 2026

    @lidel
    lidel
    on Feb 27, 2026
    ContributorAuthor

    Triage note:

  6. lidel
    added
    topic/windowsWindows specific
    status/blockedUnable to be worked further until needs are met
    kind/maintenanceWork required to avoid breaking changes or harm to project's status quo
    and removed
    need/triageNeeds initial labeling and prioritization
    on Feb 27, 2026

  7. Brent-Sanchez commented on Feb 27, 2026

    @Brent-Sanchez
    Brent-Sanchez
    on Feb 27, 2026 · Hidden as off-topic

  8. lidel commented on Mar 15, 2026

    @lidel
    lidel
    on Mar 15, 2026
    ContributorAuthor

    Triage note: Go 1.26.1 does not fix this, Go 1.26.2 should

    Go 1.26.1 shipped no Windows runtime fixes.

    The actual fix is tracked upstream as go#77975 / go#78041 ("runtime: Windows crash with Go 1.26.0, 1.26.1"), labeled Critical and release-blocker. It is cherry-pick approved for Go 1.26.2 (not yet released, milestone).

    The crash signature matches ours exactly: Exception 0xc0000005 0x8, faulting PC outside the binary's code range, corrupted return address on a goroutine stack. Same bug was independently hit by Tailscale, erigon, and easyss.

    Root cause: async preemption on Windows corrupts the high 32 bits of return addresses on goroutine stacks (GODEBUG=asyncpreemptoff=1 prevents the crash). This is not directly related to the IOCP rework -- the crash happens to surface in I/O goroutines but the bug is in the runtime's async preemption path.

    Fix: CL 753480 (internal/poll: move rsan to heap on windows), already merged to release-branch.go1.26.

    The two Go issues originally suspected here are unrelated to the actual crash:

    • go#76391 (os.NewFile hangs) -- still open, milestoned for Go 1.27
    • go#77142 (overlapped I/O interop) -- still open, in Backlog

    Plan: wait for Go 1.26.2, then switch back from Go 1.25.7 and verify the crash is gone on Windows.

  9. lidel commented on Mar 19, 2026

    @lidel
    lidel
    on Mar 19, 2026 · Hidden as off-topic
    Author

  10. lidel commented on Mar 21, 2026

    @lidel
    lidel
    on Mar 21, 2026 · Hidden as off-topic
    Author
  11. lidel
    added 2 commits that reference this issue on Apr 9, 2026

    feat: go 1.26.2, once more with feeling

    159d3c0

    feat: go 1.26.2 (#11275)

    afc8ad3
  12. lidel
    closed this as completedin #11275on Apr 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugA bug in existing code (including security flaws)kind/maintenanceWork required to avoid breaking changes or harm to project's status quostatus/blockedUnable to be worked further until needs are mettopic/windowsWindows specific

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions
      ⌂ 换网址