Kernel panic "m->m_flags & M_PKTHDR" in uipc_mbuf.c on SMB clients over 10 GbE (macOS 26)

App & System Services Core OS
sethg1 OP
Created 3d
Replies 4
Boosts 0
Views 388
Participants 2

We have a group of Macs that mount SMB shares over 10 GbE with jumbo frames (MTU 9000). Since late June, they have been kernel panicking several times a day with the same assertion:

panic(cpu N caller ...): assertion failed: m->m_flags & M_PKTHDR, file: .../xnu/bsd/kern/uipc_mbuf.c, line: 4839 @uipc_socket.c:8260
Panicked thread: dlil_input_en0
Last started kext: com.apple.filesystems.smbfs 6.0.1

Environment

  • Clients: Mac Studio (M1 Max and M1 Ultra) and Mac Pro (2019, Intel with T2), using the built-in 10GBASE-T at MTU 9000
  • macOS 26.5.1 (25F80), 26.6.2 (25G83) and 26.7 (25G229); it panics on all three
  • Servers: Samba-based NAS, SMB 3.1.1, signing on, encryption off
  • Filed as FB24912731

What we've found

  • It still panics with our third-party EDR fully uninstalled.
  • The Mac that panics needs an active SMB session. A Mac left on the network without a share mounted stayed up through several events that took down the others.
  • Panics are often simultaneous across machines: two to six Macs, with different hardware and different macOS builds, within the same minute.
  • It doesn't need sustained heavy throughput. Some panics came within minutes of reconnecting, during light editing.
  • Setting kern.skywalk.flowswitch.rx_agg_tcp_host=0 did not help.
  • The switch and server links stay up, and spanning tree doesn't change during these events. Only the Macs' ports drop.
  • In one server-side capture, the client stopped sending within about 0.2 ms of receiving a READ response made of 8948-byte frames. That fits the panicked thread being dlil_input.

Two existing threads look related

Questions

  1. Is this the same underlying issue as FB17853906, and is a fix planned for macOS 26? Our 2019 Mac Pros can't move to a later major release.
  2. Is there a known workaround, such as a sysctl, an nsmb.conf option, or a change to MTU or offload settings?
  3. Is there logging or a diagnostic we can leave enabled to capture more state at panic time? We can't reproduce this on demand, but between several machines we see it multiple times a day.

We can provide full panic reports, sysdiagnoses, and packet captures from both client and server sides.

Replies  4
Boosts  0
Views  388
Participants  2
sethg1 OP
3d

Update: likely trigger found. A switch-port mirror of an affected Mac caught a panic. The Mac was idle, with no SMB traffic for about 10 minutes. The last frames it received, about 1 second before its link dropped, were a burst of about 60 mDNS packets in 25 ms from a Windows 11 PC on the same MTU 9000 network, many of them jumbo-sized (up to 4468 bytes). They came from Windows Delivery Optimization local peer discovery (_dosvc._tcp.local), which seemed to be stuck in a name-conflict loop.

Across about 24 hours of capture, every panic lined up with mDNS traffic from that PC, and when it went quiet for 5½ hours, nothing panicked. The Macs still need an active SMB session to panic, so the mDNS traffic seems to be the trigger and the SMB session the precondition.

Mitigation: on the Windows PC, set Delivery Optimization to Simple mode (DODownloadMode = 99 under HKLM\SOFTWARE\Policies\Microsoft\Windows\DeliveryOptimization), which stopped the announcements. We're watching to confirm the panics stop. Details added to FB24912731.

If you're seeing this panic on a jumbo-frame network with Windows 10/11 PCs on it, this is worth checking.

DTS Engineer OP
Apple
18h

Filed as FB24912731

Perfect, thank you. For your reference (and all future readers), it's basically "always"[1] worth filing a bug for any kernel panic, even if you think the issue might be known or a duplicate. Kernel panic investigations can be quite difficult, so it's not unusual for the critical detail to come from one or two logs out of a large data set.

  1. Is this the same underlying issue as FB17853906,

It doesn't appear to be, as that panic was specifically noted as occurring with NFS but NOT with smb.

and is a fix planned for macOS 26?

Unfortunately, I can't really comment on our release schedule.

Is there a known workaround, such as a sysctl, an nsmb.conf option, or a change to MTU or offload settings?

No, not that I'm aware of, though what you've found here is certainly interesting/useful:

...a burst of about 60 mDNS packets in 25 ms ... They came from Windows Delivery Optimization local peer discovery (_dosvc._tcp.local), which seemed to be stuck in a name-conflict loop.

If you haven't already, I'd also recommend contacting Microsoft about that issue as well, as that sounds well outside the Bonjour spec.

Is there logging or a diagnostic we can leave enabled to capture more state at panic time?

It sounds like your main focus is on macOS 26, but if you're able to reproduce this on macOS 27, that would be the most useful information, even if you're only concerned about fixing this on macOS 26. We generally work in terms of fixing the "latest" system and then applying that bug fix "back" to earlier[2] system versions, so data from the latest system is obviously helpful to both the investigation and prioritization.

[1] The one exception I'd make here is developers working on KEXTs, DEXTs, or Endpoint Security clients. Bugs may still be warranted for those components, but those operate at a low enough level of the system that they also bear significant responsibility for its stability.

[2] This process actually starts WELL before we announce new system versions, as the primary goal here is actually to ensure that bugs don't "reappear" (because we only fixed them on the older system), not just to make sure our latest releases work as well as possible.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware
sethg1 OP
15h

Thanks, Kevin, this is really helpful.

Good to know FB17853906 is a separate issue, and understood on the release schedule.

Two things on our side:

  1. We'll report the mDNS behavior to Microsoft. Agreed it looks well outside the Bonjour spec: a rename loop emitting ~60 records in ~25 ms, many packed into jumbo-sized (4+ KB) datagrams.

  2. We'll try to reproduce on macOS 27. We have a spare Apple silicon Mac (M1 Max) we can put on 27, mount an SMB share, and hit with a synthetic version of that mDNS burst on an isolated run. If it panics there, we'll attach the full panic report, a sysdiagnose, and the packet capture to FB24912731. If 27 turns out not to panic, that's also worth knowing, and we'll note it.

One thing worth flagging from our side: because the trigger is multicast, any host on the local segment can take down every Mac on it that has an SMB session open, without touching the Macs directly. Once we have a clean repro on 27, would you want that routed to Product Security as well, or is FB24912731 the right single channel to keep everything in?

We can also share the server- and client-side captures from the original events on request.
sethg1 OP
14h

Quick update for anyone following along.

The mitigation is holding: since we set Delivery Optimization to Simple mode on the Windows PC, none of the affected Macs has panicked. That covers a weekend and the first full workday, and one Mac has kept the same SMB session open for more than 60 hours. Before the change, we saw several panics a day.

A correction to my earlier post: the "@uipc_socket.c:8260" in the panic string isn't the caller. In the published xnu source it's the panic() inside assfail(), the generic assertion handler. The assertion itself (uipc_mbuf.c:4839) is in m_add_crumb(), which stamps a debug breadcrumb on each packet as it moves up the receive path, so the real caller is one of those receive-path sites.

Next, we're symbolicating the backtraces to find out which call site trips, then reproducing on macOS 26 as a control and on macOS 27, as Kevin suggested. Details are going into FB24912731, and I'll post the outcome here.

Kernel panic "m->m_flags & M_PKTHDR" in uipc_mbuf.c on SMB clients over 10 GbE (macOS 26)
First post date Last post date
 
 
Q