CVE-2026-54995 — Microsoft Windows Reliable Multicast Transport Driver Integer Underflow

1. Overview

A vulnerability exists in the Windows Reliable Multicast Transport Driver (rmcast.sys) that implements the Pragmatic General Multicast (PGM) protocol. When processing incoming ODATA packets for a Forward Error Correction (FEC) group, the driver’s PgmHandleNewData function can decrement a parity packet counter past zero, causing an unsigned byte to underflow from 0 to 255. This corrupted counter subsequently causes out-of-bounds memory access when the driver attempts FEC decoding with an impossibly large parity count. An unauthenticated remote attacker can send specially crafted PGM multicast packets to a system running a PGM receiver application, potentially achieving kernel-level code execution. Microsoft addressed this vulnerability in the July 2026 security update.

2. Vulnerability Type

Microsoft classifies this as CWE-416 (Use After Free), but the root cause is an integer underflow (CWE-191) in the FEC parity counter. The underflow corrupts internal group metadata, which can lead to out-of-bounds write operations (CWE-787) when the corrupted counter is subsequently used to index into FEC group slot arrays during parity decoding.

3. Severity

CVSS 3.1 (from Microsoft Advisory)

Our Assessment (CVSS 4.0)

Scoring rationale:

• AC: High — The attacker must win a race condition: the duplicate ODATA packet must arrive after the FEC group is full (all k data slots filled) but before the group is consumed by the application layer. The attacker also has no visibility into internal group state.
• AT: Present — A PGM receiver application (using SOCK_RDM / IPPROTO_RM sockets) must be actively running and listening on a multicast group. The PGM/RMCAST driver is not loaded by default and requires explicit application use.
• E: Unreported — Static analysis confirms the vulnerability. Dynamic verification was blocked by PGM multicast delivery constraints (see Section 6).

4. Affected Products

Affected Products

Tested Environment (Vulnerable)

Tested Environment (Patched)

5. Root Cause Analysis

5a. Detailed Description

The vulnerability is an integer underflow in the PgmHandleNewData function within rmcast.sys, the kernel-mode driver implementing the Pragmatic General Multicast (PGM / RFC 3208) reliable multicast protocol.

PGM FEC Background:

PGM supports Forward Error Correction (FEC) where data is transmitted in groups of k original data packets (ODATA). A sender can also transmit parity repair packets for each group. The receiver tracks each group using an internal structure with three key byte-sized counters at fixed offsets:

Each group also has an array of 40-byte slot entries starting at offset +96, indexed by packet sequence within the group. Each slot tracks the packet data buffer pointer, length, and a “packet state” byte at slot offset +116 (relative to the group structure: offset 116 + 40*index).

The Vulnerable Code Path:

When PgmHandleNewData receives an ODATA packet, it determines which FEC group the packet belongs to and checks whether the group is already full. The relevant code at PgmHandleNewData+0x448 (RVA 0xFCF8):

// PgmHandleNewData — vulnerable version (rmcast.sys 10.0.26100.8521)
// FEC group slot management when group is full

v48 = *(BYTE*)(group + 59) + *(BYTE*)(group + 58);  // total = parity_count + data_count
v59 = *(BYTE*)(group + 57);                           // group_size

if (v48 >= v59)                                        // group is full?
{
    v48 = (unsigned __int8)v21;                        // reset slot search index
    if (v59)                                           // group_size > 0?
    {
        // Search for a parity slot to replace with this new data packet
        while (*(BYTE*)(group + 40 * v48 + 116) < *(BYTE*)(group + 62))
        {
            if (++v48 >= v59)
                goto LABEL_147;                        // no replaceable parity slot found
        }
        DataBuffer = ReAllocateDataBuffer(...);        // replace parity slot with data
        v58 = 0;
    }
LABEL_147:
    --*(BYTE*)(group + 59);                            // UNDERFLOW: parity_count decremented unconditionally
}

The Bug:

The decrement at LABEL_147 (--*(BYTE*)(group + 59)) executes unconditionally whenever the group is full, regardless of whether a parity slot was actually found and replaced. When a group is filled entirely with original data packets (i.e., data_count == k and parity_count == 0), and a duplicate ODATA arrives for the same group:

1. The fullness check v48 >= v59 passes because data_count (k) + parity_count (0) >= group_size (k)
2. The slot search loop iterates through all slots but finds no parity slots to replace (all slots contain original data)
3. The loop exhausts and falls through to LABEL_147
4. parity_count is decremented from 0 to 255 (unsigned byte wraparound)

Trigger Conditions for the Underflow:

1. The PGM session must have FEC enabled (parity_flags at session offset 161 is non-zero)
2. The attacker sends all k ODATA packets for a given FEC group, filling it completely with data (data_count = k, parity_count = 0)
3. The attacker then sends a duplicate ODATA for the same group sequence range

FEC Initialization Chain:

FEC parameters are established through the following network-driven initialization:

1. SPM Packet with OPT_PARITY_PRM option: The sender broadcasts Source Path Messages containing the FEC group size and parity flags. ProcessSpmPacket (RVA 0x1353C) parses these options and stores them in the address context structure at offsets 304 (group_size) and 305 (parity_flags).

2. First ODATA Packet: When the first data packet arrives for a new sender, PgmNewInboundConnection (RVA 0x10A24) creates a new receive session. If the address context has FEC parameters (offset 305 non-zero), it allocates FEC context buffers via CreateFECContext and copies:

   • group_size → session offset 160
   • parity_flags → session offset 161

3. Subsequent ODATA: ProcessDataPacket (RVA 0x12734) routes packets to PgmHandleNewData (RVA 0xF8B0), which manages the per-group slot arrays.

Attack Flow:

Network → TdiRcvDatagramHandler (RVA 0x14BE0)
       → ProcessDataPacket (RVA 0x12734)
       → PgmHandleNewData (RVA 0xF8B0)
       → Integer underflow at +0x448

Consequences of the Underflow:

With parity_count corrupted to 255:

• CheckIndicatePendedData (RVA 0xD3E0) calls DecodeParityPackets (RVA 0xEA30) when parity_count > 0
• DecodeParityPackets uses parity_count (255) + data_count (k) as an iteration bound for FEC matrix operations
• This causes out-of-bounds reads and writes into the 40-byte-per-slot group array, which was only allocated for k entries
• The out-of-bounds access corrupts adjacent kernel pool memory, potentially leading to code execution

5b. Vulnerable Assembly and Call Stack

rmcast.sys 10.0.26100.8521:

; PgmHandleNewData+0x43A: compute total = parity_count + data_count
14000FCE9  movzx  eax, byte ptr [r14+3Bh]     ; parity_count (offset +59)
14000FCEE  movzx  ecx, byte ptr [r14+3Ah]     ; data_count (offset +58)
14000FCF3  add    eax, ecx                     ; total = parity + data
14000FCF5  cmp    eax, r13d                    ; compare total vs group_size
14000FCF8  jb     short loc_14000FD44          ; if total < group_size, skip

; PgmHandleNewData+0x448: slot search loop for parity replacement
14000FCFA  movzx  eax, bl                      ; reset search index
14000FCFD  test   r13b, r13b                   ; group_size > 0?
14000FD00  je     short loc_14000FD24          ; if 0, skip to decrement

; Search loop: find a parity slot with state >= threshold
14000FD02  lea    rcx, [r14+rax*8]
14000FD06  movzx  edx, byte ptr [rcx+rax*20h+74h]  ; slot[i].state
14000FD0B  cmp    dl, byte ptr [r14+3Eh]            ; compare vs threshold
14000FD0F  jnb    short loc_14000FD1A               ; found replaceable slot
14000FD11  inc    eax
14000FD13  cmp    eax, r13d                         ; i >= group_size?
14000FD16  jb     short loc_14000FD02               ; continue search
14000FD18  jmp    short loc_14000FD24               ; no slot found → fall through

; PgmHandleNewData+0x474: CRITICAL — unconditional parity_count decrement
14000FD24  dec    byte ptr [r14+3Bh]           ; --parity_count (0 → 255 UNDERFLOW)

Call Stack (at the underflow):

00 rmcast!PgmHandleNewData+0x474
01 rmcast!ProcessDataPacket+0x12F
02 rmcast!TdiRcvDatagramHandler+0x3E6
03 tdx!TdxTdiReceiveHandler+0xNN
04 tcpip!UdpDeliverDatagrams+0xNN

5c. Fix (Patched Version)

The July 2026 patch (KB5099536 / KB5101650) introduces a WIL feature flag gate in CheckIndicatePendedData (patched address: sub_14000D808). The underflow code in PgmHandleNewData remains structurally identical — the fix instead validates group integrity before FEC decoding is attempted.

The patched CheckIndicatePendedData adds a validation gate at offset +0x85:

// CheckIndicatePendedData — patched version (sub_14000D808)
// New validation before calling DecodeParityPackets

if (group[59])   // parity_count > 0?
{
    if (sub_14000F648())   // WIL feature flag enabled?
    {
        // NEW: Validate group integrity
        if ((unsigned __int8)group[59] + (unsigned __int8)group[58] != (unsigned __int8)group[57])
        {
            // parity_count + data_count != group_size → corrupted group
            // Log error and abort processing
            goto LABEL_23;   // sets error flag 0x100, skips FEC decoding
        }
        DecodeParityPackets(session, group);   // safe: counts validated
    }
    else
    {
        DecodeParityPackets(session, group);   // legacy path: no validation
    }
}

The feature flag function (sub_14000F648) reads dword_140033870: if bit 0x10 is set (cached), it returns bit 0x1 (enabled state); otherwise it calls sub_14000F684 to initialize the flag. This is a standard WIL (Windows Implementation Library) feature flag pattern used for staged rollout.

5d. Impact

The integer underflow corrupts the parity packet counter in the kernel-mode PGM driver, causing DecodeParityPackets to operate with a parity count of 255 on a group structure allocated for only k slots (typically 128 or less). This results in out-of-bounds read and write operations into adjacent kernel pool memory. Since rmcast.sys runs at DISPATCH_LEVEL during packet processing, corruption of adjacent pool allocations can lead to arbitrary kernel code execution when those allocations are subsequently used. The most likely immediate outcome is a kernel bugcheck (BSOD), but controlled exploitation for code execution is theoretically possible.

The practical attack surface is limited by several factors. The target system must have an application actively using PGM sockets (SOCK_RDM / IPPROTO_RM), which is uncommon outside specialized multicast data distribution systems. The attacker must be able to deliver PGM multicast packets (IP protocol 113) to the target, which requires either being on the same multicast-capable network segment or having upstream routers configured to forward PGM multicast traffic. The attacker must also send packets with the correct timing relative to FEC group completion. Microsoft rates exploitation as “Unlikely,” consistent with these constraints.

6. Proof-of-Concept

6a. PoC Code

Download poc_cve_2026_54995.c (enterprise email verification required)

The following files are included in this report directory:

6b. Reproduce Instructions

Prerequisites:

• Two Windows systems on the same multicast-capable network segment
• Npcap installed on the attacking system (for packet injection)
• PGM/RMCAST protocol installed on the target system (Windows feature: “Reliable Multicast Protocol”)
• Driver Verifier enabled for rmcast.sys on the target (to catch pool corruption):

  verifier /standard /driver rmcast.sys
  

  Reboot after enabling.

Setup:

1. On the target system, compile and run the PGM receiver:

   cl /W3 pgm_recv4.c /Fe:pgm_recv4.exe /link ws2_32.lib
   pgm_recv4.exe
   

   The receiver creates a PGM socket, binds to 224.1.1.1:40000, and waits for incoming connections.

2. On the attacking system, compile the FEC sender and combined PoC:

   cl /W3 pgm_fec_sender.c /Fe:pgm_fec_sender.exe /link ws2_32.lib
   cl /W3 poc_combined.c /Fe:poc_combined.exe /link ws2_32.lib wpcap.lib packet.lib
   

Steps to trigger:

1. Start the PGM receiver on the target system
2. Run pgm_fec_sender.exe on the attacker to establish a PGM session with FEC enabled (FECBlockSize=255, FECGroupSize=128, ProActive=0)
3. Send exactly k (128) data packets to fill one FEC group completely
4. Run poc_combined.exe to inject a duplicate ODATA packet for the same group sequence, crafted with the correct GSI, source port, and sequence number range
5. Monitor the target for a kernel bugcheck (BSOD) or Driver Verifier violation

Note on packet delivery: PGM multicast traffic between separate hosts requires the network infrastructure (switches/routers) to properly forward IP multicast for protocol 113. In virtualized environments (ESXi, Hyper-V), the virtual switch must be configured for multicast forwarding or promiscuous mode. Raw socket injection (IPPROTO_PGM=113) does NOT reach rmcast.sys on the local machine — the Windows TCP/IP stack drops protocol 113 packets from raw sockets before they reach the TDI handler.

6c. Test Results

Dynamic verification of the crash was not completed. The following approaches were tested:

The integer underflow is confirmed through static analysis: the code path is unambiguously reachable when a duplicate ODATA arrives for a full FEC group with no parity slots. The patch confirms the bug by adding an explicit parity_count + data_count == group_size validation in the code path that consumes the corrupted counter.

Additional dynamic analysis confirmed:

• PGM packet delivery to rmcast CONFIRMED via raw IP sockets (SOCK_RAW, IPPROTO 113, IP_HDRINCL) from a physical host to the target VM. KDNET breakpoint at rmcast’s TdiRcvDatagramHandler (kernel address 0xfffff8046ddc3530) fired 12 times across multiple test runs, confirming crafted PGM packets reach the driver’s receive handler.
• Multicast delivery rate is ~0.1-6% due to physical switch IGMP snooping. The exploit requires sequential SPM + k ODATA + duplicate ODATA delivery, which is statistically unlikely at this delivery rate. A properly configured multicast network (no IGMP snooping, or IGMP-aware switch with ESXi Net.SendIGMPReportToUplink=1) would achieve reliable delivery.
• poc_exploit.c implements the full attack sequence: SPM with OPT_PARITY_PRM (FEC group_size=4, on_demand=1), followed by 4 ODATA to fill the group, then duplicate ODATA seq=0 to trigger the underflow. Includes a 60-second continuous blast mode with fresh FEC groups each round.
• ESXi standard vSwitches do not forward IP multicast (protocol 113) between VM ports regardless of promiscuous mode, forged transmit, or MAC change security policies
• Windows 11 26100 per-page kernel ASLR prevents locating rmcast code in memory via the kernel debugger (NtQuerySystemInformation returns obfuscated addresses, individual .text section pages are mapped at random addresses)

6d. Patched System Verification

The patched rmcast.sys (10.0.26100.33158) contains structurally identical code in PgmHandleNewData — the underflow itself is not prevented. Instead, the patched CheckIndicatePendedData validates parity_count + data_count == group_size before calling DecodeParityPackets. If the counters are inconsistent (as they would be after the underflow), the group is flagged with error code 0x100 and FEC decoding is skipped entirely, preventing the out-of-bounds memory access.

7. Detection

Note: The detection rules below are provided as a starting point. Validate and tune them in your own environment before deploying to production.

Section 7A: Network-Based Detection

Signature-Based Detection

PGM operates as IP protocol 113 (not TCP or UDP). Attack traffic consists of:

1. SPM packets with OPT_PARITY_PRM option — these establish FEC parameters for the session. An SPM with a very large FEC group size combined with ProActive parity disabled is suspicious.

2. Duplicate ODATA packets for the same sequence range within an FEC group — legitimate PGM senders do not normally retransmit the same ODATA sequence number. A burst of ODATA packets with overlapping trailing sequence edges (TSE) targeting the same group after the group is full is the attack signature.

PGM packet header structure (IP protocol 113):

ODATA-specific fields (after 16-byte common header):

Suricata Rules

# Detect PGM ODATA packets with potential duplicate sequence exploitation
# Triggers on high-rate PGM ODATA (type 0x04) targeting the same destination
alert ip $EXTERNAL_NET any -> $HOME_NET any (msg:"CVE-2026-54995 RMCAST PGM ODATA flood - potential FEC underflow exploit"; \
  ip_proto:113; \
  content:"|04|"; offset:4; depth:1; \
  threshold:type both, track by_src, count 300, seconds 5; \
  reference:cve,2026-54995; \
  classtype:attempted-admin; \
  sid:2026054995; rev:1;)

# Detect PGM SPM packets with FEC options (OPT_PARITY_PRM)
# SPM type = 0x02, options present flag = bit 6 of byte 5
alert ip $EXTERNAL_NET any -> $HOME_NET any (msg:"CVE-2026-54995 RMCAST PGM SPM with FEC options - potential exploit setup"; \
  ip_proto:113; \
  content:"|02|"; offset:4; depth:1; \
  content:"|40|"; offset:5; depth:1; \
  reference:cve,2026-54995; \
  classtype:attempted-admin; \
  sid:2026054996; rev:1;)

Byte Offset Reference

PGM common header (IP protocol 113 payload):

Section 7B: Host-Based Detection

Prerequisite Check

The PGM driver (rmcast.sys) is only present when the Reliable Multicast Protocol feature is installed (typically as part of MSMQ). Systems without this feature are not vulnerable.

# Check if the Reliable Multicast feature is installed (Windows Server)
Get-WindowsFeature -Name MSMQ | Select-Object Name, InstallState

# Check if rmcast.sys exists
Test-Path "$env:SystemRoot\System32\drivers\rmcast.sys"

Version-Based Detection

If rmcast.sys is present, check its file version against the patched versions below. A system is vulnerable if the file version is less than the listed version for its OS branch.

# Check rmcast.sys version
(Get-Item "$env:SystemRoot\System32\drivers\rmcast.sys").VersionInfo.FileVersion

8. References