[2/3] Integrate XSK maps into the core XDP datapath by ProjectsByJackHe · Pull Request #6003 · microsoft/msquic

ProjectsByJackHe · 2026-05-18T02:47:11Z

Description

Full E2E demo using tools from this PR : DEMO

Part 2 of the plan: #5982
Fixes #5972

Adds the core msquic datapath integrations for the new API contract defined in #5983

Key design decisions:

If map mode is enabled, then we do not create the WinSock or Epoll or .... datapaths. No more best effort!
Init failures in map mode hard fails the datapath initialization. No silent failures!

Testing

Added datapath unit tests. See the E2E demo.

Additionally, the plan is to leverage some supporting PRs that included code to ingest the latest XDP version, and kicking off a couple of manual CI runs based on that: https://github.com/microsoft/msquic/actions/runs/27313685557/job/80690420168

Documentation

Will come as part 3 in the plan: #5982

codecov · 2026-05-18T03:39:00Z

Codecov Report

❌ Patch coverage is 20.00000% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.80%. Comparing base (ada1bbf) to head (a5c770f).
⚠️ Report is 7 commits behind head on main.

Files with missing lines	Patch %	Lines
src/core/library.c	16.66%	10 Missing ⚠️
src/core/settings.c	33.33%	2 Missing ⚠️

❌ Your patch check has failed because the patch coverage (20.00%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6003      +/-   ##
==========================================
- Coverage   85.96%   84.80%   -1.16%     
==========================================
  Files          60       60              
  Lines       18847    18859      +12     
==========================================
- Hits        16202    15994     -208     
- Misses       2645     2865     +220

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

ProjectsByJackHe · 2026-05-19T00:10:48Z

Planning on waiting until #6006 gets resolved to address the next wave of feedback.

mtfriesen · 2026-05-20T19:01:12Z

It's not clear to me why we don't have end-to-end tests exercising everything from providing the XSKMAP before creating a registration through actual XDP data paths flowing? Since we intend to officially support this feature, it is risky for both us and our partners to supply/depend on an API that does not have regression test coverage and a paved pattern to follow.

ProjectsByJackHe · 2026-05-20T19:34:19Z

It's not clear to me why we don't have end-to-end tests exercising everything from providing the XSKMAP before creating a registration through actual XDP data paths flowing? Since we intend to officially support this feature, it is risky for both us and our partners to supply/depend on an API that does not have regression test coverage and a paved pattern to follow.

That's in the works. I took the approach of building up an E2E flow manually first to see the shape of how the datapath integration would look like. There's likely going to be some limitations in the CI environment that prevents some scenarios from being exercised, but I am currently adding coverage for the parts that can be automated now that the skeleton of the general changes is mapped out in the current PR iteration.

mtfriesen · 2026-05-20T20:56:59Z

Let's ensure we add tests with each chunk of work we do. I understand it may be a lot of work to add tests, but the best time to protect the code with automation and review the coverage is when making the change.

mtfriesen

We need to solve the testing problem - map mode should run all existing MsQuic tests that are not intrinsically incompatible with an XDP map.

Nearly all MsQuic protocol level tests should be compatible with XDP maps, so getting those exercised is a min bar.

guhetier · 2026-06-12T19:55:34Z


    CXPLAT_DBG_ASSERT(Datapath->UdpHandlers.Receive != NULL || Config->Flags & CXPLAT_SOCKET_FLAG_PCP);
-    CXPLAT_DBG_ASSERT(IsServerSocket || Config->PartitionIndex < Datapath->PartitionCount);
+    CXPLAT_DBG_ASSERT(CxPlatDpRawIsRawDatapathOnly(Datapath->RawDataPath) || IsServerSocket || Config->PartitionIndex < Datapath->PartitionCount);


Question: Why do we need this change? It isn't clear to me why xdp maps would interfere with partioning.

In raw-only mode, PartitionCount is 0 (no WinSock init), so the assert PartitionIndex < PartitionCount would always fail. Will add a comment explaining why.

Ok, and it is ok to not be partionned in raw only mode? Or is it a sign that we actually need to fix some init?

mtfriesen · 2026-06-19T16:35:55Z

Before merging this PR taking a dependency on a prerelease feature, we need to merge the upstream PR that automates prerelease publishing.

guhetier

Will go through another round for the test code.

guhetier · 2026-06-24T00:21:54Z

 }

 _IRQL_requires_max_(PASSIVE_LEVEL)
+#pragma warning(suppress:6101) // *Buffer is _Out_writes_bytes_opt_(*BufferLength); 0-length is valid.


Having to suppress those warning is probably the sign that some annotation is missing. We should try to solve the warnings, especially given that the suppression would affect every param.

Adding some "analysis_assume" might help, also, making sure there is no potential integer overflow.

guhetier · 2026-06-24T00:26:45Z

                "Allocation of '%s' failed. (%llu bytes)",
                "XDP map config",
-                BufferLength);
+                Count * sizeof(CXPLAT_XDP_MAP_CONFIG));


nit: Consider a local constant instead of repeating the multiplication, that way if we change it, there is no risk of missing one update.

guhetier · 2026-06-24T00:32:18Z

 typedef CXPLAT_DATAPATH_SEND_COMPLETE *CXPLAT_DATAPATH_SEND_COMPLETE_HANDLER;

+//
+// N.B. This data structure must be a super-set of the struct in QUIC_XDP_MAP_CONFIG in msquic.h


The comment is strange.
It is not wrong, in practice, but there is no need, it is just that we forward the parameters this way currently.
Is this a remain of an attempt of having binary compatible settings struct?

guhetier · 2026-06-24T17:57:31Z

 #include "quic_sal_stub.h"

 #define QUIC_INLINE static inline
+#define QUIC_INVALID_FILE_HANDLE         -1


These files are public API (included in msquic.h).
Since this is only used for test code, I'm reluctant about adding it here (hard to remove later).

Can we have it in the test file instead? I vaguely remember a previous discussion, I hope I am not contradicting myself.

guhetier · 2026-06-24T18:14:37Z

+        CXPLAT_DBG_ASSERT(InitConfig->XdpMapConfigs != NULL);
+        CxPlatDpRawEnableRawDatapathOnly(RawDataPath);
+        Status =
+            CxPlatDpRawInsertXskByMapConfigs(


nit: I can't help to find that name hard to understand.
Maybe CxPlatDpRawInsertXdkIntoMaps or CxPlatDpRawSetupMaps or CxPlatDpRawApplyMapConfig would be clearer? What do you think?

guhetier · 2026-06-24T18:32:14Z

+{
+    XDP_DATAPATH* Xdp = (XDP_DATAPATH*)RawDataPath;
+
+    CXPLAT_LIST_ENTRY* Entry;


nit: Declare Entry in the for loop so it is scoped + initialized on decl

guhetier · 2026-06-24T18:33:15Z

+    //
+    // In external map mode, the caller manages XDP rules; nothing to do here.
+    //
+    if (Xdp->RawDatapathOnly) {


why not use the helper here?

guhetier · 2026-06-24T18:46:21Z

+                j,
+                XskMap,
+                Queue->RxXsk);
+            return (QUIC_STATUS)Hr;


On failure, Interface->XskMap is not set, so the best effort cleanup won't remove the xsk already inserted on this interface.

Consider removing the xsk inserted in this function for the interface so that the contract of the function is clear:
either all sockets are inserted, or none (with a best effort for the none).

And the caller has a similar contract: either all interfaces inserted successfully, or none. Each deals with cleanup at its level.

guhetier · 2026-06-24T18:49:40Z

+    XDP_DATAPATH* Xdp = (XDP_DATAPATH*)RawDataPath;
+    QUIC_STATUS Status = QUIC_STATUS_SUCCESS;
+
+    CXPLAT_LIST_ENTRY* Entry;


nit: Same, move in the for (...)

guhetier · 2026-06-24T18:55:46Z

+                HANDLE XskMap = (HANDLE)MapConfigs[i].MapHandle;
+                Status = CxPlatDpRawInsertXskInMap(Interface, XskMap);
+                if (QUIC_FAILED(Status)) {
+                    CxPlatDpRawRemoveXskByMapConfigs(RawDataPath);


For clarity sake, move this after Exit, where we expect the error cleanup to be.

You can in addition try to not nest the logic as much, like in this pseudocode:

for each interfaces: def xskmap // Find the map for the interface for each mapconfig: if config match interface: xskmap = config.map break if not xskmap found: continue InsertXskInMap ...

it'll keep your control flow easier to manage.

ProjectsByJackHe added 13 commits May 14, 2026 17:46

add tools

0020948

phase 2 looks good so far

9830250

phase 2 improvements

7ba0bf1

better diagnostics

b364f47

phase 3 plumbing...

01df062

phase 3: iter 2

3097b18

bug fixes man

5750131

E2E

bbf64e0

fix comment

c62d7b1

update xdp commit

08ff04e

get rid of tools

b8285dc

get rid of preview features

4546dfe

add unit tests

ba539a7

ProjectsByJackHe requested a review from a team as a code owner May 18, 2026 02:47

ProjectsByJackHe mentioned this pull request May 18, 2026

[2/3] Add CIBIR, XDP, and XSK maps support to the sample tools #5999

Open

CLOG

d15f3a7

guhetier reviewed May 18, 2026

View reviewed changes

fix abi incompat

4fe8094

ProjectsByJackHe added 2 commits May 19, 2026 14:29

address code feedback

47f162d

Merge branch 'main' into jackhe/xdp_map_dp_int_pr_2_core

0b83796

mtfriesen reviewed May 20, 2026

View reviewed changes

Comment thread src/platform/datapath_raw_xdp_win.c Outdated

mtfriesen reviewed May 20, 2026

View reviewed changes

Comment thread src/platform/datapath_raw_xdp_win.c Outdated

ProjectsByJackHe added 3 commits May 20, 2026 14:36

fix dp tests + update logging

f68a1d8

infer xdp enablement if map mode is set

d8fe2d2

set xdpenabled to false explicitly in map mode returns error

96ded75

mtfriesen reviewed Jun 11, 2026

View reviewed changes

Comment thread src/test/bin/quic_gtest.cpp Outdated

mtfriesen reviewed Jun 11, 2026

View reviewed changes

Comment thread .github/workflows/test.yml Outdated

ProjectsByJackHe added 6 commits June 11, 2026 13:03

move stuff out of the test harness

d524666

test harness refactors

cb6db81

more feedback refactors

4c30ab2

add coverage for wildcard clients

d69d27d

rename to not ref map mode

3e997ff

use -1

f658a8b

ProjectsByJackHe commented Jun 12, 2026

View reviewed changes

Comment thread src/platform/unittest/DataPathTest.cpp Outdated

add proper preview guards

3432322

mtfriesen requested changes Jun 12, 2026

View reviewed changes

guhetier reviewed Jun 12, 2026

View reviewed changes

ProjectsByJackHe added 2 commits June 18, 2026 16:17

Merge branch 'main' into jackhe/xdp_map_dp_int_pr_2_core

ba8c194

add carve-out for xdp map mode

058c940

ProjectsByJackHe added 7 commits June 19, 2026 13:25

address feedback

1591cdf

add const to RawOnly

0251cd9

get rid of verbose comment

96eaab7

add cross platform invalid file handle abstraction

de08c99

fix CLOG generation

1d3d058

add validation checks

11be89c

update rust

9708f04

microsoft deleted a comment from github-actions Bot Jun 22, 2026

add internal cxplat ref

a5c770f

ProjectsByJackHe requested review from guhetier and mtfriesen June 24, 2026 16:22

guhetier reviewed Jun 24, 2026

View reviewed changes

Uh oh!

Conversation

ProjectsByJackHe commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Testing

Documentation

Uh oh!

codecov Bot commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ProjectsByJackHe commented May 19, 2026

Uh oh!

Uh oh!

Uh oh!

mtfriesen commented May 20, 2026

Uh oh!

ProjectsByJackHe commented May 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mtfriesen commented May 20, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mtfriesen left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mtfriesen commented Jun 19, 2026

Uh oh!

guhetier left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ProjectsByJackHe commented May 18, 2026 •

edited

Loading

codecov Bot commented May 18, 2026 •

edited

Loading

ProjectsByJackHe commented May 20, 2026 •

edited

Loading