CUDA VMM, POSIX FDs, FABRIC, and LMCache from First Principles
A visual, interactive introduction to legacy CUDA IPC, virtual memory management, POSIX file-descriptor exports, FABRIC handles, IMEX channels, and the memory-access negotiation between vLLM's extensible KV cache and LMCache MP.
Maobaolong11/11/202626 min
The first encounter with these terms often makes them look like competing transfer protocols:
VMM / FD / FABRIC / IMEX / CUDA IPC
That mental model creates more questions than it answers. Is an fd a file containing GPU data? Does a FABRIC handle mean the bytes must travel over NVLink? Does VMM place GPU memory in a virtual machine? If the vLLM worker and LMCache server run on the same host, why not send the GPU pointer directly? What exactly does isolated_ipc isolate?
This article starts from one concrete problem:
A vLLM worker owns a GPU KV cache. The LMCache server is a different process. How can the server safely see and map the worker’s memory?
Once that boundary is clear, VMM, POSIX file descriptors, FABRIC, IMEX, and the automatic detection in LMCache PR #6499 become one causal chain rather than five unrelated terms.
The implementation status in this article is pinned to November 11, 2026. vLLM’s extensible KV cache #56492 and connector access contract #66605 are merged. The exportable POSIX-fd allocation work in #66674 is still open, and LMCache’s auto-detection #6499 is still a Draft. Follow the linked code for later changes.
CUDA memory sharing · visual field guide
The data is not inside the handle; the handle is a capability to map the allocation
Separate the physical memory, each process's address, the cross-process capability, and the channel that carries the capability. Nearly every term falls into place.
Virtual address: how this process sees itPhysical memory: where the bytes liveShareable handle: who may remap it
1. Start without CUDA: why one process cannot send a pointer to another
Suppose a vLLM worker prints this address:
0x7f20_8000_0000
That number is not a global street address for a GPU location. It is meaningful only inside the current process and CUDA context. Another process that receives the same integer has no corresponding mapping. The same numeric address can refer to something else or to nothing at all.
CPU processes already work this way. Address 0x1234 in process A and 0x1234 in process B can map to different physical pages. Page tables translate each process’s virtual addresses into physical memory. GPUs have an analogous virtual-address and mapping model.
Cross-process GPU sharing therefore needs several distinct pieces:
1. Allocation
The physical GPU-memory object holding K/V bytes, not the pointer in a message.
2. Local mapping
The worker maps the allocation into its own GPU virtual address space so model kernels can use it.
3. Shareable handle
An exportable capability that lets the server refer to the same allocation.
4. Handle transport
The mechanism that carries the capability to the server; different handles require different channels.
5. Remote mapping
The server imports the handle and creates a mapping in its own address space, obtaining its own pointer.
6. Lifetime
The importer must stop using and unmap the memory before the exporter releases the allocation.
A pointer answers “where do I access it?” A handle answers “what authorizes me to map the same physical allocation?” The two processes do not need the same pointer value.
2. Legacy CUDA IPC: the traditional path for regular allocations
The classic path starts with a regular device allocation from cudaMalloc or cuMemAlloc. Process A calls cudaIpcGetMemHandle() or cuIpcGetMemHandle() to obtain an opaque IPC handle. Process B receives the handle and calls cudaIpcOpenMemHandle() or cuIpcOpenMemHandle(). The driver creates a mapping in B and returns a device pointer valid in B.
Process A Process BcuMemAlloc() receive CUipcMemHandle │ │ ├─ local pointer ├─ cuIpcOpenMemHandle() │ │ └─ cuIpcGetMemHandle() ── opaque handle ───────────┘ │ └─ B's local pointer
The handle does not contain the KV payload. Pointer values need not match, and the exporter still controls the allocation’s lifetime. Most importantly, this contract fits regular CUDA allocations; it does not describe the generic physical allocation handles created through CUDA VMM. NVIDIA documents this separately as legacy IPC. See the CUDA IPC programming guide.
When LMCache has use_vmm_api=false, a normal shared environment uses the PyTorch storage IPC wrapper, while isolated_ipc=true selects the driver-level RawCudaIPCWrapper. Both belong to the legacy CUDA IPC family. Neither one is a FABRIC handle.
3. CUDA VMM: reserve an address first, decide physical backing later
VMM means Virtual Memory Management. “Virtual” does not mean a virtual machine, and VMM does not automatically swap GPU memory to disk. It means the application explicitly controls the relationship between GPU virtual addresses and physical allocations.
A traditional cudaMalloc(size) bundles size selection, physical allocation, mapping, and pointer creation. CUDA VMM separates them:
cuMemAddressReserve reserves a virtual-address range.
cuMemCreate creates a physical allocation handle.
cuMemMap maps a physical allocation into a virtual range.
cuMemSetAccess grants device access.
Cleanup proceeds through cuMemUnmap, cuMemRelease, and cuMemAddressFree.
Separate the address from physical backing
1 / 4
The diagram omits allocation granularity, device access permissions, and the strict unmap/release sequence, but preserves the ownership model.
VMM is valuable because the address stays stable while backing can be bound later and controlled in physical chunks. NVIDIA also describes per-allocation peer access, fragmentation control, dynamic growth, and memory sharing. See the CUDA VMM programming guide.
Common mistake: a VMM allocation is not a regular `cudaMalloc` allocation with a new label. Its physical memory is represented by a `CUmemGenericAllocationHandle`, so legacy pointer-based IPC is not the right sharing contract.
4. POSIX FDs, FABRIC handles, and IMEX
A VMM physical allocation must declare its exportable handle types when it is created. LMCache currently cares about two.
4.1 POSIX file descriptors: the OS transfers the capability
With CU_MEM_HANDLE_TYPE_POSIX_FILE_DESCRIPTOR, the worker can call:
This fd is not a normal data file containing KV bytes. It is a process-local entry in the Linux descriptor table that references a driver memory object.
Serializing the integer value is insufficient. If the worker prints fd = 17, the number 17 in the server may name a log file or nothing at all. A real fd transfer uses ancillary data on a Unix-domain socket, specifically SCM_RIGHTS; the kernel installs a new descriptor in the receiving process that references the same object.
The request path may still use ZMQ or gRPC over TCP. That does not make an fd transferable over TCP. Worker and server must also see the same Unix-socket path, which generally means sharing the relevant /dev/shm in container deployments. The exact LMCache protocol is documented in vmm_cuda_ipc.md.
4.2 FABRIC handles: a driver-managed serializable capability
CUDA 12.4+ provides CU_MEM_HANDLE_TYPE_FABRIC. The exported CUmemFabricHandle is an opaque blob. LMCache carries its current 64-byte representation in the descriptor, so it does not require a shared Unix-socket path.
FABRIC is not free container-crossing magic. Exporter and importer must have device support, a running IMEX setup, access to the same IMEX channel, and an allocation created with the FABRIC handle type.
IMEX stands for inter-node memory exchange. The channel is a driver-enforced security and isolation boundary. A typical device node is:
/dev/nvidia-caps-imex-channels/channel0
NVIDIA documents an IMEX domain as either one OS or multiple OS instances securely connected through the NVIDIA IMEX daemon. On supported NVLink-fabric systems, FABRIC handles can support inter-node mapping. That does not by itself mean LMCache MP currently implements arbitrary cross-node KV-pool mapping; here the handle solves the worker/server allocation-sharing boundary.
4.3 Choose the conditions and inspect the path
Change deployment conditions and observe the sharing path
allocation
isolated IPC
same IMEX channel
vLLM workerowns KV allocationexports capability
AF_UNIX + SCM_RIGHTSPOSIX FDKV bytes stay in GPU memory
LMCache serverimports handlemaps same allocation
The compressed rule is: regular allocations use legacy IPC; VMM allocations use a shareable handle requested at creation time. A non-isolated deployment can use a POSIX fd. An isolated deployment generally needs FABRIC plus a shared IMEX authorization domain.
5. Why vLLM needs VMM: CUDA graphs want stable addresses, but the KV cache wants late sizing
The KV cache often consumes most of the available GPU memory. vLLM historically had to size it before CUDA graph capture because captured graphs bake in addresses.
The final free-memory amount is only clear after graph capture, warmup, attention workspaces, sampling buffers, and speculative-decoding initialization. Early profiling either underestimates and risks OOM, or reserves excessive slack and wastes memory that could hold KV cache.
before graph capture: reserve a large virtual-address rangeduring capture/warmup: keep those addresses stableafter final sizing: create and map only the required physical chunks
The model sees stable pointers while physical backing is chosen after final sizing. The V2 CUDA and ROCm model runners enable the extensible cache by default.
External connectors introduce another constraint. A virtually contiguous segment may contain multiple physical chunks. RDMA registration may require a single physical allocation, and LMCache MP needs to export that allocation to another process. vLLM cannot choose correctly unless each connector declares how it accesses the cache.
That is the purpose of KVCacheAccess from merged PR #66605:
Access
Connector behavior
Allocation requirement
IN_PROCESS
kernels or copies inside the worker
any allocation
REGISTERS
UCX/RDMA registration
one physical chunk per segment
EXPORTS_FD
another process imports a POSIX fd
single chunk, fd-exportable
EXPORTS_FABRIC
another process imports a FABRIC handle
single chunk, FABRIC-exportable, IMEX
LEGACY_IPC
connector only supports legacy IPC
extensible VMM cache is incompatible
The direction matters: vLLM does not parse LMCache-private flags. LMCache declares its memory-access contract, and vLLM combines that with platform support, IMEX availability, and other MultiConnector children.
As of November 11, 2026, the contract is merged, while actual EXPORTS_FD backing for extensible allocations remains in #66674. An enum value existing in main does not imply every allocator path can already produce it.
6. The LMCache MP path: export a handle, register a tensor view
The vLLM worker owns the GPU KV cache. LMCache server must map it from another process before it can store or retrieve blocks. The complete path has six steps:
1. Declare connector access
Before allocation, return `EXPORTS_FD`, `EXPORTS_FABRIC`, or `LEGACY_IPC` so vLLM knows the sharing requirement.
2. Resolve the allocation
vLLM combines platform support and all connectors to choose extensible mode and requested handle types.
3. Select the LMCache wrapper
The worker follows the resolved extensible/cuMem state and selects `VmmCudaIPCWrapper`; an explicit `use_vmm_api` remains an override.
4. Export the physical handle
The wrapper locates the allocation behind the pointer and exports a POSIX fd or 64-byte FABRIC handle together with shape, stride, and offset metadata.
5. Deliver descriptor and capability
A FABRIC handle travels with the descriptor. A POSIX fd must travel over the fd channel through `SCM_RIGHTS`, paired to the request by an export ID.
6. Import and map on the server
The server imports the same physical allocation, reserves and maps its own address, then rebuilds the tensor view. Registration does not copy the whole KV pool.
The request channel and fd channel have different jobs:
ZMQ / gRPC request channel └─ config, tensor metadata, export IDs, statusAF_UNIX fd channel (POSIX-fd path only) └─ the real descriptor capability via SCM_RIGHTSGPU allocation └─ KV bytes remain on GPU; both processes map the same object
FABRIC avoids the fd channel because its handle is copyable opaque data, but replaces the shared-filesystem dependency with an IMEX authorization dependency. Neither path is universally “more advanced”; they represent different deployment and security boundaries.
7. Why automatic detection is necessary
Previously users had to set:
{ "lmcache.mp.use_vmm_api": true}
But vLLM’s final allocation is resolved from model-runner behavior, platform capabilities, extensible mode, cuMem allocator settings, connector access, IMEX, and possibly multiple connector children. Asking the user to predict that result creates a circular dependency:
LMCache: Are you using VMM? I need to select a wrapper.vLLM: How do you need to share memory? I need to select an allocation.
Declare requirements before allocation. With no explicit override, normal IPC returns EXPORTS_FD, isolated IPC returns EXPORTS_FABRIC, and explicit false returns LEGACY_IPC.
Follow the resolved result after allocation. During worker-adapter construction, LMCache derives the real mode from vLLM’s resolved enable_extensible_kv_cache or enable_cumem_allocator state.
Explicit true/false values remain authoritative for compatibility and rollback. Older vLLM revisions without KVCacheAccess receive None and retain import compatibility.
User setting
Pre-allocation declaration
Worker behavior
Main requirement
unset
EXPORTS_FD
follow vLLM's resolved allocation
fd-exportable extensible allocation and a shared fd-channel path
unset + isolated
EXPORTS_FABRIC
follow vLLM's resolved allocation
FABRIC support and the same IMEX channel on both sides
false
LEGACY_IPC
force legacy wrapper
vLLM must not select a VMM-only sharing path
true
FD or FABRIC
force VMM wrapper
the actual allocation must export the matching handle
vLLM #56492
Reserve addresses before capture and map physical KV-cache chunks after final sizing.
vLLM #66605
Add `KVCacheAccess` so connectors declare registration, fd, FABRIC, or legacy requirements.
vLLM #66674
Create fd-exportable extensible allocations for `EXPORTS_FD` and compose FABRIC handle types.
LMCache #6499
Declare access automatically and make the worker follow vLLM's resolved allocation.
The complete production path exists only when the access contract, allocation support, LMCache export/import implementation, and deployment environment all agree.
8. Deployment and debugging checklist
Determine the allocation family first
Is vLLM using the extensible KV cache or the cuMem allocator?
Did LMCache select VmmCudaIPCWrapper from the resolved engine configuration?
Do not infer the result only from whether the user wrote use_vmm_api.
For POSIX FDs
Does the device support CU_MEM_HANDLE_TYPE_POSIX_FILE_DESCRIPTOR?
Was the allocation created with that requested handle type?
Can worker and server see /dev/shm/lmcache_fd_<port>.sock?
Did the Linux fd channel start and connect?
Never serialize the raw fd integer as if it were the capability.
For FABRIC
Does the device report FABRIC-handle support?
Is nvidia-caps-imex-channels present?
Can exporter and importer access the same channel device?
Did allocation creation request the FABRIC handle type?
For CUDA_ERROR_NOT_PERMITTED, inspect IMEX access before tensor geometry.
For legacy IPC
Is the allocation truly regular CUDA memory rather than VMM backing?
Does the exporter remain alive and retain ownership?
Does the importer close its mapping before release?
Does an isolated deployment use the raw wrapper instead of shared storage IPC?
Finally, handle success only proves that the process obtained the memory object. Rebuilding the correct tensor still requires dtype, shape, stride, storage offset, and byte span. Hybrid-model padding is a separate layer; see How a Legal Memory Gap Breaks LMCache CUDA IPC.
Four sentences to keep
VMM is an allocation model that separates virtual-address reservation from physical-memory creation and mapping.
FD and FABRIC are shareable-handle types for VMM allocations, not KV payloads or ordinary network protocols.
FD relies on the OS to transfer a capability; FABRIC relies on the driver and an IMEX authorization domain.
Auto-detection is a two-phase negotiation: LMCache declares access needs, vLLM resolves allocation, and the worker follows the resolved result.
The vLLM worker and LMCache server each have their own pointer. The handle lets both pointers map the same physical allocation. Cross-process sharing is not about sending an address; it is about securely transferring the capability to create a valid mapping.