KV Layout Should Not Be Guessed: vLLM #63699, RBLN and the LMCache Contract
17 diagrams and two interactive examples explain KV axis semantics, physical layout and encoding, trace allocation through connector registration, and show the RBLN, MLA, hybrid and LMCache integration boundaries.
Maobaolong10/26/2026Updated 10/26/202638 min
Do identical shapes and strides imply the same KV cache interpretation? RBLN provides a concrete counterexample at two blocks: [B,2,H,1,N,D] and [2,B,H,1,N,D] can have identical geometry but different K/V semantics.
This article follows vLLM #63699: declare axis semantics where the view is created, then carry them to the connector through registration. The prose, diagrams and interactive examples have complete English and Chinese versions, selected with the button in the site header.
For background on NHD, HND and LBHNC, start with What KV Layouts Are Chosen by vLLM and LMCache. Here we separate axis meaning, byte addressing and content encoding, and mark the boundary between the implemented producer API and the remaining LMCache consumer work.
Contents: background, call chains, related work and end state
Read sections 01–05 for context. Trace 06–11 against the diff. Use 12–16 to judge scope and remaining work.
01 / The change in one picture
Declare axis meaning at the point of creation.
A tensor supplies geometry. It does not label its dimensions. The new registration hook carries a separate, typed declaration of those labels.
Figure 1 · Producer → registration → consumer
Allocator / custom backendCreates an actual tensor view and knows its axis roles.
→
New typed registrationtensors[name] layouts[name].axes
→
ConnectorCan validate and consume the declaration, or inherit the legacy hook.
→
LMCache integrationFuture: retain semantics through adapters, MP transport and format resolution.
Implemented by #63699Existing / integration-specificFollow-up work
What is added
KVCacheAxis, KVCacheTensorLayout, an optional registration hook, standard view declarations, runner propagation and MultiConnector forwarding.
What stays authoritative
The actual tensor's shape, strides, dtype and storage offset. Cache specs own content encoding and state granularity.
What is not completed
Hardware-plugin adoption, LMCache consumption, a versioned MP schema, arbitrary component roles, or conversion between incompatible encodings.
Scope: registration-time metadata. This PR does not transpose tensors, change attention kernels, or claim a throughput improvement.
At two blocks, shape and strides cannot tell K from V.
RBLN has two legitimate views: custom kernels use [B,2,H,1,N,D]; the Triton path uses [2,B,H,1,N,D]. When B=2, the shape and contiguous strides are identical. A detector can choose an interpretation, but cannot recover the producer's intent.
Figure 2 · Interactive ambiguity, using the same physical values
Identical shape and strides
Illustration: H=1, N=2, D=1; scalar values are physical offsets. Color means the declared component, not information stored in the bytes.
Blocks first
BLOCKKVHEAD1STATEHEAD_DIM
K/V first
KVBLOCKHEAD1STATEHEAD_DIM
Canonical output is independently ordered by component, then block and state. At B=2, identical input geometry produces different K/V sequences. At B=3 these two shapes differ, but that does not turn arbitrary shapes into semantic declarations.
Why round-trip tests can miss this: saving with the wrong interpretation and restoring with the same wrong interpretation may reconstruct the original buffer. Comparing independently expected K and V sequences catches the semantic error.
LMCache #5748 adds support for both RBLN formats and addresses a concrete deployment regression. The two-block ambiguity explains why a durable producer contract is still valuable after that detector fix.
KVCacheTensorLayout.axes describes tensor indexing order. Added by this PR. It does not sort axes by strides or decode CONTENT.
+
2 · Physical geometry — where are the bytes?
shape + stride + dtype + storage_offset + storage. Existing KVCacheLayout selects physical permutations such as LBHNC and LBNHC. A block need not be a dense, contiguous span.
+
3 · Content / state semantics — what is stored there?
KVCacheSpec and backend contracts define packing, head slots, quantization, latent/recurrent content and tokens per stored state. This PR does not serialize all of that information.
Name
Meaning here
Do not assume
KVCacheAxis.KV
Explicit two-way selector: K at 0, V at 1.
Not number of caches; not an MLA latent-count axis.
STATE / logical N
Stored positions or recurrent states.
One stored state need not mean one original token.
CONTENT / C
Opaque payload per head slot and state.
No implicit K/V split or quantization encoding.
HEAD / H
Head slots in the exposed view.
Need not equal the model's number of KV heads.
Layer key / group
Layer key identifies the registered tensor; groups carry cache-management/spec relationships.
A cache group is not a tensor axis or necessarily a unique allocation.
LMCache format
A consumer's internal interpretation and transfer dispatch.
Not the same enum as vLLM's physical KVCacheLayout.
The standard view stays BHNC. Its strides can change.
The common allocator creates a logical [L,B,H,N,C] view, then selects each layer to expose [B,H,N,C]. Physical permutations and page padding are expressed through strides and offsets, not by relabeling the public indices.
Figure 4 · Two physical orders for the same indexing contract
LBHNC
Within one layer and block, heads precede states.
H0 N0H0 N1H1 N0H1 N1
cache[b,h,n,c]
LBNHC
Within one layer and block, states precede heads.
N0 H0N0 H1N1 H0N1 H1
cache[b,h,n,c]
Example
Per-layer shape
Per-layer strides in elements
Axis tuple
LBHNC
[8,4,16,128]
[8192,2048,128,1]
(BLOCK,HEAD,STATE,CONTENT)
LBNHC
[8,4,16,128]
[8192,128,512,1]
The small colored strips are illustrative H/N cells; the numeric table uses B=8, H=4, N=16, C=128. Neither strip implies that all backends support both orders.
Tensor strides and storage_offset are in elements; allocator planning may use byte strides. Convert units explicitly. Axis metadata supplies the meaning of B/H/N/C, not the numbers in this address formula.
Model + run configAttention kind, dtype, head dimensions, explicit backend settings.
↓
Attention backend selection
The selector calls current_platform.get_attn_backend_cls(...). Per-kind overrides can choose different backends within a model.
↓
Backend spec customizationH/N/C sizes, head slots, packing and supported block sizes.
Supported physical layoutsEach backend declares an ordered set, or no restriction.
↓
Worker intersection → engine-core resolution
Compatible backend sets, matching worker reports, mixed-spec and sharding constraints; explicit environment selection must be valid, connector preference is honored when compatible, otherwise the preferred candidate wins.
↓
Allocation creates actual views + axis declarations
For standard views, always BHNC roles. Custom allocators must declare the view they actually expose.
There is no universal “device → axis tuple” table in this PR. A hardware plugin using the common allocator gets its standard declaration. A plugin creating a custom shape must supply its own declaration. This PR does not implement or verify Ascend or MUSA adoption.
MRV2: follow the same dictionary from empty to registered.
MRV2 means Model Runner V2, the second-generation model runner. It belongs to the vLLM V1 engine; section 07 explains runner selection and why HiSparse is a cache path inside MRV2.
tensor_layouts = {} is an output container. Every call below receives the same dictionary object; the view creator mutates it in place. It is not an empty layout request sent to the backend.
Figure 6 · Click a step to inspect metadata at that point
REGISTRATION METADATA SNAPSHOT
Runner starts allocation
0
described layer keys · illustrative two-layer allocation
The trace is an illustration of the real call chain, not a runtime trace from a model run. Layer names are simplified. The example adds one shared-cache alias.
# GPUModelRunner.initialize_kv_cache
layouts = {}
caches = init_kv_cache(..., tensor_layouts=layouts)
self.kv_connector = get_kv_connector(config, caches, layouts)
# Ultimately inside create_kv_cache_views, after creating BHNC views:
layout = KVCacheTensorLayout((BLOCK, HEAD, STATE, CONTENT))
layouts.update((name, layout) for name in kv_cache_tensor.layers)
Why this belongs at view creation: an earlier draft reconstructed standard declarations later from KVCacheConfig. The current PR removes that helper. Metadata now originates where the concrete indexing contract is established.
MRV1/MRV2 are runners; HiSparse is a cache scheme.
MRV1 means Model Runner V1, and MRV2 means Model Runner V2. Both execute models inside the vLLM V1 engine. The v1 directory in their paths names the engine generation, not the runner generation.
Runner
Implementation
MRV1
vllm/v1/worker/gpu_model_runner.py
MRV2
vllm/v1/worker/gpu/model_runner.py
Figure 7 · Choose the runner, then the MRV2 cache-allocation path
Upstream GPU worker
VllmConfig.use_v2_model_runner determines which runner the standard worker constructs.
↓
MRV1
First-generation model runner.
initialize_kv_cache_tensors → common allocator → typed connector registration.
MRV2
Second-generation model runner.
Ordinary allocation
hisparse_config is None init_kv_cache → allocate_kv_cache
HiSparse allocation
hisparse_config is not None init_kv_cache → init_hisparse_kv_cache → allocate_hisparse_kv_caches
HiSparse is a branch inside MRV2, not a third runner beside MRV1 and MRV2. The standard view creator supplies axis declarations for device/host views; self-managed raw storage remains unspecified.
When does vLLM choose each runner?
The following priority follows the frozen source snapshot. Out-of-tree workers can override runner construction; this is not a universal device-to-runner table.
Condition, in priority order
Result
HiSparse is configured.
MRV2 is required. Explicit VLLM_USE_V2_MODEL_RUNNER=0 raises an error.
Watermarking is configured.
MRV2 is required and overrides an explicit value of 0.
Otherwise, explicit VLLM_USE_V2_MODEL_RUNNER=1.
Request MRV2; compatibility checks still apply. The GPU worker rejects MRV2 without Triton.
Otherwise, explicit VLLM_USE_V2_MODEL_RUNNER=0.
Request MRV1; its own feature checks still apply.
No explicit setting, and Triton is unavailable on a concrete platform.
Fall back to MRV1.
No explicit setting, and the configuration has an unsupported MRV2 feature.
Fall back to MRV1.
No explicit setting and neither fallback applies.
Use MRV2.
Examples of unsupported MRV2 combinations in this snapshot include external_launcher with pipeline parallelism, the suffix/custom_class speculative methods, and heterogeneous-vocabulary draft models. Support changes over time; inspect the selected version's checks rather than assuming an environment variable guarantees a working configuration.
What does HiSparse do?
HiSparse is an experimental, local sparse-MLA KV offload scheme. Sparse attention selects a subset of historical positions per step. HiSparse can keep historical KV in pinned host memory while the GPU uses resident pages and a hot-row cache, reducing the historical KV footprint on the GPU at the cost of host memory and transfers.
top-k historical positions selected for this step
↓
GPU resident page hit? ── yes → use the resident row
↓ no
GPU hot-row hit? ──────── yes → use the cached row
↓ no
fetch the pinned host row into the GPU hot cache
The fused GPU resolver checks resident pages, then hot rows, then host memory. HiSparse's indexer KV is managed separately; HiSparse does not automatically create its own host copy of every KV group in the model.
How is HiSparse enabled?
Configuring HiSparseConnector, directly or through MultiConnector, infers default hisparse_config. The connector supplies the host_pool_gib capacity; HiSparseConfig tunes fields such as the GPU hot-buffer size. A complete, supported configuration is required. Low free GPU memory alone does not activate HiSparse.
Snapshot constraints: NVIDIA CUDA, MRV2, and DSA models with index_topk. Pipeline parallelism and decode context parallelism are unsupported. The single cudagraph_mode=FULL setting is rejected; FULL_AND_PIECEWISE, the default, can still capture full decode graphs. These are compatibility conditions, not hardware validation performed by this PR.
HiSparseConnector manages local host/GPU residency. It can be composed with P/D or another offload connector through MultiConnector. This role is distinct from the remaining LMCache adapter/MP integration described later.
Why this matters to #63699: normal MRV2 allocation and the HiSparse host/device path must both carry matching declarations for the actual registered views. The PR propagates metadata through those paths; it does not choose the runner or enable HiSparse. Existing allocator return types remain unchanged, shared layers inherit their target descriptor, and raw buffers are not assigned a fabricated BHNC declaration.
A layer map is not a promise of one layout, one allocation or one encoding.
Figure 8 · Shared backing does not erase per-layer views
The drawing illustrates cross-layer physical packing. Hybrid cache groups can additionally overlay storage according to the allocator plan; independent layer keys do not imply independent buffers.
Registered view, illustrative
Axis roles
Additional interpretation needed
Full attention [B,8,16,C₁]
BHNC
Dense K/V packing and dtype.
Sliding attention [B,8,16,C₂]
BHNC
Its retention/group/block-table policy and spec.
Compressed MLA [B,1,4,C₃]
BHNC
Latent encoding and tokens per stored state.
Recurrent / Mamba [B,1,1,C₄]
BHNC for the standard view
State components, component dtypes and payload organization.
Raw self-managed backing
No entry
Its owner's explicit contract; no default axes assumed.
Hybrid correctness requires more than axes: combine the per-layer declaration with the actual tensor, matching cache spec/group information and block/state interpretation. This PR does not make all layers interchangeable.
Attention binding can create kernel-block views after allocation. The connector receives the registered allocation view and its matching descriptor, not an axis tuple copied from a backend-private view. A BLOCK label alone does not define how its indices map to scheduler blocks; specs and block mapping still matter.
RBLN declares the layout chosen by its own shape logic.
Figure 9 · Custom views join through the hook, not through the BHNC allocator
use_custom_kernel → existing attention_block_axis(...) selects where B is inserted.
↓
RBLN custom ops
[B,2,H,1,N,D]
BLOCKKVHEADSINGLETONSTATEHEAD_DIM
RBLN Triton ops
[2,B,H,1,N,D]
KVBLOCKHEADSINGLETONSTATEHEAD_DIM
↓
RBLN runner registers actual per-layer views + layouts
Validate shapes, preserve shared aliases, and invoke register_kv_caches_with_layouts when available. Older vLLM connectors retain the tensor-only path.
RBLN #1297 is a downstream adaptation, not a prerequisite for #63699. It remains a draft until the upstream API merges, is available in RBLN's supported dependency, and receives SDK/runtime validation. Its current dependency pin is not changed by that draft. The earlier prototype name TOKEN has been aligned to STATE.
10 / A size-one dimension is not automatically a KV dimension
MLA has no separate K/V selector in its standard view.
The descriptor's KV role means two separately addressable components, K then V. MLA's shared latent representation does not become a one-element K/V selector. MLAAttentionSpec sets head_size_v=0; its content is interpreted through the spec.
Figure 10 · Three valid MLA declarations
Standard view
[B,1,N,C]
BLOCKHEADSTATECONTENT
Size one is a head slot. Keep its semantic role.
Custom squeezed view
[B,N,C]
BLOCKSTATECONTENT
No head or K/V axis is required by the descriptor.
Custom padded rank
[1,B,N,C]
SINGLETONBLOCKSTATECONTENT
Use SINGLETON for an actual placeholder, not a semantic axis.
Invalid declaration: describing [1,B,N,C] as (KV,BLOCK,STATE,CONTENT) raises ValueError. A lone element does not identify K, V or latent content. Relaxing the size check would introduce ambiguity rather than encode MLA.
validate_shape checks rank, KV size two and SINGLETON size one. Construction rejects duplicate named roles except SINGLETON. These checks do not prove a declaration is true: equal-sized axes can still be mislabeled. Correctness starts with the producer's known view contract.
Validates and interprets declarations with its spec/group context.
↓
Retains supported semantics for transfer dispatch.
Metadata used
MultiConnector
Overrides the new hook.
↓
Forwards the tensor and descriptor maps to every child.
↓
Each child chooses its own old/new path.
Mixed children supported
A missing entry means unspecified semantics, not an implied default. The base hook allows partial mappings. A consumer requiring complete explicit semantics must reject unsupported/missing entries or use a well-defined, spec-aware legacy path. Overrides are responsible for validation before consumption.
Important scope boundary: #63699 alone does not make an existing LMCache connector stop detecting formats. It must implement the new hook and carry the declaration through its own adapters and registration transport.
Declare → preserve → validate → resolve once → transfer.
Figure 12 · The complete producer-to-consumer contract
Producer owns the truth
Actual registered tensors + axis declarations + matching cache-spec/group context. Standard allocator or custom plugin; descriptors follow the exposed view.
↓
vLLM registration boundary
Runner → typed hook → optional MultiConnector → child. This transport inside the engine is provided by #63699.
↓
Consumer adapter & MP registration
Future LMCache work: forward semantics through normalization and REGISTER_KV_CACHE; define a versioned transport schema, explicit component roles and required encoding/granularity context. Python IntEnum values alone are not a wire protocol.
↓
Validate and retain a supported format
Future: map declarations plus specs/geometry to supported EngineKVFormat/KVFormatSpec and group/transfer behavior. Explicit inconsistent or unsupported metadata should fail clearly, not be replaced by a guess.
↓
Use existing execution paths where compatible
Future: transfer with the right address geometry and kernels. Preserve matching metadata on restore, reallocation and cross-process handoff. Negotiate or reject incompatible encodings rather than promise universal conversion.
Cyan: producer/API work in this PR. Dashed gold: proposed consumer integration and broader contract work tracked by LMCache RFC #5866; not implemented or validated by #63699.
Explicit path
Producer declaration is authoritative. Consumer validates it against the exact view, retains the result, and dispatches without re-guessing axis roles on each use.
Legacy path
Keep support for older engines when interpretation is supported and unambiguous. Ambiguous cases require more information; an unrelated device-name default is not proof.
A wrong axis interpretation can become a wrong byte selection.
Figure 13 · A possible failure chain, not a claimed production incident
Unlabeled viewConsumer sees shape and strides.
→
Ambiguous detectorTwo supported interpretations fit.
→
Wrong geometryBlock/KV role or component split is wrong.
→
Wrong bytes transferredCanonical chunks can contain mismatched K/V sequences.
Visible rejection
A new producer layout may be refused as unsupported. RBLN blocks-first support in LMCache #5748 addresses such a concrete integration regression.
Silent ambiguity
Equal-sized axes can pass shape and bounds checks. Canonical-value tests demonstrate the risk; this guide does not claim an observed production output failure.
Maintenance coupling
Every producer shape change can require a new consumer heuristic, even though the producer already knows the intended interpretation.
Not every tensor-only integration is unsafe. Existing spec-aware contracts can be precise, and many layouts are unambiguous. The issue is treating geometry or backend names as a complete semantic contract when they are not.
Carry the tensor and its declaration as one logical pair.
Figure 14 · Transformations that need matching semantics
Event
Tensor change
Descriptor / context action
Permutation
[B,H,N,C] → [B,N,H,C]
Permute axes to (BLOCK,STATE,HEAD,CONTENT).
Squeeze a head slot
[B,1,N,C] → [B,N,C]
Remove HEAD from this exposed view. Do not leave a rank-four declaration.
Shared-layer alias
New layer key points at the same tensor.
Copy the target declaration under the alias key. Implemented in both runners.
Component split / reshape
One payload becomes multiple views, or dimensions are merged.
Describe new view roles and component identity; axis permutation alone may be insufficient. Broader schemas remain follow-up.
Reallocation / recovery
New storage or different views.
Re-register new matching tensor/descriptor pairs; do not rely on stale addresses or cached metadata.
Cross-process handoff
Python objects cannot be passed unchanged.
Serialize a versioned, consumer-understood contract; preserve dtype/spec/state context and reject unsupported versions.
Declaring a view once helps only if later adapters preserve its semantics. Reinterpreting, squeezing or regrouping the tensor while retaining old axis labels recreates the original problem.
Supports RBLN blocks-first format alongside K/V-first and updates transfer handling. Format support does not prove intent when dimensions are ambiguous.
Opt-in FlashInfer K/V head-slot placement for transfer interoperability. Demonstrates why BHNC axes alone do not identify packing.
Statuses are a 26 Oct 2026 snapshot, not a live dashboard. Open links for current state. Dates above are merge dates, not publication dates.
16 / A practical review route
Review the contract before the plumbing.
1. Types and validation. Is the distinction between KV selector, HEAD, STATE and opaque CONTENT clear? Does validation reject impossible fixed-size axes without pretending to prove semantics?
2. View creation. Are declarations attached where the actual BHNC view is created? Are raw-storage paths omitted?
3. Runner propagation. Follow the fresh output map through standard/HiSparse allocation and shared aliases. Verify registration describes the connector view.
4. Compatibility. Does inherited registration preserve existing overrides? Does MultiConnector pass the same semantics to all children?
5. Behavior tests. Compare independently expected K/V values, tensor identity, strides, mixed specs, aliases and error handling.
6. Downstream boundary. Check that RBLN uses the shared upstream contract, while LMCache consumer/MP work remains explicitly out of scope.
132
Passed
Registration, MultiConnector, attention-utils and GPU-connector CPU suites at 9be42ea. One existing GPU-only skip and one explicitly excluded serving test.
94
Passed
Packing, planning and allocation-scope suites at 9be42ea, after caching the required Qwen config.
3
MLA examples checked
At doc-only 3838746: all documented valid shapes accepted; size-one KV rejected. Applicable pre-commit checks passed.
Evidence limits: no GPU/model-serving E2E, RBLN NPU runtime, Ascend/MUSA integration, or LMCache consumer/MP E2E is established by these checks. RBLN targeted tests cannot collect on the local macOS host without rebel. Static checks passed for its API-alignment follow-up.
CPU reproduction commands and prerequisites
Use the vLLM development environment, Python 3.12, pytest and tblib; tests were run on macOS arm64 with PyTorch 2.14.1. The suite-tested commit is 9be42ea, based on 4b1c04e. Existing tests require cached facebook/opt-125m configuration and Qwen/Qwen1.5-7B/config.json; no model weights are needed for these CPU runs.
Seven test functions, 17 cases: what does each assertion protect?
This section reads tests/v1/kv_connector/unit/test_kv_cache_registration.py at 3838746. Each explanation covers inputs, the exercised path, independent expected values and the actual assertion boundary. Code blocks preserve the original tests and parameter decorators for comparison with the pinned source.
Test
Cases
What it protects
two_blocks…
2
K/V values and mixed-child compatibility
allocator_views…
8
Six layouts, two padding cases, aliases
invalid_descriptor…
3
Rank/KV/SINGLETON checks and side effects
unknown_layer…
1
Reject an extra descriptor key
duplicate_semantic_axes
1
Reject duplicate BLOCK
missing_descriptors…
1
Preserve unspecified semantics
hybrid_allocator…
1
Per-view roles, mixed geometry, shared backing
Start with the fixture: a real MultiConnector and two recording children.
TensorRegistrationConnector implements only the old register_kv_caches and records its input dictionary. Its inherited new hook tests compatibility with that old override. AxisRegistrationConnector calls super for validation and legacy dispatch, then records layouts as a descriptor-aware consumer. make_connector creates a CPU VllmConfig and dynamically loads both children by module path/class name with the WORKER role. This exercises real registration plumbing, not a mocked forwarding call or a serving engine.
Figure 16 · The registration path exercised by the fixture
New hook: tensors + layouts
↓
MultiConnector forwards in order
↓
Legacy child: validate → old override
Aware child: super → retain layouts
Expand the full fixture and both connector classes
1. At two blocks, forwarding a descriptor is not enough: read the right K/V values.
test_two_blocks_keep_kv_semantics_through_mixed_connectors has two parameter cases: blocks-first and kv-first. Both place arange(8) in shape (2,2,1,1,2,1), so shape and strides are identical. Only the axis declarations differ. The test consumer permutes into (KV,BLOCK,STATE,HEAD,HEAD_DIM,SINGLETON) using the received descriptor, then compares flattened values.
Figure 17 · Work out the expected values independently
Producer declaration
Original values by block
Canonical K-first result
BLOCK → KV
b0: K=[0,1], V=[2,3] b1: K=[4,5], V=[6,7]
K=[0,1,4,5] V=[2,3,6,7]
KV → BLOCK
K: b0=[0,1], b1=[2,3] V: b0=[4,5], b1=[6,7]
K=[0,1,2,3] V=[4,5,6,7]
Why hard-code the expected arrays? The oracle is independent of the implementation. Reusing the same axis-resolution algorithm to compute expected values could let both share a bug. is caches checks that both children receive the same dictionary; the value assertion checks the interpretation of K/V. Offsets 0 and 100 register two new tensor batches through the same connector, catching stale data on the second call. The declaration stays the same, so this does not test changing layouts during re-registration.
@pytest.mark.parametrize(
("axes", "expected"),
[
((Axis.BLOCK, Axis.KV), [0, 1, 4, 5, 2, 3, 6, 7]),
((Axis.KV, Axis.BLOCK), list(range(8))),
],
ids=["blocks-first", "kv-first"],
)
def test_two_blocks_keep_kv_semantics_through_mixed_connectors(
make_connector, axes, expected
):
"""Equal shapes must still give the consumer the producer's K/V values."""
connector = make_connector(KVCacheConfig(2, [], []))
layout = KVCacheTensorLayout(
(*axes, Axis.HEAD, Axis.SINGLETON, Axis.STATE, Axis.HEAD_DIM)
)
order = (Axis.KV, Axis.BLOCK, Axis.STATE, Axis.HEAD, Axis.HEAD_DIM, Axis.SINGLETON)
for offset in (0, 100):
caches = {"layer": (torch.arange(8) + offset).reshape(2, 2, 1, 1, 2, 1)}
connector.register_kv_caches_with_layouts(caches, {"layer": layout})
legacy, consumer = connector.sub_connectors
assert isinstance(legacy, TensorRegistrationConnector)
assert isinstance(consumer, AxisRegistrationConnector)
assert legacy.registered_caches is consumer.registered_caches is caches
canonical = consumer.registered_caches["layer"].permute(
*(consumer.layouts["layer"].axis_index(axis) for axis in order)
)
assert canonical.flatten().tolist() == [value + offset for value in expected]
2. Do allocator views retain their values across physical layouts and aliases?
test_allocator_views_and_aliases_keep_their_values expands into eight cases: all six KVCacheLayout members, plus padded LBHNC and BLHNC. It calls the real compute_layout_strides and allocate_kv_cache; allocation results are not mocked. Two layers share a backing allocation while exposing separate views.
The test writes 0…63 into first, fills second with -1, and aliases first. The consumer constructs semantic coordinates B=1,H=0,N=2,C=3 from the received axis tuple. In logical shape (2,2,4,4), the expected value is (((1×2+0)×4+2)×4+3)=43, independent of physical order. A descriptor/view mismatch can therefore produce a concrete wrong value.
It also verifies that second was not overwritten and that stride(0) × element_size() matches the configured block stride in bytes. A padded page is 144 bytes rather than 128, checking that registration preserves gaps between pages. Only block stride has an explicit equality assertion; layer stride participates in construction but is not separately asserted. The test manually attaches both the alias tensor and descriptor. It verifies forwarding after attachment, not automatic alias wiring inside either model runner.
3. Invalid metadata must fail before legacy registration causes side effects.
test_invalid_descriptor_is_rejected_before_legacy_registration fixes the declaration to (BLOCK,KV,SINGLETON,HEAD_DIM) and tries three invalid shapes: (3,2,1) has the wrong rank; (3,3,1,4) has KV size 3; (3,2,2,4) has structural singleton size 2.
pytest.raises(ValueError, match=message) checks both exception type and category. Afterwards, no child may have a registered_caches attribute, showing that the fixture's old hook has not run. The first child inherits base validation, which rejects the input before MultiConnector reaches the next child. This is not a general transactional rollback guarantee for arbitrary custom overrides that bypass validation.
4. A descriptor cannot refer to a layer that has no registered tensor.
test_unknown_layer_descriptor_is_rejected passes an empty tensor dictionary with a declaration for missing. It expects ValueError containing unregistered layer. This catches mismatched map keys, such as a stale layer descriptor left after a model rebuild.
This enforces that every supplied descriptor has a corresponding tensor. It does not require a descriptor for every tensor; test 6 deliberately permits missing declarations.
5. Reject repeated semantic roles when constructing the descriptor.
test_rejects_duplicate_semantic_axes constructs (BLOCK,BLOCK) and expects ValueError containing unique, before any connector runs. Otherwise, axis_index(BLOCK) would return only the first occurrence and silently ignore the other supposed BLOCK axis.
This case covers duplicate BLOCK only. The implementation allows repeated structural SINGLETON roles, but this test does not exercise that valid case or enumerate every role combination. It is a focused regression, not a complete schema property test.
def test_rejects_duplicate_semantic_axes():
with pytest.raises(ValueError, match="unique"):
KVCacheTensorLayout((Axis.BLOCK, Axis.BLOCK))
6. An absent declaration stays unspecified; do not invent a plausible layout.
test_missing_descriptors_stay_unspecified creates a configuration with a HiSparse raw spec but kv_cache_tensors=[]. With no allocation records, the allocator returns an empty dictionary and leaves output layouts empty. The test then manually registers raw and custom tensors with an empty descriptor map.
The consumer must receive the original tensor dictionary and no declarations. Rank, layer names or specs must not trigger an invented BHNC default. This case does not allocate HiSparse raw storage: it tests empty allocator output and externally supplied tensors registered without declarations. The hybrid case below actually exercises raw-storage allocation and verifies that it receives no descriptor.
7. Hybrid caches require declarations for each actual view, not one model-wide string.
test_hybrid_allocator_registers_each_created_view combines full attention, sliding attention, Mamba and HiSparse raw specs. Each KVCacheTensor uses the total backing allocation size, while its offset advances through separate regions. The resulting views share an allocation without addressing the same region.
Layer
Actual shape
Meaning in this test
full
(2,2,4,4)
float32; 2 heads, 4 states, CONTENT width 4.
sliding
(2,1,2,4)
float16; 1 head and 2 states. Shared roles do not imply shared geometry.
mamba
(2,1,1,24)
24 opaque state bytes per block: (2 + 2×2) × 4. HEAD/STATE are size-one slots.
raw
One-dimensional backing buffer
No layer view, so no tensor-axis declaration.
The assertions check three things: only full/sliding/mamba have descriptor keys; all three declare (BLOCK,HEAD,STATE,CONTENT); and separately written layer values survive registration while each layer's untyped_storage().data_ptr() equals raw's storage address. This connects per-view declarations with a shared backing allocation.
Only full and sliding have explicit dtype assertions. Mamba's byte-shaped view and values are checked, but the two float32 state components are not decoded. Nor does the test establish state-to-token granularity; that still belongs to the spec and execution logic.
Where does this evidence stop? It covers the real CPU allocator → view metadata → MultiConnector → recording consumer, plus invalid-input paths. It does not execute full MRV1/MRV2 initialize_kv_cache, HiSparse host/device binding, attention kernels, model serving or LMCache IPC. The RBLN-shaped case does not use the rebel SDK. Parameterization expands seven functions into 17 pytest cases; offset and per-layer loops add assertions inside a case, not extra pytest cases.
Run this file on its own
Use the vLLM CPU development prerequisites from the preceding section and check out 383874621325a7057190711527ad268c183e0b19: Python 3.12, PyTorch 2.14.1, pytest, tblib and the vLLM test dependencies. A fresh run of this file on 27 Oct 2026 produced 17 passed; its only warning was the missing generated vllm._version in the source environment. No model weights or GPU inference were involved.
Every implementation link pins the reviewed source.
vLLM links in this document use 383874621325a7057190711527ad268c183e0b19. RBLN links use a930b4daaf19ea083f3f7d8f80ed3fa82cc7a27d. PR and issue links remain live. If the implementation changes, review the new diff rather than treating this snapshot as current.
Reviewer's takeaway: the change moves known axis semantics across an existing ownership boundary. It provides a compatible producer API; complete layout understanding still combines that declaration with tensor geometry, specs and a consumer that actually preserves and uses it.
Discussion
Comments and Reactions
This article does not have an initialized discussion thread yet.