Why a Mamba State at Token 8768 Cannot Be Stored Under 8704
SGLang scheduling, HiCache, Mamba state, and LMCache explained from first principles—and why checkpoint alignment must be fixed before the model forward pass.
Ongoing notes on AI infrastructure, distributed systems, databases, performance work, and engineering practice.
SGLang scheduling, HiCache, Mamba state, and LMCache explained from first principles—and why checkpoint alignment must be fixed before the model forward pass.
A visual, interactive introduction to legacy CUDA IPC, virtual memory management, POSIX file-descriptor exports, FABRIC handles, IMEX channels, and the memory-access negotiation between vLLM's extensible KV cache and LMCache MP.
An animated walkthrough of LMCache multiprocess P2P: registration, peer discovery, the NIXL handshake, lookup and locking, address exchange, direct KV reads, unlock, and the exact boundaries that do or do not need authentication.
A beginner-friendly explanation of KV cache blocks, kernel pages, why LMCache confused kernel paging with compression, how cache hits could silently restore the wrong KV, and how PR #6019 fixes every MP transfer path with num_blocks, k detection, and block-to-page expansion.
17 diagrams and two interactive examples explain KV axis semantics, physical layout and encoding, trace allocation through connector registration, and show the RBLN, MLA, hybrid and LMCache integration boundaries.
A beginner-friendly guide to KV cache, paged memory, NHD/HND layouts, LMCache multiprocess kernel and direct-copy paths, and why one production-code line completes the fast path for SGLang unified MP.
Starting from LMCache Issue #5097, this article explains hybrid-model dim-0 padding, tensor strides, three CUDA IPC wrappers, and how PR #5693 and #5855 restore zero-copy transport with byte spans and as_strided.
Starting with the DMA of SSD controllers, PCIe address mapping, and GPU registers, this article uses seven animated diagrams to walk through the initialization, STORE, and RETRIEVE of LMCache MP, and explains how the backend can be extended using Phoenix.
An introduction to how LMCache multiprocess request transport evolved from ZMQ to gRPC: specific configurations for server and vLLM, implementation principles, codec design, benefits, and how to maintain compatibility when adding new service/message/rpc/field.
Starting from Prefill, Decode, and KV Cache, this article gradually explains Mamba, GDN, SWA, MLA, MTP, vLLM align-mode checkpoint, speculative block relocation, and the state misalignment issue fixed in LMCache PR #5004.
This article systematically explains why LMCache requires this layer of registration protocol, focusing on the KV cache passed after calling register_kv_caches in vLLM, client handling of the LMCache-driven multiprocess connector, server registration, EngineGroupInfo generation, and the complex model layouts of standard Attention, MLA, DeepSeek, Qwen, and Mamba.
From parsing vLLM's KVCacheLayout and the preferences of different attention backends to probing and converting boundaries of LMCache's EngineKVFormat, this system explains why there are multiple KV layouts.
An analysis of LMCache PR #3476: How the LMCacheMPConnector automatically pulls up the local MP server when vLLM starts, how the port is resolved, how worker 0 takes on the responsibility of starting, and the lifecycle boundaries of this feature.
Starting from the vLLM Step message pump and the boundaries of Scheduler/Worker/Cache Service, this explains why LMCache PR #4998 only persists in-flight lazy stores, and why the last batch of GPU blocks is delayed in release without it.
Starting from the generation and reuse of KV Cache, this article dissects the layered storage, role boundaries, Router Hint, data transportation, state machine, fault recovery of KVCC/KVCR, and whether this architecture can become a universal infrastructure in the future.
Through a review of a LocalDiskBackend disk write, analyze duplicate submissions, capacity pre-checks, failure rollbacks, and resident/in-flight handover races, as well as how to validate fixes with deterministic testing.
Starting from pageable memory, pinned memory, device support matrices, and asynchronous copying, this analysis explores how LMCache can cause silent data corruption due to raw pointer lifetimes and incorrect synchronization boundaries.
The entire process of getting sglang running on Apple Silicon Mac—why Docker is the only option, three essential aarch64 patches, the HTTP/1.1 chunked trap of Docker Desktop vpnkit, and ultimately how to obtain real inference results using curl.
Test.
test2
test3
What truly sets apart a technical blog is not the density of terminology, but whether you can clearly explain the design motivations, stress models, and key metrics together.
When encountering complex failures, the most important thing is not to immediately guess the root cause, but to quickly reduce uncertainty.
A technical site that can be updated long-term is not just about writing articles; it will also change the way you output, collaborate, and consolidate knowledge.