Pre-Compiled Pipeline Shards for Distributed LLM Inference on Intel AI PC Fleets
Modern Intel AI PCs ship capable integrated GPUs and NPUs with 16+ GB of unified memory, and they spend considerable time idle. That is not enough memory to fit a large model such as a 70B-parameter LLM. We show that a handful of AIPCs, working together over an ordinary network, can serve models beyond the capability of any single one. We use pipeline parallelism: a model is split by layer into per-stage shards, each pre-compiled into an OpenVINO graph, so that every machine runs one shard and passes activations to the next. Three techniques make this fast enough to be useful. First, we recove
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- PossiblePossibly related (embedding) · 26%vllm-project/vllm →
“Possibly related via embedding similarity 0.57 (not asserted). Timestamp check: artifact slightly before paper (-48d).”
- PossiblePossibly related (embedding) · 25%alibaba/MNN →
“Possibly related via embedding similarity 0.55 (not asserted). Timestamp check: artifact slightly before paper (-44d).”
- PossiblePossibly related (embedding) · 61%Hardware startup unveils inference accelerator →
- FuzzySimilar title/name (fuzzy) · 84%xorbitsai/inference →
“Fuzzy title match (0.92): “Pre-Compiled Pipeline Shards for Distributed LLM Inference o” ≈ “xorbitsai/inference””
- LinkedLinked via arxiv author · 85%Tate Berenbaum →
“Pre-Compiled Pipeline Shards for Distributed LLM Inference on Intel AI PC Fleets”
- LinkedLinked via arxiv author · 85%Muthaiah Venkatachalam →
“Pre-Compiled Pipeline Shards for Distributed LLM Inference on Intel AI PC Fleets”