
The computational hurdles associated with Large Language Model (LLM) inference continue to impede their widespread deployment, particularly as prompt lengths increase. Current methods for speeding up pre-filling often struggle to maintain acceptable levels of accuracy or efficiency when applied to long-context LLMs. To address the abovementioned issue, in a new paper MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attentio