The Transformer architecture has emerged as a pivotal tool in numerous domains, excelling particularly in tasks like speech recognition, machine translation, and document summarization. Yet, its efficacy often hinges on expanding the model’s size to tackle increasingly intricate challenges, thereby imposing substantial computational burdens. In the pursuit of alleviating the computational strain associated with Transformers, the exploration of linear atten