In recent years, much of the research interest in large language models (LLMs) such as OpenAI’s autoregressive GPT has shifted from what these models can do to how they do it. While LLMs have demonstrated impressive prediction consistency with factual knowledge, their computations remain opaque. Knowing how and where such factual associations are stored and retrieved and improving understanding of the mechanisms underlying autoregressive knowledge represen