Large language models (LLMs) pretrained on massive data are being used in countless real-world applications. However — as computer scientists have known for decades — not all data is equal, and this is also true with regard to the composition of LLM pretraining data, which is typically sourced from publicly available domains such as Wikipedia, book and web texts, etc. Most current data selection strategies for LLMs determine the mixture of domains (i.e. th