Instruction fine-tuning approaches — fine-tuning large language models on tasks described via instructions — have shown promising results in improving zero- and few-shot learning performance through fine-tuning objectives, task sampling strategies, instruction-tuning benchmarks, training datasets and so on. Yet there is limited understanding with regard to the performance trade-offs involved in the various choices made during the instruction-tuning process