From text and proteins to audio, images, and state sequences, decoder-only generative models have proven their ability to generate new sequences across various modalities. However, integrating multiple generative foundation models, especially those trained on different modalities, into a cohesive and superior system presents significant challenges. In a new paper Zipper: A Multi-Tower Decoder Architecture for Fusing Modalities, a Google DeepMind research t