Researchers investigated the phenomenon of mode collapse and over-dispersion in large language models (LLMs), finding that fine-tuning with sufficient data can improve diversity. The study provides a theoretical framework and experimental results demonstrating that finite-sample fine-tuning can lead to under- or over-dispersion, depending on the model and dataset. This work has implications for the development and deployment of LLMs.