A new method, probe guidance, is introduced to improve diffusion language models by using frozen internal states to construct a guidance signal, eliminating the need for an additional forward pass at inference time and providing reliable performance on unconditional generation and multiple choice question answering benchmarks.