Revealing Latent Semantics in Generative Noise

Generative diffusion models accept two primary inputs: a semantic conditioning signal that controls the output, and Gaussian noise that acts as a source of randomness. While this noise is rarely seen as a meaningful representation of the output image, our work demonstrates that, under constant general conditioning, it is actually a highly structured and compositional latent space of natural images. We expose this phenomenon by identifying semantic directions directly within the initial noise space, utilizing them for classification and for editing generated images along linear pathways. We even perform vector arithmetic in noise space in a compositional manner. These findings suggest that diffusion models naturally possess a Gaussian-type latent space rich in semantic structure. Key parts of this work were presented at CVPR 2026.

M.Sc. student under the supervision of Prof. Guy Gilboa.