Specification-Guided Generative Models

We introduce Specification-Guided Generative Models (SGMs) - a novel approach to generative models - and illustrate their versatility and effectiveness across various domains. SGMs represent an evolution in model control and expressiveness, providing a framework for augmenting existing models to achieve improved quality, controllability, and variety in generated content. By developing methods for extracting latent structures and conditioning on these structures, we refine generative processes and tailor outputs to user preferences. We demonstrate SGMs' applicability within the image, music, and poetry domains, showing how SGMs can be adapted and applied to a range of generative tasks. Finally, we establish how integrating specifications, interpretable structures, and stochastic sampling techniques helps create AI systems more closely aligned with human creativity and expressiveness.