wtf is setdlms?

This repository is the official research code for a paper accepted at ICML 2026 called Set Diffusion, which explores a new way for language models to generate text. Normal language models write text one word at a time in order, left to right. Diffusion models, by contrast, start from noise and gradually clean it up all at once. This project introduces a method that sits between the two approaches, letting the model choose different token orderings so it can decode faster and more flexibly than either extreme on its own. The codebase includes three families of models to compare: standard autoregressive models, several kinds of masked diffusion models including block diffusion, and the paper's own Set Diffusion model. Each model is built from two parts, a denoiser that decides how noisy inputs get cleaned up, and a backbone neural network that turns those inputs into predictions. Setup is done through a conda environment with pinned, locked dependencies to guarantee reproducibility, with a pip-only path available as an alternative. A helper script sets shell variables for compute clusters, and users can opt in to add their own Weights and Biases and Hugging Face credentials through a separate, untracked file. The repository is organized around reproducing the paper's experiments. Shell scripts train and evaluate models on tasks such as text summarization, mathematical reasoning on GSM8K, likelihood estimation, multiple-choice commonsense question answering, and text infilling. Evaluation scripts can pull pretrained checkpoints directly from Hugging Face or point to a local checkpoint path, and dedicated wrappers reproduce the paper's throughput measurements on H100 GPUs. This is a research codebase built for machine learning researchers and practitioners who want to reproduce the paper's results or build on the Set Diffusion approach, rather than a general purpose tool for non-technical users. Working with it requires comfort with Python, PyTorch, and GPU-based training.