wtf is deco-g?

Deco-G is the research code released alongside an academic paper about how large language models generate text. The paper's idea is to separate two things that are normally tangled together when a model writes a response: figuring out the actual answer to a task, and formatting that answer into a specific required shape, such as a particular structure or style. Deco-G lets the model first focus purely on solving the task, and then a separate piece called the Format Estimation Module steps in afterward to guide the generation so it fits whatever output format the task demands. The code is organized into a few clear pieces. The core logic for loading models, applying constraints, and running generation lives in one folder, while the experiment scripts that reproduce results on two benchmark tasks, GSM8K and SummEval, live in another. There are also folders for evaluation scripts, helper utilities for building datasets and prompts, and an optional set of scripts for a more advanced technique called HMM distillation. Some of the formatting components were adapted from an earlier project called Ctrl-G, with credit given directly in the relevant files. To run this project you need a GPU and a Python environment set up through conda, installing PyTorch along with a matching CUDA version, or alternatively a plain pip installation if you already have a compatible PyTorch and CUDA setup on your machine. The distillation feature additionally needs FAISS with GPU support. The benchmark datasets are downloaded automatically from Hugging Face, and pretrained model checkpoints for the format estimation step are also pulled from a public Hugging Face repository by default. Running an experiment means calling one of the provided shell scripts with a chosen model name, such as a Llama or Qwen variant, and it produces generation output files that can then be scored with separate evaluation scripts. This is an academic research codebase rather than a polished consumer tool, so it assumes familiarity with Python, machine learning environments, and GPU setup.