theaicatalog
Hugging Face ยท Research

Hugging Face explains decision models, which return probabilities instead of generated text

Hugging Face says decision models take a state and a set of predefined questions and return typed decisions with probabilities, instead of generating an answer token by token. The post argues that LLMs can return invalid output even with constrained decoding, and that generating a full response can be unnecessary work when a model only needs to choose an answer. Transformers now supports DecisionPipeline, AutoModelForDecision, and DecisionConfig, and llama.cpp supports decision models through the same API. Decision models can be fine-tuned with the Trainer API.