From Schema to State: Zero-Shot Scheme-Only Dialogue State Tracking via Diverse Synthetic Dialogue and Step-by-Step Distillation
Authors: Xu, H., Li, Z., Tang, W., Zhang, J.J.
Journal: Emnlp 2025 2025 Conference on Empirical Methods in Natural Language Processing Proceedings of the Conference
Publication Date: 01/01/2025
Pages: 1640-1652
DOI: 10.18653/v1/2025.emnlp-main.85
Abstract:Dialogue State Tracking (DST) is crucial for linking user intentions to appropriate services in task-oriented dialogue systems. We propose a zero-shot, scheme-only approach that tackles two main challenges: generating synthetic dialogues that balance diversity with schema alignment, and efficiently distilling knowledge from a large language model (LLM) into a smaller model. Our pipeline first creates scenarios, dialogue logic flows, and utterances via dynamic complexity prompting, eliminating reliance on handcrafted templates. We then use a two-stage distillation process to learn formalized dialogue representations and DST related chain-of-thought reasoning. This structure preserves interpretive capabilities while reducing inference overhead. Experiments on the MultiWOZ benchmark show that our method achieves state-of-the-art performance under zero-shot, scheme-only situations and generalizes to few-shot scenarios effectively, offering a practical and scalable solution for domains that lack real data. Our code is available.
Source: Scopus