Latent Maximum Entropy Approach for Semantic N-gram Language Modeling

TitleLatent Maximum Entropy Approach for Semantic N-gram Language Modeling
Publication TypeConference Paper
Year of Publication2003
AuthorsF. Peng, Shaojun Wang, D. Schuurmans
Abstract

In this paper, we describe a unified probabilistic framework for statistical language modeling--the latent maximum entropy principle--which can effectively incorporate various aspects of natural language, such as local word interaction, syntactic structure and semantic document information. Unlike previous work on maximum entropy methods for language modeling, which only allow explicit features to be modeled, our framework also allows relationships over hidden features to be captured, resulting in a more expressive language model. We describe efficient algorithms for marginalization, inference and normalization in our extended models. We then present promising experimental results for our approach on the Wall Street Journal corpus.

Full Text

S. Wang, D. Schuurmans and F. Peng, "Latent Maximum Entropy Approach for Semantic N-gram Language Modeling," Ninth International Workshop on Arti_cial Intelligence and Statistics (AISTATS), 2003
year: 2003
publishtolibrary: True
organization: WSU-CSE