Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
📜 Abstract
We present conditional random fields, a framework for building probabilistic models to segment and label sequence data. Conditional random fields offer several advantages over hidden Markov models and stochastic grammars for such tasks, including the ability to relax strong independence assumptions made in those models. Conditional random fields also avoid a fundamental limitation of maximum entropy Markov models (MEMMs) and other discriminative Markov models based on directed graphical models, which can be biased towards states with few successor states. We present iterative parameter estimation algorithms for conditional random fields and compare the performance of the resulting models to HMMs and MEMMs on synthetic and natural-language data.
✨ Summary
The paper introduces conditional random fields (CRFs), conditionally trained undirected graphical models for segmenting and labeling sequences. Its central contribution is a globally normalized sequence model that can incorporate arbitrary, overlapping, and non-independent observation features while avoiding the label-bias problem associated with locally normalized maximum-entropy Markov models. For chain-structured models, the paper gives dynamic-programming methods for inference and iterative procedures for parameter estimation, and reports comparisons with HMMs and MEMMs on synthetic data and part-of-speech tagging.
The work became foundational for discriminative sequence labeling. Subsequent research developed CRF variants, including semi-supervised CRFs for sequence segmentation and labeling, neural models that combine recurrent representations with CRF decoding, and constrained or masked CRFs for preventing illegal output sequences. CRFs were also applied to information extraction and other structured prediction tasks. These later works explicitly identify the 2001 paper as the originating formulation or as the basis for their model extensions. (cs.columbia.edu)