A simple C++ library for maximum entropy classification

Overview

This is a simple C++ library for maximum entropy classifiers (also known as "multinomial logistic regression"). The main features of this library are:

• supporting L1/L2 regularization [1,2]
• supporting fast parameter estimation algorithms (LBFGS [3], OWLQN [4], and SGD [5])
• supporting real-valued features

How to use

This software is currently tested only on linux and gcc.

1. Download the latest version

• Apr. 1 2015 maxent-3.0.2.tar.gz
• Feb. 24 2015 maxent-3.0.1.tar.gz
• Jul. 14 2011 maxent-3.0.tar.gz
• Aug. 21 2006 maxent-2.1.1.tar.gz
• Feb. 19 2006 maxent-2.0.tar.gz
• Oct. 28 2005 maxent-1.3.2.tar.gz
• Sep. 19 2005 maxent-1.3.1.tar.gz
• Sep. 13 2005 maxent-1.3.tar.gz
• Jul. 8 2005 maxent-1.2.2.tar.gz

2. Expand the archive

3. Compile

4. Run the sample program

Example

#include <string>
#include <list>
#include <cstdio>
#include "maxent.h"

using namespace std;

void make_training_data(ME_Model & model)
{
  ME_Sample s1("CAR");
  s1.add_feature("four wheels");
  s1.add_feature("blue");

  ME_Sample s2("CAR");
  s2.add_feature("four wheels");
  s2.add_feature("red");

  ME_Sample s3("BICYCLE");
  s3.add_feature("two wheels");
  s3.add_feature("red");

  ME_Sample s4("BICYCLE");
  s4.add_feature("two wheels");
  s4.add_feature("yellow");

  model.add_training_sample(s1);
  model.add_training_sample(s2);
  model.add_training_sample(s3);
  model.add_training_sample(s4);
}

int main()
{
  ME_Model model;
  make_training_data(model);
  model.train();

  ME_Sample s;
  s.add_feature("two wheels");
  s.add_feature("blue");

  // classifying a sample
  model.classify(s);
  cout << endl << "it's a " << s.label << " !" << endl << endl;
}

References

[1] Jun'ichi Kazama and Jun'ichi Tsujii. 2003. Evaluation and Extension of Maximum Entropy Models with Inequality Constraints, In Proceedings of EMNLP, pp. 137-144.

[2] Stanley F. Chen and Ronald Rosenfeld. 1999. A Gaussian Prior for Smoothing Maximum Entropy Models, Technical Report CMU-CS-99-108, Computer Science Department, Carnegie Mellon University.

[3] Jorge Nocedal. 1980. Updating Quasi-Newton Matrices with Limited Storage, Mathematics of Computation, Vol. 35, No. 151, pp. 773-782.

[4] Galen Andrew and Jianfeng Gao. 2007. Scalable training of L1-regularized log-linear models, In Proceedings of ICML.

[5] Yoshimasa Tsuruoka, Jun'ichi Tsujii, and Sophia Ananiadou. 2009. Stochastic Gradient Descent Training for L1-regularized Log-linear Models with Cumulative Penalty, In Proceedings of ACL-IJCNLP, pp. 477-485