Jointly Learning Word Representations and Composition Functions Using Predicate-Argument Structures

Kazuma Hashimoto, Pontus Stenetorp, Makoto Miwa, and Yoshimasa Tsuruoka. EMNLP 2014

  • Abstract
  • We introduce a novel compositional language model that works on Predicate-Argument Structures (PASs). Our model jointly learns word representations and their composition functions using bag-of-words and dependency-based contexts. Unlike previous word-sequence-based models, our PAS-based model composes arguments into predicates by using the category information from the PAS. This enables our model to capture long-range dependencies between words and to better handle constructs such as verb-object and subject-verb-object relations. We verify this experimentally using two phrase similarity datasets and achieve results comparable to or higher than the previous best results. Our system achieves these results without the need for pre-trained word vectors and using a much smaller training corpus; despite this, for the subject-verb-object dataset our model improves upon the state of the art by as much as $\ssim10\%$ in relative performance.

  • Material
  • Data
    • Learned model parameters used in our paper: download (93MB)
    • Word2vec embeddings (BNC) used in our paper: download (25MB)
  • Code
  • Note