Jointly Learning Word Representations and Composition Functions Using Predicate-Argument Structures
- Learned model parameters used in our paper: download (93MB)
- Github: PAS-CLBLM
A better score (ρ = 0.456) than ours (ρ = 0.42) for the SVO dataset (SVO-SVO, averaged) has been reported in the EMNLP 2014 paper ``Evaluating Neural Word Representations in Tensor-Based Compositional Settings'' by Milajevs et al. The Copy object composition seems to be nice =)
A similar model appears in the EMNLP 2014 paper ``A Neural Network Approach to Selectional Preference Acquisition'' by Van de Cruys. Evaluation on tasks of selectional preferences would be interesting.
Our new model focusing on transitive verb disambiguation has been presented in our CVSC 2015 paper.
We introduce a novel compositional language model that works on Predicate-Argument Structures (PASs). Our model jointly learns word representations and their composition functions using bag-of-words and dependency-based contexts. Unlike previous word-sequence-based models, our PAS-based model composes arguments into predicates by using the category information from the PAS. This enables our model to capture long-range dependencies between words and to better handle constructs such as verb-object and subject-verb-object relations. We verify this experimentally using two phrase similarity datasets and achieve results comparable to or higher than the previous best results. Our system achieves these results without the need for pre-trained word vectors and using a much smaller training corpus; despite this, for the subject-verb-object dataset our model improves upon the state of the art by as much as $\ssim10\%$ in relative performance.