Conseil national de recherches du Canada. Technologies de l'information et des communications
32nd International Conference on Machine Learning, July 6-11, 2015, Lille, France
artificial intelligence; brain; computational linguistics; semantics; speech recognition; speech transmission; trees (mathematics); composition layers; long distance interactions; long short term memory; machine translations; natural language understanding; recursive modeling; recursive structure; semantic composition; learning systems
The chain-structured long short-term memory (LSTM) has showed to be effective in a wide range of problems such as speech recognition and machine translation. In this paper, we propose to extend it to tree structures, in which a memory cell can reflect the history memories of multiple child cells or multiple descendant cells in a recursive process. We call the model S-LSTM, which provides a principled way of considering long-distance interaction over hierarchies, e.g., language or image parse structures. We leverage the models for semantic composition to understand the meaning of text, a fundamental problem in natural language understanding, and show that it outperforms a state-of-the-art recursive model by replacing its composition layers with the S-LSTM memory blocks. We also show that utilizing the given structures is helpful in achieving a performance better than that without considering the structures.
Date de publication
International Machine Learning Society
32nd International Conference on Machine Learning(ICML 2015)2 : 1604–1612.