> Other animals, even extremely intelligent close evolutionary relatives like bonobos and chimpanzees, treat sequences of words as sequences, not as hierarchies. The same is true for modern artificial intelligences based on deep learning.
That's only true when you think about LSTMs, but stacked CNNs, tree-LSTMs, graph neural net pooling and attention layers can do hierarchical aggregation. Hierarchical representations have been at the centre of many papers. There's even hierarchical reinforcement learning for describing complex actions as composed of simpler actions.
And trees are not good enough to represent language. Graphs would fit better because some leaf nodes in the tree resolve or refer to nodes on other branches (e.g. when you say He referring to the word John present in another place in the same text).
That's only true when you think about LSTMs, but stacked CNNs, tree-LSTMs, graph neural net pooling and attention layers can do hierarchical aggregation. Hierarchical representations have been at the centre of many papers. There's even hierarchical reinforcement learning for describing complex actions as composed of simpler actions.
And trees are not good enough to represent language. Graphs would fit better because some leaf nodes in the tree resolve or refer to nodes on other branches (e.g. when you say He referring to the word John present in another place in the same text).
http://www.arxiv-sanity.com/search?q=hierarchical