AI researchers are certainly fully aware of such issues, and there have been various models trying to encode long term dependency information, e.g. LSTM.