“most tasks can be formulated this way, including autocomplete tasks”

For autocomplete tasks, with a corpus of unlabeled documents, would you insert a separator token at an arbitrary space in each document, in order to form input/output pairs?