Looking into the full paper, it looks like the task identified inside the frontier is much more broken down, with 18 numbered steps. The outside the frontier task is two bullet points.
So for clearly scoped tasks, ChatGPT is hands down good. For less well scoped work it reduces quality.
Interestingly, the authors also looked at the amount of text retained from ChatGPT, and found that people who received ChatGPT training ended up retaining a larger portion of the ChatGPT output in their final answers and performed better in the quality assessment.
Looking into the full paper, it looks like the task identified inside the frontier is much more broken down, with 18 numbered steps. The outside the frontier task is two bullet points.
So for clearly scoped tasks, ChatGPT is hands down good. For less well scoped work it reduces quality.
Interestingly, the authors also looked at the amount of text retained from ChatGPT, and found that people who received ChatGPT training ended up retaining a larger portion of the ChatGPT output in their final answers and performed better in the quality assessment.