Copilot cannot "solve" these problems, it just knows the answer.
And for that exact reason, using LC as a hiring tool runs into the same problem as using standardized tests in colleges: Once people figure out that the tested metric isn't actually "understanding of the subject matter" but "test knowledge", and learning for the test is easier than understanding the subject, people start learning for the test.
Showing one way to get to the solution doesn't mean that it's the only way to get to the solution. It's still possible to solve these problems without the training examples that Copilot has memorized.
Copilot cannot "solve" these problems, it just knows the answer.
And for that exact reason, using LC as a hiring tool runs into the same problem as using standardized tests in colleges: Once people figure out that the tested metric isn't actually "understanding of the subject matter" but "test knowledge", and learning for the test is easier than understanding the subject, people start learning for the test.