I try to do this whenever I am the one talking through the results of the interview with a candidate but only if the candidate seems open to it. It seems like a valuable thing to do even from the company's perspective. If you are considering doing this: It isn't always received positively so you have to be prepared for that. You and your company will be judged in this conversation and it's likely that the candidate will not agree with you. Also, if the decision is final (for now or forever) make that very clear right away so no one wastes their time arguing their case. It seems best to approach it with the mindset that no interview process is perfect and your conclusion is based on your side of the exchange you had (which may just be a few hours with the person); it's possible your company's assessment is wrong for some reason but you've made your best effort to make the correct decision. It's easy to come off as arrogant when you're the side doing the rejecting.
I have successfully offered feedback and later hired a candidate (more than once!) when they returned after taking on a new project that helped them grow in a way that addressed the feedback. Our company has also changed its collective mind about some people when we have had success with another candidate or found a particular project or situation that minimizes the risk of whatever we were concerned about (for near-hires). An interview that results in a rejection isn't necessarily a permanent dead end.