Three things would have to be recorded in that case: the "you" part of the start & stop times, as well as the times themselves.
I'll take your word for it that he's a decent person. I certainly have no reason to think otherwise. But that doesn't change that his service is recording information that it ought not to be.
The way you typically record this is start time, end time, and IP assigned. It's still not enough to identify what sites or traffic was visited, but when you get a spam email forwarded to you by your upstream provider, it's enough to identify a customer.
I'll take your word for it that he's a decent person. I certainly have no reason to think otherwise. But that doesn't change that his service is recording information that it ought not to be.