The astrologer says "you are unable to father children". It is ambiguous whether the astrologer is implying that the man cannot conceive children or whether he has no ability to be a father. When the man reveals that he has seven kids, the astrologer's reply of "look after them well" could be interpreted as sarcasm. I think initially you're supposed to think that the man cannot have kids, but the joke is that he's implying he's a terrible father.

My $0.02.