The traditional household arrangement is the man is at the office 24/7, the wife gets his paychecks and uses them to raise the children and occasionally go on vacations, and the man gets an allowance (but doesn't go on the vacations because he has to be at the office). Not much room for a harem there.
There are many rich people in Japan who can afford to have multiple wives and support all of them financially. Some of them might do that in secret with a mistress but in the current social and political environment they probably wouldn't have children with their mistresses.
It's traditionally pretty normal for a married couple to drift apart over the years. People marry for appearances, social status, and to raise a family. They then seek gratification elsewhere.
The men who engage in this aren't looking to knock up more women. They already have a mother to their children. They want gratifying sex and emotional connection.
I've seen some posts by Japanese women suggesting they would be open to being the second wives of wealthy men but I'm not sure if they are the kind of women that rich men would be interested in.
The traditional household arrangement is the man is at the office 24/7, the wife gets his paychecks and uses them to raise the children and occasionally go on vacations, and the man gets an allowance (but doesn't go on the vacations because he has to be at the office). Not much room for a harem there.