You're conflating language with writing. While it is true that Japanese writing is related to Chinese writing, the languages are still from two completely different language families[1]. Anything they have in common is due to exposure, not relatedness.
The argument you're making would be the same as this:
"It's easier for an English-speaker to learn Finnish because it uses the Latin alphabet". The only reason that helps is that you don't also have to learn to decipher a new set of squiggles. It doesn't actually help you understand spoken language or speak it.
1: Mandarin is a Sino-Tibetan language, while Japanese is a Japonic language. These two language families are not related in any ancestral way known to modern linguistics.
I'm well aware that both are from different language families, however that doesn't stop the fact that because of the influence of Chinese writing a significant number of japanese words were directly imported from China. They are the words that corresponds to the on-yomi 音読み of the kanjis.
In the same way, English is a West Germanic language whereas French is a Romance language, this doesn't however stop the significant import of French vocabulary in English which likewise is a result of history and of the conquest of England by William the conqueror. This is a bit different because both English and French are the same branch of a bigger family Indo-European, so those two languages are more similar to each other in term of grammar than Chinese and Japanese but the point about vocabulary I think still illustrates the fact that besides vocabulary inherited by a common language, there can also be vocabulary imported due to historical influences between neighbours. In the case of Japanese it's significant because of their adoption of the writing system and the amount of vocabulary that was imported that way..
Another example of what I'm talking about is Korean. There's some dispute about the language family of Korean with some saying that it's also a japonic language, some saying it's a language isolate. However no one would claim that Korean is a member of the Sino-Tibetan language family. Yet, it's well known that 70-80% of Korean vocabulary is of Chinese origin. This is the same mechanism.
On the other hand, due to both languages being from different language families, the grammar of Japanese and Chinese are totally different and unrelated.
The argument you're making would be the same as this: "It's easier for an English-speaker to learn Finnish because it uses the Latin alphabet". The only reason that helps is that you don't also have to learn to decipher a new set of squiggles. It doesn't actually help you understand spoken language or speak it.
1: Mandarin is a Sino-Tibetan language, while Japanese is a Japonic language. These two language families are not related in any ancestral way known to modern linguistics.