The point of the thought experiment is to ask if Mary would interpret the experience of seeing the specific color with her own eyes as a new experience. If yes, and given the premise that she knows everything that it is possible to know about the physics of color and the human brain, the conclusion is supposed to be that an experience is not a physical fact of the world.
I have written another comment on why I think this conclusion is wrong.
> The conclusion is supposed to be that an experience is not a physical fact of the world.
I see what you're getting at and mostly agree (hence the Upvote!). But...
Whether a 1st person qualitative experience is a "physical fact of the world" depends on your level of description. In a sense, it's not an epistemically objective physical fact that I can share with others. However, it is just a brute ontological physical fact of the world that my mind experiences the color red (even if yours doesn't).
This is precisely where we need to clarify "facts of the world". We can't settle this chat until we introduce the notion of subjective vs objective, and epistemic vs ontological.
Note that I personally believe the argument is deeply flawed, and rests on some assumptions about the human mind that may turn out to be false; and that shouldn't be generalized to other potential minds (say, AGI) even if they are true of human minds.
I was just explaining what I believe to be the core of the argument.
I have written another comment on why I think this conclusion is wrong.