I assume that it's the same for "the other side".

There are others who are truly at odds with reality (in fact that's more common), but they don't have the same vibe.

The short answer is, you can't. At least not about anything that you can't test for yourself by personal experience.

The longer answer is to ask why you care about whatever particular aspect of "reality" you are thinking about testing. What difference would it make to what you choose to do, if reality were one way rather than another? If the answer is that it would make no difference, then the correct thing to do is to not care how reality is in that respect. Have no opinion at all. (There's an old engineer's joke about running tests. Before you run a test, ask yourself two questions: What will I do if the test passes? What will I do if the test fails? If both answers are the same, don't bother doing the test.)

The problem is that our minds did not evolve to be comfortable with having no opinion about something. Our minds evolved to seek answers, not to leave questions hanging. That probably made sense in the hunter-gatherer environment in which we originally evolved, but it doesn't make sense in our world now. There are simply too many questions, too much information, and too little time for anyone to check it all, and there are no "trusted" sources of information we can turn to to just tell us the answers. But it's very difficult to accept having unanswered questions; our minds keep sending us alarm signals even though we might have convinced ourselves intellectually that we should leave those questions unanswered. So many people end up accepting some answer even if it's wrong, and even if the answer makes no difference to anything they actually do.

For about two weeks I was convinced my new roommate was a hallucination and I'd finally snapped. There were a number of uncanny cases where he knew the same incredibly obscure trivia and references that I knew. Things like "trap remixes of musicians that sold their soul". That's too specific to write off as just similar interests by men in a similar demographic. There were also a number of uncanny instances of not knowing the same exact things I didn't know. I had never seen him outside of the apartment. We had no mutual friends. We were both in the same unusual living arrangement, permanent temporaries at an Airbnb partially bartering code / home automation work as barter for rent. At some point the topic of eye color came up, and someone pointed out how all of us in the room have green eyes, and how that is the least common eye color and 3 of us is very statistically unlikely. Eventually I had to ask myself which is more probable, there really are two people with all these characteristics in common that ended up at the same place by coincidence, or I'm having a schizophrenic break and this is my delusion? My possibly imaginary friend had mentioned that he lived in Russia through the first grade and speaks Russian at a first grade level. I do not speak Russian. So I ask him to teach me some Russian grammar. He agrees but then changes the topic. I ask him to teach me some Russian. He says sure but avoids the question again. I ask him to teach me a bit of Russian. He agrees and evades again. At this point I am having some very serious doubts about my grip on reality.

Eventually we do meet other people from each other's circles. After a while there's been enough mutual third parties acknowledging both of us, that either a very large cast of characters are my delusion or this man is real. I can never actually prove one way or the other, but the scales are now tipped towards real by all the people standing on them.

And so we get to the answer to your question. "Reality is shared consensus." There's a shared consensus that my roommate was real, and that consensus may as well be reality because I can't distinguish it from the case where everyone has the same delusion.

Of course reality isn't the shared consensus per se. Reality is not subject to a referendum. However, shared consensus is the multimeter we use to read reality. Where is the difference between a 9 volt battery and "if I touch the proved to the terminal it reads 9 volts"? There isn't one.

In your example, my definition of "reality is shared consensus" becomes a big problem. If there are two groups with their own consensus then those are two realities. You are free to believe anything and everything is true right up to the point of fatally erroneous belief. For things where the consequences of being wrong are not so sure, there is nothing forcing a consensus around "objective reality".

Niels Bohr allegedly had a horse shoe in his office. When asked he said it was for good luck. One audacious visitor asked "do you really believe that?" He replies "no, but they say it works even if you don't believe in it." This was cheeky of Bohr. Plenty of people don't believe in quantum mechanics, but they say it works anyway. If someone truly insists on believing magic horse shoe theory and rejecting quantum mechanics, there is nothing that will force them to acquiesce to our objectively correct answer.