The most onerous parts of the AI Act, such as the provision that says "Member States shall confer on their market surveillance authorities the powers of... carrying out unannounced remote or on-site inspections", are not yet in force yet.
I wouldn't blame them for having security/IP concerns with allowing unannounced remote or on-site inspections to their facilities.