Also just to note there is nothing to stop you from using a rotating port knock key, particularly if you are willing to assume the client's clock is reasonably accurate.
Yes, but not nearly as impossible as it should be. The amount of precision to pull of a timing attack is difficult, but the amount of computing power to refactor 4098bit RSA keys requires computers that don't exist [yet].
Nothing stopping you from HMACing the request IP+time with whatever crypto function you like and sending it in a series of encoded port knocks.
The only issue I've run into with port knocking is places that heavily restrict outbound ports/protocols. Though technically that is solvable too I just haven't bothered.
The only things in clear text are your currently IP, the time, and the AES signature. The attacker doing a packet capture already knows your IP and probably owns a watch but it's unlikely they know how to break AES to sign a modified message as you.