Most chat servers simply know who contacts whom. This is an inherent problem (or strategic advantage) in being a chat server. Signal tries to hide the information from themselves by using Intel SGX. So, while SGX may not be perfect, its better than nothing.
On top of that, even if they do run SGX, whoever hosts the SGX instances can just let the NSA into the server room and nobody would be the wiser. It's not like SGX actually works. It actually makes things worse if it allows signal admins to look the other way in good conscience.