No, unconditionally safe. The C standard exactly defines unsigned overflow while specifically leaving signed overflow undefined.
(Pedantically timing is always a crapshoot in C, a compiler only need produce the same results as the abstract machine. It could freely take all your secret data and modulate it into the timing and be conforming. -- but considering that intel/amd won't make timing promises about the instructions themselves...)
for (int i = 0; i < 32; i++) // safe branch
diff_bits |= x[i] ^ y[i];
then no, it doesn't leak, because the result of the resulting conditional branch doesn't depend on a secret. The only reason NaCl unrolls that loop is because neeed moar speeed.
If you were talking about the early return straightforward for loop:
for (int i = 0; i < 32; i++) // safe branch
if (x[i] != y[i]) // timing leak...
return -1; // ...magnified
Then yeah, it leaks.
> Longer term worry is the optimizer will figure the above out as well.
It may, but even compiler implementers realise the value of constant time code. Replacing this code with an early return doesn't just require very sophisticated optimisations, it never happens outside of a crypto library. There is little incentive for compiler writers to do this.
Problem is the optimizer is totally free to implement the 'safe' code snippet using an 'unsafe' early return. According to the standard that would be completely legal.
A compiler that did this optimization would immediately introduce a pragma guaranteed to result in constant time memcmp from some blessed source pattern.
Maintainers of crypto libraries inspect the assembly when upgrading their compilers, test with many compilers and document the versions of compilers they support.