Take a piece of assembly code generated by a compiler and try to optimize it. You'll see that you don't actually need to be that smart to obtain a significant perf improvement.
Compilers are good, but they must ensure correctness for any source code. On the other hand, you now exactly what you need thus you can drastically simplify / optimize the assembly code.
As someone who writes assembly, I think this is overstating the case. There are a lot of algorithms (particularly short ones) where good C compilers generate nearly optimal code that you would be hard-pressed to improve on.
In my experience the benefit you get from writing assembly comes largely from your ability to do better register allocation for your fast-paths, in cases where your compiler would spill registers to the stack.
Compilers are good, but they must ensure correctness for any source code. On the other hand, you now exactly what you need thus you can drastically simplify / optimize the assembly code.