I think that one safe exception is upcasting of integer types (e.g. i8 to i32) and I think it's a little awkward to program in Rust without it.

It seems also that using the "as" syntax for both upward and downward casts makes code a little harder to read. Either allowing implicit upcasting or having separate syntaxes for safe vs. unsafe numeric casts would make more sense to me.