"Response times will be higher" sounds very confusing as a way of saying we'll take less time to respond, right? So why should "response times will be lower" mean we'll take more time if the opposite construct is confusing?

Far better to just use the comparative forms that we already have for time specifically to make it perfectly clear.