The problem is, the less fat mass you have, the less good you will feel restricting calories. Dieting down to 15-20% range is very different than dieting down to 8 - 12% range (for males). The additional fat mass is like having a constant drip feed of fuel. Some other factors matter too, like leptin.