The discussion of gradient descent was excellent. So far I'm quite impressed. As others have said the question is going to be whether you can succeed at building on this base in a way that makes the later topics accessible.
A nitpick is that you use the words "matrices" and "differentiable" at the end of chapter 2. Maybe this is okay because you are signposting that these concepts will be explained but if you are aiming for high school algebra level readers with some python experience this could intimidate people.
He did say 'high school math' and not just algebra. It would be a pretty crap high school math course if it didn't cover linear algebra and beginner-intermediate calculus.
A nitpick is that you use the words "matrices" and "differentiable" at the end of chapter 2. Maybe this is okay because you are signposting that these concepts will be explained but if you are aiming for high school algebra level readers with some python experience this could intimidate people.