Are you familiar with the old Polaroid instant film? [2] You took a picture and pulled it out of the camera and it developed over the next few minutes. The picture went from dark grey nothingness to the image you took, like "magic".
Anyhow, so-called "neural" networks (they are no more neural than Kermit the frog is amphibious) are basically just giant polynomials with changeable coefficients. [3] The input variables are e.g. pixels from an image, and the polynomial is solved to generate a single output value indicating that it is e.g. a cat. (But before backprop this will be uncorrelated, random, useless.)
Now this image-to-yes/no-cat function is very difficult for a human programmer to write out by hand. I mean it's so difficult that no one can do it very well at all.
But using backprogagation, and a big pile of images (some of which are cats and some of which are not) and an index of which are cats for the computer to use (humans will have had to already sort and tag the images for this to work, TANSTAAFL), you can run a simple loop to develop a "function" (in the form of coefficients for the polynomial) that can "see" cats. The "training" phase adjusts the variable coefficients until the result correlates with the tagged "catness" of the images, and then you're done. After that, you can input new images to the function and it will be able to indicate whether they depict cats or not.
The function forms out of the data+backprop like an image developing on the film, projected there by the data set.
So yeah, this AI/ML stuff lets us generate useful functions that we have no other way of generating (literally no one knows how.) It's a big deal.
Backpropagation. [1]
Are you familiar with the old Polaroid instant film? [2] You took a picture and pulled it out of the camera and it developed over the next few minutes. The picture went from dark grey nothingness to the image you took, like "magic".
Anyhow, so-called "neural" networks (they are no more neural than Kermit the frog is amphibious) are basically just giant polynomials with changeable coefficients. [3] The input variables are e.g. pixels from an image, and the polynomial is solved to generate a single output value indicating that it is e.g. a cat. (But before backprop this will be uncorrelated, random, useless.)
Now this image-to-yes/no-cat function is very difficult for a human programmer to write out by hand. I mean it's so difficult that no one can do it very well at all.
But using backprogagation, and a big pile of images (some of which are cats and some of which are not) and an index of which are cats for the computer to use (humans will have had to already sort and tag the images for this to work, TANSTAAFL), you can run a simple loop to develop a "function" (in the form of coefficients for the polynomial) that can "see" cats. The "training" phase adjusts the variable coefficients until the result correlates with the tagged "catness" of the images, and then you're done. After that, you can input new images to the function and it will be able to indicate whether they depict cats or not.
The function forms out of the data+backprop like an image developing on the film, projected there by the data set.
So yeah, this AI/ML stuff lets us generate useful functions that we have no other way of generating (literally no one knows how.) It's a big deal.
[1] https://en.wikipedia.org/wiki/Backpropagation
[2] https://en.wikipedia.org/wiki/Instant_film
[3] https://en.wikipedia.org/wiki/Polynomial