That feels different to me. Grinding scales helps with muscle memory and technique. There’s certainly aspects of that with math, especially with algebraic manipulations. Doing math problems can yield a deeper understanding of the underlying concepts and how they behave. Thinking you understand doesn’t cut it
I’ve always preferred holding a guitar plectrum with my middle finger and thumb instead of the usual index and thumb. I always felt guilty about it since I’ve noticed professionals seem to always use the index finger.
On two occasions I have been asked, 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.
— Charles Babbage
Except that autocorrect is frequently wrong, so that many authors of hilariously wrong messages have to apologize that the messages must have been messed by autocorrect (which may be true or not).
When autocorrect is wrong, it usually is because it chooses words believed to be used more frequently in that context, so especially the authors of scientific or technical texts are affected by the wrong guesses of autocorrect, because they use less common words.
"Right" and "wrong" aren't binary states. In many cases, if the data is at least in small part correct, that small part can be used to improve correctness in an automated way.
Maybe I am failing to comprehend it. But to me the question reads “is your analytical engine, which you've described as a merely a mechanical calculator, also psychic or otherwise magical, so that it may subvert its own mechanical design in order to produce the answer I want instead of what I asked for?”.
I guess the unspoken assumption Babbage makes here is «if I put only the wrong figures into the machine». Then it is completely unreasonable to expect correct output. In ML context an LLM has been trained on much data, some «wrong» and some (hopefully more) «correct», which is why asking something incorrectly can still give you the correct answer.
For ML it goes deeper, but unfortunately discussions about it devolve into an approximation of the Brouwer–Hilbert controversy.
If you think about it from the VC dimensionality lens, in respect to learnability and set shattering is simply a choice function it can help.
Most of us have serious cognitive dissonance with dropping the principal of the excluded middle, as Aristotle and Plato's assumptions are baked into our minds.
You can look at why ZFC asserts that some sets are inconstructable, or through how Type or Category theory differ from classic logic.
But the difference between RE and coRE using left and right in place of true and false seems to work for many.
While we can build on that choice function, significantly improving our abilities to approximate or numerical stability, the limits of that original trinity of laws of thought are still underlying.
The union of RE and coRE is the recursive set, and is where not p implys p and not not p implys p holds.
There is a reason constructivist logic, lambda calculus, and category theory are effectively the same thing.
But for most people it is a challenging path to figure out why.
As single layer perceptrons depend on linearly separable sets, and multilayer perceptrons are not convex, I personally think the constructivist path is the best way to understand the intrinsic limits despite the very real challenges with moving to a mindset that doesn't assume PEM and AC.
There are actually stronger forms of choice in that path, but they simply cannot be assumed.
More trivial examples, even with perfect training data.
An LLM will never be able to tell you unknowable unknowns like 'will it rain tomorrow' or underspecified questions like 'should I driven on the left side of the road'
But it also won't be able to reliably shatter sets for problems that aren't in R with next token prediction, especially with problems that aren't in RE, as even coRE requires 'for any' universal quantification on the right side.
A LLM model will never be total, so the above question applies but isn't sufficient to capture the problem.
While we can arbitrarily assign tokens to natural numbers, that is not unique and is a forgetful functor, which is why compression is considered equivalent to the set shattering I used above for learnability.
The above questions framing with just addition and with an assumption of finite precision is why there is a disconnect for some people.
Can you help me understand the question (and context)?
Life the "machine" is a calculator, and I want to ask 5+5, but I put in the "wrong figures" e.g. (4+4), is the "right answer" 8 or 10? Is the right answer the answer you want to the question you want to ask, or the answer to the question you actually asked?
Imagine it’s not a computer, it’s a piece of paper. And the paper is a bit dirty and you can’t quite tell if it’s a 4 or a 5. You guess it’s 4, but the print-out says 5. Do you pass the exam?
Imagine you ask your friend “hey, what’s twenty divided by five?”, and they say “four” and then you realise you misspoke and meant to say “what’s twenty divided by four?” Is your friend wrong?
Just never clicked with me when I was younger. My younger brother was super into guitar so that was kind of "his thing" and I had other interests. I spent a lot of time playing keys in my late 20s / early 30s, including being in 2 different bands, one which played several shows every weekend.
...but a piano or keyboard is not very portable, and not everything is a good fit for keys. I especially love the portability of the tenor uke.
