> I use this method often for prediction applications. First, it’s a sort of hyper parameter selection, so you should obviously use a holdout and test set to help you make a good choice.
What the article is talking about is inference, not prediction. It's a different problem domain, it's not about telling a company whether design A or B leads to more engagement, it's about finding out about the (true!) causal drivers of that difference. The distinction may seem subtle but it's important. The key problems outlined all talk about common (frequentist) statistical tests and how they get messed up by variable selection. Holdout sets don't address this, because if the holdout set comes from the same distribution as the test set (as it should), the biases would be the same there. Bayesian inference isn't a panacea either, the core problem is structuring the model based on the data and then drawing conclusions about their relationships (Bayesian analysis gives you tools to help avoid this, but comes with its own set of traps to fall into, such as the difficulty of finding truly non-informative priors).
> Can you point to the parts of the document, or other resources about Mr. Beast,
> that warrant a comparison with tobacco companies?
The part where the GP says "Lot of people critiquing this, but you can't deny the success." invites counterexamples of companies that are successful but still deserving of critique.
They make money in ways that others would find morally reprehensible. The tobacco industry makes its money off of addictive substances that kill millions per year, and Donaldson makes content for entertainment that literally tortures people in the sense of being a violation of the geneva convention. In both cases they're highly efficient operations that make a lot of money, but whether or not you would call it a success story depends on your definition of success. If your definition of successful is "makes money", then the tobacco industry, Donaldson, fentanyl dealers, etc are indeed successful. If your definition is "the world is a better place for its existence", then not so much.
Regarding sources: if you're genuinely interested and not just being argumentative for argument's sake, you're capable of googling "MrBeast geneva convention" and following the sources from there.
>and Donaldson makes content for entertainment that literally tortures people in the sense of being a violation of the geneva convention
What specific acts are we talking about? "violation of the geneva convention" could mean literally anything between "putting red cross symbols on soldiers" and "summarily executing civilians", so it doesn't really narrow things down. If they're being put in uncomfortable positions, but they're not risking long term harm and it's voluntary, I don't see what the issue is.
I think of it as "Could Newton have used this to find the expressions for the forces he was analyzing (eg gravitational force = g m_1 m_2 / d^2)?". I once asked a physics prof whether that was conceivable in principle, and he said yes. It seems to me like KANs should be able to find expressions like these given experimental data. If that was true, then I don't see how that wouldn't deserve being called interpretability.
> It seems to me like KANs should be able to find expressions like these given experimental data.
Perhaps, but this is not something unique to KANs: any symbolic regression method can (at least in theory) find such simple expressions. Here is an example of such type of work (using non-KAN neural networks): https://www.science.org/doi/10.1126/sciadv.aay2631
Rephrasing: just because you can reach simple expressions with symbolic regression methods based on neural networks (or KANs) does not necessarily imply that neural networks (or KANs) are inherently interpretable (particularly once you start stacking multiple layers).
Just giving the force law hardly counts as interpret-ability. You probably know that the 1/r^2 in the force law comes from the dimensionality of space. That is the interpretation.
It seems like you're asking for quite a lot here. Are there any examples of common interpretable ML methods that would give you anything like that answer? The most common methods that are called interpretable would give you hints like "Mass and distance matter for gravity, color doesn't" or "Gravity gets stronger with mass and weaker with distance". Both are clearly less informative than the formula.
The only way I could think of to get anywhere near such an answer would be to use symbolic regression first and then ask an LLM to interpret the result. And that would probably take quite some more original research to get it anywhere near working, and even then probably primarily for problems where the answer is already known.
I agree that this kind of answer would be useful, but we also have to be honest that that's not what currently meant by interpretability. And that's what should matter for evaluating the claim - it's not misleading if it delivers what one can reasonably expect. Whether we should update our interpretability definitions is a different (interesting) discussion.
> Just giving the force law hardly counts as interpret-ability. You probably know that the 1/r^2 in the force law comes from the dimensionality of space. That is the interpretation.
I used to think the same, but don't the weak and strong forces decay differently?
another consideration: if a popular AI model hallucinates an endpoint for your API for one customer, chances are another customer will run into the same situation
Parkinson's law (humorously, but based on serious research) states that 'work expands so as to fill the available time' [0]. This just seems like a generalization to expenses and budgets.
If there was a nuclear war, you might have to do all your work from an underground bunker. Is that a reason to do all your work from an underground bunker now?
Unless you live in a city a bunker really isn't necessary. Just tape trash bags over any blown out windows so rain/dust doesn't get in and don't go outside for a couple weeks.
Fyi, it's actually easier for trains to be on time the longer the journey is - that's because sane travel times aren't calculated at max speeds. So if there is any delay, you can go at a higher speed than what was used in the calculation to catch up. On shorter journeys there are simply fewer opportunities to catch up.
Now, I wouldn't be exactly shocked if DB using too high speed assumptions in their stated travel times was part of their problem with delays.
AlphaZero is arguably like those algorithms on steroids, probably worth a mention there. Even though the goal is a bit different, finding Nash equilibria for games like Go is probably still infeasible, but you also don't need that to beat humans.
> My main problem with them is that they all turn into leaches when they are successful
And now imagine how bad that problem would be if they knew that the only way you could walk away would be by giving up your TLD, to which very likely at least some marketing materials - potentially the whole identity of your company - is tied.
What the article is talking about is inference, not prediction. It's a different problem domain, it's not about telling a company whether design A or B leads to more engagement, it's about finding out about the (true!) causal drivers of that difference. The distinction may seem subtle but it's important. The key problems outlined all talk about common (frequentist) statistical tests and how they get messed up by variable selection. Holdout sets don't address this, because if the holdout set comes from the same distribution as the test set (as it should), the biases would be the same there. Bayesian inference isn't a panacea either, the core problem is structuring the model based on the data and then drawing conclusions about their relationships (Bayesian analysis gives you tools to help avoid this, but comes with its own set of traps to fall into, such as the difficulty of finding truly non-informative priors).
reply