> curious what sentences you're specifically objecting to
Lines like this: If we're simulating the spread of measles or an outbreak of wildfire, SIR is perfect. The author could have said "this kinda-sorta looks like it might be useful in simulating wildfire but I haven't checked", but of course that would be less convincing and less exciting.
Which is fine as far as it goes. The problem is what happens when someone takes one of these models seriously without actually checking the details, or without being qualified to check them.
I think the author does a pretty good job of highlighting that there are imperfections and shortcomings while eliding over those details. They may have slipped up in some places such as your quote but I'm definitely not concerned of anyone taking any of this too seriously and extrapolating on it dangerously.
> I'm definitely not concerned of anyone taking any of this too seriously and extrapolating on it dangerously.
Not this particular article, no. But there's a whole genre of excited mathematical modelling literature where the author demonstrates a gee-whizz concept that looks like it could be really useful. The trouble is that once you start digging down into the specific details, they turn out to be really hard to get right, and at best you end up with a model that's brittle, for want of a better word.
An example that I have in mind is the literature on power law distributions. A little bit of theory showed how power law distrbutions could arise via a process known as preferential attachment, and everyone got excited and suddenly people were spotting them everywhere. The literature on this topic is massive.
The thing is, it turns out that it's quite hard to check that a given dataset follows a power law. This paper [0] showed that many of the claims were sloppy, and the researchers hadn't been careful with their statistics.
The crux of what I'm saying is that establishing that a model fits well is hard, whether it's a diffusion model, a power law distribution, or anything else. If someone wants to claim that some mathematical widget can be used to model X, they'd better be able to back that claim up with a real demonstration and carefully laid-out details. Otherwise they're just waving their hands in the air.
What happens when someone uses a hair dryer in the bathtub?
As someone who spent a fair number of hours teaching mathematics to engineering students, exciting is what you want. What use is it to state the complex hypotheses of a theorem if the student cannot feel the result coming?
> The problem is what happens when someone takes one of these models seriously without actually checking the details, or without being qualified to check them.
That's a problem with absolutely any kind of statement you can put on-line. There's only so much an author can do to prevent stupid from misreading them; the responsibility is not all on author here.
> The author could have said "this kinda-sorta looks like it might be useful in simulating wildfire but I haven't checked"
Why? Maybe they actually checked. I did, and keeping in mind that I'm not a medical scientist, it does seem that SIR model with modifications is widely used to study disease spread, and it also does look like a good first-order approximation for measles. It also does look like a good first-order approximation of wildfires.
I actually did a thesis on using cellular automata for simulating indoor fires and used some of the same ideas presented here. SIR is essentially what I'd get if I ignored the details of heat convection, conduction and radiation - which at the scale of forest fire is something you can do (unlike my work, where modelling these details was kind of the entire point).
> SIR is essentially what I'd get if I ignored the details of heat convection, conduction and radiation - which at the scale of forest fire is something you can do
How do you know? To validate this sort of model surely you need to know an awful lot about statistical modelling and about forest fires.
The main sort of insight this model would give you is that setting up clear spaces like roads through a forest would hinder the spread of fire. If you took the model literally, you might end up ignoring the fact that sparks can be carried by the wind to areas that are far away from the trees that are currently burning.
> it does seem that SIR model with modifications is widely used to study disease spread
Fair - there's a whole wikipedia page [0] about this sort of model. But like I'm saying, that page is big on theory and light on evidence. Those models are full of parameters like transmission rate that are not typically known until after the fact.
The world is full of theorists writing down academic models that are, frankly, a little useless. What I would find more convincing is a writeup of how a big practitioner health organisation like the CDC or the WHO used one of these models to gain a new insight that they couldn't have found any other way.
> How do you know? To validate this sort of model surely you need to know an awful lot about statistical modelling and about forest fires.
First-order approximation and all. You validate according to your needs, but yes, you'd also need to know a lot about statistical modelling and the domain. I'm guessing the author just read that SIR can be used for some diseases and forest fires, and that what they read was written by people who do know that. FWIW, our CA models were validated by our supervisor (who had access to firefighters) to roughly the level of "reproduces what happens on recordings of real fire" - which was good enough to show that the model has a potential to give real-time insights, but not something I'd like an actual firefighter to use on the scene.
> If you took the model literally, you might end up ignoring the fact that sparks can be carried by the wind to areas that are far away from the trees that are currently burning.
That's a very good point. However, like with all models, you need to be aware of the limitations. Maybe the author should have guarded the text for this, but I doubt the government epidemiologists and firefighters were the target audience here. I expect these professionals to understand the models in greater depth before using them (though maybe I'm hoping for too much - given how our industry is full of "professionals" mindlessly copy-pasting code from SO).
> that page is big on theory and light on evidence
Fair. I was actually going to link to a Wikipedia page in my previous comment, but I noticed a surprising lack of citations for the claims made. So instead I went and looked around Google Scholar before saying that "it does seem that SIR model with modifications is widely used to study disease spread".
> Those models are full of parameters like transmission rate that are not typically known until after the fact.
Sure. Again, I hope that no epidemiologist uses the article to develop disease spread models. But it works as giving general overview and intuition about network models, with focus on the phenomenon of criticality.
> What I would find more convincing is a writeup of how a big practitioner health organisation like the CDC or the WHO used one of these models to gain a new insight that they couldn't have found any other way.
I would absolutely love to see that.
BTW. I hope we're not just arguing about whether the word "perfect" was used in the article correctly.
> BTW. I hope we're not just arguing about whether the word "perfect" was used in the article correctly.
I guess I'm trying to make a point about theory vs. practice. I think theorists tend to be pretty cavalier about writing down models and waving away the fine details, whereas practitioners tend to appreciate how hard it is to get the details right.
As an example, the literature on finance is full of this stuff. There's a whole body of literature on how to create an optimal stock portfolio under various constraints assuming you know the joint distribution of individual stock returns. It turns out that fitting that distribution in a sensible way is extremely difficult to do. The theorists came up with a `clever' model that's mostly useless in practice, but everyone still insists that it has `applications' in finance.
Personally I find theory really interesting, and beautiful in its own right. It just annoys me when the usefulness of that theory gets overstated.
Lines like this: If we're simulating the spread of measles or an outbreak of wildfire, SIR is perfect. The author could have said "this kinda-sorta looks like it might be useful in simulating wildfire but I haven't checked", but of course that would be less convincing and less exciting.
Which is fine as far as it goes. The problem is what happens when someone takes one of these models seriously without actually checking the details, or without being qualified to check them.