There's a huge difference between what works at the professional level and what works at the collegiate level and what works at the high school level. A lot of it comes down to the difference in abilities between kickers and punters between those levels.
In high school some teams will always go for a 2 point conversion after a touchdown. Not because they will make it a large percentage of the time but because they will be lucky if their kicker will make the extra point 20 percent of the time.
In high school and college it makes sense to go for it on fourth down more often because most teams don't have punters that can consistently kick the ball 50 yards or down it inside the 10.
Professional kickers can consistently make a field goal from 40 yards, some have really good percentages from 50. An onside kick that fails almost gives the other team 3 free points in the pros. An if the receiving team is expecting it the percentage of the time it works be small. In high school, where you almost have to be at extra point distance to even think of trying a field goal, I could see it being useful.
Main point: The coaches at each level are, for the most part, acting rationally. The occasional deviation from the typical strategy is like a bluff in cards. You need to do it occasionally so that they can't gang up on your typical strategy.
By the end of it, he has a model that demonstrates when teams should punt, when they should kick a field goal, and when they should go for it. Spoiler: They should go for it a lot more often.
For example, if you have 4th & 4 (or less) and are anywhere from the 5 yard line to the 50, you should go for it.
He even has a calculator so you can now add incorrect statistical decision making to your list of reasons to scream at the TV on Sundays: http://wp.advancednflstats.com/4thdncalc1.php
You only go for it if you have better than average odds of getting it. Its not like random draws from a known distribution. Any included data on 4th down is going to be subject to some variation of survivorship bias. See, for example, also the consideration:
I used only data from the 1st and 3rd quarters to exclude situations hurried by an expiring clock and by desperate teams or teams with large leads playing differently late in games.
So, by this logic the 4th down data is similarly non-standard, no? It would clearly be a situation different than normal 1-3 downs in 1-3 quarters.
4th down data is not non-standard for 4th down data. Sure, you can't say for sure what would have happened if they had gone for it, but there's nothing particularly different about these downs that would lead to a higher conversion rate.
I think the point is that if you are using game data then the only times the offense is currently going for it on fourth down is because they have some reason to believe they can make it with a high percentage. Football is not played with random number generators, the coaches and players understand the matchups and the chances they have of making X yards with the players and plays they have available. The teams that didn't go for it might have a reason to believe that with the players on the field the probability of them getting X yards was small.
Except those numbers are optimized for winning while head coaches have other motivations, most importantly keeping their job. While there will be a lot of overlap in the two (e.g. winning means not being fired) a coach introduces a lot more risk of losing their job the further they deviate from the common wisdom. So even if the expected points suggest going for it, it might still be in the coaches best interest to kick. It is basically the opposite of moral hazard.
If the numbers truly support it, then NFL coaches, of all people, should adopt it. They are supposed to have balls and care more than anything about winning.
If the facts truly support it, I expect we will soon see NFL coaches adopt it.
An NFL coach will never risk his job by trying something so bold and untested. If he tried it for just one game, and it failed, he could lose his job. New, creative schemes always come up from minor leagues like high school and college before gaining general acceptance in the pros.
The only way it happens is if the directive or permission comes down from the owner[1]: "I will allow you to never punt for two seasons or until we have an adequate sample size to judge this method."
[1] Thinking about all the fun things I could do with billions of dollars, like buy an NFL team and run experiments like this, make me question the sanity of people who turn down $3 billion to continue working on their photo sharing app.
Burke's piece is excellent, in as a far as it goes -- however, expected value is not the only thing one should concern themselves with. Higher moments matter. In particular, in a given situation, when ahead a coach might choose to trade negative expected value for a reduced in variance, and when trailing do trade negative expected value for increased variance. (It looks like his calculator may account for this in some way through win probabilities, but I didn't see an explanation on the site.)
He mentions at the end of the piece that his EP-based analysis only applies in "typical" close games in normal weather when the clock is not yet a major factor. And that an analysis based on Win Probability could correct for most of these factors.
I'm skeptical of that last point. There are many, many variables that underlie football outcomes. Unlike baseball, where the most critical two players by far in any situation (the pitcher and the batter) are known in advance by both teams, a football play has the potential to be affected dramatically by any combination of players on the field. I am sympathetic to the idea that more teams should play aggressively on offense more often (and indeed I believe there are signs of this in today's NFL), but by no means should every team be expected to act according to the punt/FG/go-for-it chart in most situations.
I think you're absolutely right to emphasize variance tradeoffs. When coaches try to reduce variance they are often criticized as "risk-averse", but that's often a winning strategy, even in games that are close enough score-wise to be considered "typical".
This is the sort of broken thinking that prevents NFL coaches from trying this.
AFAIK the main reason NFL coaches don't try stuff that's too unusual is that although it makes a good trick a time or two, this is... well, professional football. Defenses will adjust.
Right now I'd suspect a lot of the success of going for it on 4th down, or going for two-point conversions, comes from the fact that opposing defenses aren't expecting it. But the minute you become known as "the coach who always goes for it", well, they're going to defend it (and in fact they'll have an easier time defending it since they know you're not going to try anything else).
So they would approach is like they already do for downs 1 through 3? It's not a trick, it's using all four downs to your advantage, vs wasting a down giving the ball away.
It is an unusual deviation from the normal sequence of play-calling, and the advantage is that much of the time opposing defenses aren't prepared for it. They have plays set up for when they need to make a stop -- particularly at the goal line -- but they're not used to using them in this situation and it is, I suspect, almost entirely the novelty of being forced to run the "must stop this one play" defense that makes it a successful tactic.
Which in turn implies that if people start doing it all the time, defenses will simply adjust, become accustomed to making a dozen or so fourth-down stops per game, and that will be the end of this "Opposing Defensive Coordinators HATE him -- learn this one weird trick" story.
Similarly, in basketball, the pick and roll is pretty much a flaw in the game. Teams should just pick and roll every single time, but they don't, because coaches get bored too.
He even has a calculator so you can now add incorrect statistical decision making
His methodology is suspect. The entire premise (there have been similar reductionist statistical studies that holistically look at it as pure positional numbers) is founded on the notion of the weighted point value of each 1st down position. But as others have mentioned, on-fire offenses against miserable-defenses (e.g. New Orleans taking on Dallas) are going to absolutely dominate such a ranking, because they're of course going to dominate first downs (the survivorship bias -- the ones who suck go three and out), and they're more likely to be scoring points, completely throwing off such a weighting.
If you're dominating a game but have a couple of missed passes, coaches often will go for fourth downs. If you're having little success, on the other hand, why give up good field position, and instead try to just push them back and hold them off until you can regroup. If Minnesota is 3rd and 4 at their own 35, they quite rightly -- and with full statistical backing -- aren't going to go for it if it isn't a desperate last minute drive.
We of course have actual stats on the probability of making a fourth down-
Teams have a less than 50% chance of making it on average, though of course specific teams and specific opponents will completely throw off that count, and such is what weighs on the minds of coaches making these decisions, having to do so outside of "if we were an entirely average team against an entirely average team..."
Teams have a less than 50% chance of making it on average
Ultimately, that's not what matters. Expected value is what matters.
Take the example from the video of a team on its own 5 yard line with 7 yards to go:
If they go for it and don't make it, they leave the other team with the ball on the 5 yard line, which they will convert into a touchdown 92% of the time, an expected value of -6.44 points. If they punt, the TD conversion goes down to 77%, an expected value of -5.39 points. Therefore, the break even is about 16% 4th down conversion rate. If a team can make it more than 1 in 6 times, they should go for it instead of punting. Since most NFL teams are in the 40-50% range, they should be going for it more often.
> Take the example from the video of a team on its own 5 yard line with 7 yards to go:
> If they go for it and don't make it, they leave the other team with the ball on the 5 yard line
Actually, if they go for it and don't make it, they either leave the other team with the ball somewhere between the goal line and just short of the 12, or they end up giving up a safety (which has both score and possession effects.)
Failing to convert on fourth down doesn't mean that you hand the ball over where you were before fourth down in the general case (only in the cases of an incomplete pass or zero-yard run, which aren't the only cases of failing to make a first down.)
Yes, but ... this discounts the practical consideration that the defense is going to change situationally, too. 4th and 2 is going to look a lot different than 4th and 17.
It's true, but I didn't have the data. Regardless, it decreases if they convert, which might double the break even point, but it still wouldn't be unreasonable.
I think the reason why these stories are so popular is that, from a fan's perspective, coaches rarely challenge orthodoxy head-on. Instead new schemes develop incrementally over time and are composed with other changes to systems and variables such as play calling, clock management, team speed and so on. When you are a coach that has 10-16 times a year to test your theories with your current team configuration, it makes a lot of sense not to rock the boat too hard.
So these stories kind of poke at the armchair-coach in all of us that are fans of the game. We've seen the Air Raid work since Leach and Tech, so why not the the NFL? It's fun to think about.
That said, Chip Kelly might be on to something in Philly.
A field goal from 50 yards is from the 33 yard line (33 + 10 + 7). So, I think it is extremely easy to see (without even crunching the numbers), that you should hardly ever punt if it is 4th and 1 yard and you are between the 50 and the opponent's 33.
Of course this depends on the strength of your defense and the strength of your opposing offense. Most punts will only gain you 10-30 yards from there (touchback to the 20). If your opponent has a high producing offense those 10-30 yards are not worth very much in comparison to the chance at 3 or 7 points.
In high school some teams will always go for a 2 point conversion after a touchdown. Not because they will make it a large percentage of the time but because they will be lucky if their kicker will make the extra point 20 percent of the time.
Indeed, when my son played football in Jr. High the kick was worth 2 points because so few kids could actually do it, and the run or pass play conversion was only worth 1 extra point.
This is just about the most absolutely bizarre thing in the world for me to see my High School football coach on the front page of Hacker News. I attended Pulaski Academy (PA) 6th-10th grade and played football there from 7th-10th grade, 1 under Coach Kelley who was then the Defensive Coach while I was a cornerback (and tailback).
As coaches go, Coach Kelley was smart, far smarter than any coach I had at Plano West, one of the premier school districts for football in Texas. He pushed us hard, had a great sense of humor, and was out to win. PA was also the best school I have ever attended (I also grew up in Rancho Santa Margarita in Orange County, CA).
I guess what I'm trying to say is that while this is completely bizarre to find it on a Silicon Valley tech site like Hacker News, I honestly can't say I'm all that shocked. Sometimes when you meet people, you just understand and know that there's something different about them, a potential for greatness, that they are a mold breaker. Their current circumstances may be unexceptional, but you have confidence they will go on to do great things. I think this is of course what we (HN) share culturally with a football coach from Little Rock, AR: a desire to think outside the box, to break the mold, and achieve what little greatness we believe we are capable of achieving.
I owe a great deal of my professional success to Coach Kelley and Pulaski Academy, so seeing them realize this level of success by being different, smarter, fills me with pride and warms my heart.
My dad used to coach high school and college football way back in the 60's through the 80's. At one stop, there was a rival catholic high school that almost never kicked. No punts, always going for two on extra points and no field goals unless it was the end of the half or the end of the game. Their players and coaches were cocky about it, and beat my dad's teams more often than not, so my dad never liked them.
Thinking about it further, though, their style of play broke the other teams' game plans. In general, if a defense can hold the other team to 3 yards or less they win. (Because 3 yards times 3 downs equals 9 yards and a new set of downs is 10 yards.) By calling a play on 4th down, they lowered their standard of success to 2.5 yards per play. In startup terms, they extended their runway by 33%.
I wonder how it would work out if teams actually structured their offenses around gaining only 2.5 yards per down; not averaging 2.5, but actually gaining 2.5?
Of course, they'd have to go deeper now and again just to keep the offense on its toes, but apart from that 2.5 is the goal. Everything else is a bonus.
If successful, they would also control the clock and keep the other team's defense on the field.
Yeah, well, and quick outs, slants, short passes underneath, etc.
Actually, now that I think of it, maybe the playbook wouldn't have to change too dramatically. You may run the ball a bit more, but you'd still run pass plays that spread the field out. If the defense just gives you the deep play, you take it. But, you're content to go with the first play that nets you 2.5, so you make that the goal and always have a couple of short options. The QB then prioritizes those in his progression.
It's not just football where the strategies popular at the moment are probably sub-optimal.
Consider ice hockey. Most coaches, if down by a goal, will pull the goalie for an extra attacker with about 90 seconds left in the game. It turns out that pulling the goalie significantly earlier, with about 3 minutes left in the game, would be more likely to result in a win:
However, coaches are understandably reluctant to pull the goalie so early. When they're down by a goal near the end of the game, they're probably going to lose regardless. The public accepts that. However, if they lose anyway after employing an unorthodox strategy, the critics will be vicious, much as described in the football article.
There's a similar thing going on with penalty shots. A penalty which when committed also deprives a player of a scoring opportunity can result in the non-offending team having the choice of a penalty shot or having the offending player serve the penalty. Teams will frequently choose the penalty shot, but statistically it's often a better choice to take the power play opportunity.
This really reminds me of the book Moneyball. What I found most interesting about that book and this sort of stats based management phenomenon is that it flies in the face of human nature and thus, people who are considered experts tend to reject such ideas.
What often happens is that people reject the ideas until they start winning or showing an obvious competitive advantage, then they embrace the ideas and they become the new standard way of doing things and people will parrot that as the one true way to do things as if it were always so.
I'm from a country that has zero football and soccer is the king of everything. Some time ago I started watching football and really enjoyed both the physical tactical aspects of it. Been very busy though (had kids!), and never got to really study and learn more about the game (read a for dummies book and that was it - still have last year's superbowl game on media player to be watched). What would be a good, practical way (book/video/documentary) to learn about the game? Any suggestions? Thanks in advance!
I really like rugby union but saying it is similar to football is really not true.
Many of the things that are bemoaned about gridiron football are what most of us have come to love. The play stoppage, the start/stop nature, specialization of players etc. are what make gridiron football the most strategic of ball games. Continuous play games like rugby and soccer are much more tactical less strategic. Neither is better or worse, but I enjoy the strategy of gridiron & enjoy the tactics of rugby (& aussie rules for that matter).
Getting rid of those elements of gridiron football would be a bit like letting football (soccer) players use their hands. Sure it might make for an interesting new game, but it would destroy what we love about the old.
Except the commercials. God the NFL has to get a handle on that.
The play stops a lot less often and for a lot less time, and there are no ad breaks aside from at half-time. So you get a whole lot more action per hour.
The 15 players stay on the field for the whole game, although 5 (about) can be subbed out (and can't come back).
There are four ways to restart: kick-off, Scrum (big men push), line out (thrown in from the side), penalty, (kick for goal, kick for touch or chase or tap and go), and free kick (same as penalty but no shot for goal).
The players run a lot - it's an aerobic and power game combined, which also means there is a role for players of all shapes and sizes.
The cost of playing is a ball - no pads required. It's also a lot safer as there are a lot less head injuries, as the protective gear in American Football just makes players feel safe, while increasing the impacts to the head..
The coach does not dictate plays - the players decide what to do when on the ground. It's a lot more fluid.
You can see the players' faces and bodies as they play, rather than just the coach.
It's played between countries at the top level. We have a world cup that's a genuine world cup.
> It's also a lot safer as there are a lot less head injuries...
There's a bit of debate about that point, there isn't enough data on concussions in rugby as the professional era of massive rugby players isn't as established as the NFL. Great article: http://www.bbc.co.uk/sport/0/rugby-union/24765650
For newcomers interested in rugby union, there's a fantastic subreddit of dedicated fans, really friendly to newcomers too: http://www.reddit.com/r/rugbyunion
Check out the sidebar in that subreddit, it's excellent.
Rugby to Watch
Highlights of the Rugby Championship between South africa v All Blacks (one of the best games in recent memory - it's got power, speed, skill and massive hits in abundance) - http://www.youtube.com/watch?v=f2B0EviQDB0
Keep reading Grantland. They have great weekly columns, particularly for the NFL and NBA, that have broadened my understanding for and appreciation of the sports I thought I knew. Honestly, they're the best.
You could also try fantasy sports, but they don't teach you much about the game - they just encourage the sort of singleminded obsession that makes learning any subject easier.
I was in a similar position - couldn't even watch football games on TV (there were none where I lived). However I learned the rules by playing a video game (Madden, if I remember correctly). Became a big Green Bay Packers fan, too, just by playing the game :)
Definitely playing video games and fantasy (both already mentioned). Games will teach you about the language and forms of the offensive and defensive schemes as well as the basic rules (why a penalty is awarded and what its repercussions are) and fantasy will keep you up to date with the the current players, which is nice because players rise, peak and fall very quickly in the NFL (for very sad reasons, but while it's here and on TV, I'll be watching it).
I recommend that everyone, not just new fans, read 2 books: Keep Your Eyes Off the Ball by Pat Kirwan & The Essential Smart Football by Chris Brown. Both changed the way I watch the sport for the better.
"A year ago at the Hall of Fame reception in Canton, Ohio I found myself sitting between Bill Walsh and Don Shula. I posed this question: In a day when the Bears line up five-wide and Texas Tech passes 60 times a game, are there any fundamental innovations that have not been tried? Walsh supposed someone might try using trick formations for an entire game. Shula twinkled his eyes and said: 'Someday there will be a coach who doesn't punt.'"
One glaring miss in the statistical-only approach is that the psychology of a defense on 4th down and how that can impact outcomes. If you just use average net yards-play to extrapolate expected results for 4th down, or 3rd downs as per the berkely paper, thats not going to work out. Defenses will most certainly strategy to take more risks (and decrease the median expected yards gained in return for a small risk in a huge play), crowd noise can increase, etc.
The berkeley paper completely writes this off without evidence or reason, even with his assumption that offenses will not become more conservative with play calling, and its obvious a defense and crowd would act differently on 3rd and 3 than 4th and 3:
"Since it seems unlikely that the defense has substantially more scope than the offense to affect the distribution of outcomes"
Primarily, the goal on defense is to get off the field, so other than special situations like goal-line or the edge of field goal range, its ok to give up 3 yards and even 4 is fine. You really just need one great play out of 3 - a blocked or missed pass, tackle for loss, etc. and they're 3-and-out.
The paper admits the entire basis and math would be wrong if his assumption was misplaced:
"Thus using third downs to gauge what would happen on fourth downs would lead to overestimates of the value of going for it."
On 4th down and 1-3 yards, everything changes. You try to play every play as hard as possible, but you bring something special when its 'us or them, right here, right now'. Thats partially why (the other reason is there is less room to pass) that you see teams march down the field and can't get the last yard 4 tries in a row. I know the offense also steps up, but in a short running play against relatively even teams, defensive players are usually shorter (leverage) and have an extra player (unless the offense does wildcat with a mobile quarterback).
The other missing piece is they didn't seem to look at time left in the half and timeouts left. Punting in the first quarter is not the same as punting with 37 seconds and the other team is out of timeouts.
I am from a Little Rock, AR (went to PA's rival school), and when I saw the headline, I thought "oh, I know of a coach back home that does this." Little did I know he'd be brought up in an article on HN!
That being said, I'd be interested to see various modern machine learning and statistical techniques applied to football (as they have become applied to baseball).
This strategy works so well for the same reason that the no-punt strategy works so well: because the opponents are kids.
In high-end college basketball, or the NBA, full-time full court press can become a recipe for easy fast break points. Inbounding teams can pass faster than defenders can run, and smart, disciplined players can memorize and run dozens of full-court plays.
At minimum you need a deep bench to consider it, because it is exhausting. So it's perfect for a kids team, where getting all the kids into the game is a virtue in itself.
The Bulls teams of the 90's used the full court press to destroy teams in the third quarter. I agree that it would cause too much wear and tear to run it all the time.
I found it a strange article due to "Hey, this tactic masked the lack of skills of a team with low passion. Let's use it for our core economic theory!".
Reminds me of a story I read about a girls basketball coach who was not really familiar with basketball but I think he had coached soccer. He didn't understand why you would only defend half the floor, so he trained his players to run the floor like soccer players so that they could defend the entire floor all the time. Pissed off the opposing teams something fierce, but he did win a lot of games.
"There are those kinds of people that are different, becuase they want to be different. And there are those kinds of people that are different because they want to be successful" -Coach Kelley, from the video embedded in the link.
I think that is a very valuable lesson for us to take into startups, because so often we just want to "disrupt" things, but we don't have solid reasons for why disrupting something will lead us to being successful.
I have seen this story on E:60 before and actually think about it quite a bit in regards to the NFL. I think the reality is that most coaches would get roasted in the media if they tried and failed and playing the media is a big part of the NFL. ESPN is always looking to cause a distraction and then ask all of their commentators whether or not the 'distraction" is a distraction. But you do see coaches going for it when the benefit out ways the risk or they are feeling lucky. And I do think with the more mobile quarterbacks get the more you will see teams going for it based on the ability to do more.
This last Sunday, both Detroit and Chicago went for it on 4th down back to back drives. It was like one coach was challenging the other and I thought that was pretty cool. Ultimately if you look at these punters today though, most of them can put the ball on a dime with special spin to make it go where they want. Field position is tested strategy that works.
One major flaw to this article doesn't point out is that you may have a numbers advantage, as the coach and author suggests as the reason to why this works. Or, you may have a practice advantage against your opponent.
Practice advantage means that very rarely do teams practice their onside kick coverage or their 4th down defense on the opponents 12 yard line, because it just never happens. So the week before you play this team, your coaches will over index on the amount you practice onside kicks and 4th down coverage, meaning there will be a lot less time to practice for their actual offense and defense.
Chip Kelley mentioned several times during his coaching at Oregon that the reason he lined up for two points after every initial touchdown wasn't as much for the possible extra point, but so that every opposing team would have to devote precious (very precious in the NCAA) practice time to learning how to properly line up against it.
As a fan of rugby, I've often wondered why football coaches don't try some other radically different strategies. For example, why not use a lateral pass more? Usually football fans explain it by saying it's too risky. But in Rugby they do it all the time, it's a skill that can be practiced and be almost guaranteed of success if done correctly.
Possession is more important in football, I agree. But I'm saying I suspect the risk of a lateral pass can be much lower than generally accepted because you can almost guarantee a pass completion by doing it right. In rugby, in open play, it's also very risky to loose the ball when attempting a lateral pass. The opponents can then gain possession and attack while your defensive line is not in place properly. So dropping the lateral pass is also very risky. But rugby solves that risk by making sure lateral passes are dropped very seldom (watch a top level game and see. Around 5 turnovers per team are because of a lateral pass that's dropped, if that much).
It's interesting to hear there's someone doing this. Nate Silver came to a similar conclusion at his Google Talk[1]. He proposed this as the answer to the question "What is the most statistically unsound tactic in sports?"
My feeling is that the automatic punt while you're up has also become a sportsmanship/etiquette thing now when your team is ahead.
If you're up by 2 touchdowns and you go for it on 4th, I can't think of a single announcer in the NFL who wouldn't make it seem like you were rubbing dirt in your opponent's face by doing so.
I disagree. Especially when you're trying to drain the clock with a lead, going for it on fourth-and-short is a completely orthodox strategy. I've never seen an announcer criticize a team for applying it: Far more often, I've heard some variation of "it's the defense's job to stop you, not your job to stop for the defense."
"Teams’ actual choices are dramatically more conservative than those recommended by the dynamic-programming analysis. On the 1,604 fourth downs in the sample for which the analysis implies that teams are on average better off kicking, they went for it only nine times. But on the 1,068 fourth downs for which the analysis implies that teams are on average better off going for it, they kicked 959 times."
To anyone who has been through the analysis/math: are fake punts considered "going for it"? I assume they are.
The following is possible: the surprise element of a fake punt makes going for it on 4th down more advantageous than it otherwise would be. It could make
P(good outcome | going for it on 4th down) > P(good outcome | punting) if and only if the defense expects a punt, which they no longer would if a team adopts the strategy of routinely going for it on 4th down.
I wonder how much of this is total commitment. If there is no punt, the offense is really forced to make it happen or put the defense in a terrible spot - does that change their mindset to be more gritty, more concentrated - more "these 10 yards has to happen"?
As a CFL fan, I'm wondering how these approaches would work up in Canada. It does seem that most CFL teams go on 3rd and one (easier since the defence has to give up a yard from the ball unlike US football) now, even in their own end.
Never would of thought I would see something from the Grantland channel on HN. Yet here we are. For those more into sports, especially BBall. Listen to Jalen Rose's podcast. Real insightful information and fun stories to hear.
High school play obviously has strategies that aren't appropriate for higher stakes or at a higher level of play: There are a lot of silly plays that high schools do that you'll never see in the professional leagues, aside from pro bowl level frivolity.
It's worth noting that this isn't about "going for it" on the 4th down, but instead about trying onside kicks. I mention that because one of the academic papers linked (by David Romner) is specifically about going for it on 4th down, and it doesn't say not to punt, but rather that on the average there are are many situations where teams punt when odds favor going for it on the 4th down (e.g. 4th and inches at your own 30 yard line. Aside from the last minutes of the 4th quarter down 6 points, the vast majority of teams would punt it).
In high school some teams will always go for a 2 point conversion after a touchdown. Not because they will make it a large percentage of the time but because they will be lucky if their kicker will make the extra point 20 percent of the time.
In high school and college it makes sense to go for it on fourth down more often because most teams don't have punters that can consistently kick the ball 50 yards or down it inside the 10.
Professional kickers can consistently make a field goal from 40 yards, some have really good percentages from 50. An onside kick that fails almost gives the other team 3 free points in the pros. An if the receiving team is expecting it the percentage of the time it works be small. In high school, where you almost have to be at extra point distance to even think of trying a field goal, I could see it being useful.
Main point: The coaches at each level are, for the most part, acting rationally. The occasional deviation from the typical strategy is like a bluff in cards. You need to do it occasionally so that they can't gang up on your typical strategy.