Missing from "after" is something that contributed to my career success – when you encounter problem, and you will, many, when you (small bugs etc) or your team (larger issues) solves it - stop and play it from the beginning in your head. Go through every step that you remember that you or collegues went. Pay attention to what you should have done instead, what you should have tried, now that you know where the problem was. What were the obvious things to check. What was waste of time. What kind of binary search you should have done. This alone can boost your performance above average quickly. And the weird part is that almost nobody does it; just do it and you'll see.
Great advice! It is similar to something I have been doing for close to 20 years now: whenever I come across a really tricky bug of some kind (usually one that got all the way into production without being caught), I write down some notes on it. What the problem was, what we did to find it, and what we did to fix it. Most inportant is the "Lessons" part (a few sentences) - what should be done to avoid these kinds of problems in the future.
I've done the same thing, but for more than just bugs. Pretty much any time I learn something while working that I think might be easy to overlook in the future, I write it down.
Some of my findings end up in internal training if it's something the team needs to know. If it's something for me, I'll just keep it in my own notes for the next time I encounter it until it's burned into my memory.
It is mostly for my own reference. However, almost every time I come across a bug like this, I will be discussing it with people in the team. It's often hard to keep quite about it: "You wont believe what just happened..." kind of a thing.
Agree with this - I often write little Gists mostly for myself but every once in a while others will find them useful and some have even ended up being published publicly, eg this one about mistakes to do with RDS
I found that even the concept of doing the "binary search" when debugging is an acquired skill. I often see people facing a vague (in a sense that it could be anywhere e.g. among multiple components or services) issue either think by analogy to previously seen issues without verifying; or offer various hypotheses that are random and unlikely/unwarranted (from "what if we just restart it" to "maybe RAM is defective and is corrupting the data"); or jump on a random correlate that may be spurious or actually a side-effect of the issue itself, without verifying it; or just get stumped. Just forcing yourself to think "how can I narrow it down [whether it's to one of many distributed services, or to 20 lines of code out of 200]?" / "what is the next piece of information I need?" greatly improves your debugging/live site skills.
Excellent advice. That trains your mind to avoid getting stuck and get un-stuck in the future.
When I get stuck on a problem, I'll imagine how I will soon be thinking back on what happened, after I solved the problem, hitting my fist on my forehead and exclaiming "why didn't I think of that?" It jogs me out of the rut and makes me remember other times I had to think "out of the box" to get around a problem. Often it's something I wouldn't normally think of when approaching the problem directly from the front, like something in the environment, or a broken tool, or an incorrect assumption, or a misunderstanding. So I imaging myself past the problem, looking back in the rear-view mirror at how I solved it with the 20-20 vision of hindsight. Simply reminding myself of other times I've gotten out of a rut in the past dissolves my present frustration and helps me get out of the current rut, even if the causes and effects are totally different.
