Ok one specific example. I'd say the 10x part isn't about productivity but insight into the problem. Most people solve the problem as given, but the times when I've been 10x I managed to change the definition of the problem to something simpler, and easier to solve.
Here's an example: At one company they had a 'financial calculator,' i.e. a program to help visualise how some financial math played out over time and it was giving incorrect results. When I looked at the code I saw that it had been written by composing a lot of those functions you see on actual financial calculators, (1/V, PV, PMT, etc...).
This may have seemed simpler to approach it this way, but because there were so many compositions and there was a root finding algorithm involved, it quickly got out of hand. I started from the problem definition, defined a recurrence relation for what we are trying to calculate, found a closed form formula for the recurrence relation. This part was the hard part.
Now those 3000 lines of FV1P(XYZ(ABC())) turned into like 20 lines of simple math, _and_ the formula could now be solved for the other variables, so no more root finding. I just solved for the other variables, wrote functions for them and that was it. It ended up being faster, correct, numerically more stable, and the code was smaller.
I wrote up a white-paper on how the math worked, included a proof and that was it. It took two weeks.
The value to the customer was that this bug in this one part of their 'suite' of applications was holding up a sale to major prospective customer of their's. So fixing this had an outsized valued compared to the severity of the bug.
so my bag at the moment is software is a form of literacy. And that implies that (for now) some people can read and write and some cannot.
Most business / software problems revolve around the problem of getting a person who understands the problem to explain it and work with the person who can write software.
Your example is fascinating- and suggests that it works better when one person understands both parts :-)
Another example I can give is when a client had some process represented by a graph, with boxes and arrows, but the structure of it was really ad hoc. It was _mostly_ a DAG except with some extra back edges they'd added to solve specific problems.
I suggested modifications to their structure, their algorithms and their DB storage for this thing so that it _did_ become a DAG, being careful to explain how to keep their functionality working.
After that the graph could be topologically sorted, and the upside was that instead of loading the whole graph at once as they were doing, we could load a single topological 'level' at a time.
Added an index for topological level in the DB and we were able to make it efficient.
They did have to do some work on the calling side, because they used to load it all at once, and now they had to load it in pieces, but we gained an order of magnitude or more in load time, so it was deemed worth the effort.
I am fascinated by the psychology of this as much as anything.
Any organisation is a meta-brain, and has emotion running throughout.
Your two examples seem to come after the wheels have fallen off and the magnitude of the problem is evident to all. And they call in a consultant who might see a DAG or (more often I suspect) might see opportunities to sell 120 off shore developers and a rewrite using IBM/Java
But anyway, if you had walked in a year previously with exactly the same plan would you have gotten any traction? A maths heavy white paper explaining why your ideas are the same as theirs but better would have met enormous emotional headwinds.
Maybe our scientific revolution over the past few hundred years is a blip. That does worry me. If better solutions are avoided because of emotional reactions, it costs us all.
Sorry this is starting a lot of thoughts and I should cover them more carefully. thanks for the insight and congratulations- soujds like a fun career !
> if you had walked in a year previously with exactly the same plan would you have gotten any traction?
Nope, most of the time it wouldn't and my experience bears that out ("Let's not boil the ocean...").
Thinking simple, and avoiding premature optimisation, are so often the right thing that we can forget that they don't always work.
Teams and people often prefer to learn the hard way. Mind you _I_ haven't built any companies or startups, and my up approach probably would lend itself to do that either.
There's an adage, by Fred Brooks: "Plan to throw one [implementation] away." I come in when they are ready to throw one away, but aren't sure what to replace it with yet.
As far as it being fun, sort of. First of all I have a more 'normal' job now, second of all it's a rush, and an ego boost, but ultimately exhausting.
Here's an example: At one company they had a 'financial calculator,' i.e. a program to help visualise how some financial math played out over time and it was giving incorrect results. When I looked at the code I saw that it had been written by composing a lot of those functions you see on actual financial calculators, (1/V, PV, PMT, etc...).
This may have seemed simpler to approach it this way, but because there were so many compositions and there was a root finding algorithm involved, it quickly got out of hand. I started from the problem definition, defined a recurrence relation for what we are trying to calculate, found a closed form formula for the recurrence relation. This part was the hard part.
Now those 3000 lines of FV1P(XYZ(ABC())) turned into like 20 lines of simple math, _and_ the formula could now be solved for the other variables, so no more root finding. I just solved for the other variables, wrote functions for them and that was it. It ended up being faster, correct, numerically more stable, and the code was smaller.
I wrote up a white-paper on how the math worked, included a proof and that was it. It took two weeks.
The value to the customer was that this bug in this one part of their 'suite' of applications was holding up a sale to major prospective customer of their's. So fixing this had an outsized valued compared to the severity of the bug.