You can be "technical" without a need for a concept of technical debt.

- Indie video game designer/developers, for example, live in a world where everything they write is a one-off product that will ship, at most, a v1.0.4 six years after v1.0.0. To them, code is a medium for crafting an experience, and whether the code is of quality is immaterial as long as it creates the correct experience.

- Data scientists (and bioinformatics pepole, and a few other similar groups) live in a world where everything they write is effectively a script-as-experimental-apparatus. If the script is producing output conforming to the proposed experimental method, then it's done. They run it, then discard it. It's not usually even under version control; only recently have such scripts been considered worth including for purposes of peer review and replication.

And there are other examples. I'm not saying that these kind of people are "software engineers"—engineering of any kind is never even a consideration for them; whether the product is well-engineered has no impact on their livelihoods.

But these people frequently are lumped into the category of "programmers", and that tends to confuse this sort of discussion a lot. The "software industry" doesn't just contain engineers doing engineering (well or poorly); it also contains prototypers and illusionists and tinkers and scientists, all working to their own criteria of art, none of whom will have any reason to pick up a single shred of engineering knowledge, even with decades spent in their own industries. And then these people apply for "programming jobs", and we wonder why they don't know what git is or how to properly modularize a codebase for maintainability.

It's a communication problem, I think. We need to stop calling everything we're doing "programming." Maybe we need an entirely new verb for doing software engineering, of the kind you do when you're writing something like car firmware, or Amazon S3, or the Googlebot. "Programming" is about as precise in describing that as "doing math" is in describing the invention of the lambda calculus.

I've toyed with the idea of having separate titles & job descriptions for "Software Features Engineer" (those responsible for pumping out user-visible features) and "Software Infrastructure Engineer" (those responsible for the long-term health of the code). In my experience, they really are completely different roles, down to the languages used (features engineers tend to use Python, Ruby, JS, and other dynamic languages, while infrastructure engineers tend to use C++ or Java), attitudes, personality types (infrastructure engineers tend to be much more attentive to detail), priorities, and points in the lifespan of the product when they are most useful. There's already precedent for creating a new job title to solve an organizational problem, eg. Site Reliability Engineer.

Many big companies have such a split in their org chart, with separate "infrastructure" and "features" teams reporting up to different managers, but IMHO that's the wrong split. Very frequently you want a cross-functional team with both infrastructure & features engineers on it, but they remain different roles, just like you might embed a UX designer and UX researcher on a product team even though their job descriptions are very different.

I have seen this taken further into the personality types:

Starter - someone who can solve design problems and pump out new features Journeyman - someone who can be given a spec and turn out solid, maintainable code. This kind of developer is also good a debugging and resolving edge cases. Finisher - someone who is really good at completing and polishing features/applications

A single developer is likely to be a mixture of Starter/Journeyman or Journeyman/Finisher. It is rare to find someone who is strong in all three traits.

I think I am more of a Starter (too many design ideas) with some Journeyman. I don't think I am so strong as a Finisher (too many other distracting design ideas).

Sir nostrademons,

"Software Features Engineer" vs "Software Infrastructure Engineer" is an excellent differenciation. Actually I'm an SFE and I have always felt that I am not a real programmer. SFE will always write more-shiny less-durable features, more often working under an unreasonable management. I feel like I'm tainting the reputation of real SIEs.

Question is: I've always assumed I had to evolve into an SIE or die (or become a Product Manager if I'm lucky to be better than 90% of my colleagues at understanding users). Is it a correct picture of the reality, or is it possible to strive as a "good" SFE?

No, but it will take you down a career path that exposes you to more luck & uncertainty than an SIE.

Features engineers can have lucrative, long-lived careers. It usually means they need to be the primary or lead engineer responsible for a significant user-visible product. Think of Andy Hertzfeld for the Macintosh, John Carmack for Doom/Quake, Anders Hejlsberg for Turbo Pascal and Delphi, or Aston Motes at Dropbox. Some of them cashed out directly with F-U money from a startup; others used past track records to get a startup of their own funded or a lucrative job at a big company.

The downside is that you're very exposed to the risk of your project failing and nobody caring about it. At the stage of a project where a SIE is most useful, the project is already underway, people know it's a good idea, and they just need to execute solidly. At the stage where a SFE is useful, the whole point of the project is unknown - that's why you need an engineer that specializes in rapidly testing things out. This applies in big companies as well - my role at Google could best be described as a SFE (though they don't have the title), and I had to prototype a number of features, a couple of which worked out and a good many of which did not.

There's very significant crossover between the SFE role and that of a technical cofounder - the main differences are risk tolerance, willingness to go out and interact with users personally, and product vision. It's very common for people who enjoy the prototyping aspect to end up starting their own companies, as that gives the best alignment of incentives and ability to capitalize on their own work. It's still possible to be a "professional early-employee" or even make a prototyping/R&D role work in a big company, but you need to demonstrate wins in your career.

Indeed, I have created my own company ;) It feels like a failure compared to the steady money that my SIE ex-coworkers have, but I enjoy it much more. Your insights give me good context to visualize my career, thank you.

I've always liked the way the games industry approaches this: a "game designer" (at least in healthy companies) is a developer whose job is effectively to produce a continuous stream of prototypes that encode all the business logic. You can use the prototypes for smoke testing (in the games industry, seeing whether the resulting game is fun), but can also use them to write component-level contracts based on the observed properties of the prototype.

Once you have such an "empirically-derived" contract, you can then throw that over to a larger team of actual capital-E-Engineers, whose job is to make something obeying those same observed constraints (which are really quite detailed; they basically get to do TDD where many of the tests have been written for them), then apply actual engineering to the problem, to output another, non-prototype product.

This scales up: your stream of prototypes can come from a team, instead of an individual; and you can have as many people doing engineering as you want, because they have a stable set of requirements—translating a fixed, interactive spec (the prototype) into good code—that allows for simple, static top-down software architecture.

The interesting thing is that successive prototypes will tend to be frequently ground-up rewrites (because prototype code is unmaintainable, and is also small enough that there isn't much lost in throwing it out.) The engineered product, meanwhile, will be a continuously-evolved codebase, where each new prototype marks the start of a new branch of development, new user stories and issue tickets, etc. Features might not always make it from an arbitrary successor prototype into the product!

Note also that the game designer not only encodes the requirements in their prototype, but is also the perfect place to lay the responsibility of collecting the requirements in the first place. Effectively, they're acting as a tiny embedded start-up, pivoting around and trying to find product-market fit, where the "market" is internal to the company.

Or, another way to think about it: your "features" team is like a four-piece garage band producing music (or even, perhaps, one guy with a keyboard producing chiptunes.) Those songs are played to internal stakeholders, who decide whether they like them. Then, the approved songs get thrown over to an orchestra conductor (the given team's engineering lead), whose job is to turn the motifs embedded in the piece into a symphony, and get their 100-odd performers to create a coherent, grandiose sound out of it.

A lack of understanding how to set up this kind of pipeline might be why (in the best cases, at least—let's ignore the acquihire-and-disassemble cases) so many large companies choose to continually buy start-ups. Effectively, they're doing the same thing we're talking about here: taking a small team with a finished prototype, and then reimplementing it "for real" using their engineering talent. The problem being that there's no support for the continued existence of the absorbed "start-up" mitochondrion within the host cell, so it only ever gets out the one product it was working on at the time it was acquired, rather than continuing to pump out new little ideas. (A great game-industry example: EA. They constantly buy fledgling game publishers, use their own engineering talent to produce a sequel or two, and then run out of steam because the people making up the creative engine that generated the original game designs have left, or have been reformed by their engineering culture into something non-startuppy that has no power to generate original products.)

great comment

I'm also of the school that different words or terms should have different tangible meanings.

So for example a programmer is a distinct concept from a software engineer. One can be one without the other, or both. I like to think I started as merely a programmer but... now can also say I'm a software engineer.

Because I do try to think about the full spectrum, the trade-offs, the real world concerns, as much as I think about how to see a certain computation or result occur, in isolation.

What I would call a developer (or duh-veloper, to be less diplomatic) thinks no further than how to see a certain computation or result occur, only, and only as per written reqs, then, moves on into the proverbial sunset. Even if it doesn't handle edge cases, or unwritten desires, or doesn't scale, or is hard to upgrade or support, etc. A true software engineer should consider and design for all those considerations. Not simply how to achieve 1 + 1.

I'm upfront and start the conversation with, "You employed me to give you an honest opinion. I think we are going in the wrong direction, we need more tests and some refactoring. This will cost us money now, but save money later." and I always get the reply that sure, they totally understand, but we're going to do a quick and dirty hack today and sort it all out next week...

I used to just refactor and test as I went along, but then you end up ruffling feathers and being slower that others. This is then used as ammo against you. So you lose by doing the right thing.

I think pouring energy into other things (open source) is more productive.