Mick West wrote a partly insightful and partly humorous article about how programmers develop back in January, with seven stages from Awestruck through Transcendent. I particularly liked this bit at the end.

Not all programmer take the same path, and not all aspects of the programmer will proceed at the same rate, or even direction. With this unfortunate vagueness comes the observation that you can only recognize a stage when you are two stages above that stage. Thus, the transcendent programmer will never truly know he is transcendent (although he will suspect it) and must always be in doubt as to the pragmatism of his solutions, and the pragmatic programmer will be forever conflicted regarding his usage of methodology.

That’s sort of true; you never know how dumb you used to be (and quite likely seemed to others) until you’re a little bit less dumb, and that phenomenon is not limited to programming. What I’d like to pick up on, though, is the “not all aspects” part. Just about every programmer has at some time had to explain to a friend or relative that there are a lot of specialties within computing. Just because I’ve written a cluster filesystem for Solaris doesn’t mean I can help you with your printing problem on Windows. All of programming is only one specialty within computing, which also includes hardware design and testing from chips up to systems, manufacturing, system administration, technical management, etc. each with their own challenges. Even within programming there’s a tremendous amount of fragmentation between platforms and problem domains and languages. Thus, while Mick is mostly talking about overall development, I’m thinking more about development within a particular technical area. More after the break.

Here are the stages I see of a competent programmer within one specialty coming to grips with another.

0. Ignorance
In this stage, the programmer doesn’t even realize the new area exists as a specialty. For example, a lot of people who work on highly user-centered kinds of programming such as games or web sites have only a dim appreciation of how many layers of system libraries and operating systems and boot loaders and so on have to be working before they can even begin their work. Even system programmers often fail to appreciate how many layers of programming are involved at the “other end of the wire” within a router or disk array. That’s just hardware, right? Wrong.
1. Arrogance
OK, so there’s code to do this other thing, but that code can’t be that hard, right? After all, that specialty where I just spent several years becoming an expert is the Hard Stuff, all these other bits are just afterthoughts. Wrong again. Yes, some kinds of programming are more difficult than others, and certain specialties require greater than novice-level skills just to get started. Any kernel programmer can learn basic GUI development in a week, but a novice GUI developer will take longer than that just to get something trivial to run in the kernel without crashing immediately. Nonetheless, at the journeyman to expert levels, every specialty has its own kinds of complexity. A basic GUI is one thing, but an industrial-strength GUI that scales up to thousands of components and interfaces with several external APIs is quite another. In this stage programmers just don’t even know there are hard problems to solve and established methods for solving them, so they just try to do everything ad hoc or by clinging to whatever paradigms they learned in their last specialty. Inevitably, they spend a lot more time debugging than developing.
2. Rigidity
At this stage, the programmer has fallen into enough pitfalls that they’ve become wary. They’ve found a few patterns that seem to work, and they’re extremely distrustful of any deviation. Often this comes across as being stuck trying to recreate the code they wrote at their last job. This is the only way to write a SCSI/FC driver. An Ethernet driver must be written like that. You must always do this, and never do that or else much pain will ensue. There’s only one solution to each problem, and each new project is largely a matter of redefining the problem (e.g. by contorting every other part of an architecture that involves multiple specialties) to fit that solution.
3. Flexibility
By now, the programmer has realized that, while some solutions to a problem just don’t work at all, there are multiple solutions that do. Some will be better, some will be worse, either generally or in a specific situation. The trick now is not to make the problem match the solution, but to figure out which solution or variant or combination best fits today’s problems and can be implemented with a minimum of pain.
4. Innovation
Finally, the programmer understands this kind of programming and its associated tradeoffs well enough to create solutions that are genuinely new, or to crack problems that were previously avoided as too difficult.

Of course, even within a specialty nobody fits neatly into one level. Everyone at stage 4 still spends significant time merely adapting old solutions, everyone at stage 3 still has some design rules they regard as inviolable, and so on. Nonetheless, I think it’s good to arrange things as a hierarchy like this because it can be useful sometimes to step back and think about where you are on hierarchy with respect to the problem du jour. Is the problem in front of me one that I can approach at level 4, or would I just be fooling myself? Might I be better off just accepting that I’m “only” a 3, and not trying to be fancy? Or should I even recognize that I’m only a 1, and get help?

A lot of this progression has to do with, especially fear of changing code. At levels 0 and 1 the programmer doesn’t even know enough to be afraid. At 2 he’s ruled by fear. Once something’s working, he will tirelessly defend it against change – which he views as disruption – even when changing needs or environments require it. The worst territorial behavior always involves a 2, and that’s where the greatest number of people get stuck forever. Most 2s deny that there’s anything above 2 in their field, and anyone who claims otherwise is derided as a 1 who’s deluding themselves. By 3, some of those bad attitudes have dissipated. Changes to running code are no longer rejected out of hand, but are still viewed with a mostly-healthy skepticism. At 4, work almost becomes play. Anyone fortunate enough to find an area where they can reach that level – not all do, and few get that far in more than one or two areas – knows the joy of tinkering with something familiar, confident that any breakage can be repaired quickly. It is, to a large degree, why anyone who has already reached that level continues to work in that area.

It also happens to be the case that the progress of a particular project often follows this same developmental path. Some projects start with level 0 or 1, just floundering around in an area that’s new to everyone. The first working version of anything is usually at level 2 or, if they’re lucky, 3; it works and now everybody’s afraid to touch it. Then any 4s who are around, especially if they’ve read Fred Brooks (“plan to throw one away; you will anyhow”), start their tinkering. This often provokes spasms of terror in the poor level 2s, who embark on campaigns of sniping and second-guessing to ensure that what should have been the prototype instead becomes the product. 2s are conservatives, 4s are radicals, and conflict between the two is inevitable, but that kind of conflict is really a whole new topic. The important thing is to recognize that this is normal. You should expect the product to transition between these levels just as programmers do, and prepare for those transitions. If you want to make something you can really be proud of, there will be some disruption and conflict along the way. You should define milestones and schedule development phases according to the level of both your people and your project, which you just can’t do if you don’t recognize that there are levels.