A puzzling, yet frequent phenomenon I observe in people who work intensively with computers is a neglect to do the housekeeping. This includes, but is far from limited to:
Over the last decade, I have watched many succumb to this pattern, including myself. The adverse effects ranged in severity from a sporadic itch to business-busting catastrophe. I suppose it's a ubiquitous and perfectly rational tendency to want to avoid spending time to become an expert in something we aren't likely to do very frequently. With computers, however, it is often true that although it's impossible to know a priori how much we are going to use a certain skill or system, a relatively marginal investment of time often affords us tremendous benefits. I wonder:
In the various production-oriented positions I've held in the software industry, a common organizational behaviour has been to eschew, if not outright punish, any expenditure of time that didn't fit within the arc-second-wide slice of what was considered on-task. Someone would undoubtedly invoke the cutesy, yet condescending metaphor of going down the rabbit hole. The perpetrator would plead to be permitted to finish while incurring the shame of the manager or the whole team, or a heated argument would break out about whether or not the work was necessary.
While tangents can be a real threat to productivity, the meta-baggage around them nearly always is. I understand why we want to relegate the problems we solve to those that deliver the goods, but I suspect therein lies the problem — and under extreme pressure to Get Things Done™, we mortgage our long-term success to satisfy short-term targets.
We only ever seem to really get inspired to do this kind of work when we're supposed to be doing something else, but in a business culture that only cares about that something else, when do we find the time? Being that many of these small innovations have arbitrary value, it is a tragedy that they do not get explored. It is arguably more of a tragedy, however, that simple problems become obstacles that can span for years.
A conscientious chef working at home will clean incrementally as he proceeds, so that after the meal there isn't much additional work to do. A carpenter will tidy her work area at the end of the day so that there isn't a huge mess at the end of the week. In both these examples, the mess is as conspicuous as it is hazardous, and it risks blocking progress. In software, the mess isn't physical but cognitive, and as long as there's a place to put the data, it can be ignored indefinitely.
Moreover, in a professional kitchen or construction site, it is worth noting that the maintenance work is usually delegated, likely to a junior. In software this would not be as applicable, because the person who spots the need for maintenance is also frequently the most qualified and motivated to perform it. Although, where delegation is possible, perhaps the fallow programmer pattern could apply.
I define innovation as the stuff we figure out in order to make it easier to do other stuff — purely serendipitous. It is impossible to innovate on a whim. I submit that a business that upholds a policy of staying rigidly on-task in knowledge work doesn't leave valuable innovation on the table, it throws it in the garbage.
Every little problem we solve, every little misfit we fix, increases our value as knowledge workers. If we codify that solution and make it retrievable, we are not only innovating, but we are increasing our stockpile of intellectual capital. If we share that solution with our colleagues or with the world, we make our environment that much richer.
While I could argue that maintaining and augmenting our work environments is a discipline — and it is — I think it requires greater discipline and a great deal of cognitive dissonance not to do it. Anecdotal evidence suggests knowledge workers — programmers in particular — want to incorporate it, but often feel unable to do it on company time.
I suspect the real change begins on the outside, in the deals struck around the procurement of knowledge products and software in particular. This would be a change in attitude that affects everything from bespoke contracts to venture-backed startups. It entails injecting some realism about how problems are solved, how knowledge is acquired and how information behaves. It entails considering what we value — the artifact or the solution.
In the summer of 2006, I decided I wanted to begin shaping my tools to fit me, rather than contorting myself around them. I figured there were few better ways to punctuate my endeavour than forcibly switching to Emacs after being a die-hard Vim user since about 1999. Although, aside from a few significant initial changes, I have still found the habit tremendously difficult to get into.
Since about day one of my foray into Emacs, I have used James Clark's nxml-mode. It is a major mode that provides a great many shortcuts for XML editing and validation — or at least it was supposed to. For years I have been limping through my hand-edited XML knowing that it could — and should — be doing more for me. It was only today that I was fed up enough to fix it. The magic incantation was to (load "~/.emacs.d/nxml/rng-auto.el") after I had loaded nxml-mode. I was so relieved, I even went the extra step of downloading the XHTML+RDFa DTDs and converting them to RELAX NG with Trang so that nxml-mode would properly validate RDFa.