The specificity gradient is a term I came up with for a principle that I have held for many years now, that software is, as I said on a podcast back in 2010, a very verbose, very precise incantation specifying how an information system ought to behave.
What gets written down as software code is the result of a long chain of deliberation, passing through the hands of numerous stakeholders from sharply different disciplines. In the process, the rationale for why the code behaves one way and not another is often lost, or at least misplaced. And herein lies the problem: Code has far too much detail, and is much too preoccupied with its own affairs, to be the authoritative place where business decisions are made.
Indeed, this ratcheting up of technical detail concomitates with an increasing distance from a surface
that ordinary people understand, and, I argue, less durable and more volatile with respect to time.
Interest recently resurfaced in my 2021 video outlining the specificity gradient concept. Note that initially, I had the first category as business goals, but later decided that business ecosystem was more appropriate.
As a conceptual framework, the specificity gradient rests on the shearing layer
or pace layer
concept, which has had some currency in the design community for quite some time. The essence of shearing/pace layers is that certain concerns operate at certain time scales, and more importantly, the time scales vary enough that anything trying to bridge between them will be ripped apart in the quasi-tectonic shear. So if you're going to build anything, you build within the respective layers, not across them.
The idea of the specificity gradient is to create an unbroken line from the chunkiest thoughts to the most precise ones, adding detail every step of the way. The purpose of this is twofold:
For software, a reasonable specificity gradient looks like this:
The business ecosystem is a model of the organization, situated in its immediate geographical, social, political, economic, and legal environment. Its purpose is to model how the organization interfaces with the outside world. Entities in the model can belong to categories like:
Mission-driven organizations might have constituents rather than customers, and/or members rather than shareholders; these categories are not necessarily fixed. Likewise, we might model certain relationships with particular entities, such as certain government agencies, or suppliers if they are important enough. Other categories of relationships can be represented by one or more archetypal entities within the category.
Embedded within this ecosystem—indeed, representable by the differences between current and future states of the ecosystem itself—are the organization's goals. These are the things that the organization wants to accomplish. In the case of a company, this would likely reduce in large part to earn a profit
, though other qualitative and quantitative changes will also likely show up.
We can imagine the more detailed tactical objectives turning over every quarter (or quicker), while the core, which would reference institutions, markets, and products, will be considerably more long-lived: markets and institutional milieus can last for centuries, and a product can last at least as long as the business does—so again, years, decades or centuries. My focus here indeed is less on the caprice and mercuriality of everyday business tactics, or even something like quarterly financial targets. The real value of the business ecosystem is to surface and make explicit its own (largely slow-moving) structure, to serve as an anchor for the lower rungs of this model.
Every product, software or otherwise, is going to have at least one archetypal user. It is important to recognize that a user may or may not be the same as the customer, but a user nevertheless has goals, and the purpose of the product is to help them achieve those goals, to the extent that they intersect with the business's goals.
We see here our first jump up in specificity, and down in temporality: a user only (nominally) exists in the context of a product, and so will change as the product changes.
A user task is not the same thing as a user goal. A task is a concrete thing that a user has to do to achieve their goal. To the extent that any task must have at least one concrete, specific method, the details of a task may change as say, technology, law, the commercial landscape, or fashion does. A really conspicuous example of this is the arc of computers in the large:
This is actually kind of an interesting scenario, because we kind of need to gear down at this level, temporally speaking, since a user is potentially able to complete a task using more than one, or combination of, specific methods. So we need to capture the abstract task, as well as the concrete steps in each medium or channel.
The complement of the user task is the system task, which increases once again in specificity and decreases in temporality, because it is going to be bound to the specific procedural, organizational, or technical method of achieving the user's task. A system task is the sequence of concrete steps the system has to take to support the user in carrying out their task, so that they achieve their goal, so the ultimate customer continues to pay for the product, which contributes to the satisfaction of the business's goals. Again, we're going to see a subset of these which are abstract, and are portable across specific technologies and vendor relationships, and a subset which are concrete, and get thrown away when technologies obsolesce or contracts get terminated. As a rule, we do not expect system tasks to survive as long as the longest-lived user tasks.
System behaviours are detailed, concrete, prescriptions and proscriptions about what the system must or must not do in the course of carrying out its tasks, including preventing users from doing things that harm themselves or contravene the business goals. In other words, various user interface safeguards, content moderation policies, and information security concerns live here.
This category is unusual to the extent that it has potentially the widest gamut in temporal sensitivity, eventually spanning several orders of magnitude. However the content is always going to take the form MUST or MUST NOT. What is responsible for the wide temporal range is that some of these rules will eventually abstract out into general principles and policies, while others will concern specific implementations of specific steps in specific tasks, which are peculiar to that particular implementation and will only remain relevant as long as the implementation itself does.
This step in the gradient, in addition to being the most detailed description of the system short of source code itself, also has plenty of precedent: one of these rules, at least the most granular variety, more or less maps 1:1 to a bug report or unit test case, which everybody in the industry should be familiar with.
Finally, we arrive at the most specific stratum in our gradient of specificity, executable source code. Of course what makes code the most specific is not only the fact that it has to be formally strict and internally consistent in a way that no other layer must, but also that it has to occupy itself with the mundanities of computers themselves—memory management, network connections, storage space, algorithmic efficiency, and the peculiarities of the language in which it is written—all that business which is much too detailed for any of the other layers to bother with.
Code, of course, to the extent that it is getting any attention at all, is continually undergoing changes: parts are being added, removed, moved around. Bugs are getting patched. Third-party dependencies have to be tracked, and any mutations—which on the whole are frequent—need to be compensated for. Frameworks, languages, even architectural styles, go in and out of fashion. Vanishingly few pieces of software (notably TeX) refine towards a gem-like state of stability.
In fact, I'm inclined to say that code is old, it's likely pathological: it exhibits unspecified behaviour, i.e., it does things and nobody knows how, or often even precisely what, so they're afraid to touch it, because the business depends on it. Part of the idea behind the specificity gradient is to prevent getting into this position.
Here is where I say something mildly controversial: I think way too much value is placed on code. Sure, you need an implementation, or you don't have a product. But what seems to happen so often is that all the materials that served as the basis for all the decisions that made the product possible, are left to rot. This, I believe, is a mistake.
The code, on the other hand, is elevated to sacrosanct. The attitude toward code is don't touch it if you don't have to (which is silly, because you almost always have to). I mean, I get it—code that runs correctly, and efficiently, and doesn't break, is a hard-won battle. Plus, it's the thing that makes the money. So don't screw with it. Except we screw with it all the time! (Under controlled circumstances, mind you.)
Suppose, for a moment, that the language the code was written in, or maybe some major dependency, suddenly became inviable. Plausible scenarios for this include some irreconcilable security flaw getting discovered, or some change in licensing or event in the legal domain. Or let's just say that somehow your codebase just gets deleted. How screwed would you be?
The answer, I'm guessing, is probably totally, completely screwed, because the code is the only place where most of the system's description is embodied. Part of the motivation for a specificity gradient would be to capture as much of this information as possible, somewhere other than the code.
If we treated code as what it is, namely, the substance at the very end of a long, multi-layered gradient of increasing specificity, then I don't think we would need to be so precious about it. Indeed, migrating (or expanding) platforms, frameworks, and even languages, happens a lot more frequently than we'd like to admit. I'm saying that this need not be as expensive and/or risky a proposition as it currently is, if we had an infrastructure like the one I just described. I'm talking about relieving code of the burden of being the residual source of knowledge about the system. It's a bad place for that, anyway.
Not only do the origins of this thinking—Frank Duffy's shearing layers and Stewart Brand's pace layers—represent, but programmers themselves already have pieces of this puzzle: behaviour-driven development, literate programming, design patterns, even plain-old bug reporting. The military strategy, management strategy, operations research, and systems people have models out the yin-yang, and let's not forget the UX people, who fill in the rest. What's missing is an unbroken path that takes you from one end of the edifice to the other.
I want to make it clear that the specificity gradient is not an instruction manual, in the sense that first you do the business ecosystem, then you do user goals, and so on—even though that would be a sensible start. For one, the gradient kind of zippers together a bunch of different disciplines, by system tasks at the latest, but prior to that, the disciplines are each going to be doing their own thing.
Note as well that nowhere in this description do I say the word feature.
A website, of course. But not just any website: this would be something that (naturally) was walled off from the outside world by default (that is, an intranet), and possess unprecedented power as a solvent of documents and data sources. Its job would be to make everything maximally addressable: jack into every API, decompose every document, suffuse every database.
This website would not have pages
per se; rather everything navigable—that is to say, everything—would be a piece of structured data that could be transformed and composed into more complex structures. This structuring and composition capability should be available to anybody involved, and easy enough (with no irreversible consequences) for any relevant person in the organization with any background to be trained to do.
It is a safe bet that the overwhelming majority of this forensic content—to the extent that it exists—is going to (at least initially) come from conventional office file formats (Word, Excel, Powerpoint, and their Apple and Google successors). Other elements will be addressable through database or API adapters. Some of the content will inevitably be irretrievable, be it legitimately lost, in some archaic file format, or perished due to a canceled SaaS contract. What remains will likely have to be hand-transcribed.
Eventually, documents in general will recede in importance as origins of information and will instead become targets for projecting information. The native form of the content will eventually make its way toward the shape of a dense hypermedia network: small pieces, loosely joined.
What I just described in the last section is what I would ultimately like to see, and am working towards in my own practice. In the interim I am confident that the spirit of the specificity gradient can be applied to existing tooling, assuming a modicum of discipline.
If you or your team is considering moving in this direction, this is something we can discuss.