<?xml version="1.0"?>
<?xml-stylesheet href="/transform" type="text/xsl"?>
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" xmlns:bibo="http://purl.org/ontology/bibo/" xmlns:bs="http://purl.org/ontology/bibo/status/" xmlns:ci="https://vocab.methodandstructure.com/content-inventory#" xmlns:dct="http://purl.org/dc/terms/" xmlns:foaf="http://xmlns.com/foaf/0.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:xhv="http://www.w3.org/1999/xhtml/vocab#" xmlns:xsd="http://www.w3.org/2001/XMLSchema#" lang="en" prefix="bibo: http://purl.org/ontology/bibo/ bs: http://purl.org/ontology/bibo/status/ ci: https://vocab.methodandstructure.com/content-inventory# dct: http://purl.org/dc/terms/ foaf: http://xmlns.com/foaf/0.1/ rdf: http://www.w3.org/1999/02/22-rdf-syntax-ns# xhv: http://www.w3.org/1999/xhtml/vocab# xsd: http://www.w3.org/2001/XMLSchema#" vocab="http://www.w3.org/1999/xhtml/vocab#" xml:lang="en">
  <head>
    <title property="dct:title">A Short Meditation on Testing</title>
    <base href="https://doriantaylor.com/a-short-meditation-on-testing"/>
    <link href="document-stats#EvB56k5bs8_Rr3EPy6IoeL" rev="ci:document"/>
    <link href="elsewhere" rel="alternate bookmark" title="Elsewhere"/>
    <link href="this-site" rel="alternate index" title="This Site"/>
    <link href="http://purl.org/ontology/bibo/status/published" rel="bibo:status"/>
    <link href="" rel="ci:canonical" title="A Short Meditation on Testing"/>
    <link href="lexicon/#EzqXIsriaILFcWjXdS7FbI" rel="dct:audience" title="Software Developer"/>
    <link href="person/dorian-taylor#me" rel="dct:creator" title="Dorian Taylor"/>
    <link href="person/dorian-taylor" rel="meta" title="Who I Am"/>
    <link about="./" href="3f36c30c-6096-454a-8a22-c062100ae41f" rel="alternate" type="application/atom+xml"/>
    <link about="./" href="f07f5044-01bc-472d-9079-9b07771b731c" rel="alternate" type="application/atom+xml"/>
    <link about="./" href="this-site" rel="alternate"/>
    <link about="./" href="elsewhere" rel="alternate"/>
    <link about="./" href="e341ca62-0387-4cea-b69a-cdabc7656871" rel="alternate" type="application/atom+xml"/>
    <link about="verso/" href="3f36c30c-6096-454a-8a22-c062100ae41f" rel="alternate" type="application/atom+xml"/>
    <link about="verso/" href="this-site" rel="alternate"/>
    <link about="verso/" href="elsewhere" rel="alternate"/>
    <meta content="a-short-meditation-on-testing" datatype="xsd:token" property="ci:canonical-slug"/>
    <meta content="Lots of people have opinions about testing software, so I get to have some too." name="description" property="dct:abstract"/>
    <meta content="2021-04-13T18:06:43+00:00" datatype="xsd:dateTime" property="dct:created"/>
    <meta content="2021-04-13T18:18:03+00:00" datatype="xsd:dateTime" property="dct:modified"/>
    <meta content="2021-04-21T14:14:08+00:00" datatype="xsd:dateTime" property="dct:modified"/>
    <meta content="2022-05-31T04:18:52+00:00" datatype="xsd:dateTime" property="dct:modified"/>
    <meta content="2022-05-31T15:10:50+00:00" datatype="xsd:dateTime" property="dct:modified"/>
    <meta about="person/dorian-taylor#me" content="Dorian Taylor" name="author" property="foaf:name"/>
    <meta content="summary" name="twitter:card"/>
    <meta content="@doriantaylor" name="twitter:site"/>
    <meta content="A Short Meditation on Testing" name="twitter:title"/>
    <meta content="Lots of people have opinions about testing software, so I get to have some too." name="twitter:description"/>
    <object>
      <nav>
        <ul>
          <li>
            <a href="//dorian.substack.com/p/coasean-inversion" rev="dct:references" typeof="bibo:Article">
              <span property="dct:title">Coasean Inversion</span>
            </a>
          </li>
          <li>
            <a href="document-stats#EvB56k5bs8_Rr3EPy6IoeL" rev="ci:document" typeof="qb:Observation">
              <span>urn:uuid:bc1e7a93-96ec-4f3f-b46b-dc43f2e88a1e</span>
            </a>
          </li>
        </ul>
      </nav>
    </object>
  </head>
  <body about="" id="EHgQVlo-M8MazFPuS2VrNL" typeof="bibo:Article">
    <p>My general disposition toward testing code is that it is a distinct job that needs to be carefully considered in its own right. What you are producing when you create a test suite is a product that serves the dual purpose of not only ensuring that the code is correct, but also substantively improving the experience of the developer, who includes but is certainly not limited to yourself.</p>
    <p>What this entails is that a coherent testing regime can only be considered when there is enough of an artifact to test. Test suites that exclusively grow organically, especially when combined with <q>test-first</q> policy stipulations, tend to incentivize the creation of tests that are either redundant&#x2014;as in the conditions are already part of the test suite of a given third-party dependency&#x2014;or they are tautological and do not give you any new information. As such, a testing regime should be explicitly designed.</p>
    <p>The converse of testing zeal is not testing enough. Writing tests diverts effort that could have been put toward expanding and refining the behaviour to the product itself, and does not always net positive. We should be honest about that.</p>
    <aside role="note" id="EaIwOvIQ2uhY0iz-dWm1GJ">
      <p>But then, effort put into the functionality itself does not always net positive either, even taking into account the effort saved by diverting to testing. Programming is inherently a gamble that the value of executing the code outweighs the cost of writing it. This turns out to be the case only on average, and the mode and median are very far from the mean.</p>
    </aside>
    <p>There is a natural inflection point for diverting resources to testing, that happens around the time you can no longer just eyeball your results in a <a href="https://en.wikipedia.org/wiki/REPL" rel="dct:references"><abbr title="read-eval-print loop">REPL</abbr></a>. This eyeballing situation may not be uniform across the entire artifact, but you can get quite far that way.</p>
    <p>There is a tendency for us to write tests as if we sometimes do not trust the computer itself to behave consistently; if this were actually the case, we have bigger problems than the integrity of our own software. This is often a symptom of composing test suites upward from the inside out rather than sculpting them from the outside in.</p>
    <p><strong>Therefore:</strong> Concentrate your initial testing efforts on the behaviour you <em>want</em> the artifact to exhibit before testing the behaviour you <em>need</em> for it to exhibit in order to exhibit the behaviour that you want. Backfill when you get unexpected inputs to those ultimate tests. Aim for <q>sparse coverage</q>, that is, a public interface that does what you expected it to. If there is something wrong somewhere in the plumbing then those tests are likely to fail too.</p>
    <aside role="note" id="ESs0QC4az9_DpXVpp09pFL">
      <p>Note however that they might <em>not</em> fail, so begin with the tests that probe the public interface but don't implicitly trust them.</p>
    </aside>
    <p><strong>Regarding methodology:</strong> Once you have accumulated enough of an artifact that it does something coherent, and that eyeballs in a <abbr title="read-eval-print loop">REPL</abbr> are delivering sufficiently diminishing returns, set aside some time to sketch out the entire testing regime in one go. Begin with a single file; write out declarative prescriptions and proscriptions (i.e., <q>artifact should [not] do&#x2026;</q>) as a flat list of bullet points in prose commentary. Formal language is too restrictive at this stage. It will very quickly become apparent how these expectations ought to be grouped, which expectations are redundant (i.e., they are implied by more salient assertions), and what shared resources and fixtures are required among the tests. Once you have arranged these intermediate results in a sensible manner, you can stub them in as code in a second pass, and fill in the contents as you progress.</p> 
  </body>
</html>
