Monday April 22, 2013

Explain, don't document.

Recently, on the Clojure mailing list, u1204 wrote: "Explain, don't document." I'm in general agreement with this advice, but I'd like to expand on it and point out some exceptions. In the 1970's, IBM used the following slogan, later referred to as the IBM Pollyanna Principle:
    Machines should work; people should think.
Specializing this a bit for the current discussion, how about:
    Computers should document; humans should explain.
Or, more explicitly:
    Computers should document the things they can.
    Humans get to (discover and) explain the rest.

So how would that work, exactly?

Mechanized documentation systems can harvest information from a system's files (eg, code, comments, configuration data) and behavior (eg, logs, statistics). This information can be stored in a database, then retrieved for display, follow-on analysis, and so forth. So, the humans aren't entirely off the hook, but they can avoid "documenting" things that the computer can figure out for itself. Of course, they will still need to explain some things (eg, motivation, philosophy, rationale), but most of the brain-deadening "documentation" can be avoided.

You mean documentation is still a problem?

Despite the claims of some "agile" proponents, creating and maintaining documentation is still a problem. In fact, as our systems grow in size and complexity, the need for documentation (including both facts and explanations) becomes ever more important. Fortunately, there are many interesting ideas (and even some working code) floating around. So, prospects are good for a content-agnostic, extensible "system documentation system". For example, I'm hoping for a production-ready, drop-in version of Codeq. Such a system could "listen" (in soft real-time) for events from system(s) it is documenting (eg, file commits, test or production runs, etc. The data would be stored in an immutable, highly scalable database (Datomic) and made available using assorted web technology (eg, D3, Pedestal, SVG). Putting together a production system would certainly require a lot of work, but it is largely "a simple matter of software" (:-). We have an abundance of promising components and ideas (eg, Analyze, Clojure, D3.js, Datomic, Ember.js, Lamina, Pedestal, Simulant, Storm). They "only" need to be integrated into a tool that can be dropped into an existing development shop, server environment, etc.

Would it play nicely with developers?

Many developers dislike creating or maintaining documentation, at least in the form that it gets presented to them. However, few of us would complain about having a convenient, mechanized documentation system that tracks details about the systems we're maintaining. And, if the system were truly content-agnostic, it could accept information from humans (eg, explanations, facts, files, opinions, rules, text). This could greatly enhance the value of information (eg, facts) that mechanized harvesting and analysis can provide. If humans can be cajoled, coerced, or convinced to jot down their understandings, the available "knowledge base" can be greatly enhanced over things that mechanized analysis can provide. For example, let's consider connascence.


Connascence ("born together") is a term used by Meilir Page-Jones (and popularized by Jim Weirich) to characterize types of coupling among entities in a system:
    In software engineering, two components are connascent if a change in one would require the other to be modified in order to maintain the overall correctness of the system. -- Connascence, in Wikipedia
This is a prime example of an area where mechanized analysis could be useful, particularly when augmented by human input. Unavoidably, many parts of systems are connascent. Connascence can occur when multiple components must agree on details such as:
  • identity, name, order, or type of entities
  • format, meaning, or structure of values
  • execution order or timing, algorithm, etc.
Some forms of connascence (eg, Connascence of Name) can be detected mechanically. Others (eg, Connascence of Algorithm) cannot. However, a system's developers and maintainers should be able to find out about every instance of connascence (lest they make changes with unexpected consequences). In reality, however, undocumented connascence is rife in most APIs, commercial software, and running production systems. So, some simple questions:
  • How much of it is well documented (let alone explained)?
  • How much of it could a documentation system handle?
I don't know, but I'd love to find out...


Jim Weirich explains connascence quite well in his "Grand Unified Theory of Software Design" talks. The version he gave at Aloha on Rails segues delightfully from Maxwell through SOLID to coupling and connascence in name choices, data formats, and more (:-). Unfortunately, the slides for this talk aren't shown in the video, but a set of slides is available in this GitHub download (see Connascence.key.pdf). In any case, this video has good slides (and most of the critical material).

Explain, don't document. in Clojure , Computers , Drafts , Technology - posted at Mon, 22 Apr, 13:12 Pacific | «e»

Post a comment

Note: All comments are subject to approval. Spam will be deleted before anyone ever sees it. Excessive use of URLs may cause comments to be auto-junked. You have been warned.

Any posted comments will be viewable by all visitors. Please try to stay relevant ;-) If you simply want to say something to me, please send me email.