Copyright (c) 2000-2001 by Rich Morin
published in Silicon Carny, December 2000
We have a long way to go before searching for and finding information on eclectic systems is quick and easy, at least according to Rich Morin. In this month's Silicon Carny, Rich argues that while many tools exist for proper documentation, we still need better integration, global indexing, and a reliable cataloguing system. Luckily, a few people are working hard to make these changes.
There was once a systems administrator who wanted to find out about the
But he still needed some help in understanding these files. Fortunately, the man page had a reference to "PMake: A Tutorial." That looks cool, he thought, and decided to look for it.
After going to
He then worked his way down to the
After studying the tutorial a bit, he was able to understand many of the system makefiles, and felt ready to try making some changes. Now came the big question: which of these files should he edit?
The answer was: none of the above. Looking into
The comments in the first file identified it as a repository of default makefile definitions. The second file didn't exist on his system, but he was able to figure out that if the file did exist, it would be used by the system makefiles.
So he created an
Careful readers may note a smidgen of sarcasm in the above fable. Yes, FreeBSD is a well-organized and, by and large, well-documented system. But no, it shouldn't have taken that much time, effort, and skill for our hero to find the needed information.
Nor is our hero's problem specific to FreeBSD. On other eclectic systems (e.g., Linux, Solaris), the available tools might have differed, but I suspect that the search would have taken roughly the same amount of time and effort.
The problem, as alluded to in my July column, is not a lack of documentation, or even a lack of tools. Our hero had plenty of both on hand; they simply weren't well enough integrated. First, the documentation wasn't globally indexed; second, the system files weren't cataloged.
Also, many cross-references were simply missing. Although our hero's
system actually had a copy of the document (albeit in an ugly ASCII
rendition), none of the tools enabled him to find it. As he didn't know
to look for a file named
Nor did the documentation mention the existence of
The good news is that folks are working on these issues; the bad news is
that any real solution is a ways off. In this particular case, even as
our hero was looking for the documentation, it was in the process of
being created. Another FreeBSD user, quite independently, realized the
need for a man page on
In short, the current technology is quite adequate to let users add useful documentation. At the same time, various projects are working on the larger issues of documentation indexing and integration. ScrollKeeper and Meta are two such projects. SUBHEAD: ScrollKeeper
The good news is that folks are working on these issues; the bad news is that any real solution is a ways off. ScrollKeeper is a "cataloging system for documentation on open systems." It's still in its early stages, but a number of high-powered individuals are involved, so the prospects for real results are good.
The basic notion, as I understand it, is to use XML, and, specifically, the Open Source Metadata Framework (OMF), to create an online, networked catalog for a large variety of documents. Here's the project's precis:
"ScrollKeeper is a cataloging system for documentation on open systems. It manages documentation metadata (as specified by the Open Source Metadata Framework) and provides a simple API to help browsers to find, sort, and search the document catalog. It will also be able to communicate with catalog servers on the Net to search for documents which are not on the local system." SUBHEAD: Meta
The Meta Project is my
own thought experiment in this area. Meta goes a bit further than
ScrollKeeper, in that it proposes to integrate system metadata in all
forms with documentation. Thus, Meta would have an explicit entry for
A Meta-capable browser, equipped with this sort of information, could allow our hero to locate and examine a wide range of relevant files and documents. Knowing each file's type and format, it could use the appropriate display software, saving even more time and effort.
Meta is largely science fiction at this point, but work on a proof-of-concept implementation is under way. A tree characterization filter exists; it can determine a substantial amount of information about every node in a file tree. A set of feature extraction filters for given file types is also in progress.
For pointers to other related projects, check out the ExistingWork and IntriguingIdeas pages in the Meta Wiki. Eclectic systems in general, and open source systems in particular, are powerful, robust, and flexible. Now if we can only make them a tiny bit more comprehensible...
About the author