 See
Your Files and Descriptions with "List"
Jim Pinson
The Unix/Linux world loves short file names, especially for commands.
In a world dominated by "ls", "cp", and "mv", longer filenames such
as "md5sum" seem almost extravagant. The short file names certainly
save us a lot of effort when we type commands, especially those
long pipe commands we seem to favor.
Of course, there is a price to pay. Short names for seldom-used
files, especially those we didn't write, convey little about
the files' functions. We can create long file names that clearly
spell out the purpose of a file, but the longer and more descriptive
our file name, the harder it is to type, and the long name messes
up our "ls" output. Some file names simply can't be renamed to explain
their function, because they are used by other programs, such as
those in the /etc directory.
Years ago, I saw a solution to this type of problem in a COMMAND.COM
replacement called "4-DOS". This program allowed you to give descriptions
to files, and those descriptions could be displayed when you listed
your directory contents. This functionality can be easily added
to current Unix, Linux, and Cygwin systems using a Perl script I
wrote called "list". I have found this program very useful over
the years, particularly for keeping track of all my old scripts.
The "list" program works very much like "ls". It lists your files,
but instead of displaying the normal file attributes, it displays
a file description. It also has commmand-line switches that allow
you to add descriptions and copy or move files along with their
descriptions.
"List" is quite simple to use. Just type "list" to see the files
and descriptions in the current directory, or type "list" followed
by the filenames and directories you wish to view. You can freely
mix directories and filenames on the command line. ("List" can be
obtained from the Sys Admin Web site: http://www.sysadminmag.com/code/).
To add a description to the file /mydata/serverlist.txt, you would
type:
list -add /mydata/servers.txt "This is a list of all my servers"
If you were already in the /mydata directory, you could type:
list -add servers.txt "This is a list of all my servers"
To see the description you just added, along with all the other .txt
files in the directory, you could type:
list /mydata/*.txt
Typing that command would produce output that looks like this:
Directory /mydata:
employee.txt
supervisors.txt
servers.txt This is a list of all my servers
tools.txt
All switches can be shortened to a single letter, or given in POSIX
form. For example, the -add switch can be entered as -add,
--add, or -a. However, for purposes of clarity in this
article, I will use the full switch name for all the examples.
Suppose you want to move the file to a new location. You could
type:
list -move /mydata/servers.txt /backups
This command would move the file and the description to the new directory.
You could verify that the file and its description were moved by typing:
list /backups/servers.txt
The "list" program is quite flexible and can work on multiple files
and/or directories at one time. For example, all the following commands
are valid:
list /bin /usr/bin /mydata/servers.txt
list -copy /mydata/*.txt /backup
list -copy /mydata/employee.txt /data/emp.txt /backupdir
The Description File
But, where is the description of the files kept? I debated using
databases such as MySQL or SQLite to store the description, but
decided to stick with the idea of a description file, located in
each directory. There is a certain elegance to this approach. It
means any normal tool you use to back up a whole directory will
back up the descriptions as well.
If you copy a directory to a CD, or move a directory tree, the
description goes along for the ride. "List" is not dependent on
another tool, such as a database. An advantage of the file being
plain text is that even users without "list" can view the file contents.
By default, the descriptions file is called "DESCRIPTIONS". The
format is quite simple: each line of the file contains a filename,
the separation character (a "~" ), and the description. For example:
DESCRIPTIONS~This is my descriptions file
servers.txt~This is a list of all my servers
The actual name of the file used to contain the descriptions, as well
as the separator character, can be changed in the first few lines
of the "list" program in a section I will refer to as the "config"
section. (The complete list of config options will be discussed later
in this article.)
You can of course edit the DESCRIPTIONS file directly, or use
the -add switch discussed above to create entries. I highly
recommend using the -add switch to add descriptions, particularly
when dealing with large directories with many descriptions.
Another way to get descriptions into the file is to have "list"
search files for embedded descriptions by typing:
list -scan /mydata
The -scan switch will search all text files in the specified
directory for a special markup that contains the file description.
By default, the begin/end blocks of the markup are ::bdes::
and ::edes::.
The description for servers.txt could be added to server.txt as:
::bdes::This is a list of all my servers::edes::
"List" will ignore any text before and after the begin/end
blocks, so you can protect the description with whatever "comment"
delimiters your programming language uses.
The -scan option only processes plain text files in the
directory specified; binary files are not currently supported.
The text recognized as begin/end blocks by "list" can be customized
in the config section of "list".
If everyone in your work group agrees on a uniform system to add
internal descriptions to scripts and datafiles, keeping your DESCRIPTIONS
file up to date will be much easier.
Since I practice what I preach, "list" has a description of itself
in the source. Whenever "list" alters a DESCRIPTION file, it forces
the first line to be an embedded description.
Formatting Long Descriptions
The DESCRIPTIONS file uses one line per filename/description.
Even if a long description is entered, the "list" program will not
truncate it. The "list" program will wrap long descriptions to fit
your current term window when it prints them, as displayed below:
list /mydata/*.txt
employee.txt
supervisors.txt
servers.txt This is a list of all my servers.
long.txt This is a text file that has a very very
very very very very very very very very
very, let me say this again, a very very
long description.
sales.txt A list of our sales team.
As "list" prints out files and descriptions, it uses the first 25
characters of the output for the filename and the remainder of the
display for the description. Word wraps occur on white space of the
description, and if there is no white space, the line is broken mid-word
as needed. This, of course, requires that "list" know the length of
your current term window.
Many shells have an environment setting, "COLUMNS", which contains
your current term length. Unfortunately, some shells don't export
this variable so it is not available to "list". I recommend you
export your COLUMNS environment in your shell startup script so
"list" can use it.
If "list" doesn't find COLUMNS in the environment, it will attempt
to invoke "tput cols" and "stty size" to find the setting. If these
approaches do not work, "list" will assume a default term width
of 80 characters.
The Details
To install "list", just copy the "list" program to someplace in
your PATH and change the first line (the #! "SHE BANG") to indicate
the path to your Perl program.
"List" is written for Perl 5 and only uses standard Perl Library
modules, so it should work with most Unix and Linux systems. I even
use it with Cygwin on Windows systems.
If your shell doesn't export COLUMNS, please do so in your startup
script and add any aliases you might like to reduce your typing.
For example, the .bashrc file in my home directory has these entries:
export COLUMNS
alias add="list -add"
alias move="list -move"
alias copy="list -copy"
Command-Line Switches
The following switches are recognized by "list":
-add -- Adds a description to a single file or directory.
Examples:
list -add myfile.txt "this is a description of my file"
list -add mybin "this is my private bin directory"
-copy -- Copies file(s) and descriptions. Examples:
list -copy file1 file2 (one file to another file)
list -copy file1 file2 outputdir (multiple files to a directory)
-directory -- Tells "list" you want to see the description of a directory,
not the contents of the directory. This works like the -d switch
of the ls command. Example: Suppose you add a description to the /bin
directory using the command:
list -add /bin "This is the main bin directory"
That command will add the description of /bin to the description file
/DESCRIPTIONS. However, if you type list /bin, you will not
see the description of "/bin" you just added. Instead, you will see
the files and subdirectories of /bin. To see the description of the
directory /bin and nothing else, type list -directory /bin.
-move -- Moves file[s] and descriptions. Examples:
list -move file1 file2 (one file to another file)
list -move fileone file2 file3 outputdir (multiple files to a directory)
-scan -- Scans all the text files in a directory
for embedded descriptions. Multiple directories may be specified.
Examples:
list -scan mydirectory
list -scan mydirectory1 mydirectory2
-purge -- Checks a directory or directories to
see if all the descriptions have files to go with them. If the file
doesn't exist, the "orphan" description is removed. Examples:
list -purge mydirectory
list -purge mydirectory mydirectory2
-help -- Prints program usage help.
Preserving Your Descriptions File
"List" tries to avoid damage to your DESCRIPTIONS file, since
it can represent much effort on your part to produce. Whenever "list"
updates DESCRIPTIONS, it makes a backup of the original, called
"DESCRIPTIONS.bak". Whenever you move or copy files to a new directory,
"list" updates the destination directory's DESCRIPTIONS file with
the new description, but it does not by default remove the
descriptions from the source DESCRIPTIONS file. Of course,
over time you could end up with a lot of old descriptions. This
is not really a problem, because you won't see these old descriptions
during your "list" operations. However, "list" does provide two
ways to remove old descriptions. The first is to purge the directory
using the -purge switch described above. The second method
is to set the "autopurge" feature in the config section of the "list"
source file.
When autopurge is active, "list" will automatically purge the
DESCRIPTIONS file of orphaned descriptions whenever it updates the
DESCRIPTIONS file (e.g., during -add, -move, or -copy
operations.) While autopurge may sound attractive, I generally suggest
it not be activated. Imagine, for example, that a user forgets to
use list -move and instead uses a system command like mv
to move files. That would mean the files would exist in another
directory without their descriptions. The descriptions would remain
in the old directory as orphans but would disappear from the DESCRIPTIONS
file the next time you added a description. I think it is safer
to manually purge the descriptions at times of your choosing.
As mentioned earlier, it is a wise practice to embed each description
in its respective file so that the descriptions can be recovered
using the -scan switch, no matter where the files end up.
I've mentioned the config section several times now, so let's
take a look at it:
###################### config section.######################
#change as needed
my $descriptionFile="DESCRIPTONS"; #The name used to store descriptions
my $desDelimiter="~"; #delimiter to separate filename from description.
my $nameLength=25; #The filename will be formatted to this length.
#The actual description will use
#the remainder of the screen length.
my $delimiterStart= "::bdes::"; #beginning of in-file description
my $delimiterEnd= "::bdes::"; #end of in-file description.
my $autopurge=0; #change to "my $autopurge=1;" to activate autopurge
###########################################################
Conclusion
I've found "list" to be a great addition to my tool box, I hope
it works for you. If you build a good description file for various
directories on any of the major Unix/Linux distributions, please
feel free to send it to me, and I will post it along with future
updates on: http://www.jc-research.com/downloads.
Jim Pinson is a Linux Administrator, Systems Programmer, Oracle
OCP DBA, and technical writer. He provides consulting services through
his company, J&C Research, Inc. of Havana, Florida (near Tallahassee).
He can be contacted at: jim_pinson@jc-research.com. | 
