Cover V14, i12
dec2005.tar

Simplifying SolarisTM Patches

Paul Guglielmino

Maintaining large numbers of systems over time can cause small (and sometimes large!) differences in the system configurations. Some of these differences may occur in the system's patch levels. Perhaps an ad hoc patch for the development environment never made it to production or space issues prevented a patch from getting installed. Regardless of the reason for these differences, it's good to have a way to measure the accuracy of your configurations. When issues arise, knowing these differences will aid you in troubleshooting problems.

A related patching issue for Solaris admins is finding necessary prerequisite patches (e.g., patch X needs patch Y, which in turn needs patch Z). This process can be both time consuming and frustrating. I have encountered both these problems, and I have worked on solving them with two simple Perl scripts. In this article, I'll show how to use both of the scripts as well as provide the source code so others can benefit from them. I will also briefly discuss some Sun tools that can complement these Perl scripts.

Patch Information

These scripts are made possible by a file that Sun already publishes daily. This file, called patchdiag.xref, is available from SunSolve and lists every patch Sun has released along with related information. That information includes the latest minor revision, flags indicating whether it is recommended or security related, which version of the OS it is for, necessary prerequisite patches, release date, patch architecture, packages affected, and a simple description.

Here's a sample line from patchdiag.xref. It's all in one line, wrapped here for clarity. The file can be hard to read but you can see some of the fields I described:

117702|01|Sep/01/04| ||||8|sparc;108528-29;|SUNWcsl:11.8.0, \
  REV=2000.01.08.18.12;SUNWcslx:11.8.0,REV=2000.01.08.18.12; \
  |SunOS 5.8: scsi plugin patch
Chris Josephes has written an excellent set of Perl modules, called sol-inst, to deal with installation information on Solaris systems. One module in particular deals with reading the information in the patchdiag.xref file. Unfortunately, the modules have not been updated since 2000, but I find that they are still useful.

Sun Patch Check Tool (patchk.pl) is a free Perl script from Sun. Sys admins can run Sun Patch Check to get a report of which patches are installed, which are installed but not at the current revision, and which are not installed at all. The report can categorize patches in the following ways: recommended, security, software-related, and obsolete. It can be generated in plain text or HTML. This software is no longer available, however, and Sun suggests that you transition to Patch Manager instead.

Sun Patch Manager 2.0, which grew out of the older PatchPro program, is the primary tool for managing patches on Solaris systems. It is available for both Solaris 8 and 9, but some features do not work on Solaris 8. It provides both a Web browser interface (Solaris 9 only) and a command-line interface that can analyze a system to determine missing patches. It also can be used to download patches and apply or remove patches to your systems.

Scripts

Sun recognized the problems different patch levels can cause and produced a small, free utility called Patch Comparison Utility (PCMP). The PCMP compares the output of showrev, identifies the uncommon patches between the two files, and produces an output report. Even though PCMP is a great tool, it has a few drawbacks. First, it is released as a binary, so changes and improvements can not be made. Second, it only compares two servers at a time. And last, it can be hard to find! It seems no longer to be available from Sun's Web site, but it has been archived by a few people on various Web sites (see Resources).

Those issues led me to build a new tool in Perl to compare patches (Listing 1; compare_patches.pl). This new version can compare patch levels on an arbitrary number of servers and is open so it can be improved. This script also can use the output of showrev or Sun's patchk.pl tool to compare the patch levels (although patchk.pl is deprecated, I originally wrote the script while that tool was in use). The script's structure is simple. It validates the input files and reads them into hash data structures in memory. Then it iterates over each patch on each server to check for differences in the minor revisions. It will then print the report output to the screen. Patches with differences will be marked.

Another common problem in Solaris patching is finding the necessary prerequisite patches. To solve this problem, I wrote another Perl script, called find_reqs.pl (Listing 2), that uses Sun's patchdiag.xref file to recursively find all needed patches for a single patch.

The structure of find_reqs.pl is easy to follow. It scans the patchdiag.xref file and creates a data structure mapping patch ids to necessary prerequisites. Once that is done, it's just a matter of doing a recursive search starting with the patch of interest. Then it weeds out any duplicate patch references and prints an ordered list of patches. The output from find_reqs will be a correct order in which you can install the patches.

Each script has a help option to explain the basic parameters and both scripts have a debug option to be more verbose while running. Find_reqs.pl has an option to specify a particular patchdiag.xref file; otherwise, the default is to look in the current directory. Also, compare_patches.pl has options to hide the header lines, to show only the patches with differences, or to switch the input files from showrev format to patchk format.

Here is some sample output from compare_patches.pl. This example shows the comparison of three servers whose showrev -p output is listed in files server1, server2, and server3. The output displays the patch id and the corresponding minor patch revisions on each server. The differences are marked with a "*" and are summarized at the end:

% ./compare_patches.pl server1 server2 server3
Patch ID ||  Rev #        | Rev #        | Rev #        |
109134*  ||  server1:  03 | server2:  02 | server3:  02 |
109135   ||  server1:  04 | server2:  04 | server3:  04 |
111873*  ||  server1:  01 | server2: N/A | server3: N/A |
111874*  ||  server1: N/A | server2:  03 | server3:  01 |
111879*  ||  server1:  00 | server2:  03 | server3:  03 |

Different patches: 4 out of total 5
Here is another output sample from the same data but only showing the differences and without the header line. The "noheader" option also omits the trailing summary line:

% ./compare_patches.pl --diff --noheader server1 server2 server3
109134*  ||  server1:  03 | server2:  02 | server3:  02 |
111873*  ||  server1:  01 | server2: N/A | server3: N/A |
111874*  ||  server1: N/A | server2:  03 | server3:  01 |
111879*  ||  server1:  00 | server2:  03 | server3:  03 |
Here is some example output from running find_reqs.pl on kernel patch 108528. Patch 108528 requires patches 108987, 111111, and 111310. Patch 108987 requires 112396, and the others do not require any prerequisites:

% ./find_reqs.pl -xref=/patches/patchdiag.xref 108528
| 111310 111111 112396 108987 108528 |
Improvements

For the compare_patches.pl script, I think an additional cross-check with the patchdiag.xref file to see whether the patches are at the latest revision would be very useful. Also, I would like the Perl scripts to run with warnings and strict turned on. Find_reqs is there, but compare_patches needs a bit more tweaking. Future versions of these scripts can also be written to take advantage of functionality in the sol-inst modules.

Resources

Patchdiag.xref -- http://sunsolve.sun.com/patches

Patch Comparison Utility (PCMP) -- http://www.scn.rain.com/pub/solaris/

Sol-inst modules -- http://www.cpan.org/modules/by-category/04_Operating_System_Interfaces/ \
Solaris/sol-inst-0.90a.tar.gz

Sun Patch Manager -- http://www.sun.com/download/products.xml?id=3f9d714b

Paul Guglielmino is a consultant in the Boston area. He is always looking for a new script to write or problem to solve. Please send comments and improvements on these scripts to him at: paulg@nepd.com.