Article
Listing 1
Listing 2
Listing 3
Listing 4
Listing 5
Listing 6
Listing 7
Listing 8
Listing 9
Listing 10
Listing 11
Listing 12

dec2004.tar

Entrap: A File Integrity Checker

Ed Schaefer and John Spurgeon

Verifying the integrity of files is an important systems administration task. Well-known systems administration authority Æleen Frisch says that "minimally, you should periodically check the ownership and permissions of important system files and directories." One method for verifying files is to take a snapshot of the system in a pristine state and compare it against subsequent snapshots.

You can use a product such as Tripwire (http://www.tripwire.com), or create your own such as our Entrap utility. Entrap is a suite of Korn shell scripts that compares two snapshots of a system and reports the differences. When two snapshots are compared, Entrap reports information about files that have been added, deleted, and modified.

An Entrap snapshot includes the file characteristics displayed by the command ls -ild, as well as optional file signatures, such as md5. Filtering rules may be set up to instruct Entrap to ignore specific files and/or attributes when comparing two snapshots.

In this column, we'll explain Entrap's configuration file. We'll discuss the commands used to take a snapshot, filter snapshots, and compare two snapshots. We'll review the directory structure, present an Entrap example, and include a high-level description of the Entrap scripts. We conclude with what's in the tarball and possible Entrap enhancements.

Configuration File

Entrap uses a configuration file to describe the directory trees to be processed. Each line of the configuration file contains an ordered pair. The first element of the pair is a pathname, and the second is a pathname alias. For example, the default configuration file, /opt/entrap/etc/conf/entrap, looks like this:

/  root
/usr  usr
/var  var
/opt  opt
/export/home  home

You can override the default, and specify an alternate configuration file with the -c option:

-c    config_file

Snapshot Command

The snapshot command creates a snapshot:

entrap snapshot [-o] snapshot_name

A snapshot is composed of a directory containing one or more files. The files in a snapshot directory are named the same as the directory aliases listed in the configuration file that was used to create the snapshot. Existing snapshot files are not overwritten without using the -o option.

Filter Command

entrap filter [ -f filter_name] snapshot

The filter command applies filtering rules to the specified snapshot. A given snapshot only needs to be filtered once for a given filter. After filtering, the snapshot may be compared against other snapshots multiple times without being filtered again. However, if the filtering rules change, the snapshot must be re-filtered.

A filter is implemented as a directory containing one or more filter files. The files in a filter directory must be named the same as the directory aliases listed in a corresponding configuration file. If a filter file does not already exist, Entrap creates an empty file with the appropriate name when the filter command executes. When using multiple filters, make sure the filter being applied to a particular snapshot is compatible with the configuration file used to create the snapshot.

Each line in a filter file is formatted as follows:

PATHNAME MATCHTYPE ATTRIBUTES

PATHNAME is either the complete pathname of a file or directory, or a pathname prefix.

Valid values for MATCHTYPE are "exact" and "prefix". If MATCHTYPE equals "exact", the filtering rule applies only to the file or directory whose pathname matches PATHNAME exactly. If MATCHTYPE equals "prefix", the filtering rule applies to all files and directories whose pathnames begin with the prefix PATHNAME.

ATTRIBUTES is a string of one or more characters. Certain alphanumeric characters instruct Entrap to ignore specific attributes of a given file or directory object when comparing two snapshots. The * and ! characters are wild cards with special meaning.

Entrap supports the following characters:

i = Inode

p = Protection mode

l = Number of links

o = Owner

g = Group

s = Size (in bytes)

m = Modification time

1 = First file signature (md5 by default)

2 = Second file signature (e.g. cksum)

n = Nth file signature

* = Ignore all attribute changes, but detect file creations and deletions

! = Ignore all attribute changes as well as file creations and deletions

Compare Command

entrap compare [-fv] snapshot1 snapshot2

The compare command compares two snapshots summarizing the differences. When using the -f option, Entrap compares filtered snapshots and may not report all differences. Without the -f option, unfiltered snapshots are compared and all differences are reported. The filter command must be run before comparing filtered snapshots.

When using the -v option, Entrap produces a verbose report, which includes a list of all files that have been added, modified, or deleted. Without the -v option, Entrap produces a summary report, which only lists the number of files.

Entrap Directory Structure

By default, Entrap is installed in the /opt/entrap directory. If Entrap is installed in another directory, modify the global constant ENTRAP_DIR in ~/bin/entrap (Listing 1). In the following description, assume ~ is the default:

~/bin -- The bin directory contains the Entrap shell scripts including the top-level entrap script. The entrap script is the only program directly called.

~/etc -- The etc directory contains two subdirectories: conf and filters.

~/etc/conf -- The conf directory contains one or more Entrap configuration files. The default configuration file entrap resides here.

~/etc/filters -- The filters directory contains one subdirectory for each defined filter. The default filter entrap resides here.

~/var -- The var directory contains two subdirectories: snapshots and tmp. You may want to link ~/var to another directory such as /var/opt/entrap/var.

~/var/snapshots -- The snapshots directory contains one subdirectory for each snapshot.

~/var/tmp -- The tmp directory is where Entrap stores various temporary files that are removed when the program terminates normally. Files may remain in this directory if Entrap is interrupted unexpectedly. Since these files can be relatively large, Entrap attempts to clean up old temporary files the next time the program executes.

~/man -- The man directory contains the source for the man page and a README file explaining how to install the man page.

Entrap Example

To demonstrate Entrap, we'll set up a test directory tree and track changes using Entrap snapshots. We'll follow these steps:

Set the configuration file.
Create the first snapshot.
Create the filter file.
Filter the first snapshot.
Create changes to the directory tree.
Create the second snapshot.
Filter the second snapshot.
Compare the first snapshot with the second and view the changes.

Setting the Configuration File

The following describes the contents of our test directory tree /home/eds/entrapdir:

drwxrwxrwx   2 eds      people    512 Aug  5 14:41 dir1
drwxrwxrwx   2 eds      people    512 Aug 23 13:32 dir2
drwxrwxrwx   2 eds      people      0 Aug 23 13:32 dir2/johns.file
-rw-rw-rw-   1 root     other       0 Aug 24 16:15 delfile
-rw-rw-rw-   1 eds      people      3 Aug  6 10:40 file1
-rw-rw-rw-   1 eds      people      0 Aug  5 14:40 file2
-rw-rw-rw-   1 eds      people      0 Aug  5 14:40 file3

Our custom configuration file ~/etc/conf/testconfig contains one line composed of a directory pathname and alias:

/home/eds/entrapdir  hee
Create the First Snapshot

Executing:

./entrap -c testconfig snapshot testsnap1

creates the first snapshot file ~/var/snapshots/testsnap1/hee. The snapshot file is named after the alias (hee, in this case), and resides in a directory created for the snapshot name (testsnap1).

Create the Filter File

Suppose we don't want to report changes associated with editing a file. When a file's contents are modified, the file's modification time, size, and message digest typically change.

To ignore the modification time, file size, and md5 value for /home/eds/entrapdir/dir2/johns.file, set up custom filter file ~/etc/filters/testconfig/hee as such:

/home/eds/entrapdir/dir2/johns.file exact ms1

Remember that the filter file exists in a directory with the same name as the configuration file (testconfig, in this case) and is named after the alias (hee, in this case).

Filter the First Snapshot

Next, filter the first snapshot:

./entrap -c testconfig filter testsnap1

This command creates filter file hee.f (alias with an "f" extension) at the same directory level as the snapshot file.

Create Changes to the Directory Tree

Execute the following commands:

rm /home/eds/entrapdir/delfile
touch /home/eds/entrapdir/file4
vi /home/eds/entrapdir/dir2/johns.file (make some changes)

Create the Second Snapshot

Analogous to creating the first snapshot, executing:

./entrap -c testconfig snapshot testsnap2

creates the second snapshot file ~/var/snapshots/testsnap2/hee.

Filter the Second Snapshot

Similar to filtering the first snapshot, filter the second:

./entrap -c testconfig filter testsnap2
Compare the Unfiltered Snapshots
Compare the two unfiltered snapshots by executing this command:

./entrap -c testconfig compare -v testsnap1 testsnap2

Note that with the verbose option turned on, the three changes to the test directory are each listed:

Date: Thu Aug 26 14:49:56 PDT 2004
Comparing testsnap1 vs. testsnap2

Filtering is turned OFF

Checking: /home/eds/entrapdir

1 file has been added

/home/eds/entrapdir/file4

1 file has been deleted

/home/eds/entrapdir/delfile

2 files have been modified

/home/eds/entrapdir
      snapshot: testsnap1                         testsnap2
      last modified: Aug-26-14:47                 Aug-26-14:49
      /usr/bin/md5: 393e1248f538cb7f30ec043fa4    93b7b067d94b \
                                                  574e579cb2a35f

/home/eds/entrapdir/dir2/johns.file
      snapshot: testsnap1                         testsnap2
      size in bytes: 0                            14
      last modified: Aug-26-14:47                 Aug-26-14:49
      /usr/bin/md5: d98f00b204e9800998ecf8427e    85c97ff7b0af3 \
                                                  ab020575f3e77

Compare the Filtered Snapshots

To ignore attributes that changed as a result of editing johns.file, execute the following compare with the filtered (-f) option:

./entrap -c testconfig compare -fv testsnap1 testsnap2

Program Description

The following is a high-level description of the 12 scripts in the ~/entrap/bin directory:

entrap, Listing 1 -- This script is the driver that calls the other Entrap scripts. Several environment variables are set, the existence of important files and directories is verified, and the user is prompted to create certain directories if they do not exist. If an invalid operand or no operand is supplied, a usage message prints and the program terminates.

snapshot, Listing 2 -- This script creates a snapshot file. If the snapshot already exists, it is overwritten if the -o flag is used, otherwise the program terminates.

compare, Listing 3 -- This script compares two separate snapshots and generates a report.

filter, Listing 4 -- This script creates a filtered snapshot file. This filtered file is used during a filtered compare.

filter.sh, Listing 5 -- This script applies filtering rules to data read from standard input. In this case, the standard input is the snapshot file to be filtered.

attributes.sh, Listing 6 -- This script uses the ls -ild command to record a file's attributes: inode number, number of links, owner, group, file size, and modification date.

The script also creates a file's message digest signature. Entrap is designed to handle an arbitrary number of file signatures. Four signatures are illustrated, but three are commented out:

#       Signature 1: /usr/local/bin/md5
        /usr/local/bin/md5 $1 ...

#       Signature 2: /usr/bin/sum
#       /usr/bin/sum $1 ...

#       Signature 3: /usr/ucb/sum
#       /usr/ucb/sum $1 ...

#       Signature 4: /usr/bin/cksum
#       /usr/bin/cksum $1 ...

By default, Entrap uses only the md5 message digest command. Creating file signatures is time consuming and resource intensive. A clever hacker might spoof one of the message digest commands so a neurotic systems administrator might consider using more than one. Simply uncomment your signatures of choice in the attributes.sh script. Also, if you want to change or add more signature commands, review the instructions in the script.

list_added.sh, Listing 7 -- This script takes a file generated by the compare command and lists the files that were added.

list_changed.sh, Listing 8 -- This script takes a file generated by the compare command and lists the files that were modified.

list_deleted.sh, Listing 9 -- This script takes a file generated by the compare command and lists the files that were deleted.

count_added.sh, Listing 10 -- This script takes a file generated by the compare command, and counts the number of files that were added.

count_changed.sh, Listing 11 -- This script takes a file generated by the compare command, and counts the number of files that were changed.

count_deleted.sh, Listing 12 -- This script takes a file generated by the compare command, and counts the number of files that were deleted.

What's in the tarball

The tarball (http://www.samag.com/code/) contains all source code, documentation, and configuration files found in the /opt/entrap directory. The tarball was created relative to the entrap subdirectory to ease movement. To move Entrap to a different location, simply change the ENTRAP_DIR variable in the file ~/bin/entrap.

Conclusion

Is Entrap the epitome of a file-integrity checking process? Certainly not; consider these possible enhancements:

Add support for spaces in file and directory names.
Provide a method for updating an existing snapshot.
Eliminate the "ls -ild" dependency. Rewriting ~/bin/attributes.sh in "C" could improve Entrap's portability.

While Entrap works on our Solaris 7 system, it might not on some other Unix variant. If you have specific questions concerning portability or if something in the code requires clarification, email us at: sascripts@yahoo.com.

References

Frisch, Æleen. Essential System Administration. Sebastopol, CA: O'Reilly & Associates, 1995.

Resources

Tripwire -- http://www.tripwire.com

AIDE (Advanced Intrusion Detection Environment) --http://sourceforge.net/projects/aide

ICU (Integrity Checking Utility) -- http://www.algonet.se/~nitzer/ICU/

Osirus (a file integrity verification system) -- http://www.shmoo.com/osiris/

Samhain (a file integrity and intrusion detection tool) --http://samhain.sourceforge.net/

md5 (file signature utility) -- http://www.fourmilab.ch/md5/

The Solaris Fingerprint Database --http://www.sun.com/blueprints/0501/Fingerprint.pdf

CERT: List of Security Tools -- http://www.cert.org/tech_tips/security_tools.html

CIAC System Monitoring Tools -- http://ciac.llnl.gov/ciac/ToolsUnixSysMon.html

John Spurgeon is a software developer and systems administrator for Intel's Factory Information Control Systems, IFICS, in Aloha, Oregon. Outside of work, he enjoys turfgrass management, triathlons, and spending time with his family.

Ed Schaefer is a frequent contributor to Sys Admin. He is a software developer and DBA for Intel's Factory Information Control Systems, IFICS, in Aloha, Oregon. Ed also edits the monthly Shell Corner column on UnixReview.com. He can be reached at: shellcorner@comcast.net.