Security Forensics Using DTrace

Boris Loza, PhD

Solaris 10 has introduced a new tool for Dynamic Tracing in the OS environment -- dtrace. This is a very powerful tool that allows systems administrators to observe and debug the OS behavior or even to dynamically modify the kernel. Although this tool has been designed primarily for developers and administrators, in this article, I will explain how to use dtrace as a security forensics tool for analyzing suspicious files and processes.

Using DTrace

Dtrace has its own C/C++-like programming language called "D language" and comes with many different options. Dtrace is easy to use, once you familiarize yourself with D language and know a little bit about Solaris internals. For example, the following construction using dtrace could serve as a "primitive" strace command for the process ID 510:

# dtrace -n syscall:::'/pid == 510/ {}'

I will explain the syntax of such constructions in a moment.

To understand how to use dtrace as a security forensic tool, I will present a case study, as follows. Let's assume that we are going to investigate the process ID 968 from the suspicious "srg" application that we found running on our system.

Typing the following at the command line will list all files that this process opens at the time of monitoring. We'll let this run for a while and terminate with Control-C:

# dtrace -n syscall::open:entry'/pid == 968/
{ printf("%s%s",execname,copyinstr(arg0)); }'

dtrace: description 'syscall::open*:entry' matched 2 probes
^C
CPU     ID                    FUNCTION:NAME
  0     14                    open:entry srg /var/ld/ld.config
  0     14                    open:entry srg /lib/libdhcputil.so.1
  0     14                    open:entry srg /lib/libsocket.so.1
  0     14                    open:entry srg /lib/libnsl.so.1

D language comes with its own terminology, which I will address here briefly. The whole 'syscall::open:entry' construction is called a "probe" and defines a location or activity to which dtrace binds a request to perform a set of "actions". The 'syscall' element of the probe is called a "provider", and, in this case, permits probes on 'entry' (start) to any 'open' Solaris system call ('open' syscall is sent to file when it is about to be opened). The so-called "predicate" (/pid == 968/) uses the predefined dtrace variable 'pid', which always evaluates to the process ID associated with the thread that fired the corresponding probe.

The 'execname' and 'copyinstr(arg0)' are called "actions" and define the name of the current process executable file and convert the first integer argument of the system call (arg0) into a string format, respectively. The printf's action uses the same syntax as in C language and serves the same purpose -- to format the output.

Each D program consists of a series of clauses, with each clause describing one or more probes to enable and an optional set of actions to perform when the probe fires. The actions are listed as a series of statements enclosed in curly brackets { } following the probe name. Each statement ends with a semicolon (;).

In general, each probe clause has the general form:

probe descriptions:
/ predicate /
{
action statements
}

You may want to read the dtrace(1M) man pages and the Introduction to the Solaris Dynamic Tracing Guide at:

http://docs.sun.com/app/docs/doc/817-6223

for more options and further explanation of the syntax.

As the name suggests, the dtrace (Dynamic Trace) utility will show you the information about a changing process -- dynamically. That is, if the process is idle (doesn't do any system calls or opens new files), you won't be able to get any information. To analyze such a process, either restart it or use "static" methods and utilities, such as mdb(1), or process analyzing commands, such as pfiles(1).

Next, we will use the following command-line construction to list all system calls for "srg". Again, we'll let this run for a while and terminate it with Control-C:

# dtrace -n 'syscall:::entry /execname == "srg"/ \
  { @num[probefunc] = count(); }'
dtrace: description 'syscall:::entry ' matched 226 probes
^C
  pollsys                                                   1
  getrlimit                                                 1
  connect                                                   1
  setsockopt                                                1
...

You may recognize some of the building elements of this small D program. Additionally, this clause defines an array named 'num' and assigns the appropriate member 'probefunc' (executed system call's function) the number of times these particular functions have been called (count()).

Using dtrace, we can easily emulate utilities traditionally being used to analyze suspicious binary files and processes. However, dtrace is a much more powerful tool and may provide more functionality; for example, you can dynamically monitor the stack of the process in question:

# dtrace -n 'syscall:::entry/execname == "srg"/{ustack()}'
  0    286                lwp_sigmask:entry
              libc.so.1'__systemcall6+0x20
              libc.so.1'pthread_sigmask+0x1b4
              libc.so.1'sigprocmask+0x20
              srg'srg_alarm+0x134
              srg'scan+0x400
              srg'net_read+0xc4
              srg'main+0xabc
              srg'_start+0x108

Based on all our investigation (see the list of opened files, syscalls, and the stack examination above), we may positively conclude that srg is a network-based application. Does it write to the network? Let's check by constructing the following clause:

# dtrace -n 'mib:ip::/execname == "srg"/{@[execname]=count()}'
dtrace: description 'mib:ip::' matched 412 probes
dtrace: aggregation size lowered to 2m
^C
  srg                                      520

It does. We used a 'mib' provider to find out whether this application transmits to the network.

Could it be just a sniffer or a netcat-like application that is bounded to a specific port? Let's run dtrace in the truss(1) fashion to answer this question (inspired by Brendan Gregg's dtruss utility from users.tpg.com.au/adsln4yb/DTrace/shellsnoop):

#!/usr/bin/sh
#
dtrace='

 inline string cmd_name = "'$1'";
 /*
 **  Save syscall entry info
 */
 syscall:::entry
 /execname == cmd_name/
 {
      /* set start details */
      self->start = timestamp;
      self->arg0 = arg0;
      self->arg1 = arg1;
      self->arg2 = arg2;
 }

/* Print data */
 syscall::write:return,
 syscall::pwrite:return,
 syscall::*read*:return
 /self->start/
 {
    printf("%s(0x%X, \"%S\", 0x%X)\t\t = %d\n",probefunc,self->arg0,
    stringof(copyin(self->arg1,self->arg2)),self->arg2,(int)arg0);
         
      self->arg0 = arg0;
      self->arg1 = arg1;
      self->arg2 = arg2;

 }
'
# Run dtrace
        /usr/sbin/dtrace -x evaltime=exec -n "$dtrace" >&2

Save it as truss.d, change the permissions to executable, and run:

# ./truss.d srg
0     13       write:return write(0x1, "    sol10 -> 192.168.2.119 
  TCP D=3138 S=22 Ack=713701289 Seq=3755926338 Len=0 
  Win=49640\n8741 Len=52 Win=16792\n\0", 0x5B)            = 91
0     13       0     13                     write:return write(0x1, 
  "192.168.2.111 -> 192.168.2.1  UDP D=1900 S=21405 LEN=140\n\0", 
  0x39)          = 57
^C

Looks like a sniffer to me, with probably some remote logging (remember the network transmission by ./srg discovered by the 'mib' provider above). You can write pretty sophisticated programs for dtrace using D language. Take a look at /usr/demo/dtrace for some examples.

You may also use dtrace for other forensic activities. The following is an example of a more complex script that allows monitoring of who fires the suspicious application and records all the files opened in the process:

#!/usr/bin/sh

command=$1

/usr/sbin/dtrace -n '

inline string COMMAND  = "'$command'";

 #pragma D option quiet
 
 /*
 **  Print header
 */
 dtrace:::BEGIN
 {
     /* print headers */
     printf("%-20s %5s %5s %5s %s\n","START_TIME","UID","PID", \
     "PPID","ARGS");
 }

 /*
 **  Print exec event
 */
 syscall::exec:return, syscall::exece:return
 /(COMMAND == execname)/
 {
        /* print data */
         printf("%-20Y %5d %5d %5d %s\n",walltimestamp,uid,pid,ppid,
         stringof(curpsinfo->pr_psargs));
        s_pid = pid;
 }
/*
 **  Print open files
 */
 syscall::open*:entry
/pid == s_pid/
 {
        printf("%s\n",copyinstr(arg0));
 }
'

Save this script as a wait.d, change the permissions to executable 'chmod +x wait.d' and run:

# ./wait.d srg
START_TIME             UID   PID  PPID ARGS
2005 May 16 19:51:20     0  1582  1458 ./srg
/var/ld/ld.config
/lib/libnsl.so.1
/lib/libsocket.so.1
/lib/libresolv.so.2
...
^C

Once the srg is started, you will see the output.

The real power of dtrace comes from the fact that you can do things with it that otherwise aren't possible without writing a comprehensive C program. For example, the shellsnoop application written by Brendan Gregg allows you to use dtrace at the capacity of ttywatcher; see:

http://users.tpg.com.au/adsln4yb/DTrace/shellsnoop

It is not possible to show all the capabilities of dtrace in this article. Dtrace is a very powerful as well complex tool with virtually endless capabilities. Although Sun insists that you don't need a "deep understanding of the kernel for DTrace to be useful", knowledge of Solaris internals is a real asset. Looking at the include files in /usr/include/sys/ directory may help you write complex D scripts and give you more of an understanding of how Solaris 10 is implemented.

Conclusion

When monitoring your systems, be creative and observant. Apply all your knowledge and experience for analyzing suspicious binary files and processes. Also, be patient, have a sense of humor, and learn how to use dtrace.

Boris Loza, PhD, CISSP is an author of UNIX, Solaris and Linux: A Practical Security Cookbook, which deals with securing UNIX OS without any third-party applications. Boris is also a contributor to several industry magazines and has been quoted in many information security books and Web sites. He loves nature, reading books, watching movies, and enjoys scuba diving and entomology.