Integrating Nessus with MySQL

Tony Howlett and Lorell Hathcock

Nessus is an open source, network-based vulnerability scanner available for the Linux platform. It is based on a client-server architecture with the server running on UNIX and Linux, with UNIX, Linux, and Windows clients available. If you are unfamiliar with vulnerability scanners, they are software packages that interrogate a machine over the network and determine whether it is vulnerable to any number of security holes. Since the number of exploits grows daily, and keeping up with the patches is a never-ending task, a vulnerability scanner can be a quick way to determine whether there are any vulnerabilities in your systems. Nessus has appeared in previous Sys Admin articles. See "Nessus -- A Powerful, Free Remote Security Scanner," by Alan Laudicina, May 2002 (http://www.samag.com/documents/s=7121/sam0205c/), or "Proactively Protecting VPNs with Nessus," by Edward L. Haletky, March 2003 (http://www.samag.com/documents/s=7835/sam0303e/).

There are commercial vulnerability scanners available, including Internet Scanner by ISS, STAT Analyser by Harris, and Retina by eEye. Nessus offers a robust, open source vulnerability scanner. Besides being free, it is modifiable and extendable, as we will demonstrate in this article.

The plug-in architecture of Nessus makes it easy to add new security checks using its own built-in scripting language. However, like many open source projects, it falls down in comparison to its pricier brethren on the output side. Although Nessus produces some decent-looking HTML reports and offers a fair number of export formats, analyzing the results of a scan, especially on large networks, can be daunting. A single Nessus scan of a medium-size network (200-300 hosts) can generate hundreds of pages of output. The ability to compare past scans and track your network's status over time is very difficult with standard Nessus reports. While some of us like the potential of being able to create our own front ends, many overworked and understaffed IT managers would prefer output that's ready for management presentation. Furthermore, many consultants and contractors who track multiple customers and networks use Nessus, and the ability to easily sort and browse scan data would be useful to them.

Nessus and MySQL

Mods and extensions are continually being created for Nessus, and there are several projects working on Nessus-to-database integration. There are now some new tools to integrate your Nessus data with a MySQL database using PHP. In this article, we will describe how to create and populate a MySQL database with your Nessus results.

We looked for tools that would use PHP and MySQL on Apache Web servers. MySQL is the most logical database to use with Nessus for several reasons. It's also open source and is the most common database that open source tools talk to. Because it is an SQL-based database, it's fairly easy to convert its records to other formats, and it comes with most Linux distributions. PHP makes the most sense for the interfacing language because it, too, is open source and offers a fairly simple scripting language.

Nessus PHP Interface (NPI)

We found two open source projects trying to integrate Nessus into a MySQL database, one called "Nessquik" and another called "Nessus PHP Interface". There are also several commercial projects, including "Java Nessus Report Manager" by NetShroud Technologies. We chose the Nessus PHP Interface because it seemed to be more functional and was written under GPL so there are no issues with using it or modifying it. The Nessus PHP Interface (NPI) was designed by Kristofer T. Karasof and released under the GPL. It can be obtained at:

http://enterprise.bidmc.harvard.edu/pub/nessus-php/

We loaded NPI on a Sun Cobalt RaQ 4r running Cobalt OS 6.0, which is basically Red Hat Linux with some vendor-specific modifications. None of those changes should matter for purposes of this install; it looks and feels just like Red Hat. Our Web server was Apache 1.3.20 running PHP version 4.1.2. We also installed a fairly recent version of MySQL (3.23.27) and Nessus 1.2 to generate the scans we imported. Obviously, your mileage may vary with different configurations.

Before discussing installation, we'll cover some of the elements described in this article. Figure 1 shows the different servers/clients that interact to produce the end result, which is a searchable Nessus scan database. The diagram shows the logical parts as separate entities, though they could certainly exist all on one physical server.

To begin, determine which target host or network needs scanning. The Nessus server will do the actual security scanning. The Nessus client software initiates and controls the scan, and the end results are stored on the client, not the server. You will also need a Web server running PHP and a server running MySQL. Again, these can be two separate boxes or one. The configuration of the MySQL settings in Nessus PHP differs depending on which way you choose to go. Finally, you have a Web browser that will be logging into the Web server to run queries against the database.

Installation

After downloading and unzipping the contents of the file, you'll find a pretty standard set of files. The INSTALL file is well written and contains all the necessary directions, with a few exceptions. The instructions give seven steps to installation, assuming you already have all the above elements up and running. We will go through these seven steps, only noting additional or helpful information for each stage, rather than regurgitating the INSTALL file.

1. Because it's written in PHP or shell scripts, unpacking the distribution didn't involve the extra steps of a compiled program, which was nice. Make sure you create a separate subdirectory in a commonly known place and copy all the files from the distribution there.

2. Follow the instructions for editing the "nsr-php" script with your MySQL information. One of the useful variables here is $your_domain, which takes the domain name listed and strips it out of any output. This is nice if you want to take out full domain name listings for purposes of security or readability.

3. Edit the "nsr" script. There are a few variables to edit here similar to the ones in the script above. However, we also had to edit the nessusphp.inc, which is an include called from "nsr-php" (which wasn't mentioned in the docs).

4. When creating the "Nessus" database, you can replace "Nessus" with whatever you want to name the database. For our purposes, we created different databases for each customer so that none of the customer data could get mixed up. If you are using NPI for a single entity, you will probably want to use one large database so you can run summary reports for the whole enterprise. The script didn't create the password for our database user so we had to set it manually. (See the "Creating a Password with MySQL" sidebar.)

5. Add the Web interface to the Web server directory from which you want to run your NPI. Then copy the Web tree included with the program files to the directory from which you want the Web interface page to run. We suggest adding .htaccess protection or another form-based login right away for this directory. Protection is important on this page because you don't want your scan information available to the whole world. Also, when creating the directories, we found that we had to chmod the directory contents to 775 rather than the 640, as the author listed. Be sure you are in that directory and not chmoding it from above, which will just give that directory those properties and cause the program to bomb.

6. Upon successful completion of the previous steps, you can populate your Nessus database with scan data. If you don't already have scan data, you will need to run some scans, making sure to save the file in the native .nbe format. Note that if you have historical scans in the older .nsr format, NPI will still convert that data properly. Also note that you must have the file on the server where the PHP will run. If the files are on different machines from where your Nessus client runs, you will need to either copy them over manually or write a short script to automatically copy them up to your Web server. A crontab that regularly runs the PHP conversion script on the contents of that directory might also be a good idea. Run the conversion script, making sure you are in the same directory as your Nessus results file. If it works correctly, you should see a listing of each element as it gets imported.

One downside of the import function is that importing multiple scan files from the same day will cause previous ones to be overwritten. This isn't a downfall of the script, but rather an innate issue with Nessus as it only tracks dates and not the time of each item.

7. You will now be able to browse your scan data as with any other database -- sorting, and running queries to separate out specific vulnerabilities or hosts or other criteria.

There was a small bug in the interface in our installation. When you select a category of tests, it automatically enters a specific test in the next field so that you are searching for results on a specific test. Generally we wanted to search by port or by host for all vulnerabilities. However, the "Reset" control did not seem to erase this entry, and we had to manually erase it in order for our queries to work properly. It would be nice to be able to pull down a list of all the tests, but that feature didn't seem to be working in our installation.

Other than that, the basic interface that Kris wrote is nice. We do, however, have a few items on our wish list, one of which we coded ourselves and describe below. Because it's in PHP, it's relatively easy to modify the interface and the controls for your own use. However, because it's a GPL program, any changes you make must be GPL'ed as well, and because all the data is in a database, you can easily write custom queries.

The NPI Date-of-Scan Pull-Down Patch

We wanted to be able to view each finite scan by date and flip between them to see the results over time so that we could tell whether the client's network security was improving or worsening on a relative basis. We could have used the date entry fields, but that was a little clunky for us since we wouldn't know the dates of the scans and we didn't want to enter them. We wanted it to automatically display the data for the latest scan and give us easy access to the other ones.

Because the date of scan is stored for each record, we could easily write a query to sort it by date. We took index.php, which is the PHP script that creates the index file on the front page. We noticed the query that creates the category results basically did the same thing, so we modified it and inserted it, using the same code for creating the pull-down object that was used for the other entry fields. Listing 1 shows this code. Obviously, with a little more work, this could be done for any of the entry fields. Using the same logic, you can redo the interface to your own specifications or needs.

Conclusion

Thanks to Kris for creating NPI, which is a great way to import your Nessus data into a database and view it. It is also a great start for users looking to create a custom control panel to better analyze and manage their scan data. Enjoy using it and happy coding to all you Nessus users out there who are probably already at work creating improvements and additions to it. We eagerly look forward to the next version.

Tony Howlett is President of Network Security Services, a network consulting firm (www.netsecuritysvcs.com). He was previously a founder and CTO of a regional CLEC/ISP. He holds the CISSP and GSNA titles as well as a BBA in MIS.

Lorell Hathcock is an IT Manager with more than 10 years experience in Internet, telecom, network security, and database technologies. He currently hails from Houston, TX where he lives happily with his wife and four kids.