Intrusion Prevention and Active Response: Implementing an Open Source Defense

Angela Orebaugh and Eric Cole

The term "intrusion prevention" has become prevalent in marketing materials and sales presentations as commercial vendors develop an abundance of products (both good and bad) under this umbrella term. While commercial intrusion prevention products are often technologically diverse and contain a rich feature set, they also often come with a hefty price tag.

This article will describe free, open source alternatives for implementing intrusion prevention systems. It looks at intrusion prevention from a defense in depth approach, including not only network methods but also system and application methods.

Introduction

Traditionally, networks and systems have been protected by perimeter defense methods such as routers with access control lists (ACLs) and firewalls. Network and system intrusion attempts and attacks have been detected by network-based intrusion detection systems (NIDS) and host-based intrusion detection systems (HIDS). These types of devices continue to provide a strong and robust defense in depth strategy; however, advances in technology have made it possible for these technologies to work together. Intrusion prevention systems (IPS) combine the features of a firewall and IDS not only to detect attacks but, more importantly, to prevent them.

One important distinction to make is the difference between intrusion prevention and active response. "Active response is the dynamic reconfiguration or alteration of network access control mechanisms, sessions, or even individual packets based on alerts generated from an IDS." (Snort 2.1 Intrusion Detection, 2nd Ed., Syngress). Active response happens after the event has occurred, thus a single packet attack will be successful on the first attempt and blocked in future attempts.

While active response devices are beneficial, this one aspect makes them unsuitable for an overall perimeter solution. Intrusion prevention devices are typically inline devices on the network that inspect packets and make decisions before forwarding them on to the destination. This type of device has the ability to defend against single packet attacks on the first attempt by blocking or modifying the attack packet. There are several methods of intrusion prevention technologies:

• System memory and process protection -- This type of intrusion prevention strategy resides at the system level. Memory protection consists of a mechanism to prevent a process from corrupting the memory of another process running on the same system. Process protection consists of a mechanism for monitoring process execution, with the ability to kill processes that are suspect attacks.
• Session sniping -- This type of intrusion prevention strategy terminates a TCP session by sending a TCP RST packet to both ends of the connection. When an attempted attack is detected, the TCP RST is sent and the attempted exploit is flushed from the buffers and thereby prevented. Note that the TCP RST packets must have the correct sequence and acknowledge numbers to be effective.
• Gateway interaction devices -- This type of intrusion prevention strategy allows a detection device to dynamically interact with network gateway devices such as a router or firewall. When an attempted attack is detected, the detection device can direct the router or firewall to block the attack.
• Inline network devices -- This type of intrusion prevention strategy places a network device directly in the path of network communications that has the capability to modify and block attack packets as they traverse the device's interfaces. This acts much like a router or firewall combined with the signature-matching capabilities of an IDS. The detection and response happens in real time before the packet is passed on to the destination network.

Although there are many methods of intrusion prevention and active response, using a combination of methods to build a strong defense in depth strategy is still the best approach.

Implications of Intrusion Prevention and Active Response

There are several deployment risks when deploying intrusion prevention and active response technologies. Most notable is the recurring issue of false positives in today's intrusion detection systems. On some occasions, legitimate traffic will display some attack characteristics similar to those of malicious traffic. This could be anything from inadvertently matching signatures to uncharacteristic, high-volume traffic. Even a finely tuned IDS can present false positives when this occurs.

When intrusion prevention and active response are involved, false positives can create a denial of service (DoS) condition for legitimate traffic. Additionally, attackers who discover or suspect the use of intrusion prevention methods can purposely create a DoS attack against legitimate networks and sources by sending attacks with spoofed source IP addresses. A simple mitigation to some DoS conditions is the use of an exclude list, also called a whitelist. A whitelist contains a list of the network sources that should never be blocked. It is important to include systems such as DNS, mail, routers, and firewalls in the whitelist.

Session sniping system identification is another concern when deploying IPSs. When systems terminate sessions with RST packets, an attacker may be able to discover not only that an IPS is involved, but also the type of underlying system. Readily available passive operating system identification tools, such as p0f, analyze packets to determine the underlying operating system. This type of information allows an attacker to potentially evade the IPS or direct an attack at the IPS.

Another risk with IPSs involves gateway interaction timing and race conditions. In this scenario, a detection device directs a router or firewall to block the attempted attack. However, due to network latency, the attack has already passed the gateway device before it received this direction from the detection device. This could be combined with another scenario that creates a race condition on the gateway device between the attack and IPS. In either case, the attack has a high chance of succeeding.

When deploying an IPS, you should carefully monitor and tune your systems and be aware of the risks involved. You should also have an in-depth understanding of your network, its traffic, and both its normal and abnormal characteristics. It is always recommended to run IPS and active response technologies in test mode for a while to thoroughly understand their behavior.

Snort Flexible Response Plug-in

Snort can perform session sniping through its flexible response plug-in. This plug-in adds the response and react keywords to rule creation. When a rule is triggered, the appropriate action is taken based on the keywords. If you are using Snort in stealth mode, you will need an additional interface to send the responses. Also, make sure that the libnet library is installed because it is used to create and send packets on the network.

To configure Snort with the flexible response plug-in, use the ./configure --enable-flexresp option when building Snort. The response keyword uses the following format:

resp: <resp_mechanism>[,<resp_mechanism>[,<resp_mechanism>]...];

rst_snd -- Sends a TCP RST packet to the sender of the packet.
rst_rcv -- Sends a TCP RST packet to the receiver of the packet.
rst_all -- Sends a TCP RST packet to both the sender and receiver.
icmp_net -- Sends an ICMP_NET_UNREACH message to the sender.
icmp_host -- Sends an ICMP_HOST_UNREACH message to the sender.
icmp_port -- Sends an ICMP_PORT_UNREACH message to the sender.
icmp_all -- Sends all three ICMP messages to the sender.

The following example attempts to block telnet by resetting any TCP connection to port 23:

alert tcp any any -> any 23 (msg:"Attempted Telnet"; flags:S; resp:rst_all;)

react: <react_mechanism>[,<react_mechanism>[,<react_mechanism>]...];

block -- Blocks connection access.
warn -- Sends a warning notice to the user's Web browser.
msg -- Includes the msg option text as the warning notice.
proxy: <port_number> -- Specifies a proxy port number on which to send the warning notice.

The following example attempts to send a warning notice and block Web access to sites that contain the string "mp3":

alert tcp any any <> 192.168.1.0/24 80 (content: "mp3"; \
    msg: "Don't download illegal music!"; react: block, msg;)

SnortSam

SnortSam is an active response plug-in for Snort that performs gateway interaction with various router and firewall devices. SnortSam acts at the network layer by instructing the gateway to alter or block traffic for specified amounts of time based on IP address. SnortSam consists of two parts: an intelligent agent that runs on the gateway device and accepts commands, and an output plug-in for Snort that sends commands based on triggered rules. The communication between the output plug-in and agent is secured by an encrypted TCP session. SnortSam supports the following gateways:

• Checkpoint Firewall-1
• Cisco PIX firewall
• Cisco routers
• Netscreen/Juniper firewalls
• IPFilter (ipf)
• OpenBSD's Packet Filter (pf)
• Linux IPchains
• Linux IPtables
• Linux EBtables
• WatchGuard Firebox firewalls

The SnortSam agent provides several features including:

• The ability to specify a whitelist of IP addresses that will never be blocked.
• The ability to provide per-rule blocking and time interval.
• The ability to prevent repetitive blocking of the same IP address.
• Twofish encrypted sessions between Snort and SnortSam.
• The ability to multi-thread for faster processing and simultaneous blocking on multiple devices.
• The ability to log events and send email notification.
• The ability to scale to larger distributed networks using a client/server architecture.

You must download and install both the SnortSam agent and the Snort plug-in patch. Once they are installed, you will need to add the output plug-in command for alert_fwsam in the following format to the snort.conf file on the Snort IDS:

output alert_fwsam: <SnortSam server IP address:<port>/<pre-shared key>

output alert_fwsam: 192.168.1.1/str0ngpassw0rd

fwsam: <host to block>, <duration>

The following example blocks the source address for 1 hour:

alert tcp any any -> any 23 (msg:"Attempted Telnet"; flags:S; fwsam: src, 1 hour;)

The SnortSam agent is configured using a simple text file called snortsam.conf. Configuration options include the firewall interaction parameters, the encryption key (once again in cleartext), the Snort sensors to receive commands from, the whitelist, the logging path, the port number to listen on, and the IP address the agent should listen on. The README.conf file contains a complete listing of configuration file parameters. The following is an example snortsam.conf file:

accept 192.168.10.1, str0ngpassw0rd
bindip 192.168.1.1
dontblock 192.168.0.0/16
logfile /var/log/snortsam.log
iptables eth0

/usr/sbin/snortsam /etc/snortsam.conf

Fwsnort

Fwsnort functions as a transport layer inline IPS, because it is deployed directly within the iptables firewall. It works by translating Snort signatures into their equivalent iptables rulesets; hence, it will only stop attacks for which there are Snort signatures. Not all Snort rules are easily translated, but fwsnort does a good job at translating about 70% of them. Fwsnort also accepts Snort rules by the SID value, so you can add specific rules to your iptables ruleset. Iptables can then either log or block the attacks.

Before installing fwsnort, you must install the iptables string match kernel patch and then recompile the kernel. Once fwsnort is installed, it references the configuration file /etc/fwsnort/fwsnort.conf. You must make some initial changes to this file to assign firewall interfaces. The configuration file also contains areas for whitelists where you can exclude host and networks from blocking. Use the following command to run fwsnort with active response:

fwsnort --ipt-reject

fwsnort --snort-sid 301 --ipt-reject

Fwsnort is not bulletproof. Since it uses string matching, it can easily be evaded with simple evasion techniques such as fragmentation, URL encoding, and session splicing. It is still a good tool to use as part of your defense in depth strategy, however.

Snort Inline

Snort Inline is a true IPS that is deployed between network segments with the capability to alter or drop packets in real time as they flow through the system. It runs on a Linux system and uses iptables packet queuing to collect and make decisions about packets as they traverse the system's interfaces. It can also be used in stealth mode as a bridge between network segments, so it will not be detected as a hop in the network. One of the best features of Snort Inline is its ability to mitigate attacks by altering application layer data as the packet traverses the system.

Snort Inline installation requires several specific versions of utilities and a kernel patch. Once it is installed, there are several configuration steps that must be made including configuring iptables and Snort. Snort Inline adds three new rule actions for rules:

drop -- Drops the packet using iptables and logs via Snort.

reject -- The communication is closed either by TCP RST for TCP sessions or by an ICMP port-unreachable message for UDP and dropped by iptables and logged.

sdrop -- Drops the packet using iptables but does not log it.

Snort Inline also adds a new rule option:

replace -- Substitutes text matched by the content keyword with text specified by the replace keyword (the new data must be the same length as the original data).

Use these new keywords to modify Snort rules accordingly. Snortconfig is a utility that allows you to easily update your rules to work with Snort Inline. Snort Inline does not include whitelist functionality. However, you can achieve the same thing by configuring the EXTERNAL_NET variable in the snort.conf file. For example, if your network has hosts, such as DNS and mail, with external addresses on 10.11.11.0/24 and you never want them blocked, use the following:

var EXTERNAL_NET !10.11.11.0/24

Hogwash

Hogwash is a Gateway IDS and packet scrubber based on the Snort IDS that can detect attacks and filter them out. It is still in development and has mostly been superseded by Snort Inline; however, it is still in use and worth mentioning.

Hogwash works in three different modes: IDS, inline packet scrubber, and honeypot. In IDS mode, it can detect attack traffic and send TCP resets to end sessions. In inline packet scrubber mode, it can actively filter exploits from network traffic. In this mode, it can send resets, drop the packet, or modify the packet as it traverses the system. The honeypot mode is still experimental; however, the concept is that Hogwash can route attackers to one of several honeypots that are behind the Hogwash system while each of these honeypots can have the same IP and MAC address. Hogwash also has the ability to perform multi-packet signature matching and port-scan detection.

Hogwash runs on a Linux system that can be transparently connected to the network. It has the capability of managing up to 16 different interfaces, thus protecting several network segments with a single system. Hogwash handles the packet forwarding for each network segment, so remember to disable the kernel IP forwarding.

When running Hogwash, the only required arguments are the configuration file and the rules file:

hogwash -c "config file" -r "rules file" [options]

-t -- Parse rules and exit.
-n "count" -- Process count packets and exit.
-l "log dir" -- Write logs to specified directory.
-d -- Run in daemon mode.

The config file contains information specific to your network and setup. The rules file contains all of the Snort rules, modified accordingly. Hogwash adds the following new keywords to Snort rules:

drop -- Drops the packet, sends a TCP reset and logs the packet.
ignore -- Drops the packet without sending a TCP reset.
sdrop -- Drops the packet, sends a TCP reset, and does not log the packet.
replace -- This keyword tells Hogwash to replace a detected string with another string.

Hogwash also uses a Stackless Control Protocol to remotely control the Hogwash system. The transactions are secured with either Twofish or AES encryption. Remote actions that can be performed include pinging, gathering statistics, and transferring files.

Modsecurity

Modsecurity is an Apache Web server module that acts as an intrusion detection and prevention engine for Web applications. It increases Web application security by protecting applications from both known and unknown attacks. Modsecurity sits inline between the Web client and server to detect attacks. If it identifies a potential attack, it can reject the request or perform any number of built-in active responses. Modsecurity integrates with the Web server and provides the following features:

• Request filtering -- Incoming Web requests are analyzed inline before being passed to the Web server or other modules.
• Anti-evasion techniques -- Paths and parameters are normalized before analysis takes place. This includes removing multiple forward slash characters (//), treating backslash and forward slash characters equally (Windows only), removing directory self-references (./), removing null characters (%00), and decoding URL encoded characters.
• Understanding of the HTTP protocol -- The engine has a deep understanding of the HTTP protocol, allowing it to perform very specific and granulated filtering.
• POST payload analysis -- The engine will intercept and analyze POST methods contents.
• Audit logging -- All requests are logged in full detail for later analysis.
• HTTPS filtering -- The engine can operate with encrypted sessions because it has access to the request data after decryption occurs.
• Built-in checks -- Other special built-in checks include URL-encoding validation, Unicode-encoding validation, and byte-range verification to detect and reject shellcode.
• Rule support -- Modsecurity also supports any number of custom rules for attack detection and prevention. These rules are formed using regular expressions. Negated rules are also supported.

Modsecurity rules can analyze headers, cookies, environment variables, server variables, page variables, POST payload, and script output. Rules are configured in the following two formats:

SecFilter keyword [action]
SecFilterSelective "variable list separated with |" keyword [action]

Once rules are triggered, several actions can be taken:

Deny -- Deny the request.
Allow -- Stop the rule processesing and allow the request.
Status:nnn -- Respond with an HTTP status nnn.
Redirect:url -- Redirect the request to the absolute URL url.
Exec:cmd -- Execute the script cmd.
Log -- Log the request to the error log.
Nolog -- Do not log the request.
Pass -- Ignore the current rule match and go on to the next rule.
Pause:nnn -- Stall the request for nnn milliseconds.
Chain -- Evaluate the next rule in the chain.
Skipnext:n -- Skip the next n rules.

Modsecurity is configured by modifying the Apache http.conf file with the appropriate settings and rules. Listing 1 shows an example http.conf configuration with example rules. (Listings are available from the Sys Admin Web site: http://www.sysadminmag.com.)

Modsecurity includes a Perl script to convert Snort rules to Modsecurity rules. Snort classifies rules into Web attacks, which are converted to reject incoming requests, and Web activities, which are converted to only log the activity. The following is an example of converted Snort rules:

# WEB-ATTACKS ps command attempt
SecFilterSelective THE_REQUEST "/bin/ps"
# WEB-ATTACKS uname -a command attempt
SecFilter "uname\x20-a"
# WEB-CGI testcgi access
SecFilterSelective THE_REQUEST "/testcgi" log,pass
# WEB-IIS /iisadmpwd/aexp2.htr access
SecFilterSelective THE_REQUEST "/iisadmpwd/aexp2\.htr" log,pass
# WEB-MISC http directory traversal
SecFilter "\.\.\\"

1. Parse the request -- Modsecurity starts by parsing the request.

2. Perform canonization and anti-evasion actions -- Modsecurity performs a series of transformations on the input to make it suitable for analysis. This mitigates various evasive techniques such as: null byte attacks, self-referencing directories, multiple slash characters, using backslash characters on Windows, etc.

3. Perform special built-in checks -- Modsecurity performs more complicated validations such URL-encoding validation, Unicode-encoding validation, and byte-range verification to detect and reject shellcode.

4. Execute input rules -- Modsecurity executes custom rules that you have created using regular expressions. Additionally, several rules can be combined for more complex analysis.

5. Execute output rules -- The request is transferred to the handler where output rules are applied to the response body. They are very useful to prevent information leaks.

6. Log the request -- Modsecurity logs the complete request consisting of input and output headers, and the request body. Modsecurity can be configured to log specific requests and responses to prevent excessive logging.

The Modsecurity audit log feature captures data and logs it in text format. This is easy and convenient to work with; however, when analyzing large quantities of data, a better method is needed. Modsec2mysql is a PHP script that parses the Modsecurity log and imports the data into a MySQL database. This tool can be downloaded at:

http://www.hackinthebox.org/misc/modsec2mysql.tar.gz

LIDS

The Linux Intrusion Detection System (LIDS) is an intrusion detection and prevention system that resides within the Linux kernel. It is a security enhancement to the Linux kernel consisting of a kernel patch and some admin tools. LIDS implements mandatory access control (MAC), file protection, and process protection on the Linux system by restricting file access, network operations, raw device access, memory use and access, and I/O access. LIDS provides the administrator the ability to define and finely tune access controls. LIDS also contains a port-scan detector.

LIDS provides protection, detection, and response within the kernel of the Linux system. It provides protection in the following ways:

• Full file system protection of files and directories from unauthorized users and programs including protection from root.
• Protection of important processes from being terminated.
• Protection of RAW I/O operations from unauthorized programs including hard disk and master boot record (MBR) protection.

LIDS provides detection via the port-scan detector and by monitoring any unauthorized system activity. The port-scan detector functionality is built into the kernel. It will detect half-open scans, SYN stealth scans, FIN scans, Xmas, and Null scans. It can easily detect tools like Nmap and Nessus. The port-scan detector works with raw socket disabled, making it more secure than standard sniffers.

LIDS can provide response in the following ways:

• When a rule violation occurs, LIDS logs a detailed message about the violation to the system kernel log file, which is also protected by LIDS. LIDS logging has an anti-flooding capability.
• Sending log messages via email.
• Automatically terminating a user's session that is in violation of the rules.

The LIDS configuration file is stored in /etc/lids.conf. However this file should never be edited by hand. Managing LIDS consists of two programs: lidsadm and lidsconf. Lidsadm is the administration utility used to administer the LIDS system, including enabling and disabling LIDS, sealing the kernel, and viewing the status of LIDS. Lidsconf is used to configure the access control lists (ACLs) and set the LIDS password. Running lidsconf -h displays the help syntax for the lidsconf program. It can be used to protect files as read only or append only.

The LIDS functionality extends the existing Linux kernel "immutable" attribute by allowing you to grant or deny specific rights on a more granular basis with ACLs. You can also use the capabilities, discussed later, to remove the Linux "immutable" feature all together, thus using LIDS for all file system protection. Read-only files are protected from changes, for example /etc/passwd. On multi-user production systems, making the /etc/passwd file read only may not be practical since users will want the ability to change attributes such as their shell. The following shows the usage for protecting read only files:

lidsconf -A -o filename_to_protect -j READONLY

#lidsconf -A -o /sbin/ -j READONLY
#lidsconf -A -o /etc/passwd -j READONLY

lidsconf -A -o filename_to_protect -j APPEND

# lidsconf -A -o /var/log/message -j APPEND
# lidsconf -A -o /var/log/secure -j APPEND
# lidsconf -A -o /etc/httpd/logs/ -j APPEND
# lidsconf -A -o /var/log/httpd/ -j APPEND

# lidsconf -A -s /usr/local/bin/rotatelogs -o /var/log/message -j WRITE

CAP_LINUX_IMMUTABLE -- Protects the files and file systems from being written to when marked "immutable".
CAP_NET_ADMIN -- Prevents modification of network configurations. For example, it prevents route table entries and firewall entries from being modified.
CAP_SYS_MODULE -- Prevents insertion and removal of kernel modules.
CAP_SYS_RAWIO -- Prevents raw disk/device I/O.
CAP_SYS_ADMIN -- Prevents a large range of other systems administration functions.

Capabilities are enabled and disabled by editing the /etc/lids/lids.cap file. This file contains a list of all capabilities. If a capability has a "+" sign in front of it, it is enabled; if it has a "-" sign, it is disabled. You must reload the configuration file after making changes by using the following:

# lidsadm -S -- +RELOAD_CONF

# lidsadm -S -- +CAP_SYS_ADMIN

Grsecurity and PaX

Grsecurity is a Linux security project that uses a multi-layered detection, prevention, and containment model. It uses a Role-Based Access Control (RBAC) system that can generate least-privilege policies for the entire system. It also provides the following additional features:

• Change root (chroot) hardening
• /tmp race prevention
• Full-featured fine-grained auditing
• Address space modification protection provided by the PaX project
• Additional randomness in the TCP/IP stack and process IDs
• All alerts and audits support a feature that logs the IP address of the attacker with the log
• Restricted viewing of processes
• Integrated local attack response on all alerts

PaX is a separate project that is included in grsecurity as part of its security strategy. The PaX project researches various defenses against the exploitation of software bugs that give the attacker arbitrary read/write access to the target's address space (e.g., buffer overflows and user-supplied format string bugs). PaX doesn't focus on finding and fixing the bugs, but rather on prevention and containment of exploit techniques. Exploit techniques can affect the target at three different levels:

• Introduce or execute arbitrary code.
• Execute existing code out of original program order.
• Execute existing code in original program order with arbitrary data.

PaX is a patch for the Linux kernel that implements least-privilege protections for memory. It flags data memory as non-executable and program memory as non-writable, and randomly arranges the program memory. Prevention is implemented through PaX and hardening certain sections of the kernel. This should be used as part of your defense in depth strategy. You must monitor your log files to look for intrusion attempts reported by PaX. An execution attempt of a service could signify an attack or compromise. For effective security, these logs should be correlated with your network logs.

LAk

LAk is an open source intrusion prevention systems project. It houses a collection of programs, scripts, and whitepapers on implementing and operating an open source IPS. The goal of the project is for users to be able to easily install and run an IPS in a short amount of time. It also aims to combat the current media hype generated by commercial vendors about IPS.

LAk currently consists of a "Getting Started" guide, a list of prerequisites, and a whitepaper with installation and configuration instructions. The LAk IPS is based on iptables and Snort Inline. It assumes that iptables is installed with IP queuing enabled. LAk also assumes that Snort Inline is installed (although it includes a precompiled binary) and that the latest set of Snort signatures has been downloaded. LAk includes the following set of scripts to make the installation and configuration process more efficient:

convert-IPS.sh -- Converts Snort rules to Snort Inline rules. The latest Snort signatures must be downloaded and unzipped in the LAk installation directory. This file must be copied to the unzipped Snort rules directory.
LAk-NAT.sh -- Called by the LAk-startup.sh script to configure iptables in NAT mode.
LAk-snort_inline.sh -- Snort Inline startup script that is called by LAk-startup.sh.
LAk-startup.sh -- The main startup script that configures the network interfaces and calls the appropriate subscripts.
rc.firewall.IPS-NAT -- A configurable script used to configure iptables.
snort_inline -- A precompiled binary of Snort Inline from the Honeynet Project.
snort_inline.conf -- A configuration file for Snort Inline.

LAk IPS is a handy tool that automates the process of setting up iptables and Snort Inline; however, it has not been updated for more than a year. We hope to see more activity on this site in the future with an updated release and additional features.

PortSentry and PSAD

PortSentry was developed to detect and respond to port scans on a host. Because port scans are often the first step of the attack process, PortSentry monitors the TCP and UDP ports on a system and responds when a scan is identified. It has the ability to detect various types of scans including stealth scans.

PortSentry provides three active response choices:

• Insert a null route into the hosts routing table. This will re-route the scan from the attacker to a non-existent IP address. The disadvantage to this type of response is that it increases the size of the routing table on the host, which uses more memory. If the attacker is using random, spoofed source addresses as part of the attack, this could lead to a DoS condition on the host.
• Insert a firewall rule to block traffic from the scanning IP address. PortSentry supports ipfw, ipfilter, ipfwadm, ipchains, and iptables. When it detects a scan, PortSentry can add the appropriate rule to the firewall to block the IP address of the scanning host. Once again, this can also be used to create a DoS condition for the host or network. An attacker could spoof the source address to prevent legitimate connections.
• Add a TCP wrapper rule for the attacking IP address to the /etc/hosts.deny file. This will prevent the attacker from connecting to the target host's services. Although this protection mechanism isn't as strong, it does alleviate the potential DoS conditions from the other options.

The Port Scan Attack Detector (PSAD) runs on Linux and analyzes iptables firewall logs to detect port scans and other suspicious traffic. It is designed to function as a network IDS that uses iptables firewall log data to block and log packets. It consists of three system daemons written in Perl and C.

PSAD features alert messages that include the source, destination, scanned port range, start and end times, TCP flags and corresponding nmap options, as well as email alerting, DShield reporting, and automatic blocking of the attacking IP address via iptables. It also utilizes Snort signatures to detect backdoor scans, DDoS tools, and advanced port scans such as stealth, FIN, and Xmas.

PSAD also has the ability to passively fingerprint the remote operating system of the attacker by using TTL, IP id, TOS, and TCP window sizes. Combining PSAD with FWSnort and the iptables string match extension allows you to detect about 70% of all Snort signatures including application-level attacks.

PSAD addresses some of the limitations of PortSentry including:

• Better firewall integration. PortSentry listens on ports to detect scans, which causes more administration on the firewall.
• Implemented scoring mechanism for scans to prioritize actions and responses.
• Implemented passive fingerprinting.
• ICMP probe detection.
• Backdoor and DDoS probe detection.
• Integrated Whois lookups.
• Integrated email alerts.

PortSentry is very portable across many different Unix systems. Note that PortSentry's original Web site under Psionic has been redirected to Cisco after their acquisition. It is uncertain whether PortSentry is still in development; however, PSAD is actively developed and maintained.

References

Caswell, Brian. 2004. Snort 2.1 Intrusion Detection, Second Edition. Syngress Publishing.

Cox, Kerry and Christopher Gerg. 2004. Managing Security with Snort and IDS Tools. O'Reilly & Associates.

fwsnort -- http://www.cipherdyne.org/fwsnort/

Grsecurity -- http://www.grsecurity.net

Hogwash -- http://hogwash.sourceforge.net/

Honeynet Project -- http://project.honeynet.org/

LAk-IPS -- http://lak-ips.sourceforge.net

Libnet -- http://libnet.sourceforge.net/

LIDS -- http://www.lids.org

ModSecurity -- http://www.modsecurity.org/

PSAD -- http://www.cipherdyne.com/psad/

Sentry Tools -- http://sourceforge.net/projects/sentrytools/

Snort -- http://www.snort.org/docs/snort_manual/node21.html

Snortconfig -- http://www.shmoo.com/~bmc/software/snortconfig/

SnortSam -- http://www.snortsam.net/

Snort Inline -- http://snort-inline.sourceforge.net/

Online Articles

Understanding IDS Active Response Mechanisms --http://www.securityfocus.com/infocus/1540

An Introduction to Gateway Intrusion Detection Systems: Hogwash GIDS -- http://www.cansecwest.com/core02/hogwash.ppt

Securing an Unpatchable Webserver... HogWash! -- http://www.securityfocus.com/infocus/1208

Introducing mod_security -- http://www.onlamp.com/pub/a/apache/2003/11/26/mod_security.html

Web Security Appliance with Apache and mod_security --http://www.securityfocus.com/infocus/1739

Better Living Through Mod Security -- http://www.hackinthebox.org/article.php?sid=12867

Focus on Linux: Intrusion Detection on Linux --http://www.securityfocus.com/infocus/1416

Overview of LIDS, Part Two -- http://www.securityfocus.com/infocus/1502

PortSentry for Attack Detection: Part One -- http://www.securityfocus.com/infocus/1580

PortSentry for Attack Detection: Part Two -- http://www.securityfocus.com/infocus/1586

Angela Orebaugh is a Senior Scientist in the Advanced Technology Research Center of The Sytex Group, Inc., where she works with a specialized team to advance the state of the art in information systems security. She has more than 10 years experience in information technology and is pursing her Ph.D. in Information Security. Angela is also a researcher, writer, and speaker for SANS Institute. Angela is the author of the Syngress best seller Ethereal Packet Sniffing and O'Reilly's Snort Cookbook. More information on Intrusion Prevention Systems can be found in her latest book Intrusion Prevention and Active Response by Syngress Publishing.

Eric Cole is a speaker at national and international conferences on network security, including SANS. An information security expert for more than 10 years, Eric holds several professional certifications. Eric is currently Chief Technology Officer for The Sytex Group, Inc., where he heads cutting edge research in technology and various areas of network security. He is the author of Hackers Beware, Hiding in Plain Sight, co-author of SANS GIAC: Security Essentials Toolkit, and co-author of Security Essentials.