Linux-Hosted Frame Relay
Anthony L. Cook The frame relay protocol is gaining popularity and has the potential to become the standard for wide area networking in North America. Although it is similar to X.25, frame relay is faster, cheaper, and generally more capable than both X.25 and typical leased-line approaches to WAN building. These factors, coupled with easier maintenance, higher reliability, and wide-spread availability, make frame relay an ideal framework for expanding existing networks or installing new ones. Parallel to the gains by frame relay, Linux, with the release of the 2.0 kernel, has been making rapid inroads into corporate IS departments. Often serving as an integral part of the institutional intranet, Linux has already proven itself a capable network server and is well-positioned to help growing networks handle expanding needs while meeting demands for low cost. Therefore, Linux should be seen as an ideal, if not natural, partner to frame relay for expanding or installing wide area networks (WANs). In this article, I will describe how the Linux/frame relay partnership was used to set up a WAN for one of my company's clients. Many of our customers have offices in several cities and require fast and reliable communications with the central database server from each of them. When one such customer expanded their operations from their home in Manhattan, they opened offices in Baltimore, Houston, and Philadelphia. They demanded low-cost yet reliable communications from each of their new offices to their Linux-based database server. Because each office would have a combination of networked PCs, ASCII terminals, and high-speed forms printers, Linux communications servers were specified for frame relay and installed in each location. Server Design Each comm server needed to accommodate a multitude of routing and print service requirements, both within the local environment and with the New York database server. Additionally, any one of them could be called into further service as a local file server, email/fax server, or Internet gateway. For these reasons, Intel Pentium-class machines were chosen as the base platform. Each comm server is built to these specs: Intel P5 or AMD K5 CPU running at 133 Mhz or above, 32 MB fast EDO RAM, 2 GB fast IDE hard-drive, S3 PCI video w/2 MB, 8X IDE CD-ROM, 1.44 floppy, and 230 W power supply in a mini- or mid-tower case. With a SVGA monitor, keyboard, mouse, and an NE2000-compatible Ethernet card, this system can be built for around $1200 to $1500, depending on area and source, with off-the-shelf components. For local serial-based services comprising "dumb" terminals, high-speed forms and laser printers, and modem pools, multiport controllers were chosen for speed, reliability, and native support by the Linux kernel. For these purposes, I chose Cyclades' Cyclom Ye-16/RJ-45 for ISA bus (Cyclades Corporation, Fremont, CA). Cyclades was the first multiport vendor to provide support to the Linux OS for their range of controllers. Thus, the Cyclom driver is built into the Linux kernel and exhibits high reliability and efficiency. Although there are faster alternatives now available for Linux, Cyclades' RISC-based Cyclom series can operate at speeds up to 115 Kbs per port, offering plenty of speed for most uses. It's expandable to 32 ports per card, and up to 4 cards may be added, bringing the possible number or ports per server to 128. For frame relay support, Sangoma's S508 multiprotocol router cards were chosen (Sangoma Technologies Inc., Markham, Ontario, Canada). Sangoma's Linux drivers offer a choice of PPP over leased-line, frame relay, and X.25 communications protocols. Although the latest 2.0.x kernels offer native frame relay support (at the time of writing, Sangoma's cards are the only ones supported by the native driver), Sangoma's wantools package, freely downloadable from their ftp or Web sites, is continually being expanded and improved and offers greater ease of configuration, maintenance, and monitoring via the /proc filesystem. With the 2.1.x series of kernels, Sangoma's drivers are now an integral part of the Linux OS. However, if using one of these kernels, the wantools package should still be downloaded for Sangoma's configuration and monitoring tools. Although Sangoma has recently released versions of their cards with integrated CSUs, separate-CSU models were chosen for this installation for their ease of visual status monitoring. Sangoma's cards support speeds from 56/64 Kbs to T1/E1. Although any pre-2.0.x kernel and up should work for the native driver, a 2.0.27 or greater kernel is recommended for improved stability and robustness. However, if using Sangoma's driver package, 2.0.27 or greater is required, and 2.0.30 is a must if using a card with an integrated CSU. Any of these kernels and their patches may be downloaded from a number of Internet sites, along with their required release versions of supporting packages, but a lot of time and effort can be saved by simply purchasing a prepackaged distribution. If considering kernel/software versions alone, the latest releases of all the major distributions now install kernel version 2.0.30 and support, and should work equally well. However, with multi-site installations, administration requirements can also play a significant factor in choosing a package. Several distributions offer easy to use X Window-based administration tools that can be a boon to local administrators. However, they can be a tremendous bane to off-site or remote administrators who must often connect via modem to perform their tasks. Even over "low bandwidth X", remote administration using X Window can be painfully slow, even at 33.6 Kbps modem speeds. A few Linux packages offer text-based administration tools, but they tend to be limited in their capabilities for large-scale administration. Consider the alternatives carefully to avoid an installation that you will regret later. For these purposes, I chose a Linux distribution with kernel version 2.0.27 and X Window-based administration tools, then downloaded Sangoma's wantools-1.0.2.tgz driver package. Setup and Configuration Depending on the versions of the cards selected, some jumper settings may need to be changed on the Cyclom and/or Sangoma cards. On the ISA version of the Cyclom card, both the I/O port and interrupt are card-configurable via a bank of DIP switches. Sangoma's S508 router card uses jumpers to select the I/O port, but interrupts, interface type, and other pertinent factors are all driver-configurable at runtime. Once Linux is installed in the base machine, compatible IRQ and IOP settings for these cards can be chosen by checking against those already allocated in a running system. To check allocated settings, cat /proc/interrupts for IRQs and /proc/ioports for IO addresses. Here are a few things to keep in mind when selecting IRQs for your cards:
1. Do not use IRQ 9. This is the same as IRQ 2 on most systems, and IRQ 2 is usually used by the kernel for its "cascade" device. Selecting IRQ 9 will create a conflict with IRQ 2 and cause the board to behave unreliably, at best. 2. The kernel seems to miss PCI video cards when registering interrupts with devices at bootup. If using such a video card, be sure to pay attention to the IRQs posted by the BIOS at bootup, as it will not show up under /proc, making its assigned IRQ seem available for use. Choosing a video card's assigned interrupt for another installed card can be an unending source of frustration, as your installed card simply will not work. 3. Most NE2000-compatible Ethernet cards, such as those from Linksys or MaxTech, should be configured with their DOS-based setup programs prior to use with Linux. Alternatively, at NASA's Beowulf Project home page is a Linux native configuration tool for some NE2000 compatible Ethernet cards. Downloading and compiling this tool will allow you to configure boards based on National Semiconductor's AT/LANtic chipset. However, if using an NE2000 board based on Winbond's chips (or other than the AT/LANtic), then don't expect this program to be of much use for your particular card. Boot from a DOS diskette to configure these cards before installing Linux. The Linux kernel driver for this class of card almost never detects them with autoprobe and so will require compiling as a module. At a minimum, this card's IO port address can then be passed to it, with its "io=0xnnn" parameter at bootup.
After installing the chosen distribution set and configuring the cards, download and install an appropriate version of Sangoma's wantools driver package. After downloading the wantools package, login as superuser and untar the package from the root directory (/). This will install the main setup and configuration files in /usr/lib/router. cd to /usr/lib/router and execute the ./Setup shell script. Answer "y" to both install and apply the kernel patches when prompted, and the script will install Sangoma's driver patches into the Linux kernel for reconfiguration and compilation. Now change to /usr/src/linux and run make config to configure the kernel for the new cards. At a minimum, the following options should be selected:
* * Loadable module support * Enable loadable module support (CONFIG_MODULES) - 'Y' Set version information on all symbols for modules (CONFIG_MODVERSIONS) - 'N' Kernel daemon support (e.g. autoload of modules) (CONFIG_KERNELD) - 'Y' Sangoma's driver compiles as a loadable module, but must be recompiled with each kernel change per their documentation. Typically, the wanpipe module is inserted and removed with router starts and stops, yet it's a good idea to always enable module autoloading when configuring a system for module support.
* * General setup * Networking support (CONFIG_NET) - 'Y' * * Networking options * TCP/IP networking (CONFIG_INET) - 'Y' IP: forwarding/gatewaying (CONFIG_IP_FORWARD) - 'Y' IP: optimize as router not host (CONFIG_IP_ROUTER) - 'Y' WAN router (CONFIG_WAN_ROUTER) - 'Y' * * Network device support * Network device support (CONFIG_NETDEVICES) - 'Y' Dummy net driver support (CONFIG_DUMMY) - 'M' Frame relay DLCI support (EXPERIMENTAL) (CONFIG_DLCI) - 'N' PPP (point-to-point) support (CONFIG_PPP) - 'M' WAN drivers (CONFIG_WAN_DRIVERS) - 'Y' Sangoma WANPIPE(tm) multiprotocol cards (CONFIG_VENDOR_SANGOMA) - 'Y' Maximum number of cards (CONFIG_WANPIPE_CARDS) - '1' WANPIPE Frame Relay support (CONFIG_WANPIPE_FR) - 'Y' Ethernet (10 or 100Mbit) (CONFIG_NET_ETHERNET) - 'Y' Other ISA cards (CONFIG_NET_ISA) - 'Y' NE2000/NE1000 support (CONFIG_NE2000) - 'M' You may notice that "Frame relay DLCI support" is answered "N". This is the kernel's native frame relay support and will conflict with the Sangoma driver if selected "Yes." Reportedly, this driver works at least as well as Sangoma's, but has different configuration requirements that are incompatible with the information presented here. See the sidebar listing of recommended sites for where to find additional information on setting up this driver.
* * Filesystems * /proc filesystem support (CONFIG_PROC_FS) - 'Y' * * Character devices * Cyclades async mux support (CONFIG_CYCLADES) - 'Y' The remaining configuration options should be answered according to a site's particular requirements. Now, recompile the kernel with make dep ; make clean; make zlilo. Once finished, reboot and recompile the modules with make modules; make modules_install. Rebooting before recompiling modules helps to avoid module versioning conflicts. The system should now be rebooted again to ensure that any modules required by the OS are properly loaded. Also, if using Red Hat or a Red Hat-based distribution, the kernel Makefile may need to be edited to uncomment the line INSTALL_PATH=/boot. Otherwise, the new kernel and system map file will not be installed into the correct location. Once the new kernel has been built and booted, configuring the new router/terminal server can proceed. Before the Cyclom controller can be used, device files need to be created for it. These files can be created with the /dev/MAKEDEV script. Change to /dev and execute ./MAKEDEV cyclades to build the appropriate device files. However, this will only work for the first 16 or 32 ports (depending on the version of MAKEDEV being used). If more than this is required, Cyclades' supplied driver disk has a script that will build device files for all possible 128 ports. The router's /etc/inittab file can now be edited to accommodate any ports that will be used for terminals or dial-in modems (/dev/ttyC* or /dev/cub*, respectively). Configuring the WAN card is bit more involved but no more difficult. Two files must be configured before the driver will function properly: /etc/router.conf and /etc/router.rc. The /etc/router.rc file is the driver's configuration meta-file and installs with acceptable defaults for most options. You just need to add the appropriate WAN driver to its "WAN_DRIVERS" option. However, as these drivers can be added with Sangoma's "Configure" script, no direct editing of the file is required. /etc/router.conf is the driver's main configuration file. This file must be edited to set up DLCI and card configuration options. To do so, login as superuser, change to /etc and invoke your preferred editor on this file. In most cases only three areas need be edited: 1. Devices section: This section begins with the label "[devices]" and requires one line for each physical connection to the router. The format is device = config_id, description where "device" is the name of the connection and will be listed under /proc/net/router by this name; "config_id" is the protocol type used by this device; and "description" is an optional description for this device. For frame relay, this section should resemble:
[devices] wanpipe = WAN_FR, Our site's frame-relay link 2. Interfaces section: A WAN link isn't much good if doesn't allow for communication across it. In this section, the WAN interfaces are named for configuration by .../sbin/ifconfig and .../sbin/route. DLCI's are also established in this section, one DLCI per interface. Each line in this section follows the section label "[interfaces]", and for frame relay has the form name = device, address, description where "name" is the name of the interface as seen with ifconfig; "device" is the name of the device controlling this interface as established in [devices]; "address" is the DLCI for the site connected via this interface; and "description" is an optional description for this interface. For the above frame relay device, this section should resemble:
[interfaces] fr_site1 = wanpipe, 116, This site's link to remote office 1 (Baltimore) fr_site2 = wanpipe, 117, This site's link to remote office 2 (Houston) fr_site3 = wanpipe, 118, This site's link to remote office 3 (Philly) .... 3. Device configuration section: Parameters for the physical link(s) are configured here including all remaining card specific parameters. Each link subsection follows a label with the same name as the configured device and is composed of multiple statement lines of the form parameter = value. Configuration for the above frame relay device should resemble:
[wanpipe] #####This Site's Frame-relay Configuration#### # ----- Hardware configuration --------------- IOPort = 0x300 # I/O port base IRQ = 12 # interrupt request level Firmware = /usr/lib/router/wanpipe/fr508.sfm # adapter firmware # ----- Dual Port Memory Base Address -------- #Memaddr = 0xD0000 # Commenting this out enables Auto Memory # selection. Valid Memory addresses are: # 0xA0000,0xA2000...0xAE000 / 0xC0000,0xC2000... xD0000,0xD2000...0xDE000 / 0xE0000, # 0xE2000...0xEE000 # ----- Physical interface configuration ----- Interface = RS232 # physical interface type, RS232/V35 Clocking = External # Tx/Rx clock source, External/Internal BaudRate = 56000 # data transfer rate in bps, 1200..2048000 # ----- Media-specific configuration --------- MTU = 1600 # maximum data transfer unit, bytes UDPPORT = 9000 # for UDP management Station = CPE # station type, CPE/Node Signalling = LMI # In-channel signalling, ANSI/LMI/Q933 # -CRITICAL DATA: CONTACT SANGOMA TECHNOLOGIES- T391 = 10 # Link Integrity Verification Timer T392 = 15 # Polling Verification Timer N391 = 6 # Full Status Polling Cycle Counter N392 = 3 # Error Threshold Counter N393 = 4 # Monitored Events Counter TTL = 0x7F # Time To Live parameter for UDP packets Pay particular attention to "Signalling," especially LMI (Local/Link/Line Management Interface) type, when establishing a carrier account. Though there are several specifications for LMI, 'Annex A' and 'Annex D' remain the most widely used. Older and more limited 'Annex A,' also called "Cisco LMI," is the original LMI specification proposed by Cisco Systems, IBM, and Digital Equipment Corporation. It remains in widespread usage largely because it is supported by large base of existing Cisco equipment. However, as Sangoma's card, like many newer products, supports the 'Annex D' LMI specification, be sure your frame carrier knows to use this specification for your link(s). Following the previous sections are "[interface]" configuration sections, but these are generally not required unless the installation site has unusual connection requirements. Sangoma's documentation should be consulted if this is the case. Once this file has been edited, finish the installation by configuring the routing information for each interface. Change to /usr/lib/router and execute ./Configure. First, add a driver to the /etc/router.rc meta-file. Select option "4) add driver" and enter "wanpipe", which is the name of the loadable module built during kernel configuration. Now select option "2) add interface" to configure routing information for each of the interfaces set up in /etc/router.conf. Router interface files are kept under /usr/lib/router/interfaces and have the form of typical network config files used by SysV init scripts. While these can be edited directly, there should not be any need to unless the installation site has unusual requirements, such as extensive subnetting or router IP aliasing. To establish a frame relay link, the following minimums should be configured: interface name, interface IP, point-to-point IP, and interface network address. The specifics of these options as well as any others that may be required, such as netmask or gateway address, depend upon the installation site's particular network topology and link requirements. Once all necessary interfaces have been configured, the Configure script may be exited by returning to the main menu and selecting "q) quit". Now all that should remain is to start the router and test the link. Sangoma's Setup script installs a SysVinit compatible script called router into /usr/sbin and may optionally establish a link to it from the installation site's /etc/rc.d/rc<initdefault>.d directory for automatic startup at boot time. Execute /usr/sbin/router stop before starting up for the first time then /usr/sbin/router start. This script brings up all configured interfaces and adds them to the system routing table. If the installation site's frame relay node switch is not active within 1-2 minutes, the link connection should be verifiable with cat /proc/net/router/Status. The configured device(s) should be listed as "connected." Then ping each of the point-to-point connections to verify the channels to the remote sites are "alive". Troubleshooting If the link does not operate as expected, check /var/log/messages and /var/log/router for clues. Failing that:
Linux is already a well-respected platform for LAN-based services. With the increasing availability of WAN adapter cards and drivers for Linux, it should be seen as an equally viable platform for so-called "PC hosted routing." Reliable Linux-based router boxes matching the capability of traditional WAN routers can be assembled, installed, and configured, usually at a lower cost. Linux routers can also provide an organization with additional capabilities that most traditional routers cannot:
The relatively simple installation and configuration process outlined in this article should simplify getting your Linux-based WAN router operating. About the Author
When not attending his family or playing with his computers at home, Tony doubles as a Systems Engineer/Administrator for Advanced Transportation Systems, Inc. in Colorado Springs, Colorado. He can be contacted at his personal email address: vschade@mindless.com.
|