Farther and Faster at the Speed of Light

As disk-drive interfaces increased in speed during the 1990s, the distance limitations of the parallel small-computer-systems interface (SCSI) bus became apparent. Each time the speed of the SCSI bus increased, the supported distance decreased because of the limitations of the copper cabling. To overcome these distance limitations and offer a networked architecture, the American National Standards Institute (ANSI) decided to define Fibre Channel and, thus, storage-area networks (SANs). The Fibre Channel name implies the use of optic fiber to interconnect storage devices and servers via channels (high-bandwidth connections).

Fiberoptic technology has always been a key element of Fibre Channel, as it enables connections between storage devices and servers to span long distances. As corporate and campus computing resources grew to support thousands of servers, data centers grew to multiroom and multibuilding facilities spread across metropolitan areas. Optical communications technology and Fibre Channel, in particular, enable communications at data rates on the order of gigabits per second between these computing resources over a variety of distances.

Approximately 90% of Fibre Channel links span less than 100 m and are known as shortwave (SW). SW links use multimode fiber and vertical-cavity surface-emitting lasers (VCSELs) within the data center. Another 9% of Fibre Channel links are known as longwave (LW). LW links use singlemode fiber and Fabry-Perot lasers to span distances up to 10 km in campus-type environments. The remaining 1% of Fibre Channel links use singlemode fiber and distributed feedback (DFB) lasers to connect data centers separated by 50 km or more. The pairing of lasers and fiber for certain links meets the requirements for a variety of SAN applications (see figure 1).

Figure 1. The fiber and equipment for SANs vary depending on the distance.

The Fibre Channel community has stan-dardized a variety of gigabit-per-second trans-ceivers. Starting with the gigabit link module, designers progressed to the gigabit interface converter (GBIC) using SC-type fiberoptic connectors in the late 1990s. In 2000, the small-form-factor pluggable (SFP) transceiver incorporated the LC-type connector and the combination quickly dominated the market. The SFP cage that holds the transceiver is less than half the size of the GBIC cage (see figure 2).

Figure 2. The delta points of the SFP transceiver are the electrical interfaces to the transceiver. The gamma points are the optical interfaces.

The SFP was designed to work only at speeds of up to 5 Gb/s, however, so when the 10 gigabit Fibre Channel (GFC) standard came out in 2002, a transceiver known as the 10(X) gigabit small-form-factor pluggable (XFP) transceiver was created to handle the higher speed. Several new transceivers are currently under development and will be discussed later.

The SFP transceiver is the most popular Fibre Channel transceiver. It has been standardized at two points in the Fibre Channel physical interfaces standard (FC-PI-2).¹ The delta points are the electrical interfaces to the transceiver; they have been set for the line rates of 1, 2, 4, and 10 Gb/s. ANSI defined the delta points so well that the Institute of Electrical and Electronic Engineers used most of the same standards to create the Gigabit Ethernet physical layer. In an odd twist, ANSI used most of the 10 Gigabit Ethernet standard to define 10 GFC. The same transceivers can be used in either Fibre Channel or Gigabit Ethernet products.

The second part of the architecture standardized by FC-PI-2 consists of the optical interfaces, or the gamma points. The gamma points constitute the main region of interest to optical engineers. The gamma points are defined for a number of optical-signal characteristics at both the transmitter and the receiver. The standard specifies the quality of the optical signal by characteristics such as jitter, rise time, fall time, optical-modulation amplitude, and spectral qualities at each speed and distance. The gamma-point specification determines the maximum distance over which a link can operate with a bit-error rate of less than 1 X 10^-12. Actual measured bit-error rates are usually much lower, but 1 bit error in a trillion is practical to measure.

A key result of these specifications is that a link should function virtually error-free over a given distance. Multimode fiber links with VCSELs operating at about 850 nm have limited bandwidth-length products (BLPs) due to intermodal dispersion. The BLP is a useful guide for multimode fibers. We can roughly calculate the supported distance L in kilometers by

L = BLP/BW

where BLP is in megahertz kilometers and BW represents the bandwidth in megahertz. Above the calculated distance, signal integrity decreases until an unacceptable bit-error rate manifests. The BLP of 62.5-µm-core fiber, for example, is 200 MHz km. If we increase transceiver speed from 1.06 Gb/s (1 GFC) to 10.52 Gb/s (10 GFC) on this fiber, the distance supported by the link drops from 300 to 33 m (see table). Because of the low BLP of 62.5-µm-core fiber, designers rarely choose it for SANs.

The multimode workhorse for SANs is 50-µm-core fiber with a BLP of 500 MHz km. When operating speeds rose above 10 Gb/s, the supported distance of 82 m left many users dissatisfied. To solve this problem, manufacturers developed enhanced fibers with BLPs of 2000 MHz km. The fibers were designed to operate at up to 300 m at 10 Gb/s.

Singlemode fiber, by definition, cannot have intermodal dispersion, but its performance is limited by other factors such as chromatic dispersion. The first LW transceivers used Fabry-Perot lasers that operated at 1310 nm with a spectral width of up to 6 nm. Different wavelengths travel at different speeds in the fiber, temporally broadening the pulses and causing unacceptable bit-error rates. To support 10-km distances at data rates of up to 4 GFC, the spectral bandwidth require-ment for lasers was tightened in the ANSI standard so that most Fabry-Perot lasers do not achieve the necessary spectral performance. Supporting 10-km distances at 4 and 10 GFC requires another type of laser.

To transmit 4.25-Gb/s (4 GFC) signals over 10 km and farther, the ultimate transceivers use DFB lasers that achieve spectral linewidths of less than 0.1 nm. Since spectral bandwidth is narrow enough to eliminate chromatic dispersion effects, the link is only limited by optical attenuation. These DFB lasers can reach distances of more than 100 km at any speed and are only limited in output power by eye-safety requirements. Telecom lasers can go farther because they are not eye-safe, Class-1 lasers. All SAN transceivers are eye-safe and produce limited power to minimize the risk of eye injury to installers. Although the singlemode fiber-transceiver pairs are designed for long distances, some high-end users future-proof their short-distance installations by using singlemode fiber on all links. When an old transceiver is replaced with a faster device, the installed singlemode fiber will still work and does not need to be replaced.

The mantra of the transceiver industry is smaller, faster, and cheaper. Each given design eventually runs into limitations. The SFP transceiver was designed to run at up to 5 Gb/s only. ANSI is currently developing standards for operation at 8.50 Gb/s (8 GFC), which is incompatible with 10 GFC encoding schemes. The 8 GFC standard will essentially involve doubling the line rate of the 4 GFC standard.

Many in the Fibre Channel commu-nity would like to use 8 and 10 GFC data rates in the SFP form factor. To support these higher speeds, the industry is redefining the SFP interfaces as the SFP+. These enhanced interfaces will have new electrical, mechanical, and electromagnetic properties compatible with 8 and 10 GFC data rates while continuing to support the traditional SFP transceivers.

Another interesting development in standardized transceivers is the use of parallel optics. The Quad SFP (QSFP) transceiver will use the SNAP-12 fiberoptic ribbon cable. Only eight of the 12 fibers in the ribbon will be used in the QSFP, corresponding to four transceiver pairs. The QSFP is bor-rowing designs from both the XFP and the peripheral component interface-express (PCI-E) standards. The cage for the QSFP will be the XFP cage and the electrical connector of the QSFP will be the same as the PCI-E 4X cable. The combination of tried and true components with a fiberoptic ribbon offers some of the highest throughputs available in a standardized transceiver.

As SANs grow in speed, the SAN community continues to develop transceivers that meet the performance needs of every application. From 1 to 10 GFC, gigabit SAN transceivers are being designed as low-cost components while remaining eye-safe. As speeds increase, system designers are using higher-bandwidth multimode fiber alongside singlemode fiber to increase the distance of the links. As the size and scale of data centers continue to grow, ANSI will continue to standardize multigigabit connections over tens of kilometers to increase adoption. oe

1. Fibre Channel Physical Interfaces-2 (FC-PI-2), www.t11.org (2005).