Data centers (DCs)—facilities that are used to centralize the IT operations and equipment of an organization—represent a critical piece of modern networked applications, in both the private and public sectors. The trend toward DCs has been driven by a number of key factors, e.g., economies of scale, reduced management costs, better use of hardware (via statistical multiplexing), and the ability to elastically scale applications in response to changing workload patterns. In particular, a robust network fabric is fundamental for the success of DCs, i.e., to ensure that the network does not become a bottleneck for high-performance applications. In this context, the design of a DC network must satisfy several goals, including high performance (e.g., high throughput and low latency), low equipment and management costs, robustness to dynamic traffic patterns, incremental expandability to add new servers or racks, as well as other practical concerns (e.g., cabling complexity, and power and cooling costs). Currently available DC network architectures, however, do not provide satisfactory solutions to these requirements.
There are two main problems with traditional static (wired) networks. They are either ‘overprovisioned’ to account for worst-case traffic patterns and thus incur high costs (e.g., with fat trees or Clos architectures), or they are ‘oversubscribed’ (such as with simple trees or leaf-spine architectures). Although the latter networks have low costs, they offer poor performance because of their congested links. In recent studies, attempts have been made to overcome these limitations by augmenting a static ‘core’ with some flexible links (radio-frequency or optical wireless). These augmented architectures do show some promise, but they provide only a small improvement in performance. Moreover, all these architectures involve high cabling costs and complexities.
In our work,1 we envision an extreme design point to meet the requirements of modern DC networks rather than trying to incrementally improve the cost-performance tradeoffs, high cabling complexity, and rigidity of current DC architectures. In our architecture vision—known as FireFly—we use free-space optics (FSO) communication links to realize a fully flexible, all-wireless inter-rack fabric. FSO communication technology is particularly well suited to our aim because it offers very high data rates (tens of Gb/s) over longranges (more than 100m), while having low transmission power and a small interference footprint.
A conceptual overview of our FireFly architecture vision is shown in Figure 1. In our design, each top-of-rack (ToR) switch has flexible (steerable) FSO links that can be dynamically reconfigured to connect to other ToRs. The controller reconfigures the topology to adapt to current traffic workloads. This vision provides several benefits over current DC architectures. For instance, our topological flexibility (if achieved correctly) provides a low-cost option (i.e., few switches and links) with performance comparable to more expensive overprovisioned networks. In addition, our all-wireless fabric eliminates cabling complexity and associated overheads (e.g., obstructed cooling). Our approach can also facilitate new and radical DC topology structures that would otherwise remain at the ‘paper design’ phase because of their cabling complexity. Lastly, our method of flexibly turning links on or off brings us closer to realizing the aim of energy-proportional DCs (and the flexibility enables easier incremental expansion of a DC).
Figure 1. Schematic illustration of the FireFly architecture. FSO: Free-space optics. ToR: Top of rack.
The unique characteristics of our approach (i.e., the FSO-based inter-rack links and the fully flexible topology) give rise to novel algorithmic, networking, and system-level challenges that need to be addressed before our vision can be made into a reality. First, we need to develop cost-effective and robust link technologies that have small form factors and that can be steered at short timescales to impart flexibility. Second, we require algorithmic techniques to design the efficient and flexible networks. Third, we need new network management solutions. These may include algorithms for the joint optimization problem of runtime topology selection and traffic engineering, as well as data-plane mechanisms to guarantee various consistency and performance requirements.
In our recent work,1 we have demonstrated the viability of our FireFly architecture by building a proof-of-concept prototype (with commodity components) for a steerable, small-form-factor FSO device (see Figure 2). We have also developed practical heuristics to address the algorithmic and system-level challenges in the network design and management of our architecture. In addition, we have developed techniques to provide line-of-sight for FSO links in the FireFly architecture. For our steerable, small-form-factor FSO device, we have been exploring the use of microelectromechanical systems (MEMS) mirrors as a steering technology to steer the FSO beams with minimal latency. In this device, we use a collimated laser beam that is transmitted from the fiber collimator of an FSO terminal. The laser beam passes onto a gimbal-less two-axis MEMS micromirror (2mm diameter) and thus steers the beam in an ultrafast manner within a large optical deflection (10°) over the entire device bandwidth (1.2kHz). The MEMS mirror deflects the beam into a wide-angle lens that magnifies (about three times) the optical scan angles of the system. This magnification is linear and therefore results in an overall scan capability field of view of more than 30°. The power consumption of this system is less than 1mW and thus several orders of magnitude lower than that of galvanometer mirrors. The outgoing FSO beam from our MEMS beam-steering mechanism passes through autopoints and onto a receiving aperture (where it is efficiently coupled into a fiber collimator). With this fast and precise MEMS steering mechanism, we can switch the FSO from one rack to the next for reconfigurable networking. It also enables an autocorrection mechanism for fixing any misalignments (in real time).
Figure 2. Photographs of the MEMS (microelectromechanical systems)-based proof-of-concept prototype assembly used to realize steerable FSO beams.
In summary, we have designed the novel FireFly architecture for radically improving modern DC networks. Our vision includes unique characteristics, such as FSO-based inter-rack links and a fully flexible topology. Such features give rise to a number of algorithmic, networking, and system-level challenges that we are working to address. We have recently demonstrated the feasibility of our architecture with a proof-of-concept prototype for a MEMS-based steerable, small-form-factor FSO device. There are, however, various challenges that we need to address before we can realize commercialization of our architecture. In our current work we are thus building a small testbed for the FireFly architecture, which includes autoalignment through the use of galvanometers and MEMS steering technologies. We now plan to demonstrate the resilience of our FSO-link technologies against indoor effects, e.g., rack vibrations and temperature variations.
This project was supported by the National Science Foundation award 1513866 (NeTS: Medium: Collaborative Research: Flexible All-Wireless Inter-Rack Fabric for Datacenters using Free-Space Optics) and represents a collaboration between faculty members, postdoctoral fellows, research associates, and graduate students at Pennsylvania State University, Stony Brook University, and Carnegie Mellon University.
Electrical Engineering and Computer Science
Pennsylvania State University
University Park, PA
Mohsen Kavehrad has been the W. L. Weiss Chair Professor of Electrical Engineering since 1997, and is the founding director of the Center for Information and Communications Technology Research. He has previously worked for Bell Laboratories and is a fellow of the IEEE. He is the author of more than 400 papers, several books and book chapters, and holds several patents.
Sami R. Das, Himanshu Gupta, Jon Longtin
Department of Computer Science
Stony Brook University
Stony Brook, NY
Samir Das received his PhD in computer science from Georgia Institute of Technology. He previously studied at Jadavpur University, India, and the Indian Institute of Science. He has also worked briefly at the Indian Statistical Institute. He moved to Stony Brook in 2002 and was previously a faculty member at the University of Texas at San Antonio and then at the University of Cincinnati.
Himanshu Gupta obtained his PhD in computer science from Stanford University in 1999 and his BTech from the Indian Institute of Technology in 1992. In his recent research he focuses on theoretical issues associated with wireless networking. In particular, he is interested in sensor networks and databases. His other research interests include database systems and theory, e.g., materialized views, (multiple) query optimization, and data analysis.
Jon Longtin joined the mechanical engineering faculty in 1996. He is the author of more than 130 technical research publications, including a number of book chapters. He also holds six issued and three pending US patents. His expertise is in the thermal sciences, with a focus on the development of laser-based optical techniques for the measurement of temperature, concentration, and thermal properties. He is also interested in the use of ultrafast lasers for precision material processing and micromachining, and the development of sensors for harsh environments (e.g., direct-write thermal spray technology). He has been the recipient of a Japan Society for the Promotion of Science postdoctoral fellowship, the National Science Foundation's Faculty Early Career Development award and the Presidential Early Career Award for Scientists and Engineers, and the Stony Brook Excellence in Teaching award. He is a registered professional engineer in New York State.
School of Computer Science
Carnegie Mellon University
Vyas Sekar is an assistant professor in the Electrical and Computer Engineering department. He received his PhD from Carnegie Mellon University in 2010 and earned his bachelor's degree from the Indian Institute of Technology Madras (during which he was awarded the President of India Gold Medal). His research interests lie at the intersection of networking, security, and systems. He has also received a number of best paper awards, e.g., at the Association for Computing Machinery's SIGCOMM, CoNext, and Multimedia conferences.
1. M. Kavehrad, Firefly: reconfigurable optical wireless networking data centers. Presented at SPIE Photonics West 2017.