The problem: The explosive growth of the Internet spurred by high-speed access networks and flat-rate mobile billing plans has caused ferocious competition in the telecom service provider market, as well as an alarming disconnect between traffic growth and carrier revenue. It's impossible to charge more for ever-increasing access speeds, and service providers are faced with an impossible dilemma: how to increase network throughput and reduce costs at the same time.
Solutions: One potential answer is IP over dense wave division multiplexing (DWDM). In that vein, router vendors, including those that market their routers as layer 3 or network-layer switches), are trying to position themselves as the saviors. Their message is simple: 1) You need to deploy DWDM in the network core to increase throughput without incurring additional fiber-related costs; 2) We can help you save money by eliminating all the components between the core DWDM system and the router ports.
Their target is also obvious: Vendors want to persuade service providers to eliminate the legacy SONET/SDH boxes (or at least push them aside) in favor of new investments in DWDM-enabled routers and switches.
When to use IP over DWDM in metro networks
In some cases, eliminating legacy SONET/SDH makes perfect sense. For example, if you use a metropolitan DWDM core in a design that does not need ultrafast convergence or signal regeneration within the DWDM system, it's cheaper to generate DWDM-compliant signals on the router/switch ports.
You can also realize significant savings when designing DWDM-based backhaul of residential Internet traffic. You can replace a short-reach (multimode) Gigabit Ethernet connection between an Gigabit Ethernet switch or router and a transponder card in your DWDM shelf with a DWDM-compatible SFP transceiver (small form-factor pluggable (SFP) is a specification for a new generation of optical modular transceivers) in the Gigabit Ethernet port, and an add-drop multiplexer card (which you need anyway) in the DWDM shelf. As long as these connections are used for point-to-point links between compatible devices, you don't exceed the optical budget (and thus require no regeneration), and you don't need very fast convergence (as offered by SONET/SDH or G.709). Similar solutions exist for 10 Gigabit Ethernet (10 GE) backhaul links.
The lack of G.709 framing on these DWDM connections results in several significant drawbacks:
- Lack of Forward Error Correction (FEC) and low-quality optics used in some SFP transceivers reduce the maximum reach of these links;
- Lack of performance monitoring and in-band alarms significantly increases failure detection time and rerouting/convergence time.
The low cost of these solutions might persuade you to sacrifice the benefits of G.709, implement signal loss detection with higher-layer protocols (for example, the Unidirectional Link Detection mechanisms) and use IP routing protocols or Multi-protocol Label Switching (MPLS) Traffic Engineering for decently fast rerouting.
IP-over-DWDM in core networks: Interfaces pose complex problems
The situation is completely different in the high-speed network cores where you need ultrafast failure detection and robust and fast rerouting around failed links or nodes. If you want to implement these features in a (potentially) multivendor environment, you have to enforce strict standard compliance. This is where the IP-over-DWDM story gets complex.
Let's start from the outside, where the routers (including layer 3 switches) usually offer these connectivity options:
- Gigabit Ethernet (GE)
- 10 GE
- Packet-over-SONET (POS) interfaces with speeds ranging from optical carrier levels of OC-3/STM-1 (155.52 Mbps) to OC-192/STM-64 (10 Gbps).
Most of these interfaces cannot be patched directly into a DWDM core owing to lack of G.709 (a standardized method for transparent transport of services over optical wavelengths in DWDM systems, also known as the Optical Transport Hierarchy, OTH, standard) framing. A transponder card is needed in the DWDM shelf to encapsulate the raw data (payload) received from the client devices (routers or switches) into a G.709 Optical Transport Unit (OTU).
Not surprisingly, some vendors want you to spend less money on DWDM transponders and invest more in high-end routers. They offer you various IP-over-DWDM schemes using 10 Gigabit Ethernet or 10 Gigabit (OTU2) or 40 Gigabit (OTU3) Packet-over-SONET/SDH interfaces. The POS interfaces should not present a significant challenge; after all, they're nicely integrated into the Optical Transport Network (OTN) framework defined by G.709. You just have to make sure of the following:
- The router's interface can insert the payload into the G.709 OTU;
- The router's hardware supports G.709 FEC and EFEC;
- The router's software recognizes the performance and alarm data carried in the G.709 header.
Interfaces for 10GE are trickier. The 10GE standard (IEEE 802.3ae) defines two line rates: WAN PHY (9.95328 Gbps), which matches the OC-192/STM-64 line rate and LAN PHY (10.3125 Gbps), which results in 10 Gbps throughput. The addition of the G.709 wrapper around the LAN PHY-encoded data results in an overclocked signal at 11.0975 Gbps. This signal passes through the (passive) optical part of the DWDM network just fine, but might not be recognized correctly by a third-party optical-electric-optical (OEO) regenerator.
IP over DWDM and end-to-end management systems
Last but not least, you need an end-to-end management system that simplifies network provisioning and monitoring. Ideally, the elements in the end-to-end solution should be able to communicate and establish on-demand paths across the network based on true bandwidth needs.
The ultimate network management solution in IP-over-optical environments is (Generalized Multiprotocol Label Switching (GMPLS), the signaling protocol that unites packet switching, TDM, wavelength switching and fiber switching (optical cross connects). Vendors already deliver GMPLS-enabled equipment (for example, Juniper's M-series routers and Cisco's CRS-1).
But there's a long way to go before you'll be able to deploy a seamless multivendor solution and provision it automatically through GMPLS. In the meantime, you're left with a patchwork similar to the early days of ATM networks: proprietary network management systems, customized integration solutions and manual configuration.
About the author: Ivan Pepelnjak, CCIE No. 1354, is a 25-year veteran of the networking industry. He has more than 10 years of experience in designing, installing, troubleshooting and operating large service provider and enterprise WAN and LAN networks and is currently chief technology advisor at NIL Data Communications, focusing on advanced IP-based networks and Web technologies. His books include MPLS and VPN Architectures and EIGRP Network Design. Check out his IOS Hints blog.
This was first published in January 2010