Years ago, incumbent service providers offered virtual circuits on top of Frame Relay, ISDN or ATM networks, and enterprises used Ethernet to provide LAN connectivity within their private networks. The two worlds never mixed and usually interacted through routers providing layer-3 switching services.
The ubiquity and widespread popularity of Ethernet resulted in versatile low-cost solutions. Ethernet chipsets are manufactured in huge quantities, and the relatively simple technology allows for cost-effective implementation. So service providers trying to offer lower-cost high-speed services started to replace the traditional technologies (from DSL access to the SONET core) with Ethernet, which can provide speeds up to 10Gbps.
Ethernet enters service provider networks
Ethernet technology is all over the place now, particularly in three major areas in modern service provider networks:
- Access Networks: Low-cost concentration of high-speed access links usually implemented as fiber-to-the-building (FTTB) or fiber-to-the-home (FTTH).
- Core Networks: Long-range Ethernet (Ethernet on single-mode fibers) is used as the high-speed point-to-point technology replacing SONET/SDH.
- Transport: Site-to-site transparent Ethernet services are offered as a replacement for DWDM services or as a low-maintenance solution for service providers that have not yet mastered VPN services.
Service provider network Ethernet limitations
In all cases, the early Ethernet adopters in service provider networks have faced...
To continue reading for free, register below or login
To read more you must become a member of SearchTelecom.com
significant problems. Why? Because Ethernet technology and the cost-effective products offered by networking vendors addressed the needs of private enterprise-wide deployment, not large-scale public network deployment. Some of the basic limitations of using Ethernet in a service provider network include:
- Ethernet doesn't provide virtual circuits, but a shared bridged domain where every participant can communicate with and affect any other participant.
- Ethernet doesn't provide end-to-end signaling similar to Frame Relay Local Management Interface (LMI). In general, it's impossible to indicate to a customer end-point that the other end of the link is no longer available.
- Ethernet expects network-wide uniqueness of physical (MAC) addresses, which cannot be enforced when a service provider offers transparent Ethernet services to numerous customers.
- The Ethernet virtual LAN (VLAN) technologies do not scale. For example, the 802.1Q standard has a network-wide limitation of 4,096 VLANs.
- Ethernet has no inherent security architecture. Even the authentication services provided by 802.1x address the needs of authenticating a user connected to a shared virtual LAN infrastructure.
On top of the technology issues, service providers were faced with equipment limitations. The traditional routers or layer-3 switches coupled with large-scale, high-speed layer-2 aggregation were simply not capable of offering the required per-user services. For example, the high-performance per-user services in a Cisco 7600 router were made possible only after the introduction of ES20 and ES40 line cards.
Standards address Carrier Ethernet fixes
The industry was quick to address the shortcomings of the Ethernet technology. New standards in the IEEE's 802.1 Working Group address specific service provider needs:
- 802.1ad (Q-in-Q) defines two-level VLAN encapsulation, which the service providers can use to transport customer VLANs across service provider backbones.
- 802.1ah defines truly scalable VLAN-in-VLAN solution (where the whole customer Ethernet frames are encapsulated in a VLAN envelope).
- ITU recommendation Y.1730 defines Ethernet OAM requirements.802.1ag defines the Connectivity Fault Management and associated MIB.
Vendor carrier-grade platforms
>Major networking vendors have implemented these standards, giving service providers a versatile technology deployed on carrier-grade platforms. You have to be careful when buying the equipment, however, because the carrier-grade additions to Ethernet are posing an additional burden on Ethernet switches, resulting in more expensive boxes.
For this reason, most vendors offer two groups of Ethernet equipment. The lower-cost equipment implements the traditional enterprise-focused Ethernet functionality, while the higher-cost equipment provides carrier-grade services (for example, Cisco calls the carrier-focused devices Metro Ethernet Access switches). If your acquisition process focuses solely on the equipment cost without specifying the minimum required functionality, you might run into unpleasant surprises when trying to deploy Ethernet-based service.
Carrier Ethernet's other issues
Remember that the Carrier Ethernet standards address solely the shortcomings of the Ethernet technology. You'll need to address numerous other issues you need to address in a large-scale Ethernet deployment, including:
- User identification. Your physical infrastructure might extend to end-points that are not currently using your services. For example, if you're rolling out FTTH infrastructure, you'll try to reach every home in the neighborhood while laying the fiber, not just those that happen to be your customers. It is vital to have a mechanism to authenticate and authorize your customers. Some incumbent service providers have opted to deploy PPPoE over high-speed Ethernet infrastructure, transforming the new fiber-based infrastructure into yet-another dialup solution (and incurred huge overhead because they needed powerful access devices to support high-speed PPPoE sessions). It's way more cost-effective to deploy aggregation switches that support DHCP option 82 to identify the customer port.
- End-user protection. Consumer-focused Carrier Ethernet is usually deployed as a large logical LAN (single IP subnet) to connect individual users. By default, Ethernet switches allow these users to communicate directly without the overhead of going through the central aggregation router. While this approach definitely reduces the load of the aggregation router in environments with a large percentage of peer-to-peer traffic, it also exposes all end-users to direct layer-2 attacks (including ARP spoofing) from other end-users connected to the same LAN segment. The layer-2 attacks have to be stopped at the network ingress point; the first aggregation switch has to support ARP inspection.
Denial-of-service protection. IP uses additional protocols (ARP and DHCP) on Ethernet to assign IP addresses to clients and establish mappings between MAC and IP addresses. While it's always been possible to impact a router's operation with a flood of ARP packets (and use mechanisms like Control Plane Policing to protect the router), the aggregation scale deployed in modern carrier networks (thousands of customers aggregated onto a 10GigE port on a router) makes the task significantly harder. A single dissatisfied customer can create a denial-of-service attack that can affect thousands of other customers, unless the attack is stopped at the point where it's still manageable: on the first aggregation switch, which has to support DHCP snooping and DHCP/ARP rate limiting.
The bottom line on Carrier Ethernet
With the additional functionality provided by new 802.1 standards and implemented by major networking vendors, Carrier Ethernet became ready for production-grade deployment in service provider networks and you should seriously consider its use in your network infrastructure if you haven't deployed it yet.
The migration from traditional technologies (SONET in the core and DSL in the access) to Carrier Ethernet should be well-designed, planned and tested, as Ethernet does not provide a cheaper one-to-one replacement for current technologies.
When replacing core SONET links with Carrier Ethernet, be aware that you'll lose the fast link loss detection and rerouting inherent in SONET. These functions have to be migrated to layer-3 devices (for example, using the Fast Reroute functionality of MPLS traffic engineering). Likewise, it's not efficient to deploy Carrier Ethernet over FTTx fiber links and emulate DSL circuits by running PPPoE over the new high-speed infrastructure. To benefit from Carrier Ethernet deployment, you should implement a provisioning and accounting solution that supports DHCP, VLANs and Netflow-based accounting (or its equivalent).
About the author: Ivan Pepelnjak, CCIE No. 1354, is a 25-year veteran of the networking industry. He has more than 10 years of experience in designing, installing, troubleshooting and operating large service provider and enterprise WAN and LAN networks and is currently chief technology advisor at NIL Data Communications, focusing on advanced IP-based networks and Web technologies. His books include MPLS and VPN Architectures and EIGRP Network Design. Check out his blog for more on networking.
