A White Paper by Steve Tang, Senior Engineer, TC Communications
Significant new standards introduced by the IEEE 802.3 Working Group over the past two decades have enabled Ethernet to emerge as the technology of choice for most networking requirements.
Ethernet, which has been the ubiquitous standard of choice for Local Area Networks (LAN), is rapidly replacing legacy technologies such as SONET/SDH (TDM), Frame Relay and ATM in Metro Area Networks (MANs), Wide Area Networks (WANs), and industrial communications.
Industrial and Utility networks, traditionally dominated by SONET/SDH and other proprietary technologies, are migrating to Ethernet en masse. There are many reasons behind this migration.
Ethernet's simplicity, interoperability, and cost efficiencies bring tangible benefits. Utilities, ever mindful of implementing Smart Grid communications networks, are discovering the additional benefits of TDM over IP/Ethernet technologies and multi-service Ethernet platforms.
Recent Ethernet standardization efforts have focused on improving Quality of Service (QoS) and Operation Administration and Management (OAM). As a result, Ethernet addressed some weaknesses and now provides strengths previously only associated with SONET/SDH.
For example, Ethernet’s real time (deterministic) capabilities are now comparable to SONET/SDH for virtually every application including Smart Grid, SCADA, and teleprotection relay communications in power utilities.
Because Ethernet was designed for data transport and SONET/SDH for voice, each is inherently superior to the other for certain applications. However, for the purpose of this article, the list below includes most of the important areas in which Ethernet surpasses SONET/SDH systems:
- Bandwidth Efficiency
- Finer Granularity
- Protection Bandwidth
- Flexible Topologies
- Ubiquitous Connectivity
- Operational Simplicity
- Lower Cost
- Future Proof Investment Protection
SONET/SDH networks are designed to use fixed bandwidth circuits which contribute to overall complexity and operation inefficiencies. In short, SONET/SDH bandwidth allocation is rigid; Ethernet is flexible.
For example, users often over-subscribe to accommodate for traffic bursts during peak usage. This over-provisioning, combined with the network redundancy required for self-healing, results in inefficient use of available network capacity. In contrast, as shown in figure 1, the statistical multiplexing nature of traditional Ethernet provides more efficient network utilization for service providers.
The need for more bandwidth efficiency is most noticeable in mobile backhaul applications. Changing cell phone habits and the emergence of 4G technologies have triggered an exponential increase in data content for mobility backhaul. This has created an ongoing need to add more transmission capacity. The rub is that voice centric SONET/SDH-based networks no longer offer a viable and cost-effective solution to handle the increase in data traffic for backhaul transport.
In the past, addressing increased voice traffic involved simply deploying larger number of T1/E1 circuits between the cell sites and the network core. The generous revenue stream from these voice calls easily justified the cost of additional T1/E1 circuits.
In contrast, adding additional data traffic is more complex. Installing more T1/E1 circuits to handle increased data traffic is not cost effective because data traffic generates a much smaller revenue stream per Mbit of traffic than voice. The cost effective solution is Ethernet.
Ethernet is a connectionless packet technology that inherently supports statistical multiplexing. Ethernet-based backhaul transport can easily accommodate data increases with a much smaller incremental increase of backbone bandwidth through the proper use of over-subscription and statistical multiplexing. Industrial Ethernet layer 2 switches can be deployed in several applications to increase overall bandwidth efficiency including:
- Terminating T1/E1 circuits over Ethernet at the cell sites takes advantage of Ethernet's statistical multiplexing capabilities to reduce bandwidth needs for backhaul.
- Using Ethernet switches instead of SONET/SDH ADMs at the aggregation points to reduce overall Capital Expenditure (CAPEX) as well as Operational Expenditure (OPEX).
Recently introduced standards such as Virtual Concatenation (VCAT), Link Capacity Adjustment Scheme (LCAS) and Generic Frame Protocol (GFP) are designed to make SONET/SDH more suitable for data transport.
However, Ethernet still offers superior transparency, scalability, and finer granularity to support industrial applications. This includes applications such as SCADA, native Ethernet and delay-sensitive serial traffic such as teleprotection in substations.
In many cases, a SONET/SDH Add-drop-Mux's (ADM) ability to add and drop traffic depends on the implementation of its cross-connect fabric and whether it supports high order Virtual Tributary (STS-1) switching or low-order (VT1.5) VT switching. This remains true regardless of the backbone capacity.
Many ADMs that support only high-order VT switching offer diminished granularity for lower speed interfaces common to many industrial applications.
SONET/SDH technologies typically use n+1 protection. This means network capacity designed for protection is entirely dedicated to protection and not used for carrying any real traffic.
With sub-second recovery, Ethernet supports Rapid Spanning Tree Protocol (RSTP - IEEE 802.1w) and Multiple Spanning Tree Protocol (MSTP - IEEE 802.1s) which offer load balancing between working and protection backbones to more efficiently use network resources.
Most SONET/SDH networks are deployed in fixed ring-based designs making changes to the network cumbersome. Adding a node requires reconfiguring the entire system. Because Ethernet supports ring, mesh, tree, and string based topologies, it offers significantly more design flexibility and deployment options for industrial communication networks.
Ubiquitous Connectivity (Industrial & Utilities)
With the proliferation of Ethernet-capable IEDs and RTUs for Utility substation automation, and the emergence of IEC 61850 as the de facto standard for substation communication devices, Ethernet is the natural migration path for networking in substations.
In a like manner, large numbers of manufacturers, standards organizations and equipment venders are embracing Industrial Ethernet technology to provide economies of scale and pave the way for migration to Ethernet backbones for factory floor applications. These organizations include:
- Industrial Ethernet Association (IEA)
- Open DeviceNet Vendor Association (ODVA)
- Modbus.org, Fieldbus Foundation
- Profinet and Profibus International (PI)
Manufacturers recognize Industrial Ethernet as an ideal means to improve productivity, quality and reduce costs. Industrial Ethernet also allows them to control information, configuration, diagnostics, safety, synchronization and motion on the same network while seamlessly connecting to information and enterprise systems.
SONET/SDH hardware elements require complexity and precision that not only affect the initial capital expenditure, but also the cost and configuration of operating and scaling the network.
Even minor modifications to SONET/SDH bandwidth require major hardware reconfigurations that can sometimes take weeks to implement as opposed to hours or even real-time for similar changes using Ethernet. Because all generations of Ethernet are functionally compatible, users typically enjoy "plug-and-play" provisioning and scaling capabilities.
Because of the ubiquitous nature of Ethernet and large numbers of suppliers, production volumes and a highly competitive market environment, Ethernet will continue to enjoy economies of scale and lower equipment costs (CAPEX). In contrast, semiconductor manufacturers are beginning to phase out production of some components for older SONET/SDH systems.
In addition, operational and maintenance costs (OPEX) are also lower for Ethernet. Compared to the complicated hierarchies of SONET/SDH based systems, Ethernet reduces operational expenses by eliminating unnecessary hardware layers. This is a huge benefit because it significantly lowers the skill level (and related costs) required for installation, maintenance and troubleshooting.
Personnel qualifications, experience levels, and training requirements are more stringent for SONET/SDH systems. Personnel with basic IP training can learn relatively quickly how to configure, manage and maintain most Ethernet systems. On the other hand, SONET/SDH system personnel typically must have stronger technical backgrounds and be able to perform at higher levels of technical sophistication.
It is important to point out differences of the total cost of ownership (TCO), including initial costs and hidden costs. Initial costs include equipment outlays, training and commissioning costs; hidden costs include operational and maintenance. Because Ethernet has a distinct advantage over SONET/SDH for both initial and hidden costs, the TCO is much lower.
Future-Proof Investment Protection
Because the scalability of Ethernet allows easy migration to 10GE and beyond, the bulk of investment in Ethernet NIC cards, switching infrastructure and industrial control devices is protected.
This offers measureable ROI from Ethernet's ability to dramatically increase bandwidth without equipment upgrades (especially fork-lift upgrades). Further, the modular design inherent in Ethernet based solutions offers "pay-as-you-grow" flexibility to minimize upfront capital expenditures.
Ethernet’s inherent modular design has been augmented by the maturation of multi-service Ethernet and its TDM over IP/Ethernet capabilities. Because TDM over IP/Ethernet is an evolutionary (as opposed to revolutionary) approach, investment protection is maximized.
Modern multi-service Ethernet networks now support virtually all legacy applications and interfaces. As a result, industrial communications network managers in particular can upgrade their legacy networks and applications at a pace that fits both their needs and budgets.
For example, a network manager can migrate to Ethernet without spending time and money on details such as replacing a dial-up serial or modem network link by simply deploying the appropriate TDM over IP/Ethernet interface card(s).
In the past, SONET/SDH systems were the only choices when it came to building out industrial communications networks. These legacy systems, although reliable and well understood , are voice-centric, relatively inflexible, difficult to scale, and most of all, not optimized for packet data transmissions.
Today, modern network designs increasingly center around Ethernet systems. Ethernet offers an efficient, low cost alternative to SONET/SDH network solutions for most applications and promises simple scalability for all network topologies and sizes.
Ethernet is less expensive than SONET/SDH in terms of initial equipment costs, ongoing maintenance and provisioning. Moreover, the complicated technical nature of SONET/SDH networks typically requires several highly trained personnel for ongoing system configuration and maintenance. Ethernet networks, on the other hand, can be configured and maintained by fewer Ethernet-trained personnel.
As the size and range of industrial communication networks increase, the search for more bandwidth will become paramount. Ethernet and multi-service Ethernet networks will become even more popular as power utilities, service providers, etc. seek to transport diverse traffic such as SCADA, office data, voice, multiple types of TDM circuits and video packets.
With data requirements inundating network traffic, legacy voice-centric SONET/SDH platforms no longer offer viable or cost-effective backbone solutions when compared to Ethernet.
Driven by improvements in control, computing and communications technologies, future network deployments will be dominated by Ethernet and simplified by the inevitable blurring of the boundaries between LANs, MANs, WANs and Industrial networks.
Steve Tang (firstname.lastname@example.org)
949-852-1972 ext. 117