At the March 2016 OFC conference, Inphi announced its delivery of a 100G, QSFP28, PAM-4, pluggable transceiver with 80km reach. PAM technology has been utilized for 100G transmissions (Inphi is a specialist in this area) before but at much shorter distances. Pulse-amplitude modulation (PAM) is an analog transmission scheme similar to NRZ but with multi-level signaling, with PAM-4 utilizing four levels to signal one of four possible symbols (2 bits per symbol). During the announcement, Microsoft also publicly announced that it will begin sourcing the pluggable PAM-4 technology from Inphi for interconnection of its regional, metro-distributed data centers, which by definition are within 70km of each other. Coherent technology will continue to be used elsewhere. The metro-distributed data center deployment model builds and interconnects a number of smaller data centers within a metropolitan area instead of deploying a single hyperscale data center in the region. Microsoft also divulged that it was their intention to turn up all 40, 100G wavelengths at one time (4Tb/s with each carrier occupying 100GHz channel spacing) on a fiber pair, utilizing all available colors in the fixed-wavelength portfolio.
Some at the conference reacted to the Inphi/Microsoft announcement by declaring the obsolescence of existing optical DCI/coherent DWDM solutions. Although the Inphi/Microsoft announcement is exciting news, ACG thinks the PAM-4 technology is far more complementary to existing coherent DWDM solutions than competitive for multiple reasons.
Figure 1. Optical Reach for 100G Technologies
Reach. The PAM-4 solution covers a portion of the optical reach needed to interconnect data centers. Below 10km, IEEE 802.3ba 100G pluggable optics are readily available with 100GBASE-LR4 supporting 10km reach in a QSFP28 package for cost-effective point-to-point connectivity. The 100GBASE-ER4 specification for 40km reach has been more challenging for optics suppliers to deliver and remains either in larger packages (example, CFP, CFP2) or in nonstandard formats, meaning non-interoperable across vendors. So where does the PAM-4 technology fit? In general, its initial fit appears to be in the <40km range as an alternative to existing, suboptimal pluggable solutions. We believe there is limited overlap with coherent DWDM solutions in this range. The solution also plays in the 40–80km range as an alternative to optical DCI/coherent DWDM solutions for some deployment scenarios.
So, based solely upon reach, a logical question is how much of the optical DCI/coherent DWDM market is covered by 40–80km? ACG Research recently completed a worldwide survey of data center service providers, including network service providers, cloud service providers, Internet content providers and Internet eXchange providers. This research will be available in a published report later this month (April). One of the questions we asked the service providers was the proportion of optical reach needed to cover their data center interconnections today and in 2019. What we found is that service providers on average believe that 30–80km optical reach is needed for approximately 30% of their data center interconnections. The results indicate a modest increase between today and 2019. Based upon this preliminary research, we have a sense of the addressable optical DCI market for this technology. However, we also believe that service providers will consider at least three other factors in making their DCI deployment decisions.
Figure 2. Data Center Interconnect Optical Reach
Operations. Every data center deployment is not like Microsoft’s plan for metro-distributed data centers, which is to turn up all 4Tb/s of connectivity in a point-to-point fashion on day one of data center activation. By deploying all 40 wavelengths at once, Microsoft could reduce the incremental cost per wavelength of deploying dispersion compensation on the fiber, which is required for PAM but not for coherent DWDM solutions. Dispersion compensation costs include both the capital equipment as well as the operational costs associated with installing and tuning the compensators. Microsoft also avoids the operational complexity of deploying fixed wavelength pluggable optics incrementally, where inventory and on-site resources are required every time a change or a wavelength addition is needed.
Other service providers that have existing metro optical networks may not want to deploy in this manner. They may not want the added complexity of dealing with dispersion compensation for PAM deployments. Some may want to utilize existing metro optical infrastructure and/or deploy in a mesh architecture. Still other service providers may not have the same visibility as Microsoft with regard to their data center connectivity needs. They may need to be more agile and utilize a pay-as-you-go/pay-as-you-grow deployment model where they add interconnection capacity over time and in alignment with their data center compute/storage capacity and revenue generation. An incremental deployment model is just more operationally complex with fixed-wavelength pluggable optics.
Fiber Scarcity. When fiber is scarce or expensive, fiber optic transmission efficiency (bits per Hz) increases in importance. The PAM-4 solution delivers an efficiency ratio of 1 with 100Gb/s transmission occupying 100GHz channel spacing. 16-QAM coherent DWDM modulation offers 200Gb/s in 50GHz channels or an efficiency ratio of 4. Recent flexible grid implementations have an even greater efficiency ratio approaching 7. If more than 4Tb/s of connectivity is needed and incremental fiber is scarce or expensive, service providers may need to utilize the more efficient coherent DWDM system to squeeze more bandwidth through their limited fiber resources.
Programmability. Fixed-wavelength pluggable optics do not advance the broader drive toward a programmable, agile, SDN enabled optical underlay. SDN and NFV are changing all aspects of the ICT industry, including optical solutions. Service providers are looking to utilize intelligence, automation and programmability to reduce operational costs and ensure that network resources adapt to changing business and networking conditions across protocol layers, including optics and IP. Many demonstrations at OFC utilized SDN control and service automation combined with a programmable optical layer to showcase network efficiency and adaptability. The ONS 2016 conference had similar demonstrations with ONOS and ODL controllers programming in near real-time optical and IP networking infrastructure.
Figure 3. Example of a Mixed Technology DCI Deployment
The Inphi PAM-4, QSFP28 solution is an exciting achievement and addresses a very real need in the sub-80km 100G market. We believe the solution is actually far more complementary than competitive to existing optical DCI/coherent DWDM solutions. Most service providers will utilize an all-of-the-above approach to their 100G DCI deployments just as they did before with dark fiber, IEEE pluggables and coherent DWDM options. PAM-4 meets the needs of data center operators, such as Microsoft, that intend to turn up 4Tb/s of transmission capacity in a point-to-point fashion between data centers in a ~70km metro-distributed network. However, if a provider needs longer reach or more than 4Tb/s per fiber pair or an incremental growth operational model or if a service provider is looking to advance its programmable, SDN enabled network, then a tunable, coherent DWDM solution is a better fit. PAM-4 or coherent DWDM for data center interconnections? Yes!
For more information about ACG’s DCI services or ACG’s 4Q15 DCI Market Share report, contact info@acgcc.com.
Click for more information about Tim Doiron and his recent articles.
