Next generation data networking system requires combination of high bandwidth at faster data rate to maximize the system throughput. Current 50 T switch system with 512 lanes running at 112 Gbps will double the capacity to 100 T if the data rate increases to 224 Gbps. PAM4 signaling has been widely adopted due to below described advantages: 1) Transistor and process node can be scaled to PAM4 baud rate; 2) Better SNR performance when the SERDES can support PAM4 bandwidth; 3) PAM4 is more efficient and simpler in coding scheme than PAM6, therefore requires less power and area; 4) Backward compatibility to 112 Gbps; 5) Optical link operating with PAM4 also prefers PAM4, hence compatible electrical and optical signaling for better power and cost in the host-to-host link with optical medium. 224Gbps-PAM4 signaling, however, faces more challenges of bandwidth in the electrical channel design. It is observed that when the operation frequencies exceed 45 GHz, the electrical performance of most passive components drops off considerably, and the traditional design can hardly meet the 224 Gbps system requirements. The already challenging electrical requirements of high bandwidth are further complicated by the consideration of high SERDES I/O density.
In this paper, the signal integrity design challenges of 224Gbps-PAM4 networking system are analyzed, and the key enablement solutions are proposed to meet the end-to-end (E2E) loss budget of ≤ 40 dB. The link budget requires a reasonable distribution among the bi-direction package, board, cable, and connectors. In this paper we will describe how to break down the link budget among the system components and how to design these components within their respective loss budgets.
Package design needs to consider higher-order mode propagation and dispersion, plane resonance, transmission loss, cross talk, vertical transition, and BGA ball pitch and ball pattern. Desired next generation package trace loss target for interpretation flexibility is 0.123 dB/mm at Nyquist frequency to allow up to a total of 60 mm (TX+RX) package trace routing. This can be achieved through a.) skip-layer trace routing; b.) use of low loss material; c.) advanced copper surface treatment for smooth surface roughness. Package vertical loss target is 1 dB. This requires ≤ 0.8 mm BGA ball pitch and < 1 mm package core thickness. Thick core will introduce dramatic loss beyond 60GHz. Small BGA ball size can further reduce package discontinuities and package loss. A 224Gbps-PAM4 package design practice was described in our 2021 DesignCon paper [1] that used 0.5 mm BGA ball pitch. This paper discusses the 0.8 mm ball pitch package and PCB design to address the large-formfactor package reliability concerns.
PCB design requires a careful architectural planning for the channel placement, trace breakout/break-in and via optimization to minimize the horizontal and vertical loss, and the via-to-via and trace-to-via coupling, especially when dealing with the small ball pitch high-density board design. Via coupling due to the deep BGA row arrangement in the high-IO count board design requires novel via configuration to diminish both the trace-to-via and via-to-via coupling. Via stub length significantly impacts the PCB loss beyond 45 GHz and should be controlled to be < 6 mils in the 224Gbps-PAM4 board design. A desired next generation PCB trace loss target for interpretation flexibility is 0.95 dB/inch at Nyquist frequency in the global routing area to allow up to a total of 10-inch (TX+RX)
PCB trace routing. This can be achieved through the skip-layer trace design and use of ultra-low loss material as well as HVLP copper surface treatment for a smooth copper surface roughness. PCB skip-layer trace routing requires more routing layers and therefore a deeper via transition. PCB via vertical loss should not exceed 1 dB when the via is connected to the longest trace. This requires the via length to be less than 65 mils and the via stub length smaller than 6 mils. Well-controlled variations of the dielectric material properties, dielectric thickness and copper geometry are also important for a successful design. A 224Gbps-PAM4 PCB breakout design practice was described in our 2021 DesignCon paper [1] that utilized 0.5 mm BGA ball pitch for FPGA applications. Similar optimization methodology was leveraged in the 0.8 mm ball pitch channel breakout and via transition design.
The package and PCB are often designed separately, and the optimized package and PCB models are then cascaded for performing a link simulation. The BGA ball is included in the package model and should not be double counted in the PCB model. How to terminate the PCB ball pad is critical for an accurate PCB modeling. In this paper, a coax port at the package-PCB interface is designed that accurately captures the pad capacitance yet not introduces any artificial discontinuities. The cascaded package and PCB models are well correlated with the integrated package-PCB model in the frequency range of DC-80GHz.
Several cable/connector configurations were investigated, and it is believed that due to the backward compatibility limitation, the projected cable assembly loss target based on current cable and connector characteristics is ~ 15 dB (1 m cable + 2 connectors). The cable assembly design is not within the scope of this paper, however, with some technological breakthroughs and the well-controlled manufacturing tolerance, the 1 m cable + 2 connector configuration should target to meet ~ 10 dB loss target to the Nyquist frequency. An example of the connector mating interface optimization is described in this paper that showed 3.2 dB loss improvement in a cable assembly with 2 connectors.
The above stated approaches drive the key enablement solutions to a successful 224Gbps- PAM4 high-performance and high-density system design.
The paper referenced here received the Best Paper Award at DesignCon 2022. To read the entire DesignCon 2022 paper, download the PDF.
