Thread Links Date Links
Thread Prev Thread Next Thread Index Date Prev Date Next Date Index

Re: [STDS-802-3-400G] 400G Electrical Channels



Chris,

We are no longer a study group, we are a task force with stated approved objectives. Much of this conversation in your prior email was primarily not about 400GbE, but about 50G and other rates that might use it. Hence, my request to take 50GbE, 100GbE, or 200GbE discussion onto DIALOG, where it would belong.

 

Regards,

 

John

 

 

From: Chris Cole [mailto:chris.cole@xxxxxxxxxxx]
Sent: Monday, January 05, 2015 8:23 PM
To: DAmbrosia, John; STDS-802-3-400G@xxxxxxxxxxxxxxxxx
Subject: RE: 400G Electrical Channels

 

John

 

The thread is about 8x50G CDAUI-8 specification and how it should be written. It is entirely within scope to consider full range of applications. We have learned from past projects that when we don’t consider the full range of applications, we cause ourselves grief.

 

This thread is also consistent with discussion during Study Group when developing our project objectives. We agreed that lower speed break-out applications are important and can be considered as part of the specifications even if they are not formal objectives.

 

Chris

 

From: John_DAmbrosia@xxxxxxxx [mailto:John_DAmbrosia@xxxxxxxx]
Sent: Monday, January 05, 2015 4:45 PM
To: Chris Cole; STDS-802-3-400G@xxxxxxxxxxxxxxxxx
Subject: RE: 400G Electrical Channels

 

All,

This thread started out talking about topics related to 400GbE and an 8x50 approach, but then digressed into 1x50 and 4x50 (200GbE) discussions.

 

If the conversation is going to move onto these topics (which are not part of the scope of the 802.3bs project), I request that you move such threads over onto the 802-3 DIALOG reflector.  See http://www.ieee802.org/3/dialog_reflector.html for further information on subscribing to the Dialog reflector.

 

Regards,

 

John D’Ambrosia

Chair, IEEE P802.3bs 400GbE Task Force

 

From: Chris Cole [mailto:chris.cole@xxxxxxxxxxx]
Sent: Monday, January 05, 2015 7:22 PM
To: STDS-802-3-400G@xxxxxxxxxxxxxxxxx
Subject: Re: [STDS-802-3-400G] 400G Electrical Channels

 

Hi Scott,

 

You raise a valid question as to whether CAUI-2 (2x50G) electrical I/O will be interesting.  

 

ASICs with 50Gb/s SerDes can support 100Gb/s Ethernet with two configurations.

 

1.      Reduce the SerDes rate to 25Gb/s and support 100G with 4x25G ( CAUI-4) I/O

2.      Keep the SerDes rate at 50Gb/s and use new 2x50G (CAUI-2) I/O

 

Reducing the SerDes rate to 25Gb/s results in half the 100G ports supportable by operating at 50Gb/s rate. This wastes half the ASIC switching bandwidth. Therefore if ASICs with 50Gb/s SerDes do not use 2x50G I/O for 100G ports, 200G ports become dominant and 100G becomes legacy. This is similar to how ASICs with 25G SerDes are now used to support 40G. The 25G SerDes are operated at 10G to support 4x10G legacy links and not as a 2x20G I/O to double 40G ports.


There is no disagreement that SFP has lower cross-talk than QSFP. However, the issue is how do we develop a 50Gb/s per single lane spec. If the specification only supports SFP level cross-talk, than 4x break-out using CFP4/QSFP or 8x break-out using CFP2 applications are precluded. This is too restrictive. The more general specification assumes multi-channel crosstalk (either 4x or 8x), which support 4x and 8x break-out modules. SFP applications can then take advantage of the loss allocated to multi-channel cross-talk to support longer host reach.

 

So future 1x50G, 2x50G (if of interest), and 4x50G electrical specifications should be similar or same as 8x50G CDAUI-8 specification we develop in 802.3bs.

 

Chris

 

From: Scott Kipp [mailto:skipp@xxxxxxxxxxx]
Sent: Monday, December 29, 2014 9:37 AM
To: Chris Cole; STDS-802-3-400G@xxxxxxxxxxxxxxxxx
Subject: RE: 400G Electrical Channels

 

Chris,

 

I haven’t heard of much interest in 2X50G 100GbE, but IEEE could consider a future project that includes 1X50, 2X50 and 4X50.  I’d be glad to hear if others are interested in 2X50G 100GbE.  200GbE, based on 4X50G, has been added to the Ethernet Alliance roadmap that can be viewed at:

http://www.ethernetalliance.org/subcommittees/roadmap-subcommittee/

 

My claim is that there is more crosstalk in a QSFP28 than an SFP28.  I think this is backed up in all simulations that have been submitted to this group.  In the case of 32GFC vs 128GFC, Avago has proposed that there is a 2.5dB electrical crosstalk penalty for using QSFP28 vs an SFP28.  Others, like John Petrilla and Jonathan King, are more versed in these crosstalk simulations and I hope that they can comment on the differences.  Molex or TE should also be able to discuss the differences in the crosstalk between the two connectors.  I didn’t think this assumption that crosstalk is worse in a QSFP28 was really up for debate, the question is how much worse it is. 

 

I’ll try and get some data that shows that electrical channels from the same SERDES in an ASIC support different lengths of traces depending on if they channels are routed through a QSFP28 or an SFP28.  The SFP28 should support longer channels with the same BER than a QSFP28 on a similar board. 

 

My second claim is that the crosstalk penalties between 1X50G in SFP56 and 8X50G in CFP2 will be even larger than 2.5dB.  I don’t have data to back this assumption up yet and would like to hear if anyone has such data.

 

I agree that the standard should specify loss and not distance. 

 

Kind regards,

Scott

 

From: Chris Cole [mailto:chris.cole@xxxxxxxxxxx]
Sent: Saturday, December 20, 2014 11:25 PM
To: Scott Kipp; STDS-802-3-400G@xxxxxxxxxxxxxxxxx
Subject: RE: 400G Electrical Channels

 

Hi Scott,

 

Thank you for bringing into the discussion the perspective that we should be cognizant of future 1x50G and 4x50G I/O when developing 8x50G CDAUI-8 specifications. 2x50G should also be included as this will double ASIC and front panel 100G port density.

 

However, the conclusion that 1x50G I/O will have lower cross-talk penalty than 8x50G is not supported by our experience with 1x10G and 1x25G I/O. 10G I/O is now used in break-out configurations, like 4x10G QSFP+ and 10x10G CFP2. 25G I/O is now used in break-out configurations like 4x25G CFP4 followed by QSFP28, and may be used in 16x25G CDFP. This strongly suggests that we are likely to see 50G I/O break-out configurations like 4x50G CFP4 followed by QSFP28, and 8x50G CFP2. Therefore the same cross-talk penalty is appropriate for 1x50G, 2x50G, 4x50G and 8x50G specifications.

 

With respect to material loss, the normative specification should specify loss, allowing the designer to trade-off material quality (and cost) against reach. This means we should write the 8x50G specification so that it can be used for a variety of applications. Precluding 8x50G I/O Switch applications until 4x100G I/O becomes available seems too restrictive.

 

Chris

 

From: Scott Kipp [mailto:skipp@xxxxxxxxxxx]
Sent: Friday, December 19, 2014 8:05 AM
To: STDS-802-3-400G@xxxxxxxxxxxxxxxxx
Subject: [STDS-802-3-400G] 400G Electrical Channels

 

400G Task Force,

 

In the electrical ad hoc yesterday, I objected to using the materials loss tool because I thought it could be misused.  After Beth explained how people should use the tool, I could not retract my objection before the conversation went out of control.  I agree to use the tool to predict losses, but would like for people to use the tool appropriately.

 

I would like to build consensus to what kinds of electrical channels this project should support.  The question that arises is:

 

Should we define the same electrical channels for 1X50G, 4X50G and 8X50G when they are serving different markets and volumes?

 

I propose that 8X50G (CDAUI-8) electrical channels should be different than channels that will eventually be defined for 1X50GbE (LAUI) because:

 

1.      Crosstalk - Crosstalk will impact the channel by more than 3dB.  I base this off some excellent work that Avago presented to T11 earlier this month.  In 14-409v0, Avago showed how going from 1X28G (32GFC) to 4X28G (128GFC) caused a 2.5dB decrease in the channel mainly due to crosstalk in the QSFP28 connector.  I would expect more crosstalk than 2.5dB when the speed and number of channels are doubled.

2.      Small market -  The 400GbE market based on 802.3bs standards will be small compared to single lane 50GbE channels that won’t be standardized for a few years.  802.3bs needs to recognize that this project is targeting a very small market for high-end router interconnects.  The complete router market was less than 2M ports/year according to leading analyst firms and not growing much.  16X25G and 8X50G will never be deployed in millions of ports/year because it will be made obsolete by a future 4X100G 400GbE project.  50GbE will go after a larger market and be designed into high-volume, server interconnects by the end of the decade while 4X100G technology will be in the next decade.

3.      High cost – Because of the low volume of 8X50G and the high quality required by the router market, the cost will be high for 400GbE.  Because of the premium nature of the router market, electrical channels will be made of high quality, high cost materials that can be absorbed in high end router products.

4.      Better Materials – The group should model channels based on the better materials supported by the high-end router market in 2018.  As others said, the materials being targeted in 802.3bs should be different than the materials targeted for high volume servers in 802.3bj and a future 50GbE project.

 

Based on these 4 factors that are related, we need to agree on a distance that needs to be supported and consider crosstalk and the loss/inch of better materials.  Based on these difference, I bet that CDAUI-8 electrical channels will be 5dB different than the eventual LAUI-1 electrical channels.

 

We should make sure that CDAUI-8 is optimized for our application, not for LAUI-1 that might be optimized for server and switches.

 

I’d like to hear if you disagree with these claims.

 

Kind regards,

Scott Kipp