Re: [HSSG] 40G MAC Rate Discussion
Hello,
Please take these comments as from an end customer. I'm the Group Lead
for Networking and Security for NERSC (www.nersc.gov) that would best be
described as a high end data center.
40G Ethernet interfaces in servers is something that we would never
consider buying for a few specific reasons.
1) We can bond 4x10G today. Trunk hashing causes a set of problems but
not enough that I would roll a major upgrade for no bandwidth increase.
2) It will be too little to late (for us). At NERSC we feel that 100G
is already 2 years behind schedule and that this tardiness is feeding
Infiniband and Infiniband attached storage. You can translate this to
"your Ethernet development $$$ is going to IB" and IB is the next logical
step for a data center interconnect. Ethernet is playing catch-up and
stopping at 40Gig won't help.
3) 40 Gig and 100 Gig don't align gracefully. You have to match 5x40
to 2x100. We've already seen issues with lan-fi/wan-fi bandwidth
mismatches with a minor speed difference and don't want to go through
the growing pains of a mixed 40/100 environment.
If someone wants to give me a sales pitch on why 40G out of a server
is going to make our center better, please do so. All of our hopes
right now are on getting the 100G spec out ASAP or it will be competing
with 160G IB.
We also feel for WAN community that is sitting on OC768 fiber and hut
layouts that won't easily adapt to 100G. That may be a reason to do
a 40G interface for the wan but not in the data center.
On a personal note: I want to thank the HSSG. While we were hoping for
100G earlier we understand the monumental task that you are undertaking.
I really respect the effort that you are putting into 100G Ethernet.
Thanks,
Brent Draney
NERSC Networking and Security
www.nersc.gov
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Shimon,
I agree that developing a standard for 40GigE SR in data centers should
help drive demand for 100GigE at aggregation points deeper in the
network.
While initially I could appreciate the hesitation by some that working
on 40GigE might delay 100GigE standardization, after thinking about this
more I tend to agree with you that whatever SR PHY might be defined for
40GigE should be able to be leveraged to a large extent for a 100GigE SR
PHY.
For example, since we're looking at parallel PHYs to begin with, a
4x10Gig over 100m of MMF could have a similar 10Gig lane PMD defined as
for a 10x10Gig SR PHY. And the same for a 2x20Gig and 5x20Gig SR.
Perhaps the PCS/PMA approach presented by Mark Gustlin could also be
applied with either fewer virtual/CTBI lanes or slower rate virtual/CTBI
lanes.
I'd be interested in hearing other views regarding the viability of
using similar technical approaches for 40GigE and 100GigE SR PHYs.
...Dave
David W. Martin
Nortel Networks
dwmartin@nortel.com
+1 613 765 2901 (esn 395)
~~~~~~~~~~~~~~~~~~~~~~~~~
-----Original Message-----
From: Shimon Muller [mailto:Shimon.Muller@SUN.COM]
Sent: Monday, April 02, 2007 7:52 PM
To: STDS-802-3-HSSG@listserv.ieee.org
Subject: Re: [HSSG] 40G MAC Rate Discussion
Thank you for the clarification, John.
On that point, as someone who started the 40Gb discussion back in
January, I would like to make it clear in no uncertain terms what our
motivations were and still are: in no way do we intend to impede or
slow down the development and/or the adoption of 100Gb Ethernet.
We believe that doing a standard for 100Gb is important but not enough,
for the following reasons:
- The 40Gb speed will enable us to get the most out of our servers
until 100Gb becomes technically and economically viable for server
connectivity. We believe that there is a 5-year window of opportunity
for this market.
- The two speeds should be addressing two distinct markets. This can be
accomplished by defining the PMDs based on reach, with 40Gb defined for
short-reach datacenter connectivity only.
- 40Gb connectivity at the server will require a faster aggregation
speed
even in the datacenter. This will increase the market potential for
100Gb.
- Unless the 100Gb effort starts today, the technology will not be ready
when we need it for servers in 2015. I am sure your heard me say this
at the last meeting, and I meant it.
- The development of a 40Gb standard will be highly leveraged: either
from the work that was done in other standards bodies, or from the
work that needs to be done for 100Gb anyway. Therefore, 40Gb should
in no way slow down the 100Gb effort.
What has been puzzling to me in this debate ever since it started is:
how can 40Gb server connectivity in the datacenter hurt those of you who
believe that 100Gb is the right speed for aggregation links in service
provider networks? I am certainly at a point where I understand and
respect the needs of your market. All I am asking in return is the same.
Any comments that will help me understand and address the above concerns
would be very much appreciated.
Regards,
Shimon.
John DAmbrosia wrote On 03/30/07 19:48,:
> All,
>
> From discussions I have had, I sense that there may be some confusion
> regarding the proposal for adding a 40Gb/s MAC rate objecitve. 40 Gb/s
> has been proposed as an additional MAC rate, not as a replacement for
> the current objective of 100 Gb/s.
>
> The presentations given have focused on the needs of servers / end
> stations:
>
>
http://grouper.ieee.org/groups/802/3/hssg/public/jan07/muller_01_0107.pd
f
>
>
http://grouper.ieee.org/groups/802/3/hssg/public/mar07/frazier_01_0307.p
df
>
>
http://grouper.ieee.org/groups/802/3/hssg/public/mar07/vandoorn_01_0307.
pdf
>
> I encourage all to review the above presentations, and use the
> reflector to further discuss them and / or the proposed objective.
> This will help to assist the SG in preparing for the April Interim, as
> well as making decisions regarding the project's objectives.
>
> Best Regards,
>
> John D'Ambrosia
>
>
------------------------------------------------------------------------
>
> *From:* John DAmbrosia [mailto:jdambrosia@FORCE10NETWORKS.COM]
> *Sent:* Monday, March 26, 2007 5:11 PM
> *To:* STDS-802-3-HSSG@listserv.ieee.org
> *Subject:* [HSSG] 40G MAC Rate Discussion
>
> Dear HSSG Members,
>
> As discussed in the action plan pulled together at the end of the
> plenary week, further discussion on adding a 40G MAC rate as an
> objective is needed. Topics of interest include: economic feasibility,
> broad market potential, and what pmds are desired for this rate.
>
> I would like to encourage all to use the reflector for this subject
> matter, which could assist individuals in their preparations for the
> interim meeting next month.
>
> Best Regards,
>
> John D'Ambrosia
>