Thread Links Date Links
Thread Prev Thread Next Thread Index Date Prev Date Next Date Index

Re: [802.3_100GNGOPTX] Emerging new reach space



Hi Petar,

I welcome your call for innovation.

Did I understand you to say that cost/performance has decreased for the HPC hardware elements other than the data interconnect ? That sounds strange, since a 10x performance for 3x cost would be consistent with lower cost/performance.

Is it true that in HPC, if the number of processors goes up as N, the number of data links goes up as ~N squared ? ie the performance demanded from the total interconnect is increasing at the square of the processing power ?

Best wishes

jonathan

 

 

From: Petar Pepeljugoski [mailto:petarp@xxxxxxxxxx]
Sent: Wednesday, November 16, 2011 12:53 PM
To: STDS-802-3-100GNGOPTX@xxxxxxxxxxxxxxxxx
Subject: Re: [802.3_100GNGOPTX] Emerging new reach space

 

Dan,

While I understand that the discussion is in the context of LR4, I'd like to comment on the 10x performance for 3x the cost. This model does not apply to the HPC application space any more and I believe other applications will reach the same point soon. Since the number of links continues to grow, bandwidth costs have already reached critical point relative to the rest of the system and further increases are not sustainable. If you look at the rest of the hardware elements of a system (memory, cpu's, drives, I/O ports etc.) over the years their costs have remained the same or declined while the performance has continued its upward trend.
 
I think it is time for innovation, to identify the costs and propose alternative solutions to address this issue, like putting the modules deep inside, simplified packaging and integration. The benefit will be not just the cost, but also power consumption and higher density.

Regards,

Peter

Petar Pepeljugoski
IBM Research
P.O.Box 218 (mail)
1101 Kitchawan Road, Rte. 134 (shipping)
Yorktown Heights, NY 10598

e-mail: petarp@xxxxxxxxxx
phone: (914)-945-3761
fax:        (914)-945-4134




From:        Daniel Dove <ddove@xxxxxxx>
To:        STDS-802-3-100GNGOPTX@xxxxxxxxxxxxxxxxx
Date:        11/16/2011 02:55 PM
Subject:        Re: [802.3_100GNGOPTX] Emerging new reach space





Hi Ali,

I like your first point.

Your last point was the first thing that came to my mind. We really need to understand customer sensitivity to 8x fiber runs of that length.

As for the cost modeling, today we use 10x performance for 3x the cost as a goal. Given the 2.5x performance relationship between 100G and 40G, I would say that a reasonable goal should be 2.5x the performance for equivalent cost.

As people consider ways to solve the 500m to 1Km reach, they should be cognizant of the relative cost/performance range to fit within, and consider customer sensitivity to fiber installation costs/impacts as well.

Regards,

Dan



From: Ali Ghiasi <aghiasi@xxxxxxxxxxxx>
Reply-To:
Ali Ghiasi <
aghiasi@xxxxxxxxxxxx>
Date:
Wed, 16 Nov 2011 09:03:16 -0800
To:
100G Group <
STDS-802-3-100GNGOPTX@xxxxxxxxxxxxxxxxx>
Subject:
Re: [802.3_100GNGOPTX] Emerging new reach space


Jack

If there is another LR4 PMD out there the best starting point would be 40Gbase-LR4, look at its cost structure, and build a 40G/100G compatible PMD.

We also need to understand the cost difference between parallel MR4 vs 40Gbase-LR4 (CWDM).  The 40Gbase-LR4 cost with time could be assumed  
identical to the new 100G MR4 PMD.  Having this baseline cost then we can compare its cost with 100GBase-LR4 and parallel MR4.  The next step is to take
into account higher cable and connector cost associated with parallel implementation then identify at what reach it gets to parity with 100G (CWDM) or
100G (LAN-WDM).

In the mean time we need to get more direct feedback from end users if the parallel SMF is even an acceptable solution for reaches of 500-1000 m.

Thanks,
Ali



On Nov 15, 2011, at 8:41 PM, Jack Jewell wrote:

Thanks for this input Chris.
 I'm not "proposing" anything here, rather trying to frame the challenge, so that we become better aligned in how cost-aggressive we should be, which guides the technical approach. As for names, "whatever works" :-)  It would be nice to have a (whatever)R4, be it nR4 or something else, and an english name to go with it. The Structured Data Center (SDC) links you describe in your Nov2011 presentation are what I am referencing, except for the restriction to "duplex SMF." My input is based on use of any interconnection medium that provides the overall lowest-cost, lowest-power solution, including e.g. parallel SMF.
Cost comparisons are necessary, but I agree tend to be dicey. Present 10GbE costs are much better defined than projected 100GbE NextGen costs, but there's no getting around having to estimate NextGen costs, and specifying the comparison. Before the straw poll, I got explicit clarification that "LR4" did NOT include mux/demux IC's, and therefore did not refer to what is built today. My assumption was a "fair" cost comparison between LR4 and (let's call it)nR4 - at similar stage of development and market maturity. A relevant stage is during delivery of high volumes (prototype costs are of low relevance). This does NOT imply same volumes. It wouldn't be fair to project ER costs based on SR or copper volumes. I'm guessing these assumptions are mainstream in this group. That would make the 25% cost target very aggressive, and a 50% cost target probably sufficient to justify an optimized solution. Power requirements are a part of the total cost of ownership, and should be considered, but perhaps weren't.
The kernel of this discussion is whether to pursue "optimized solutions" vs "restricted solutions." LR4 was specified through great scrutiny and is expected to be a very successful solution for 10km reach over duplex SMF. Interoperability with LR4 is obviously desirable, but would a 1km-spec'd-down version of LR4 provide sufficient cost/power savings over LR4 to justify a new PMD and product development? Is there another duplex SMF solution that would provide sufficient cost/power savings over LR4 to justify a new PMD and product development? If so, why wouldn't it be essentially a 1km-spec'd-down version of LR4? There is wide perception that SDC's will require costs/powers much lower than are expected from LR4, so much lower that it's solution is a major topic in HSSG. So far, it looks to me like an optimized solution is probably warranted. But I'm not yet convinced of that, and don't see consensus on the issue in the group, hence the discussion.
Cheers, Jack

From: Chris Cole <chris.cole@xxxxxxxxxxx>
Reply-To:
Chris Cole <
chris.cole@xxxxxxxxxxx>
Date:
Tue, 15 Nov 2011 17:33:17 -0800
To:
<
STDS-802-3-100GNGOPTX@xxxxxxxxxxxxxxxxx>
Subject:
Re: [802.3_100GNGOPTX] Emerging new reach space


Hello Jack,
 
Nice historical perspective on the new reach space.
 
Do I interpret your email as proposing to call the new 150m to 1000m standard 100GE-MR4? J
 
One of the problems in using today’s 100GE-LR4 cost as a comparison metric for new optics is that there is at least an order of magnitude variation in the perception of what that cost is. Given such a wide disparity in perception, 25% can either be impressive or inadequate.

What I had proposed as reference baselines for making comparisons is 10GE-SR (VCSEL based TX), 10GE-LR (DFB laser based TX) and 10GE-ER (EML based TX) bit/sec cost. This not only allows us to make objective relative comparisons but also to decide if the technology is suitable for wide spread adoption by using rules of thumb like 10x the  bandwidth (i.e. 100G) at 4x the cost (i.e. 40% of 10GE-nR cost) at similar high volumes.

 
Using these reference baselines, in order for the new reach space optics to be compelling, they must have a cost structure that is referenced to a fraction of 10GE-SR (VCSEL based) cost, NOT referenced to a fraction of 10GE-LR (DFB laser based) cost. Otherwise, the argument can be made that 100GE-LR4 will get to a fraction of 10GE-LR cost, at similar volumes, so why propose something new.
 
Chris
 
From: Jack Jewell [mailto:jack@xxxxxxxxxxxxxx]
Sent:
Tuesday, November 15, 2011 3:06 PM
To:
STDS-802-3-100GNGOPTX@xxxxxxxxxxxxxxxxx
Subject:
[802.3_100GNGOPTX] Emerging new reach space
 
Following last week's meetings, I think the following is relevant to frame our discussions of satisfying data center needs for low-cost low-power interconnections over reaches in the roughly 150-1000m range. This is a "30,000ft view,"without getting overly specific.
Throughout GbE, 10GbE, 100GbE and into our discussions of 100GbE NextGenOptics, there have been 3 distinct spaces, with solutions optimized for each: Copper, MMF, and SMF. With increasing data rates, both copper and MMF specs focused on maintaining minimal cost, and their reach lengths decreased. E.g. MMF reach was up to 550m in GbE, then 300m in 10GbE (even shorter reach defined outside of IEEE), then 100-150m in 100GbE. MMF reach for 100GbE NextGenOptics will be even shorter unless electronics like EQ or FEC are included. Concurrently, MMF solutions have become attractive over copper at shorter and shorter distances. Both copper and MMF spaces have "literally" shrunk. In contrast, SMF solutions have maintained a 10km reach (not worrying about the initial 5km spec in GbE, or 40km solutions). To maintain the 10km reach, SMF solutions evolved from FP lasers, to DFB lasers, to WDM with cooled DFB lasers. The 10km solutions increasingly resemble longer-haul telecom solutions. There is an increasing cost disparity between MMF and SMF solutions. This is an observation, not a questioning of the reasons behind these trends. The increasing cost disparity between MMF and SMF solutions is accompanied by rapidly-growing data center needs for links longer than MMF can accommodate, at costs less than 10km SMF can accommodate. This has the appearance of the emergence of a new "reach space," which warrants its own optimized solution. The emergence of the new reach space is the crux of this discussion.
Last week, a straw poll showed heavy support for "a PMD supporting a 500m reach at 25% the cost of 100GBASE-LR4" (heavily favored over targets of 75% or 50% the cost of 100GBASE-LR4). By heavily favoring the most aggressive low-cost target, this vote further supports the need for an "optimized solution" for this reach space. By "optimized solution" I mean one which is free from constraints, e.g. interoperability with other solutions. Though interoperability is desirable, an interoperable solution is unlikely to achieve the cost target. In the 3 reach spaces discussed so far, there is NO interoperability between copper/MMF, MMF/SMF, or copper/SMF. Copper, MMF and SMF are optimized solutions. It will likely take an optimized solution to satisfy this "mid-reach" space at the desired costs. To repeat: This has the appearance of the emergence of a new "reach space," which warrants its own optimized solution. Since the reach target lies between "short reach" and "long reach," "mid reach" is a reasonable term
Without discussing specific technical solutions, it is noteworthy that all 4 technical presentations last week for this "mid-reach" space involved parallel SMF, which would not interoperate with either 100GBASE-LR4, MMF, or copper. They would be optimized solutions, and interest in their further work received the highest support in straw polls. Given the high-density environment of datacenters, a solution for the mid-reach space would have most impact if its operating power was sufficiently low to be implemented in a form factor compatible with MMF and copper sockets.
Cheers, Jack