Re: Equalization and benefits of Parallel Optics.
Chris,
You had sent me a request for information similar to this. I have not been
very busy with other things so could not respond properly. Hopefully this
will also help and add weight to our, the customers, concerns.
I had a meeting with a major technology consumer last week. I will not
state any names, but you can guess who. They were very interested in 10GbE
early on as part of their data facility overbuild plans. It is in this
context that I want to make these comments.
As part of my role in the design and implementation of advanced
architecture data networks, I have been involved with the design and
implementation of data facilities for about ten years. This e-mail is a
simple overview of how these are designed and implemented.
Large data facilities take a long time to plan and build. As such the
initial design and implementation is based on existing mature
technology. The initial construction within the data facility is grouped
in a common area, generally one end of room. If the routers are located in
the same room as the servers, they will generally be along a wall in the
data room. The servers and data storage systems are put in the room next
to the area where the routers were installed. Data switches which
aggregate the traffic and manage the server traffic flow are sometimes
located with the routers, and sometimes located with the servers. Where
there is planed router growth, the switches are installed adjacent to the
servers. From time to time, the routers are located in a different room,
with longer reach connections between the aggregation data switches and the
routers.
In most cases, the rows of equipment are at most about 20 equipment
racks/cabinets long. For 24in racks that is about 40 feet (12.2m), for
19in racks that is about 32 feet (9.75m). Most of the time data switches
will be in the same row as the servers, to reduce the amount of cable trays
and cable handling between rows. Often the aggregation data switches will
be in the middle of the row to reduce the distance of the interconnect
cabling. The row to row distance is about 8 feet (2.5m). Even with the
riser from the bottom of one rack/cabinet at one end of the row to the
bottom of the rack/cabinet at the other end of adjacent rows, the
interconnections are less than 20m.
For new technology overbuilds of existing data rooms, the new technology
systems are grouped together in a different area of the data room than the
mature technology. The data switches to support the new technology systems
are co-located in the same row with those systems. In these situation, the
vast majority of the new technology interconnections are within the row of
the new technology overbuild, less than 20m. By some estimates, data rooms
designed specifically around 10GbE will be at least two years away. Given
that the initial deployment of 10GbE will be in new technology overbuilds
of these data rooms, it is very important that the ability to understand
and use the same construction techniques and technologies, such as the type
of fiber and fiber management systems.
It is a personal estimation on my part that the high capacity data switches
will be at about 500+ Gb aggregate bandwidth per bay/cabinet by about
2002. As such, they will handle a total of 50 10GbE links. With a limit
of 19 racks for servers, even at single non-redundant 10Gb link each that
is 19 links. For servers with redundant links that is 38 ports, or about
380Gb aggregate bandwidth which would exceed the ability of the data switch
interconnect with any outside communications systems. In the case of
exceeding the aggregate bandwidth of any one switch, multiple switches are
interconnected. These switches could be located next to each other or, as
is more likely, at equal distances long the row of servers. As more and
more and more servers come on line, the number of supporting data switches
increases along with the interconnections between the data switches. In
this situation, the implementation of the interconnections will be about
1/3 (33%) of the data switch ports will be connected to the supported
servers/storage systems; 1/3 (33%) of the data switch ports will be
interconnections between the aggregation data switches; and 1/3 (33%) of
the ports on the aggregation data switches will be to outside
communications systems. From this simple model it is easy to see that
potentially 66% of the initial 10GbE links will be less than 20m.
Thank you,
Roy Bynum
At 05:45 PM 7/28/00 -0400, Chris Diminico wrote:
>
>Corey,
>
>A personal thanks for the invaluable customer input. I believe that if we
>had more customers
>coming forward with their detailed requirements it would help break the
>current stalemate in the PMD
>selections. This is the type of debate that I hoped to stimulate in
>proposing that we should
>re-address the objectives of the PMD s; we need to clearly resolve any
>ambiguity in the
>objective statements in regards to application-space media-distances and
>the usage of the word
>"installed" to represent MMF fiber performance.
>
>As a supplier of Internet infrastructure product for Ethernet customer
>applications,
>I hear requests such as yours each day. I ll paraphrase in bullets here,
>borrowing from your e-mail.
>
>----My reason for wanting MMF (a 10G interface over MMF) is primarily cost
>simplicity, and
> compatibility with my current applications (technology and distances).
>----Cost - overall cost for the total installation.
>+++Labor LAN installers familiar with multimode terminations produce
>higher yields per unit time
> versus single mode.
>+++Materials: Connectors, tools, patch cables, test equipment, Laser/LED
>transceivers, etc...
>
>Other customers of 10 Gb/s Ethernet addressing the reflector and the task
>group have
>voiced strong support for the inclusion of a low-cost short-reach
>multimode fiber objective
>even if it included the use of higher bandwidth MMF. The task group
>responded to these
>clearly stated customer requirements by including in the current set of
>objectives a physical
>layer specification for operation over 300 m of MMF. Omission of the word
>"installed" was to
>implicitly allow for the new higher bandwidth MMF fiber. The usage of the
>word "installed" in the
>100 meter objective was to identify the MMF with the MMF currently
>specified in 802.3z.
>
>In order to clearly identify the current implicit differences in the MMF
>objective fiber types,
>
>I offer the following definitions.
>
>+++++Installed MMF MMF as specified in 802.3z.
>+++++MMF Either installed MMF or the Next Generation MMF fiber specifications
>currently proposed in both TIA and ISO. The development for these
>specification
>was supported in a Liaison letter issued from IEEE.
>
>A low-cost serial 850 nm PMD option coupled with the benefits of the
>higher bandwidth
>300 meter multimode fiber solution will addresses your requirements for
>cost, simplicity, and compatibility
>with your current Ethernet (10 Mb/s-100 Mb/s-1 Gb/s) distances and for the
>10 Gb/s Ethernet
>distances. Additionally, the new MMF coupled with the right PMD would
>allow for next generation
>40 Gb/s Ethernet applications.
>
>The impact of media selection on technology deployment can be severe.
>The debate over driving single mode versus higher performance multimode for
>new "in the building" LAN installations has the same flavor as coax versus
>twisted-pair.
>Before coming to CDT, I had worked at Digital Equipment Corporation for
>almost 20 years.
>DEC lost the Ethernet repeater business (coax) primarily due to its
>slowness in responding
>to the customer requirements for Ethernet over twisted-pair. DEC said,
>"coax is technology
>proof and will meet all of your long term application needs", the customer
>said, "but my
>reason for wanting twisted-pair is overall cost (installation, testing,
>materials), simplicity, and
>compatibility with my current applications (technology and distances). The
>rest is history.
>
>
>Chris Di Minico
>Cable Design Technologies (CDT) Corporation
>Director of Network Systems Technology
>Phone: 800-422-9961 ext:333
>e-mail: <mailto:cd@mohawk-cdt.com>cd@mohawk-cdt.com
>
>
>>----- Original Message -----
>>From: <mailto:Corey@citgo.com>McCormick, Corey
>>To: <mailto:stds-802-3-hssg@ieee.org>stds-802-3-hssg@ieee.org
>>Sent: Thursday, July 27, 2000 12:31 AM
>>Subject: RE: Equalization and benefits of Parallel Optics.
>>
>>I also may be a bit confused. From a business perspective I have this view.
>>
>>My reason for wanting a 10G interface over MMF is primarily cost and
>>simplicity. Most of the servers I have installed are within 100M and
>>most of the core and distribution switches are as well. If there is a
>>low-cost way to use some new-fangled media, then fine, but it seems to me
>>that improving ASIC technologies and economies of scale are the primary
>>downward factors in interface technologies.
>>
>>If the MMF limit is 100M or less then the pain incurred for me installing
>>new MMF is relatively minor, as the distance is not that large. This
>>means the number of labor-intensive obstacles encountered will be
>>small. It is work and cost to be sure, but if the runs were for
>>200-500M+ then the labor costs would be *much* higher. However, I
>>believe the costs for the tooling, cables, certification gear and
>>connectors will increase if we choose some new radically different
>>technology as the only choice. In our experience the SFF connectors are
>>not significantly less in overall cost. (there are exceptions, but for
>>the majority of them the costs are quite similar to ST/SC) We still have
>>*much* more difficulty with the SFF installations due to primarily lack
>>of available cables, field termination components, and conversion
>>cables. Also, there is the major problem of field Zip<->Dual fiber MM
>>adaptation to our installed ST/SC infrastructure (yuk!).
>>
>>I really do not care which technology is selected/specified, but for the
>>short-haul standard my primary goal is lowest overall cost for the total
>>installation. (Labor, connectors, tools, patch cables, test equipment,
>>Laser/LED transceivers, etc...) I care very little about which form
>>factor, mostly the cost and ease of use.
>>
>>If such relatively simple Net things as the broken 10/100 Autoneg Phy and
>>LX mode adaptation/conditioning cables are such a problem in the wide
>>acceptance of new technologies, then it seems like the KISS principle
>>should be a strong factor. I do not care how complicated it is
>>internally, but it needs to be simple for the end user.
>>
>>I also seems to remember that the goal was 3X the cost of 1G. If the
>>cable length limits are going to be <100M, then the
>>real-world-end-user-makes-the-comparison will be with 1000Base-TX copper,
>>not SX. This might make it much more difficult to complete the 3X cost
>>target unless there are *significant* savings in the
>>Phy/Xceiver/cable/connector/tools area.
>>
>>My engineering hat does not always agree with this, but then it is
>>business that pays the bills.
>>
>>What do you good folks think?
>>
>>Corey McCormick
>>CITGO Petroleum
>>
>> -----Original Message-----
>>From: Booth, Bradley
>>[<mailto:bradley.booth@intel.com>mailto:bradley.booth@intel.com]
>>Sent: Wednesday, July 26, 2000 8:30 PM
>>To: stds-802-3-hssg@ieee.org
>>Subject: RE: Equalization and benefits of Parallel Optics.
>>
>>I have one question:
>>
>>Which of our distance objectives is satisfied with parallel fiber and
>>parallel optics?
>>
>>It has been my interpretation that when we talked about 100m of installed
>>base of MMF, that we were referring to the MMF fiber currently available for
>>use by 802.3z. Parallel optics does not operate over this installed base.
>>
>>Or am I missing the point here?
>>
>>Cheers,
>>Brad
>>
>> -----Original Message-----
>> From: ghiasi [SMTP:Ali.Ghiasi@Eng.sun.com]
>> Sent: Tuesday, July 25, 2000 8:32 PM
>> To: stds-802-3-hssg@ieee.org; Daljeet_Mundae@mitel.com;
>>hakimi@lucent.com
>> Cc: Ali.Ghiasi@Eng.sun.com
>> Subject: RE: Equalization and benefits of Parallel Optics.
>>
>> Sharam
>>
>> > From: "Hakimi, Sharam (Sharam)" <hakimi@lucent.com>
>> > To: stds-802-3-hssg@ieee.org, "'Daljeet_Mundae@mitel.com'"
>> <Daljeet_Mundae@mitel.com>
>> > Subject: RE: Equalization and benefits of Parallel Optics.
>> > Date: Tue, 25 Jul 2000 21:04:49 -0400
>> > MIME-Version: 1.0
>> > X-Resent-To: Multiple Recipients
>><stds-802-3-hssg@majordomo.ieee.org>
>> > X-Listname: stds-802-3-hssg
>> > X-Info: [Un]Subscribe requests to majordomo@majordomo.ieee.org
>> > X-Moderator-Address: stds-802-3-hssg-approval@majordomo.ieee.org
>> >
>> >
>> > Although parallel fiber is technically an easier solution, the
>>major reason
>> > for support of 850nm has been to consider the installed base, and
>>cost. If
>> > users have to pull new fiber, IMHO, parallel fiber would not
>> be on
>>top of
>> > the list and most of installed base is single fiber.
>>
>> I did not suggest to pull any new fiber. Limit the shortwave
>>variant
>> including parallel optics to the data center with 100 m radius.
>>
>> Thanks,
>>
>> Ali Ghiasi
>> Sun Microsytems
>>
>> >
>> > Sharam Hakimi
>> > Lucent Technologies
>> >
>>