Thread Links Date Links
Thread Prev Thread Next Thread Index Date Prev Date Next Date Index

Re: [802.3BA] [HSSG] Reach Ad Hoc

Chris and others,

As I noted in my previous e-mail, I send out a survey among a large
end user group (AMS-IX customer base). The responses I got (some 30)
were more or less similar to what you describe below. Based on these
responses I would support your conclusion below.

  - Henk Steenman

On Jan 7, 2008, at 9:34 PM, Chris Cole wrote:

> Following the 12/06/07 Reach Ad Hoc conference call, I sent an email  
> to
> the HSSG reflector (appended at the end of this email) requesting
> continued end user input on reach distribution for 10km and shorter
> applications.
> My email was forwarded to several end user email lists, for example
> NANOG (North American Network Operators Group). I subsequently  
> received
> a number of emails from different data center operators commenting on
> their needs. This gave me the opportunity to follow up with some more
> detailed exchanges to better try to understand how optics are being
> used. While it would have been preferable for those emails exchanges  
> to
> have occurred on the HSSG reflector, at least I can share the  
> insights I
> gained.
> More then half the emails objected to reducing the 10km reach  
> objective.
> There were three broad categories for why a 3km or 4km reach objective
> was seen an unacceptable. For a portion of the end users, 3km or 4km  
> was
> not enough reach; they needed something a bit more then 4km for  
> example
> 5km. For another portion, the extra 10km link budget was used to
> accommodate much higher number of connectors / patch panels then
> nominally in the standard, or just to deal with sloppy patch cord
> handling. The last portion of the end users had a significant fraction
> of links between 4km and 10km.
> The emails that preferred a 3km or 4km reach objective were in three
> broad categories of what was seen as the volume sweet spot, for which
> minimum cost was paramount. The three sets of reaches that were seen  
> as
> capturing most of the volume were: 1) 500m to 1km, 2) <2km, 3) <4km.
> I received permission from several of the end users to share their
> emails with the HSSG reflector, and they are appended below. These
> emails are meant to be exemplary and there is no intent to imply that
> they accurate reflect the distribution of opinions for any of the end
> user groups.
> My conclusion from these exchanges is that there are many different  
> ways
> in which optics are used. End users have adopted the existing 10km  
> reach
> standard to a variety of applications, and we should be very careful
> before making changes that may break many applications that we are not
> even aware of.
> At this point, I support keeping the existing 10km reach objective,  
> and
> do not support changing to a shorter reach.
> I would also encourage anyone on the HSSG reflector, who is also a
> member of any of the end user groups, like NANOG, to share their  
> views,
> or summarize their take on the views of their group on the HSSG
> reflector. This will add to the credibility of this discussion.
> Chris
>> -----Original Message-----
>> From: Sam Silvester []
>> Sent: Thursday, December 13, 2007 3:13 PM
>> To: Chris Cole
>> Subject: Re: IEEE 40GE & 100GE
>> Hi Chris.
>> Working for a reasonably small (geographically speaking - within a
>> single state / capital city) carrier, as long as (per your earlier
>> comments) 10km optics are happy talking to each other with no
>> attenuator required, I see very little value in a 4km reach version.
>> If we're talking about a cost ratio of between 1.15 and 1.3, this  
>> just
>> reinforces the fact in my opinion.
>> Internally i.e. within our PoPs, we use 10km optics currently at 1GE
>> and 10GE (even for < 500m runs). We have some deployment of 40km but
>> in general due to geography, 10km does the job nicely.
>> I'd suggest even if a 4km optic came along, I'd not bother as the  
>> "end
>> user price difference" most likely wouldn't excite me enough to have
>> to carry another line of spares. I realise this is more or less in
>> line with other comments, but I figured you might like the input
>> anyway.
>> Excluding "short runs" of 500m, I'd say the ratio of <4km links vs
>> <10km links is about 50/50.
>> Note that I expect in an enterprise-style deployment the number may
>> well stack up with the 90% figure you mentioned in your email. I'm
>> still not sure a 1.15 - 1.3 ratio makes that worthwhile...
>> Regards,
>> Sam
>> -----Original Message-----
>> From: Justin M. Streiner []
>> Sent: Wednesday, December 12, 2007 4:51 AM
>> To:
>> Cc: Chris Cole
>> Subject: Re: IEEE 40GE & 100GE
>> On Wed, 12 Dec 2007, Robert E. Seastrom wrote:
>>> A practical question here: does anyone know offhand if 4km reach is
>>> adequate for interbuilding access (i.e., DC[124] to DC3) access at
>>> Equinix Ashburn, including worst-case interior wiring and cross
>>> connects?  I'm thinking that's cutting it close.  The enterprise
>>> people are substantially less likely to find themselves with a lot  
>>> of
>>> interconnections in a GCE (Ginormous Campus Environment) than we  
>>> are,
>>> and I suspect that skews the 90% number a bit.  Folks who are more
>>> familiar with the layout of other facilities may wish to chime in
>>> here.
>> I'm not in any of the Equinix facilities, but I do run a decent-sized
>> urban campus network and a 3km-4km distance limitation would be
>> cutting it really close for me in some cases.  Some of the 10G links
>> on my backbone today do require multiple physical cross-connects,
>> which would eats into the link budget. Most of my backbone  
>> connections
>> work find with 10G-LX4 ptics, but there are a few places where 10G-ER
>> is needed.
>> I haven't read the draft spec yet to see what's being proposed for a
>> link budget at 3/4/10km, but that's just as important as the physical
>> distance.
>> jms
>> -----Original Message-----
>> From: Randy Whitney []
>> Sent: Wednesday, December 12, 2007 8:46 AM
>> To: Chris Cole
>> Subject: Feedback on 100GE Proposed
>> Chris,
>> Per cross-posting to lists I am on I am providing you feedback on the
>> proposed change to the 100GE standard.
>> For my deployment needs, 3km or 4km would *not* be sufficient in many
>> cases, so I would be strongly opposed to reducing the std down to 3km
>> or 4km.
>> Best Regards,
>> --
>> Randy S Whitney                                      +1 703 886 5642
> (0)
>> Verizon Business, Global Peering Manager             +1 703 886 0512
> (F)
>> 22001 Loudoun County Pkwy, Ashburn, VA 20147         +1 202 997 8174
> (M)
>>                     uursw (AIM,
>> -----Original Message-----
>> From: Robert E. Seastrom []
>> Sent: Wednesday, December 12, 2007 11:20 AM
>> To: Chris Cole
>> Cc: Justin M. Streiner;
>> Subject: Re: IEEE 40GE & 100GE
>> "Chris Cole" <> writes:
>>> One of the points made by Ted Sealy from Sprint is that they take
>>> advantage of the extra link budget in 10GBASE-LR 10km link budget to
>>> account for extra connector loss, etc.
>> Ted Seely and I are of the same mind on this.  2 dB sounds like  
>> plenty
>> for connector loss right up until you have to deal with multiple  
>> patch
>> bays in a structured system with amateurishly applied mechanical
>> splices.  The difference between noting that the loss is a little  
>> high
>> but the link still works so you roll with it, and having to spend  
>> time
>> on the phone arguing with someone who thinks 24 dB link loss is A-OK,
>> will make the slight additional up front cost for the better grade
>> optics look very inexpensive indeed...
>>                                        ---rob
>> -----Original Message-----
>> From: Dale W. Carder []
>> Sent: Wednesday, December 12, 2007 12:53 PM
>> To: Chris Cole
>> Subject: Fwd: IEEE 40GE & 100GE
>> Hi Chris,
>> I appreciate the opportunity to provide some end-user perspective  
>> from
>> our environment, the University of Wisconsin @ Madison and WiscNet.
>> Most of what I saw in the links below was about link budget, but what
>> about ORL?  We got bit pretty hard migrating from 1G to 10G through
>> old hand-polished SMF panels which had never been a problem before.
>> We ended up cutting & fusion splicing on new pigtails in numerous
>> locations.
>> Data Center / super-computing environments:
>> - high performance connections are all within 300 meters,
>>   with both FDDI grade MMF (largely abandoned) and also SMF-28.
>> Campus environment:
>> - about (200) buildings all within 3km of our central
>>   aggregation points on SMF-28.
>> - roughly 40% of point-to-point links have (2) patches
>> - roughly 60% of point-to-point links have (4) patches
>>   (via an intermediate building's patch-panel)
>> Metro Ethernet environments:
>> - about (10) links 8-10km, (4) 10-12km.
>> - We are running 10G-LX today on a few links beyond 10km.
>> - most have 6-8 patches
>> Regional Long-Haul environment:
>> - (30) 80-100km links
>> We are quite interested in what happens with HSSG, let me know if
>> there's anything else we can help with.
>> Dale
>> --
>> Dale W. Carder - Network Engineer
>> University of Wisconsin - Madison / WiscNet
>> -----Original Message-----
>> From: Brian Dickson []
>> Sent: Tuesday, December 11, 2007 6:17 PM
>> To: Chris Cole
>> Subject: Re: [HSSG] Reach Ad Hoc
>> Hi, Chris,
>> Bora Akyol forwarded your summary message on the 10 km vs 3/4 km
> issue,
>> to the NANOG mailing list, indicating that the IEEE needs operator
>> feedback on this issue.
>> I'm pretty familiar with who needs to be deploying 40G and 100G, and
>> where and how they are likely to need these interfaces.
>> There are primarily two places we expect to see *lots* of these
>> connections.
>> One is - service provider backbones. These are the internal network
>> links that form major ISP/NSP network infrastructure. Those will fall
>> into a number of categories, ranging from DWDM, to CWDM, to
>> direct-over-fibre.
>> The other is, the interconnections between ISPs. The number of
>> connections on these ingress/egress points is likely to be higher by
> an
>> order of magnitude - anywhere from 2x to 10x the links,  
>> proportionally
>> speaking.
>> It is worth noting that even when traffic levels may only be
> approaching
>> 10Gbps, that 100G is *very* attractive, notwithstanding the price.
> Burst
>> capacity is what IP network engineering is all about, not average
> usage.
>> And, even if an ISP is small compared to his neighbor, the links have
> to
>> operate at one speed, which *needs* to be the highest-common- 
>> multiple,
>> not the lowest-common-denominator. When in doubt, go high, is the
> rule.
>> And, also worth noting is, that once the technology is available, the
>> switching capacity is anticipated to meet the line rate head-on.
>> Line-rate many-port switches will be the norm, and access to those is
>> best achieved at 100G.
>> Getting those cards affordable is a huge deal, and getting them
>> delivered in a timely fashion by the parts manufacturing community is
>> something we are *very* worried about. Even 10G cards are late to the
>> game.
>> The places where ISPs interconnect, are a few dozen locations
> worldwide,
>> and those have the following common characteristics:
>> A very large building, with cages, suites, and cabinets
>> A meet-me area where all interconnections generally run
>> Some big colocation providers who re-sell some of the building
>> real-estate, with their own meet-me areas
>> Hundreds to tens of thousands of companies colocating their equiment,
>> especially ISPs
>> Interconnections run from ISPs to common switching equipment (an
>> Internet eXchange, or IX)
>> Interconnections directly between ISPs bypassing the IX
>> The former (IX connections) typically reach much further down the  
>> list
>> of customers, many 10s to a few hundreds
>> The latter (private peering) typically use the same media type (SMF,
> or
>> sometimes MMF) regardless of speed - 100M, 1000M, 10G, 40G, 100G
>> The buildings will typically be telco hotels (vertical, up to 30 or  
>> so
>> floors), no more than 50m long and 30m wide, max fibre distance of
> 300m
>> typically, but as many as 8-10 patches per link, or as few as zero.
>> Some other buildings are more warehouse type - one floor only, but as
>> big as 300m x 500m or more, max fibre distance typically 1.5-2km,  
>> with
>> probably fewer patches (2, 4, 6 are typical numbers).
>> Based on this info, I would suggest 4km over 3km, but that 4km is  
>> very
>> much of interest to the buying community, who (based on volume of
> links)
>> are definitely interested in keeping costs down. Many of those folks
>> were bitten in the @ss by OC-192 costs that were about 5x higher than
>> they should have been.
>> Hope this information is helpful, and if you would like me to put you
> in
>> touch with some folks who operate the neutral data centers and run  
>> the
>> fibres at them, I would be happy to oblige.
>> Thanks for allowing me to provide some input,
>> Brian Dickson
>> Afilias Canada
> -----Original Message-----
> From: Chris Cole [mailto:chris.cole@FINISAR.COM]
> Sent: Tuesday, December 11, 2007 4:22 PM
> Subject: Re: [HSSG] Reach Ad Hoc
> During the November HSSG meeting, optics vendors made a presentation
> proposing changing the 10km reach objective to 3km or 4km. One of my
> motivations for working on the proposal was informal input from a  
> number
> of 100GE end users that >90% of their data center and short  
> interconnect
> needs would be met by a reach objective less then 4km (versus 10km.)
> With such a reach distribution, a 4km or less optimized reach  
> objective
> would result in overall cost savings.
> As part of the HSSG effort to review this proposal, numerous requests,
> both informal as well as from the HSSG chair and Reach Ad Hoc chair,
> have been made for contributions to quantify the 10km and under reach
> distribution. While the optics vendors as suppliers can accurately
> represent the relative costs of optics alternatives, they can not
> represent end user requirements.
> To date, we have seen no end user presentation or data supporting
> changing the 10km reach objective to 4km or less. Unless such
> contributions are forthcoming, it is likely that there will be no
> motivation to make the change. This sentiment can be seen in the 12/7
> Reach Ad Hoc conference call minutes.
> I would encourage any HSSG participant that views their volume 100GE  
> needs as better met by a 4km or shorter reach objective to make a
> contribution containing reach distribution data in support of this
> position. Otherwise we will move forward with the existing approved
> objectives.
> Chris
> ________________________________________
> From: Andy Moorwood [mailto:amoorwood@EXTREMENETWORKS.COM]
> Sent: Tuesday, December 11, 2007 6:03 AM
> Subject: [HSSG] Reach Ad Hoc
> Colleagues, the meeting notes from our call last week are now posted  
> on
> the IEEE website
> Thank you for your contributions
> Andy