RE: Long distance links
Hi Paul,
First off, thanks for the White Paper reference. I found it informative
and have forwarded copies off to some people here that are wondering why
10 Gigabit Ethernet is necessary. I have some questions about the need for
"several megabytes of memory plus half-switch" on the PHY if a 10.0000 MAC
is used with an IPG stretching HOLD function though.
I believe that a single packet is all you would have to buffer in that
configuration. You could begin receiving the packet from the MAC, insert the
data into the OC-192 stream (including headers) and hold the MAC from
sending any more packets until you have completed sending the complete
frame.
As for the "half-switch", that is quite an overstatement when you consider
that this device is strictly a rate-matching mechanism. There is no address
hashing, L3 protocol, or other forwarding calculation.
Based on my experience with memory cost and ASIC complexity, I can't believe
that such elements pose more than a few dollars of cost in a system that
will have PHY optics that run many hundreds of dollars.
In addition, the 9.58 vs 10.0000 MAC speed aspect should be clarified
a bit. If I have a switch that runs 100/1000/10000, I am using a 125MHz
clock oscillator that is PLL multiplied up to 1.25GHz for 1Gig and I only
have to widen the bus to operate at 10Gig. Everything in the MAC chip is
synchronous except for the incoming FIFOs which are QUICKLY brought into
that same clock domain. If I had my druthers, the PHY would perform
frequency conversion as well to bring everything at the MAC into a single
clock domain. Using a 9.953280G clock rate, or a 9.58464G clock rate will
require FIFOs on my transmit side and complicate that ASIC design quite a
bit. Since that clock is only required for an OC-192 line rate, and the PHY
has a FIFO to accomodate headers and rate adjustments anyway, it seems like
the PHY is a better place to add the functionality.
This way, those PHYs that are running in a campus can operate with the same
clock rate at a lower overall system cost because we are using a single
clock domain for the ASIC and no FIFOs for the transmit side.
Either way, the cost addition is negligible, but the complexity should go
where it really belongs (IMO) which is on the PHY that requires it.
Thanks again,
Dan Dove
___________ _________________________________________________________
_________ _/ ___________ Daniel Dove Principal Engineer __
_______ _/ ________ dan_dove@hp.com LAN PHY Technology __
_____ _/ ______ Hewlett-Packard Company __
____ _/_/_/ _/_/_/ _____ Workgroup Networks Division __
____ _/ _/ _/ _/ _____ 8000 Foothills Blvd. MS 5555 __
_____ _/ _/ _/_/_/ ______ Roseville, CA 95747-5555 __
______ _/ ________ Phone: 916 785 4187 __
_______ _/ _________ Fax : 916 785 1815 __
__________ _/ __________________________________________________________
> -----Original Message-----
> From: pbottorf@nortelnetworks.com [mailto:pbottorf@nortelnetworks.com]
> Sent: Monday, August 30, 1999 3:18 PM
> To: DOVE,DANIEL J (HP-Roseville,ex1); HSSG
> Subject: RE: Long distance links
>
>
>
> Dan:
>
> At 02:11 PM 8/30/99 -0600, DOVE,DANIEL J (HP-Roseville,ex1) wrote:
> >
> >Hi All,
> >
> >Have we come to concensus yet?
> >
> >I keep seeing things flying across the reflector about how
> we need SONET
> >OAM&P, SONET's 9.58464 data rate at the MAC/PLS interface,
> etc... I thought
> >that the initial issue was having SONET's signalling at the PHY PMD
> >interface.
>
> I believe management and rate are separate topics. They are
> coupled in the
> sense that both become an issue when considering Ethernet in the WAN.
>
> >
> >For those who want a 10.0000 Gigabit Ethernet that operates
> as a "Local Area
> >Network" (LAN) and allows us to hook our switches together in wiring
> >closets, buildings, and campuses cost effectively, we have
> an architecture
> >that will work.
> >
> > [ 10 Gig MAC ]->10 Gig MII-> 10 Gig PHY (SMF/MMF/Cu)
> >
> >For those who want their long-distance WAN solution
> >
> > [ 10 Gig MAC ]->10 Gig MII-> 9.58464 SONET PHY (SMF)
> >
>
> The WAN PHY is not SONET. SONET does not specify a method for carring
> Ethernet frames. SONET requires full management. SONET has
> very tight clock
> requirements. The reason we are getting confused is that we
> are calling the
> WAN PHY SONET and, in so doing, are carrying more bagage than
> Ethernet can
> stand. I believe what is being proposed is the use of some OC-192
> technology for the WAN PHY which might be called 'SONET
> light' or 'Thin
> SONET', but is not OC-192. The issue we are debating is how
> much of SONET
> needs to be retained to fill the market requirement for easy
> interface and
> operation over existing WAN networks. To make it clear we are
> not talking
> about SONET we should refer to the PHY as the WAN PHY.
>
> >which uses HOLD as a mechanism to prevent overflow of the SONET PHY
> >on outbound traffic. Now, I have a few questions about this
> SONET PHY that
> >are due to my inexperience with SONET signalling. I am willing to
> >expose my naivete in the interest of gaining knowledge on
> the specific
> >problems with my vision on this matter.
> >
> >1) Assuming the HOLD mechanism, what is essentially wrong
> with using a
> >10.0000 Gbps MAC/PLS interface?
>
> What is wrong with this is it forces the PHY to use a
> different transmit
> clock than the MAC since the PHY wants to send a 9.584640
> Gbps data stream
> on a 9.953280 Gbaud carrier. A better solution is to use a
> 9.953280 Gbaud
> clock at the MAC with a HOLD mechanism. This will allow the
> MAC and PHY
> transmitters to be synchronous.
>
> If a word-by-word HOLD is used, then the PHY will not require any
> buffering. Using any of the IPG stretching techniques,
> including IPG HOLD,
> will force the PHY to have about 1 packet worth of transmit
> and 2 packets
> worth of receive queue. The reason these queues are much larger than
> required for the rate match is because the transmitter must
> synchronously
> transmit overheads mixed in the data stream. While the
> overheads are being
> transmitted the PHY must either be able to HOLD the MAC or
> must buffer as
> many bytes as the overhead area. On reception the PHY must
> either be able
> to put the receiver in HOLD over the overhead area or must
> have sufficient
> packet data buffered to feed an entire packet to the MAC
> synchronously.
>
> >
> >2) Why can't OAM&P functionality be performed in the SONET
> "PHY" where
> >it renders the least impact on the overall system architecture?
>
> Yes, the WAN PHY could perform OAM&P without impact on the
> upper layers.
> The issue is not implementation, but is cost. For connections
> over photonic
> networks OAM&P function are required, however for LAN
> connections they are
> an overkill. It is very desirable to have a PHY which can
> operate with or
> without OAM&P so those who need it pay for it, but those who
> do not can
> have cheap interfaces.
>
> >
> > 2a) Can't we use MDIO/MDC to allow management access to the
> necessary
> > PHY parameters in case processing is required?
>
> Yes
>
> >
> >Now I totally realize that I am using the term "PHY" loosely
> as the SONET
> >PHY function may require extensive processing capability. But from an
> >architectural/specification perspective, it makes it much
> more clean than to
> >burden a short-haul system with the same complexities as a
> long-haul system.
>
> This is why SONET is not proposed for the PHY. The OAM&P function does
> require considerable processing which is expensive for in building
> applications while very valuable for WAN applications.
>
> >
> >Now, if I have a vision that is workable (subject to debate)
> why are we
> >continuing to discuss a 9.58464 MAC/PLS interface or OAM&P
> functionality in
> >the MAC?
> >
> >Thanks,
> >
> >Dan Dove
> >HP Workgroup Networks
> >
> >PS: Is there a good white-paper on SONET signalling on the web?
>
> Have a look at:
>
http://www.nortelnetworks.com/products/library/collateral/87003.25-08-99.pdf
>
>
>
Paul A. Bottorff, Director Switching Architecture
Enterprise Solutions Technology Center
Nortel Networks, Inc.
4401 Great America Parkway
Santa Clara, CA 95052-8185
Tel: 408 495 3365 Fax: 408 495 1299 ESN: 265 3365
email: pbottorf@NortelNetworks.com