Thread Links Date Links
Thread Prev Thread Next Thread Index Date Prev Date Next Date Index

Re: On two clock issue




Roy,

Ethernet is (so far) asynchronous.

Each transmitter runs at its own speed and phase (within the allowed
tolerances, which are huge compared to SONET, typically +/-100 ppm), and
each receiver recovers its clock from the received data.

And, Yes, they free run (each Gigabit Ethernet receiver switches over to its
local transmit clock in the absence of incoming signals to minimize receive
PLL acquisition time), and, Yes, the transmit and receive data rates differ.
One of the main purposes of the Inter-Packet Gap (IPG is to allow for this).

The only exception I know of is 1000BASE-T where the timing/phase accuracy
of the echo cancellation and line compensation filters (simultaneous
transmit/receive on same copper wires) is so tight that one end of the link
has to supply clock for both directions, and a master-slave relationship is
negotiated and set up for the link (but not synchronous to the cosmos--
still +/-100 ppm for the clock used for both directions).

The POS line cards I have seen need considerable buffering and flow control
to get into/out of the synchronous world.

Some of you guys are trying to hammer Ethernet into SONET/SDH in a big way.

This interface between the synchronous and asynchronous worlds, as has been
pointed out by many here, will not go away, but will simply move into
greater or less costly or inconvenient points in the data paths.

Since the synchronous world requires terrifically accurate time bases
(single Ethernet links can tolerate very large timing differences in
comparison, like 10X that of SONET without even thinking about it much--
hey, it's A-synchronous) you should expect and will get a lot of resistance
to forcing the Ethernet world into the expensive SONET performance
requirements and costs.

Cheers,

Larry Miller
Nortel Networks

-----Original Message-----
From: Roy Bynum <rabynum@xxxxxxxxxxx>
To: rtaborek@xxxxxxxxxxxxxxxx <rtaborek@xxxxxxxxxxxxxxxx>
Cc: HSSG <stds-802-3-hssg@xxxxxxxx>
Date: Monday, July 19, 1999 8:50 PM
Subject: Re: On two clock issue


>
>Rich,
>
>Just a note of clarification.  If the receive clock is separate from the
system clock,
>what controls the transmit clock.  Does each link transmit bit rate provide
the clock
>to the other system's receiver?  Would this tend to cause free running and
separate
>transmit and receive data rates? How do existing OC48C POS interfaces use
line timing
>from the carrier LTE to control receive as well as transmit speeds?  If
802.3 were to
>be used on anything other than enclosed LANs, what would be available to
provide line
>timing?  Are there alternatives?
>
>Thank you,
>Roy Bynum
>MCI WorldCom
>
>Rich Taborek wrote:
>
>> Devendra,
>>
>> We're taking a few too many tangents here. To bring everyone else up to
speed, the
>> root issue is a MAC/PLS rate of 10.0 Gbps vs. 9.584640 Gbps. I asserted
that a
>> 9.584640 Gbps would complicate the clock design of Ethernet equipment due
to the
>> requirement to support a rate which is not a multiple of slower Ethernet
MAC/PLS
>> local clocks.
>>
>> Receive clocks are separate from local clocks. I believe that most
Ethernet
>> equipment supporting multiple interfaces DOES NOT use the receive clock
from any
>> interface as its local clock.
>>
>> Devendra Tripathi wrote:
>>
>> > Rich,
>> >
>> > One clarification here. The use of two clocks is almost always there.
>> > Ofcourse
>> > some designs carry the retreived clock all the way to host fifo
interface
>> > and some
>> > terminate it early. I am doubtful, though, if anyone uses that clock
for
>> > transmit
>> > purpose. Because in that case, the reference clock (which is taken as
transmit
>> > clock by PHYs) also has to be retrieved one, thus making a loop in the
PLL.
>> >
>> > Thanks,
>> > Tripathi.
>> >
>> > At 10:17 AM 7/19/99 -0700, you wrote:
>> > >
>> > >Roy,
>> > >
>> > >This one's pretty simple. The MAC/PLS clock for 10 Mbps Ethernet is
based
>> > on the
>> > >transport of bits. At 100 Mbps, it's nibbles (4 bits). At 1 Gbps, it's
>> > octets. At 10
>> > >Gbps, several proposals suggest 8, 16 and 32 bit chunks. The local
clock
>> > for Ethernet
>> > >equipment which performs speed matching is easily derived from MAC/PLS
>> > clock of the
>> > >slowest interface supported. Deriving a 9.584640 is not so
>> > straightforward. Reverse
>> > >deriving lower speed clocks is also not so straightforward. The
simplest
>> > solution is
>> > >to use two clocks. This solution increases implementation cost and
will
>> > significantly
>> > >complicate clocking design.
>> > >
>> > >Best Regards,
>> > >Rich
>> > >
>> > >--
>> > >
>> > >Roy Bynum wrote:
>> > >
>> > >> Rich,
>> > >>
>> > >> I hear much about the single system clock issue.  In the past, the
data
>> > transport
>> > >> signal clock and the data system bus and processing clock were
separate.
>> >  For many
>> > >> systems the data processing would exceed the transport signal in
order
>> > to maintain
>> > >> control.  If that is the case, then your argument about a single
clock
>> > does not
>> > >> hold.  I could be wrong.  Would one of the system implementation
people
>> > please
>> > >> define how a data signal clock is derived in today's systems. Is
>> > internal 802.3
>> > >> frame processing done at a higher rate than the transport signal
rates
>> > in today's
>> > >> GbE L2 switches?  Is there a real issue with a separate transport
signal
>> > rate or
>> > >> even a separate transport data rate?
>> > >>
>> > >> Thank you,
>> > >> Roy Bynum
>> > >> MCI WorldCom
>> > >>
>> > >> Rich Taborek wrote:
>> > >>
>> > >> > Hon Wah,
>> > >> >
>> > >> > Thanks for kicking this issue off again!
>> > >> >
>> > >> > Hon Wah Chin wrote:
>> > >> >
>> > >> > > Perhaps a difficult number to remember, but with the +- 100ppm
>> > tolerance
>> > >> > > and a bit rate that needs only to fit within about 200ppm of the
>> > nominal
>> > >> > > SONET number we should be able to choose a round number with 4
>> > digits in it.
>> > >> > >
>> > >> > >   ---
>> > >> > >
>> > >> > > As I understand the presentations in Montreal on speed,
>> > >> > > a strong advantage of choosing this OC-192 payload rate is
>> > >> > > to transport the signal over SONET OC-192 equipment.  This would
>> > >> > > be from a "10Gb/s Ethernet" port out to SONET gear, which is
really
>> > >> > > a PMD external interface rather than a definition for the
MAC/PLS
>> > interface
>> > >> > > and data rate.
>> > >> >
>> > >> > In my conversations with several folks on both sides of the issue
>> > during the
>> > >> > Montreal meetings, I've come to the conclusion that the root
reasons
>> > to select
>> > >> > either a 10 or 9.584640 Gbps are purely ease-of implementation
based
>> > and have no
>> > >> > architectural basis whatsoever. I believe this to be true on both
>> > sides of the
>> > >> > argument with the choice of one over the other, rendering the
>> > implementation
>> > >> > (i.e. product cost) of the losing side only slightly more
difficult.
>> > Please
>> > >> > allow me to explain the basis of this contention:
>> > >> >
>> > >> > 1) SONET, and specifically synchronous transport, is legacy in the
MAN
>> > and WAN,
>> > >> > will never be replaced by Ethernet completely or even quickly.
>> > Ethernet will
>> > >> > make inroads into "green-field" applications, but SONET will be
king
>> > for some
>> > >> > time to come;
>> > >> >
>> > >> > 2) Ethernet, and specifically packet-based transport, is legacy in
the
>> > LAN, is
>> > >> > growing in its dominance in the LAN, and will likely gain market
share
>> > in the
>> > >> > LAN as well as encroach on other non-traditional Ethernet
transports
>> > including
>> > >> > MAN, SAN, and some WAN. I don't include WAN access in WAN. Instead
I
>> > include WAN
>> > >> > access in LAN or MAN;
>> > >> >
>> > >> > 3) The existing WAN infrastructure does a great job of
transporting
>> > Ethernet
>> > >> > packets end-to-end today. However, much protocol conversion and
>> > equipment to map
>> > >> > between packets and TDM bits exists in mapping Ethernet to the WAN
at
>> > each end.
>> > >> > Considerable savings can be realized by architecting a more
seamless
>> > Ethernet to
>> > >> > SONET connection. This issue seems to be at the root of the 10 vs.
>> > 9.584640 Gbps
>> > >> > issue.
>> > >> >
>> > >> > 4) There seems to be no intent by either side to consider any
other
>> > changes but
>> > >> > speed as a HSSG objective. Therefore, Ethernet will remain a
simple,
>> > general
>> > >> > purpose, packet-based transport, and SONET will remain a specific
purpose
>> > >> > (MAN/WAN), synchronous transport no matter which way the decision
goes.
>> > >> >
>> > >> > 5) Consider a Ethernet to OC-192 line card (feeding a fiber or
>> > wavelength) in
>> > >> > operation. Assume that receive and transmit paths are separate on
the
>> > SONET side
>> > >> > and related (i.e. full duplex) on the Ethernet side:
>> > >> >   a) Ethernet -> SONET @ 9.584640 Gbps: The Ethernet side can
>> > continuously feed
>> > >> > the SONET link with no flow control required.
>> > >> >   b) Ethernet -> SONET @ 10 Gbps: The Ethernet side must be flow
>> > controlled to
>> > >> > prevent over-feeding the SONET link
>> > >> >   c) SONET -> Ethernet @ 9.584640 or 10 Gbps: The Ethernet side
can
>> > continuously
>> > >> > source SONET data but will flow control or drop packets downstream
>> > whenever the
>> > >> > network is congested.
>> > >> >
>> > >> > Therefore, the issue boils down to one of implementation of
existing
>> > Ethernet
>> > >> > mechanisms such as 802.3x flow control or a reasonable facsimile
on
>> > the line
>> > >> > card versus complicating the implementation of all Ethernet
products
>> > which must
>> > >> > support a MAC/PLS rate which is not a multiple of 10. These
>> > implementation
>> > >> > difficulties include multiple clocks which may "beat" against each
>> > other, not
>> > >> > being able to easily feed 10 slower links into one faster one, and
>> > numerous
>> > >> > other difficulties which are best listed by Ethernet product
>> > implementers.
>> > >> >
>> > >> > My intention is not to make light of the problem but rather to
agree
>> > with a
>> > >> > solution direction along the line proposed by Dan Dove of HP at
the
>> > Montreal
>> > >> > meeting. I believe that Dan's general direction was to tradeoff a
simple
>> > >> > architectural change with respect to MAC operation to enable cost
>> > effective 10
>> > >> > Gbps to SONET implementations. I don't particularly agree with
resolving
>> > >> > implementation cost issues between two dominant legacy protocols
by
>> > tweaking
>> > >> > with the underlying architecture, but I'll raise my hand in
support of
>> > this
>> > >> > solution to the problem.
>> > >> >
>> > >> > Such a solution would enable the implementation of a 10 Gbps
Ethernet
>> > to SONET
>> > >> > OC-192 line card without requiring a full MAC.
>> > >> >
>> > >> > I'll let Dan fill in the details of his proposal so I don't get it
>> > wrong if it
>> > >> > is still applicable.
>> > >> >
>> > >> > Best Regards,
>> > >> > Rich
>> > >> >
>> > >> > --
>> > >> >
>> > >> > > Given a raw continuous bit stream at the PMD, some scheme for
>> > >> > > framing packets would be needed.  10M used a carrier, 100M used
coding,
>> > >> > > 1000M used coding.  Using coding where the PMD speed is fixed at
>> > 9.58Gb/s
>> > >> > > would mean a further speed reduction (probably 10-20%) at the
MAC/PLS
>> > >> > > interface. The discussion at the meeting has already started to
>> > consider ways
>> > >> > > of
>> > >> > > reducing the useful throughput at the MAC/PLS below the data
>> > clocking rate.
>> > >> > > An
>> > >> > > alternative framing scheme presented to HSSG, which has a
smaller
>> > throughput
>> > >> > > reduction, requires a packet length header -- a departure from
>> > previous 802
>> > >> > > practice.
>> > >> > >
>> > >> > > In considering the advantage of leveraging SONET OC-192
transport
>> > >> > > we should also consider the issues which come up in actually
getting
>> > >> > > the hoped-for benefits.  It would also be worthwhile to
carefully
>> > consider
>> > >> > > what volume forecasts for the OC-192 components can be
documented, in
>> > >> > > evaluating the advantage to be gained.  Counting IEEE802.3
10Gb/s data
>> > >> > > ports (however the definition works out) to get 2 million ports
sounds
>> > >> > > good, but I found the forecast of 2,000,000 OC-192 ports in 2000
rather
>> > >> > > surprising.
>> > >> > >
>> > >> > > -hwc
>> > >
>> > >-------------------------------------------------------------
>> > >Richard Taborek Sr.    Tel: 650 210 8800 x101 or 408 370 9233
>> > >Principal Architect         Fax: 650 940 1898 or 408 374 3645
>> > >Transcendata, Inc.           Email: rtaborek@xxxxxxxxxxxxxxxx
>> > >1029 Corporation Way              http://www.transcendata.com
>> > >Palo Alto, CA 94303-4305    Alt email: rtaborek@xxxxxxxxxxxxx
>> > >
>>
>> --
>>
>> Best Regards,
>> Rich
>>
>> -------------------------------------------------------------
>> Richard Taborek Sr.    Tel: 650 210 8800 x101 or 408 370 9233
>> Principal Architect         Fax: 650 940 1898 or 408 374 3645
>> Transcendata, Inc.           Email: rtaborek@xxxxxxxxxxxxxxxx
>> 1029 Corporation Way              http://www.transcendata.com
>> Palo Alto, CA 94303-4305    Alt email: rtaborek@xxxxxxxxxxxxx
>
>