Thoughts on Gigabit Ethernet Physical (or what we can learn from FC-0 and FC-1) =============================================================================== Jonathan Thatcher IBM AS/400 Division Optical Interconnect Technology April 1, 1996 Fibre Channel "PHY" Layer Background ==================================== Originally designed as a bus extender for AS/400 ------------------------------------------------ o Ran at 220 Mb/s o Changed to 266... for FC o Adopted by S/390 for clustering (531 & 1063) o AS/400 upgraded to 1063 Do you want me to bring the museum to the next meeting? Hardware does not exactly match the FC layers --------------------------------------------- o Gigabit Link Module (GLM) implements all of FC-0 and some of FC-1 o Protocol chips implement the rest of FC-1 and above o 10b Chips with transceivers effectively maintain this same hardware partition o Fibre Channel "PHY" Layer Background Mantras from the beginning of time ================================== Find the cheapest way to do everything (eschew status quo) ---------------------------------------------------------- o Utilize CD Lasers (a megaunit per month device) o Develop Pre-Amps with Integrated photoDetectors from GaAs (PAID) o Put the parallel logic into cheap CMOS solutions o Use OFC to widen margins on optics o Use duplex SC connectors (and special wrap plugs) o Support both 62.5 and 50 mm fiber. Provide customer with parallel I/F ---------------------------------- o Isolate system integrator from all those nasty high speed signals o Isolate system integrator from the related EMI problems Make it reliable ---------------- o Improve reliability of CD Lasers o Use 8b/10b encoding o Low BER (nominal <10E-15) Watch the power --------------- OFC -- PROS & CONS ================== PROS ---- o Proven reliable and safe o Lower transceiver cost + Less sensitive receiver + Less precise laser optics + Less need to throw away optical power o Useful for both SW and LW CONS ---- o Only good for point-to-point, duplex links o Must be integrated into transceiver (i.e. not in auto-negotiation logic) o More complex host logic; more difficult to debug; greater system cost? o Requires clock (race condition on POR); more failure mechanisms o Slower start-up (10.1 seconds w.c. delay) Link Acquisition: 3 Steps ========================= 1. OFC (as applicable) + Completion indicated by loss-of-light (loss-of-signal) = LOL (LOS) = 0 + Erroneous K28.5's will likely be detected prior to bit synchronization 2. Bit Synchronization + Requires continuous input of valid 8b/10b code (DC balanced data); this is an AC coupled system + Receiver PLL takes 2500 bits to synchronize following an instantaneous phase or frequency change (Example: a simple cross-point switching will lose both bit and word synchronization 3. Word Synchronization + Shared by FC-0 and FC-2 hardware + Reference FC-PH Clause 12 + Loss of word synchronization (8b/10b errors will be detected prior to LOL). + ESD strikes tend to cause bursty errors; protocols tend to over-react. Open Fibre Control (OFC) - How it works (simplified) ==================================================== [This page does not work well in text] Link Acquisition Sequence ========================= The following sequence should be followed to get a Transceiver/SERDES in full synchronization with a companion undergoing a similar sequence. 1. Power on. The Transmit Byte Clock (TBC) should be running. 2. The OFC will drive the Link Unusable line to a high state indicating that its receiver is not detecting any signal from the link. 3. Drive the Transmit Data lines to a 0101010101. 4. Drive the Enable Wrap low (will not be changed) 5. If a link is properly connected and the companion card is in an equivalent state of readiness the laser will turn on in 10.1 seconds as indicated by the Link Unusable line going low. This transition indicates that the OFC is operational and in control. 6. After 2500 bit times (2.4 ms), the OLM-1063-SW/LW should be in bit synchronization (but not yet byte synchronization). The Receive Byte Clock frequency should now be running at 53.125 MHz and the Comma Detect line is ready to indicate reception of the Comma Character. 7. Drive the Transmit Data lines with a K28.5 (Byte Sync) character. 8. As soon as the OLM-1063-SW/LW receives the K28.5 character from the other side of the link, the clocks will align to the byte boundary and all the Receive Data lines will have valid data. This will be indicated by the activation of the Comma Detect line 8b/10b ====== 8b/10b encoding is required by Fibre Channel (ref: FC-PH Clause 11.2) The 8b/10b encoding scheme has the following practical benefits: + Maintains a 50% DC balance point. + No more than 5 running 0's or 1's. + K28.5 character is unique regardless of combination of data/phase. + Maintains a fixed encoded length, 10 bits, and therefore a fixed data rate. + Improves data reliability. + Provides commands. + Has a simple, fast encoding/decoding scheme. 8b/10b Code Characteristics + There are 256 data values (8 bits) and 12 special characters (e.g. command, Byte Sync) yielding a total of 268 encoded sequences. + The 8 data bits split into 3 + 5 bits. The 3 bit field is encoded into 4 bits, the 5 bit field is encoded into 6 bits totalling 10 encoded bits. 8b/10b (continued) ================== Disparity + All codes have a minimum of 4 and a maximum of 6 logical ones. + The Running Disparity (see below) guarantees that the DC balance point is met. + Disparity is a measure of the number of ones minus the number of zeros in the encoded binary data (or command). + If there are an equal number, then the disparity is zero and the code is disparity neutral. + The disparity within the 4-bit, the 6-bit, or the combined 10 bit encoded data will always be -2, 0, or +2. The following chart represents all combinations. Bits 0:3 Bits 4:9 Bits 0:9 -------- -------- -------- -2 -2 unused -2 0 -2 -2 +2 0 0 -2 -2 0 0 0 0 +2 +2 +2 -2 0 +2 0 +2 +2 +2 unused 8b/10b (continued) ================== Of the 268 encoded characters, 134 are disparity neutral. Each of the characters has an analog with the opposite sign + The sum of the disparity of both versions of an encoded character is always 0. + Which of the two versions of the characters sent on the link is dependent on the current value of the running disparity. Running Disparity + Running disparity is the net sum of the 10-bit disparity for all codes sent from the time of reset. + The running disparity must start at -1. + The encoding scheme guarantees that any encoded character will always leave the running disparity at a value of -1 or +1. The 8B/10B code has several error detection mechanisms. A nonexhaustive list includes: + Validation of correct codes (only 268 of the possible 1024 10-bit codes are used). + Validation of correct running disparity. + Validation that running number of 1's or 0's does not exceed 5. Begin (and end?) every packet with a -K28.5 + Otherwise error detection may not be isolated to the packet. Top 10 Reasons For 1.25 Gb/s (10X) ================================== 1. I have 10 fingers and 10 toes... 2. My customers can only multiply/divide by 10 3. 10 is different (everyone else uses factors of 2/4/8); FC has already done 1.063 Gb/s 4. 802.3 wants to go faster than 802.12 5. ...and faster than SONET at 1.24 Gb/s. 6. Some technology will have no problem doing it 7. Some company can demonstrate that they already can 8. We want to level the playing field 9. We need to delay until we make money on the 100 Mb/s version 10. Well, we have always done it that way Top 10 Reasons For 1.0625 Gb/s ============================== 1. You'll get to market faster. 2. It will cost less. 3. There is less risk. 4. The various budgets are not going to scale simply. 5. It requires less aggressive technologies. 6. You probably won't build a hub with ten, 100 Mb/s inputs anyway. 7. 1.25 Gb/s doesn't include all the overhead.... 8. What standard bus is going to give you 125 MB/s of data throughput in 1997 or 1998? 9. Your friends in Fibre Channel will greatly appreciate your reducing their costs. 10. Your friends in Fibre Channel will greatly appreciate your simplifying their lives. [Author note: In hindsight, it is clear that number 7 is wrong. I therefore offer as substitute: because you will be able to use the gigaBYTE parallel optical links currently under development). Link Budget Comparisons ======================= Description Unit SW (NOFC) LW-L (NOFC) LW-I (NOFC) FC IEEE FC IEEE FC IEEE ------------------------------------------------------------------------------- ------ General ------- Data Rate MBytes/s 100 125? 100 125? 100 125? Bit Rate MBaud 1063 1329? 1063 1329? 1063 1329? Distance (50 mm) m 500 400? Distance (62 mm) m 300 240? Distance (9 mm) m 10,000 = 2,000 = Receiver -------- Receiver Power (min) dBm -16 ** -16? -25 -25? -20 -20? Receiver Power (max) dBm 0 0? -3 -3? -3 -3? Return Loss (rec) dB 12 = 12 = 12 = Optical Rise/Fall ns 0.6 0.48? NA NA? NA NA? Transmitter ----------- Wavelength (min) nm 770 = 1285 = 1275 = Wavelength (max) nm 860 = 1330 = 1335 = Spectral Width nm 4 = 3 = 6 = Launch Power (max) dBm -5 -5? -3 -3? -3 -3? Launch Power (min) dBm -10 ** -10? -9 -9? -12 -12? Extinction Ratio dB 9 = 9 = 9 = RIN (max) dB/Hz -116 -117? -116 -117? -116 -117? Eye Opening %p-p 57 57? 57 57? 57 57? Deterministic Jitter % 20 = 20 = 20 = Optical Rise/Fall ns 0.45 0.36 0.37 0.37? 0.37 0.37? ** Set high under the assumption that the IEC would vote to approve 1995 draft that would have raised safety limits. In 20/20 hind-sight, this might have been a mistake. ------------------------------------------------------------------------------- ------------------ ++++++ ++++++++ +++ +++ R. Jonathan Thatcher +======+ +====== +==+ +==+ IBM Optical Interconnect Technology + == + + == == + ==+ +== + OEM Technical Support, D/516A + == + ++=++==+ + = =+= = + 3605 HWY 52 North + == + + == == + = = = + Rochester, MN 55901-7829 +======+ +====== +== ==+ (507) 253-2867 FAX: (507) 253-1438 ++++++ ++++++++ ++ ++ E-Mail: jonathan_thatcher@vnet.ibm.com