Thread Links Date Links
Thread Prev Thread Next Thread Index Date Prev Date Next Date Index

Re: HARI: Question regarding byte (column?) versus word striping


Column/Byte-Striping and Hari heavily leverages InfiniBand technology from its
SIO, NGIO and FIO roots. The old NGIO supported "thin" and "fat-pipes" of 1 and
12 lanes, respectively, running at a line rate of 2.5 Gbaud, 8B/10B encoded. The
technology for this architecture, in turn, was leverage from Gbe whose
architecture, in turn was leveraged from Fibre Channel. InfiniBand has recently
added 4 lanes to its 1/12 repertoire.

An InfiniBand specification tutorial is available at:

The last presentation,,
has the most relevant information. See page 7 of this presentation for the
striping direction which is:

"Supporting 1, 4 or 12 physical lanes by byte striping packet on to all
available lanes"

A key feature of Column-Striped Hari is its scalability.

Starting a packet on lane 0 is nether required for Hari nor Column-Striping. It
is simply a direction.

When Hari is scaled to higher speeds, Column-Striping can function with a
smaller IPG than can Word-Striping. Two requirements of the IPG are:

1) Link synchronization - commas required
2) Clock tolerance compensation - insert/remove protocol required

Column-Striping requires only a single column of commas (K) for link
synchronization and utilizes a single column of skips (R) for clock tolerance
compensation. The minimum IPG size based on starting a new packet in lane 0 is
3n, where n=number of lanes. That's one n for the K column, another for the R
column and a third possibly complete column containing the End-of-Packet
delimiter plus pad characters.

Word-Striping uses words containing commas which may be inserted/deleted during
the IPG. The minimum IPG size is 5n, where n=number of lanes. That's 4n for 4
deletable Idles and one more word containing the End-of-Packet delimiter plus
pad characters. 

Best Regards,


"Simon L. Sabato" wrote:
> Gents,
> Should be be considering the scalability of the two proposals?  It would be
> nice if the protocol (including idle insertion, etc) could extend easily in
> terms of the number of lanes, as well as frequency of operation, for future
> versions of Ethernet or for Infiniband.
> I could be wrong, as I haven't been following every detail of the debate,
> but my first instinct tells me that the word-striped proposal would more
> easily scale with the number of lanes.  It is also possible that it is not
> customary (or taboo!) to discuss future scalability when defining an IEEE
> standard, although I think that it is reasonable to at least consider the
> ramifications.  For example, an early Hari proposal mentioned that since
> packets must start on lane 0, odd length packets (i.e. 65 bytes) would have
> a larger than normal IPG.  What happens if, for 100Gig, we move to 20 lanes
> at 6.25GHz?  Not only does the IPG get extended even further, but it would
> be extended for "normal" length packets such as 64 bytes.
> The word-striped proposal works by putting one "word" of data (the word
> width given at the XGMII interface) on each lane.  The number of lanes
> required is solved by dividing the total data rate (10Gbps) by the rate of
> each line (2.5Gbps).
> Byte-striping, in contrast, works by putting one "byte" (fixed at 8 bits) of
> data on each lane.  The number of lanes required is solved, again, by
> dividing the total data rate by the rate of each line.  In this case,
> through, the XGMII interface width is dependant on the number of lanes,
> fixed to (8 bits)*(number_of_lanes).  Parts of the protocol (for example,
> IPG extension) are dependant on number of lanes.  This seems to be slightly
> less flexible.
> Basically, my point is that if byte- and word-striping are quite comparable
> in terms of complexity at 10Gbps, should we examine the future scalability
> of both proposals to help us select one?
> **************************************************
>  Simon L. Sabato                ph (408) 765-7814
>  Level One Communications
>  An Intel Company        
> **************************************************


Best Regards,

Richard Taborek Sr.         Tel: 408-330-0488 or 408-370-9233       
Chief Technology Officer                   Cell: 408-832-3957
nSerial Corporation             Email: rtaborek@xxxxxxxxxxxxx  
2500-5 Augustine Dr.           Alt email: rtaborek@xxxxxxxxxx 
Santa Clara, CA 95054