Thread Links Date Links
Thread Prev Thread Next Thread Index Date Prev Date Next Date Index

Re: [HSSG] 40G MAC Rate Discussion

> There is a 32 lane version.  The question is how viable is that from 
> an implementation stand-point given that a 32 lane interface is 128 
> PCB traces.

Brad is correct. I am not aware of even one x32 PCI-Ex
implementation. It is not a spec issue, but rather what
is practical for real-life implementations. Pin efficiency
is of essence in I/O bridges, and given the choice, they
will go "faster" rather than "wider".

I have seen multiple bridge implementations that have
several x16 spigots. A server could allocate one of them
for a high-speed device, and use the other for fanout. I
expect this to remain the case in the future. Pins do not
follow Moore's Law.

> There is also overhead associated with moving data from the adapter 
> over the PCIe host bus and into the host memory.

Correct. Our analysis shows 70% efficiency at best,
depending on packet size. Which is why I pointed out
in the past that practically we can expect to get ~45Gb
out of PCI-Ex 2.0.

> Therefore, the bandwidth capability of PCIe is not just what can be 
> calculated.

Very true.

> Looking at when 10GbE first showed up, the only host bus available was 
> PCI-X 133.  It is okay for the network to exceed the capability of the 
> host bus, because over time, the host bus and the servers will catch 
> up, just as they are starting to do now.

Yes, it will. But until then the server will not deliver on
its promise. This is where we differ.