In addition to using TCP and UDP sockets over IP as the underlying medium, it was also decided to bypass the IP stack and use the data link layer to implement the protocol. Unfortunately, the current APIs provided along with the Myrinet distribution do not allow more than 2 channels (similar to ports on TCP/UDP sockets) on a single Myrinet interface. This prevents us from running more than one daemon ( either HEC/SEC ) on a host. It is expected that this problem will be alleviated in the near future.
Results of the establishment procedures, in terms of setup time taken over different communication mechanisms, are provided in Section 6.