In HHH-L mode, Hadoop MapReduce runs over HDFS that is integrated with parallel file system (e.g. Lustre). This is a hybrid mode that stores data in both local storage and Lustre. Lustre provides fault-tolerance in this mode. Therefore, HDFS replication is not required. We evaluate our package with this setup and provide performance comparisons here. For detailed configuration and setup, please refer to our userguide.
TestDFSIO Read and Sort
TestDFSIO Read
Sort Execution Time
Experimental Testbed: Each node of our testbed has two 4-core 2.53 GHz Intel Xeon E5630 (Westmere) processors and 24 GB main memory. The nodes support 16x PCI Express Gen2 interfaces and are equipped with Mellanox ConnectX QDR HCAs with PCI Express Gen2 interfaces. The operating system used was RedHat Enterprise Linux Server release 6.4 (Santiago).
These experiments are performed in 8 DataNodes with a total of 32 maps. Sort runs with 14 reducers. Each DataNode has a single 1TB HDD, a 300GB SSD, and 12GB RAM Disk. HDFS block size is kept to 256 MB. Each NodeManager is configured to run with 6 concurrent containers assigning a minimum of 1.5 GB memory per container. The NameNode runs in a different node of the Hadoop cluster and the benchmark is run in the NameNode.
The RDMA-IB design improves the read throughput of TestDFSIO by up to 3x over Lustre-IPoIB (32Gbps). The execution time of Sort is reduced by 26% - 55% compared to HDFS-IPoIB (32Gbps) and 31% - 58% over default MapReduce running on top of Lustre-IPoIB (32Gbps).
