RDMA for Apache Spark Change Log -------------------------------- This file briefly describes the changes to the RDMA for Apache Spark software package. The logs are arranged in the "most recent first" order. Release 0.9.5 - 2018-02-19 NEW FEATURES - Initial support for POWER architecture - Performance optimization and tuning on OpenPOWER clusters - Support for various communication thread binding policies - Support for RDMA Device Selection - Tested with - Various multi-core platforms (e.g., x86, POWER) - OpenJDK and IBM JDK BUG FIXES - Fix an issue for running spark-shell with RDMA-Hadoop 1.3.0 - Thanks to John Garbutt@StackHPC for reporting the issue Release 0.9.4 - 2017-03-26 NEW FEATURES - Based on Apache Spark 2.1.0 - Compliant with Spark 2.1.0 APIs and applications BUG FIXES - Fix issues for Spark SQL workloads Release 0.9.3 - 2016-11-12 NEW FEATURES - Built with Apache Hadoop 2.7.3 - RDMA support for Spark SQL - Integration with HHH in RDMA-Hadoop - Support pre-connection mechanism - Support for Mellanox EDR HCA BUG FIXES - Fix a hang issue in finalize stage Release 0.9.1 - 2016-01-05 NEW FEATURES - Based on Apache Spark 1.5.1 - Built with Apache Hadoop 2.6.0 - High performance design with native InfiniBand and RoCE support at the verbs level for Spark - RDMA-based data shuffle - SEDA-based shuffle architecture - Efficient connection management and sharing - Non-blocking and chunk-based data transfer - Off-JVM-heap buffer management - Compliant with Spark 1.5.1 APIs and applications - Easily configurable for native InfiniBand, RoCE, and the traditional sockets based support (Ethernet and InfiniBand with IPoIB) - Tested with - Mellanox InfiniBand adapters (DDR, QDR, and FDR) - RoCE support with Mellanox adapters - Various multi-core platforms - RAM Disks, SSDs, and HDDs