Loading…
Monday, June 20 • 2:00pm - 2:25pm
Accelerating Complex Data Transfer for Cluster Computing

Sign up or log in to save this to your schedule and see who's attending!

The ability to move data quickly between the nodes of a distributed system is important for the performance of cluster computing frameworks, such as Hadoop and Spark. We show that in a cluster with modern networking technology data serialization is the main bottleneck and source of overhead in the transfer of rich data in systems based on high-level programming languages such as Java. We propose a new data transfer mechanism that avoids serialization altogether by using a shared clusterwide address space to store data. The design and a prototype implementation of this approach are described. We show that our mechanism is significantly faster than serialized data transfer, and propose a number of possible applications for it.

Monday June 20, 2016 2:00pm - 2:25pm
Denver Marriott City Center 1701 California Street, Denver, CO 80202

Attendees (1)