Title: MadLINQ: large-scale distributed matrix computation for the cloud
Speaker: Zhengping Qian, Microsoft Research Asia (MSRA)
Time: 15:30pm, Thursday, Dec 27, 2012
Venue: Conference Room, 4th Floor #5 Building, General Department of ISCAS
Contact: Kai Wang (wangkai@iscas.ac.cn)

Abstract:
The computation core of many data-intensive applications can be best expressed as matrix computations. The MadLINQ project addresses the following two important research problems: the need for a highly scalable, efficient and fault-tolerant matrix computation system that is also easy to program, and the seamless integration of such specialized execution engines in a general purpose data-parallel computing system.

MadLINQ exposes a unified programming model to both matrix algorithm and application developers. Matrix algorithms are expressed as sequential programs operating on tiles (i.e., sub-matrices). For application developers, MadLINQ provides a distributed matrix computation library for .NET languages. Via the LINQ technology, MadLINQ also seamlessly integrates with DryadLINQ, a data-parallel computing system focusing on relational algebra.

The system automatically handles the parallelization and distributed execution of programs on a large cluster. It outperforms current state-of-the-art systems by employing two key techniques, both of which are enabled by the matrix abstraction: exploiting extra parallelism using fine-grained pipelining and efficient on-demand failure recovery using a distributed fault-tolerant execution engine. We describe the design and implementation of MadLINQ and evaluate system performance using several real-world applications.

Bio:
Dr. Zhengping Qian is an associate researcher in the System Research Group at Microsoft Research Asia (MSRA).  His research interests are in distributed and mobile computing with an emphasis on emerging applications. Dr. Qian joined MSRA in July of 2009, and won the best paper award in EuroSys 2012. Dr. Qian has been very fortunate to work with quite a few super talented intern students in MSRA and many of them are now pursing their PhDs in MIT, Princeton, CMU, UIUC, UT Austin, etc. and he is actively looking for the next one.