Greetings, You are invited to the PhD prelim defense for Jingoo Han. Title: Towards a Resource Efficient Framework for Distributed Deep Learning Applications Who: Jingoo Han Date: Monday, February 8, 2021 Time: 2:00~3:30pm Zoom: https://virginiatech.zoom.us/j/84630292147 Abstract: Distributed deep learning is widely used for solving critical scientific problems with massive datasets. However, to accelerate the scientific discovery, resource efficiency is also important for the deployment on real-world systems, such as high-performance computing (HPC) systems. Deployment of existing deep learning applications on these distributed systems may lead to underutilization of HPC hardware resources. In addition, extreme resource heterogeneity has negative effects on distributed deep learning training. However, much of the prior work has not focused on specific challenges in distributed deep learning including HPC systems and heterogeneous federated systems, in terms of optimizing resource utilization. This PhD dissertation addresses the challenges in improving resource efficiency of distributed deep learning applications, through (1) Performance analysis on deep learning for supercomputers, (2) GPU-aware deep learning job scheduling, (3) Topology-aware Virtual GPU training, and (4) Heterogeneity-aware adaptive federated learning.