Hi all,
The first Stack@CS meeting of Spring 2025 will take place this Wednesday (1/22) at 220 Gilbert Street (4211). The speaker of this meeting is Trey Woodlief, who is a Ph.D. Candidate at the University of Virginia. Details about the talk and bio of the speaker are given below -
Title:
A Differential Testing Framework to Identify Critical AV Failures Leveraging Arbitrary Inputs
Abstract:
The proliferation of autonomous vehicles (AVs) has made their failures increasingly evident. Testing efforts aimed at identifying the inputs leading to those failures are challenged by the input’s long-tail distribution, whose area under the curve is dominated by rare scenarios. Leveraging emerging open-access datasets can accelerate the exploration of long-tail inputs. However, having access to diverse inputs is not sufficient to expose failures; an effective test also requires an oracle to distinguish between correct and incorrect behaviors. Current datasets lack such oracles and developing them is notoriously difficult. This talk introduces DIFFTEST4AV, a differential testing framework designed to address the unique challenges of testing AV systems: 1) for any given input, many outputs may be considered acceptable, 2) the long tail contains an insurmountable number of inputs to explore, and 3) the AV’s continuous execution loop requires failures to persist in order to affect the system. DIFFTEST4AV integrates statistical analysis to identify meaningful behavioral variations, judges their importance in terms of the severity of these differences, and incorporates sequential analysis to detect persistent errors indicative of potential system-level failures. Our study on 5 versions of the commercially-available, road-deployed comma.ai OpenPilot system, using 3 available image datasets, demonstrates the capabilities of the framework to detect high-severity, high-confidence, long-running test failures.
Bio:
Trey Woodlief is a Ph.D. candidate at the University of Virginia advised by Sebastian Elbaum and Kevin Sullivan. His research interests lie at the intersection of software engineering and robotics focusing on autonomous system safety by developing novel verification and validation techniques tailored to the unique constraints of the robotics domain. Results of his work have appeared in top venues in software engineering and robotics including ICSE, ICRA, and IROS.
Location: Gilbert Street (4211)
Date: 1/22/2025
Time: 10am-11am
Zoom: https://virginiatech.zoom.us/my/mahir.kabir
The meeting schedule of this semester can be found here<https://sites.google.com/vt.edu/stackcs-spring25/home>. If you want to present your work, please contact me or Dr. Na Meng (nm8247(a)vt.edu<mailto:nm8247@vt.edu>). Looking forward to seeing you this Wednesday.
Sincerely,
Mahir
Md Mahir Asef Kabir
PhD Candidate
Department of Computer Science
Virginia Tech
