About the Talk: The advent of large-scale machine learning models ("foundation models") has been paradigmatic in the fields of computer vision and natural language processing. What would a similar consolidation look like for robot learning, where learned models are typically trained on data from a single research group and on a specific robot embodiment? In this talk, there will be presented author's thesis research on training machine learning models from cross-embodiment robot data, collected entirely in the real world, for the task of visual navigation. In particular, it will be discussed why and when learning from real-world robot data is beneficial (or inevitable) over learning from simulation or passive data sources. As a recipe, it will be discussed how sharing data across robots and tasks can enable remarkable zero-shot generalization, as well as serve as a "starting point" for downstream tasks beyond the training data. Lastly, it will also be discussed how we can adapt pre-trained "base" models with real-world interaction data for learning socially-compliant robot behaviors and learning challenging tasks like off-road racing that push the robot to its physical limits.

About the Speaker: Dhruv Shah is a final year PhD candidate in EECS at UC Berkeley, where he is advised by Sergey Levine. His research spans the fields of machine learning and robotics, with the general goal of enabling real-world robotic systems perceiving and acting “in-the-wild”. His research is supported by the Berkeley Fellowship for Graduate Study, and was nominated for the Best Systems Paper Award at RSS 2022. Earlier, he graduated with honors from IIT Bombay, where he received the Undergraduate Research Award and the Institute Academic Prize