This video demonstrates how our fully-learned local planner can navigate complex environments by recognizing different terrains and their traversability.

Our paper introduces a novel planner design that combines depth and semantic information. It employs the imperative learning paradigm for optimizing the planner weights end-to-end based on the planning task objective. The optimization uses a differentiable formulation of a semantic costmap, enabling the planner to differentiate between different terrains and accurately identify obstacles.

Trained entirely in simulation, ViPlanner can be applied to real-world scenes in a zero-shot manner. Moreover, our experimental results demonstrate resistance to noise and a significant decrease in terms of traversability costs compared to purely geometric approaches.

For more information:
- Visit our Project Website at https://leggedrobotics.github.io/vipl...
- Read our Paper https://arxiv.org/abs/2310.00982
- Checkout our Code https://github.com/leggedrobotics/vip...