Dobb-E

Introducing Dobb·E, an innovative open-source framework that empowers robots to master household tasks through imitation learning. Designed to overcome the challenges faced by contemporary home robotics, Dobb·E offers an affordable and user-friendly solution for collecting task demonstrations.

At the heart of the framework is an ingenious tool called the Stick, which combines a $25 reacher-grabber stick with 3D-printed components and an iPhone. This simple yet effective device enables the collection of rich data from a unique dataset named the Homes of New York (HoNY). This comprehensive dataset features 13 hours of interactions captured from 22 different homes across New York City, providing a wealth of RGB and depth videos along with action annotations that detail the gripper's 6D pose and opening angle.

Using the data gathered, Dobb·E trains an advanced representation learning model known as Home Pretrained Representations (HPR). This model, based on the ResNet-34 architecture and trained through self-supervised learning, sets the stage for robots to learn and execute new tasks in unfamiliar environments. Remarkably, Dobb·E has achieved an impressive 81% average success rate in tackling new tasks within just 15 minutes, utilizing only five minutes of data collected in a new home.

Dobb·E isn't just about performance—it's also about accessibility. The framework offers a wealth of resources, including pre-trained models, source code, and detailed documentation, all available on GitHub. For those interested in the underlying methodologies and findings, an open-access paper titled "On Bringing Robots Home" provides further insights into the project.

Embark on a new era of home robotics with Dobb·E, where learning and adaptability meet everyday household tasks!