Green Screen Augmentation

Overview

Green Screen Augmentation is a data augmentation framework designed to achieve robust visual scene generalisation for robot manipulation policies without requiring expensive multi-environment data collection.

Vision-based policies frequently overfit to visual background textures, table surfaces, and ambient lighting in their training setup. Green Screen Augmentation addresses this by segmenting the robot and task-relevant objects from the background using green-screen-inspired chroma-keying and synthetic background compositing. By training models with diverse background augmentations, the learned policies exhibit zero-shot generalisation to novel visual backgrounds, lighting conditions, and cluttered tabletop distractors in the real world.

Summary Poster

Overview poster for Green Screen Augmentation.

Authors

Eugene Teoh · Sumit Patidar · Xiao Ma · Stephen James

Publications & Resources

Preprint (2024)

E. Teoh, S. Patidar, X. Ma and S. James, “Green Screen Augmentation Enables Scene Generalisation in Robotic Manipulation,” arXiv preprint arXiv:2407.07868, 2024.

• arXiv Preprint (arXiv:2407.07868)
• Project Website
• Video Demonstration

Citation

@article{teoh2024green,
  author  = {Teoh, Eugene and Patidar, Sumit and Ma, Xiao and James, Stephen},
  title   = {Green Screen Augmentation Enables Scene Generalisation in Robotic Manipulation},
  journal = {arXiv preprint arXiv:2407.07868},
  year    = {2024}
}