How can we use objects in the real world to control creative content on the screen? VisionPlay is a framework to help designers extract information from a video and link it realtime to a world on the screen. The framework provides tools to extract, identify, and enhance applications for scenarios of digital puppetry, seeing yourself in the story, and play with objects at a distance. I implemented four examples during my fellowship with Hasbro Toy Company, as examples of ways to use physical objects to create digital content: A green-screen glove that places content in a 2.5d perspective world, a tracking engine for puppets and action figures, a laptop green-screen theatre, and telepresence puppetry platform. They are all grouped here to illustrate the broad creative possibilities for creative improvisation, play, and performance with objects supported by emerging technologies.