Autonomous Marionettes

The project is divided into three main research tasks:

Puppet Choreography with Motion Programs

Much like Hollywood and Broadway writers, puppeteers write scripts that decribe the elements that make up their performance. These elements include stage setups, characters, timing, and actions. Consequently, puppeteers use these detailed scripts to control the characters (puppets) in an expressive way while simultaneously adhereing to the structure and timing of the play.

Since the plays are described by discrete actions based on a clock mechanism, they may be modeled by a MDL, specifically the variation MDLp. Our research focuses on automating the process of translating puppet play specifications in MDLp to a sequence of valid puppet control modes. Not only are we interested in the specificaiton of these plays, we also wish to optimize the control sequence based on a model of a real puppet in order to create more realistic motion on the real platform.

Check out a movie of our joint work with Northwestern University!

Imitating Human Motions

What does it mean when people say that two motions are “similar?” This is an important question in the area of human-robot interactions, where robots must interpret human movements in order to act in an equivalent manner. Alternatively, suppose we would like a puppet to do “the same thing” as a human. The puppet has different degrees of freedom, different dimensions, and different dynamics from the human: What is the appropriate way to compare puppet motions to human ones, or to control the puppet so it does “the same thing?” This work addresses these questions from two different directions:

  1. by directly measuring the preferences of human judges.In this approach, the idea is that when people compare alternatives, they are implicitly making reference to an objective function “inside their heads;” the goal, then, is to reverse-engineer this objective function based on peoples’ expressed preferences.
  2. by using similarity measures based on homeomorphisms.In this second approach, we “warp” one motion in time and space so that it resembles another motion, and use the amount of warping as our measure of dissimilarity.

Distributed Protocols for Coordinating Puppets

In this work, we consider the problem of coordinating multiple pendula attached to mobile bases. In particular,the pendula should move in such a way that their motion is synchronized, which calls for two problems to be solved simultaneously, namely a constrained optimal control problem for each pendulum, and a constrained agreement problem across the network of pendula. Novel ways of manipulating the initial conditions in the consensus equation are employed to deal with the latter of these problems.


In collaboration with Northwestern University:

and the Center for Puppetry Arts:

  • John Ludwig
  • Jason Hines


  • P. Kingston, M. Egerstedt. Comparing Apples and Oranges through Partial Orders: An Empirical Approach. To appear atAmerican Control Conference, June 2009.
  • P. Martin and M. Egerstedt. Timing Control of Switched Systems with Applications to Robotic Marionettes. To appear inJournal of Discrete Event Dynamic Systems: Theory and Applications, 2009.
  • P. Martin and M. Egerstedt. Optimization of Multi-Agent Motion Programs with Applications to Robotic Marionettes.Hybrid Systems: Computation and Control, pp. 262-275, Springer-Verlag, San Francisco, USA, April 2009.
  • R. Chipalkatty, M. Egerstedt, and S. Azuma. Multi-Pendulum Synchronization Using Constrained Agreement Protocols.ROBOCOMM, Odense, Denmark, March 2009.
  • P. Martin and M. Egerstedt. Optimal Timing Control of Interconnected, Switched Systems with Applications to Robotic Marionettes. Workshop on Discrete Event Systems, Gothenburg, Sweden, May 2008.
  • T. Mehta and M. Egerstedt. Multi-Modal Control Using Adaptive Motion Description Languages. Automatica. Accepted for publication. To appear 2008.
  • M. Egerstedt, T.Murphey, and J. Ludwig. Motion Programs for Puppet Choreography and Control. Hybrid Systems: Computation and Control, Springer-Verlag, pp. 190-202, Pisa, Italy April 2007.


US National Science Foundation

Comments are closed.