Session 3: Tutorial in Movement Notation

Interactive Design Tutorial: Using Shared Infrastructure to Notate Movement Across Mediums
Alexandra Bacula and Cat Maguire

1:45 PM Lecture / Movement class / Demonstrations

Topics covered: movement notation formats (including historical development) action stroke, phrasing, BESST System concepts, including Laban Effort

3:15 PM Practice in observation / Working time for pairs

Topics covered: observational guides and methods, iteration through physical rehearsal, ideation across disciplines

3:45 PM Sharing (and trading) movement notations
Topics covered: multitude of interpretive possibilities, complexity of movement, high-level abstractions for imitation across disparate bodies, intention in movement

Session 4: Break Out Sessions

4:30 PM Break out sessions
5:00 PM Report back to group
5:15 PM Close meeting

Breathless: Catie and the Robot (an eight-hour modern dance homage to  physical labor)

Ken Goldberg, Professor, Industrial Engineering and Operations Research, William S. Floyd Jr. Distinguished Chair in Engineering, UC Berkeley, Chief Scientist, Ambi Robotics

Catie Cuan, Postdoc, Stanford University 

This durational duet created by dancer/choreographer/engineer Catie Cuan and artist/researcher Ken Goldberg pairs Catie with an industrial robot arm for an eight-hour dance performance that unfolds over the timespan of an American workday. With music including original compositions by Peet van Street, the performance will contrast the beauty, strength, and frailty of the human body with the precision of machinery. After, Cuan and Goldberg joined NS+ Curator Elena Park for conversation and audience Q&A.

Bios

An engineer, researcher, and artist, Dr. Catie Cuan is a pioneer in the nascent field of ‘choreorobotics’ and works at the intersection of artificial intelligence, human-robot interaction, and art. She is currently a Postdoc in Computer Science at Stanford University. Catie recently defended her PhD in robotics via the Mechanical Engineering department at Stanford, where she also completed a Master’s of Science in Mechanical Engineering. The title of her PhD thesis is “Compelling Robot Behaviors through Supervised Learning and Choreorobotics”, which was funded by the National Institutes of Health, Google, and Stanford University. During her PhD, she led the first multi-robot machine learning project at Everyday Robots (Google X) and Robotics at Google (now a part of Google Deepmind). She has held artistic residencies at the Smithsonian, Everyday Robots (Google X), TED, and ThoughtWorks Arts. Catie is a prolific robot choreographer, having created works with nearly a dozen different robots, from a massive ABB IRB 6700 industrial robot to a tabletop IDEO + Moooi robot. Catie is also a 2023 International Strategy Forum (ISF) fellow at Schmidt Futures and the former co-founder of caali, an embodied media company.

Ken Goldberg has been interested in robots, rockets, and rebels since he was a kid. He’s skeptical about claims that humans are on the verge of being replaced by Superintelligent machines yet optimistic about the potential of technology to improve the human condition. Ken developed the first provably complete algorithm for part feeding and the first robot on the Internet. In 1995 he was awarded the Presidential Faculty Fellowship and in 2005 was elected IEEE Fellow: “For contributions to networked telerobotics and geometric algorithms for automation.” Ken founded UC Berkeley’s Art, Technology, and Culture public lecture series in 1997 serves on the Advisory Board of the RoboGlobal Exchange Traded Fund. Ken is Chief Scientist at Ambidextrous Robotics and on the Editorial Board of the journal Science Robotics. He served as Chair of the Industrial Engineering and Operations Research Department and co-founded the IEEE Transactions on Automation Science and Engineering. Short documentary films he co-wrote were selected for Sundance and one was nominated for an Emmy Award. He lives in the Bay Area and is madly in love with his wife, filmmaker and Webby Awards founder Tiffany Shlain, and their two daughters.

Ways of Knowing: The tools of art and the art of tools

Daniel Koditschek, Kod*lab, Alfred Fitler Moore Professor, Electrical & Systems Engineering, University of Pennsylvania 

 Kat McDermott, Artist, Industry Assistant Professor of Integrated Design & Media at NYU Tandon 

 Dan and Kat participated in the Robotics Art Residency, a collaboration between the GRASP Lab and Weitzman School of Design, awarded a 2020 Sach Program for Arts Innovation Grant, and further supported by the GRASP Lab, Kod*lab and the Weitzmann School of Design. In 2022, three artists – Sam Hensley, Nikkita Stagg and Kat McDermott – spent four weeks at the GRASP Lab at PERCH, developing creative projects that address contemporary social, cultural, educational and/or environmental emergencies. Kod*lab, a subsidiary of the GRASP Lab, directed by Dan Koditschek, specifically hosted and mentored the artists. In November – December 2022, the residencies culminated in an art exhibition, Experiments in Robotic Gestures, at The Arts League in Philadelphia. 

 Bios

Dan Koditschek received his bachelor’s degree in Engineering and Applied Science and his M.S. and Ph.D. degrees in Electrical Engineering in 1981 and 1983, all from Yale University. He served on the Yale Faculty in Electrical Engineering until moving to the University of Michigan a decade later. In January 2005, he moved to Penn as Chair of the recently formed Electrical and Systems Engineering Department, a position which he held through 2012.Koditschek’s research interests include robotics and, more generally, the application of dynamical systems theory to intelligent mechanisms. His more than 200 archival journal and refereed conference publications have appeared in a broad spectrum of venues ranging from the Transactions of the American Mathematical Society through The Journal of Experimental Biology, with a concentration in several of the IEEE journals and related transactions. Various aspects of this work have received mention in general scientific publications such as Scientific American and Science as well as in the popular and general lay press such as The New York Times and Discover Magazine. Koditschek was awarded the 2016 IEEE Robotics and Automation Society Pioneer Award and the 2017 Penn Heilmeier Award for Excellence in Research. 

 Kathleen McDermott is an interdisciplinary artist with a background in installation, prop-making and sculpture, currently based in Brooklyn, NY. She combines her knowledge of fabrication with open source hardware to build a language of absurdity that merges new media, design, performance, and video. She is interested in technologies that are not productive, robots badly suited to absurd purposes and electronic creations beyond her control.

 She received a BFA from Cornell University, MFA in Creative Media from City University of Hong Kong and Ph.D. in Electronic Arts from Rensselaer Polytechnic Institute. She is currently an Industry Assistant Professor of Integrated Design & Media at NYU Tandon. Her work has been exhibited internationally, including at the Museum of Art and Design in New York, The Tides Institute and Museum of Art in Maine, the Wende Museum in LA, and Ars Electronica in Linz, Austria; and has been featured in publications such as The Wall Street Journal, Huffington Post, Fast Company, and Dezeen. In addition to her artistic practice, she is an advocate for accessible technology education who actively seeks opportunities to give workshops, collaborate with community partners, and shares tutorials for working with DIY electronics at urbanarmor.org.

From Pig Iron to Futurist Theater
Mark Yim, Ruzena Bajcsy Director of the GRASP Lab, Asa Whitney Professor of Mechanical Engineering, University of Pennsylvania 

 Simon Kim, Associate Professor of Architecture, University of Pennsylvania 

Mark and Simon co-founded and direct an ongoing collaborative research project Immersive Kinematics. This group is a unique team of engineering and design that expands the roles of architecture and engineering focusing on integrating robotics, interaction, and embedded intelligence in our buildings, cities, and cultures. In this capacity, new devices and new environments are continuously developed and deployed in the production of culture and art. The group has continually tested these prototypes within the domain of theatrical performance. The pair will share selected projects from their history of collaboration. They have worked with award winning organizations such as Pig Iron, Carbon Dance, The Dufala Brothers, Grace Kelly Jazz, and Opera Philadelphia, Immersive Kinematics has produced and designed multiple bespoke stage designs, nonhuman performers, and interactive instruments. With titles such as Nervous Matter, Music of the Planets, Orpheus and Eurydice, the collaborations have been presented to the public at venues such as the Slought Foundation, Traction Company, and the Annenberg Centre.

Immersive Kinematics will host visionary artist and architect Vanessa Keith of Studioteka as an artist in residence. Keith’s work, influenced by Eco-, Afro-, and Indigenous Futurism, addresses climate issues through emerging technologies including Extended Reality (XR) that foster new and more equitable forms of community. She is the author of the book 2100: A Dystopian Utopia / The City After Climate Change, which foregrounds the design issues facing an Anthropocene planet. From 2019 – 2022 Keith was a Lecturer at the Weitzman School of Design and taught a research seminar, Envisioning Climate, which focused on climate fiction and storytelling using emerging technologies and virtual reality. Immersive Kinematics is inviting Keith back to Penn to teach a series of workshops on worldbuilding that centers African-American, Black diaspora and other excluded perspectives in forging sustainable, just and equitable futures for us all.

Hips Do Lie: When Robots Do The Twist

Monica Thomas in conversation with Amy LaViers

Monica Thomas was the choreographer for Boston Dynamic’s “Do You Love Me” dance video on YouTube (currently up to 39 million plus views) featuring Atlas, Spot and Handle. 

https://www.dancemagazine.com/boston-dynamics-dancing-robots/

https://spectrum.ieee.org/how-boston-dynamics-taught-its-robots-to-dance

Monica Thomas is a choreographer and dancer. The majority of her creative work has been with Mad King Thomas, a collaboration with Tara King and Theresa Madaus that spanned over a decade. Together they created and performed small and large works nationally and internationally. Monica also choreographed for Boston Dynamics, where she collaborated with engineers to develop movement for robots. Monica is trained as a dance/movement therapist and works as a licensed clinical mental health counselor.

Format:

An introduction to movement notation will be given via a hybrid between a lecture and a movement class. Slides and short exercises will demonstrate the mechanics of a system of notation presented in a recently published MIT Press text Making Meaning with Machines (Oct 2023). Then, interactive movement assignments will provide pairs a chance to get to know one another and use their newly gained terminology to discuss movement. Pairs will find a movement source (either movement from the hybrid lecture or in a video, either online or sourced from prior work, the pairs agree on) of interest, create a simple notation of the aspect of the movement that is of interest (with help from facilitators), and use this crystallization of common interest to begin creating other media (choreography, visual sketches, musical elements, robot sketches, and so on) that explore this concept further. 

Discussion:

The goal of this session is to begin new collaborations between roboticists and artists through an initial working session identifying and refining movement concepts of interest via movement notation. At the core of any collaboration in robotics is movement, but identifying and describing movement phenomena of interest can be a challenge, especially across disparate disciplines. To address this central challenge, the tutorial gives an introduction to a system of movement notation that may be used to bridge disjoint ways of measuring, perceiving, and talking about movement, and moreover, it is an opportunity for participants to interact with each other doing meaningful work. 

This aspect of the event is funded by an AI infrastructure grant from the National Science Foundation that aims to create a large, shared database to improve the speed with which researchers and developers can utilize human movement perception in their work. The grant funds education for roboticists around movement notation and annotation as well as collecting feedback on pain points in this domain. 

Bios

Catherine (Cat) Maguire is a movement educator, dance artist, Certified Movement Analyst (CMA), Registered Somatic Movement Educator through ISMETA (International Somatic Movement Education and Therapy Association), and a master teacher of the Laban/Bartenieff Movement System (LBMS). She is a member of WholeMovement, a coterie of Movement Analysts working together to promote the LBMS globally, and has taught and co-coordinated Movement Analysis Certification Programs in the US, Europe, Mexico and China. Cat is also a collaborator and consultant for the Robotics, Automation and Dance (RAD) Lab where she works with roboticists on embodied movement training and analysis to support the development of expressive robotic systems.

Alexandra Bacula is a dancer, artist, and roboticist working as a research scientist in the Robotics, Automation, and Dance (RAD) Lab. She completed her PhD in Robotics at Oregon State University in the CHARISMA Lab under Professor Heather Knight. 

Discussion: Each group will be given access to a shared google doc with the questions to facilitate discussion. They will be asked to designate a scribe to take notes during the discussion in the shared google doc and a speaker to communicate key takeaways from their discussion to the remaining groups at the end of the session.  Discussions will be framed by the “live” ideas that come up across the day. 

Tentative questions to be provided for discussion:

• What are the best practices and methodological means for creating successful artist-roboticist collaborations?
• How are these collaborations funded? What are typical funding amounts / rates for this work?
• How do artists help make better robots? 

• What are the measures of success for artists and for roboticists?
• How are artist processes and engineering processes similar and different?
• When is an art work or a research paper ready for the world?
• Are robots a new tool for art? Does art just help roboticists communicate their work? Are robots replacing creatives? Consider collaboration rather than one field serving the other.