Designing for Gesture and Tangible Interaction

Interactive technology is increasingly integrated with physical objects that do not have a traditional keyboard and mouse style of interaction, and many do not even have a display. These objects require new approaches to interaction design, referred to as post-WIMP (Windows, Icons, Menus, and Pointer) or as embodied interaction design.

This book provides an overview of the design opportunities and issues associated with two embodied interaction modalities that allow us to leave the traditional keyboard behind: tangible and gesture interaction. We explore the issues in designing for this new age of interaction by highlighting the significance and contexts for these modalities. We explore the design of tangible interaction with a reconceptualization of the traditional keyboard as a Tangible Keyboard, and the design of interactive three-dimensional (3D) models as Tangible Models. We explore the design of gesture interaction through the design of gesture-base commands for a walk-up-and-use information display, and through the design of a gesture-based dialogue for the willful marionette. We conclude with design principles for tangible and gesture interaction and a call for research on the cognitive effects of these modalities.

Mary Lou Maher is Professor and Chair of Software and Information Systems at UNC Charlotte and Honorary Professor of Design Computing in the Design Lab at the University of Sydney. At UNC Charlotte, she has established a concentration in HCI in the B.A. in Computer Science and in the M.S. in Information Technology. She has established a Design Studio and a Maker Space in the Department of Software and Information Systems. While at the University of Sydney, she created a new undergraduate degree called a Bachelor of Design Computing. While a Program Director at NSF, she started a funding program called CreativeIT. Dr. Maher's research interests include: cognitive effects of embodied interaction modalities, computational and cognitive models of creativity, and design patterns for active learning strategies in CS education. Lina Lee is a Ph.D. student in the College of Computing and Informatics at UNC Charlotte. She received a B.S. and a M.S. in Housing and Interior Design from Kyung Hee University in South Korea. After then, she received a dual degree: M.S. in Architecture and M.S. in Information Technology Software at UNC Charlotte. She is a member of the InDe Lab and HCI Lab at UNC Charlotte. Her research interest is in the area of Human-Computer Interaction with particular focus on designing an interactive information system with gesture-based interactive techniques that enhance public engagement with interactive systems. John M. Carroll is Distinguished Professor of Information Sciences and Technology at Penn State University, and was a founder of human-computer interaction. He served on the program committee of the 1982 Bureau of Standards Conference on the Human Factors of Computing Systems that in effect inaugurated the field and was the direct predecessor of the field's flagship conference series, the ACM CHI Conferences. Through the past two decades, Carroll has been a leader in the development of the field of Human-Computer Interaction. In 1984 he founded the User Interface Institute at the IBM Thomas J.Watson Research Center, the most influential corporate research laboratory during the latter 1980s. In the 1994, he joined Virginia Tech as Department Head of Computer Science where in 1995 he led the effort to form the university's Center for Human-Computer Interaction. That year, Virginia Tech was invited to join the Human-Computer Interaction Consortium, a group of the leading corporate and academic HCI research organizations in the world. He has written more than 250 technical papers, more than 25 conference plenary addresses, and 13 books, including HCI Models, Theories, and Frameworks, and Usability Engineering (with Mary Beth Rosson). He serves on 10 editorial boards for journals and handbooks, and is the current editor-in-chief of ACM Transactions on Computer- Human Interaction (ToCHI). He has won the Rigo Career Achievement Award from ACM (SIGDOC), received the Silver Core Award from IFIP, and is a member of the CHI Academy. In 2003 he became the fifth recipient of the CHI Lifetime Achievement Award, the most prestigious research award in HCI.