Skip to main content

The Movement Undercommons: Movement Analysis as Meaning Making in a Time of Global Migrations: The Movement Undercommons: Movement Analysis as Meaning Making in a Time of Global Migrations

The Movement Undercommons: Movement Analysis as Meaning Making in a Time of Global Migrations

The Movement Undercommons: Movement Analysis as Meaning Making in a Time of Global Migrations

The Movement Undercommons: Movement Analysis as Meaning Making in a Time of Global Migrations

Grisha Coleman Brenda McCaffrey

School of Arts, Media & Engineering Arizona State University Tempe, AZ 85281, USA

Keywords

Movement, migration, motion capture, annotation, repository, database, pattern analysis, open-source, translation, community, somatic movement.

Abstract

While migration studies are generally approached in geographical/statistical/geo-political terms [time, histories, routes], this project considers migratory movement at the scale of individual human movement. movement as a marker of identity expressed through qualities of posture, rhythm, gesture, tempo, orientation. Each person’s movement is unique, an individual’s movement ‘fingerprint’, and this project seeks to reveal and honor the specific, fluid, complex qualities of a people in motion of body and location, while adding to a critical discourse surrounding issues of contemporary migration. This is a position paper describing the research framework behind a new project which proposes an exploration of movement and mobility amongst internal migratory populations within two pilot areas; South Africa and Greece. This work develops our previous work [1] [Iyengar, V., Coleman, G. et. al. 2016], creating a repository for a growing collection of high- resolution motion-capture ‘portraits’. This repository will not only hold the source documentation of movement sequences, but also serve as an open platform for those recorded. It will become a space for discussion, creative interpretation, translation, annotation, and analysis. The repository opens a public space for artists, researchers, dancers, ethnographers, humanists, and somatic movement educators to respond and add diverse layers of meaning; creative interpretation, social and historical context, and technological and somatic analysis. Thus, we build an expandable platform for exploring the linguistics of movement through a range of responses.

Introduction

Cultural identities can be said to be emergent of movement, and can be clearly and immediately recognized from extraordinarily small and sparse sets of digital information. As kinesthetic data ‘portraits’, this information can contribute to visualizing individual and group narrative in new ways, including narratives of migration, typically approached through statistical or textual study. Human movement is intrinsically expressive, and can convey affective, visceral understandings of another. And because

digital movement information can at once represent the specific movement patterns of an individual while remaining an abstraction, it has the power to express an intimate and sensorial image of human effort and engagement.

Recent advances in movement capture technology (mocap) allow for the translation of human movement into digital data. If we imagine that the field of dance can include a range of everyday movement, mocap technology can extend an exploration of human movement at dimensions of scale, space, and the instantaneous simultaneity of networked information. This project proposes a situated approach for sourcing and capturing movement data that incorporates and foregrounds environmental, cultural, technological, economic and historical contexts in which our movement is inextricably embedded to approach legacies of expulsion, displacement, migration, diaspora, and dense cultural diversity.

As one of the goals for this research is the development of an open-source digital movement repository built in collaboration with a diversity of communities, the process of both data collection and use is of the utmost importance. Issues of cultural (and physical) appropriation are rampant and complex in the histories of marginalized communities, continental Africa and the Diaspora, and digital technologies can be easily used to propagate these injustices. At the same time, we seek to record people and patterns who are typically not documented, to find alternative ways to embody those who have not been seen.

We re-purpose Ivan Illich’s concept of ‘vernacular work’ [14] here to refer to those everyday forms of dance and organized movement that are informal, refractory (resistant to formal analysis), yet are socially reproduced and derived from commons. The project challenges the notion of digital neutrality and investigates movement vernaculars through the development of a computational approach that highlights both similarities and differences, thereby revealing the specificities of each individual mover.

Pilot Applications

To investigate this proposed work, we take on applications through two unique invitations. The first is a month-long workshop residence in collaboration with LATRA [2] an innovation design studio in Lesvos-Greece organized in

response to the European refugee crisis. LATRA has established a technology driven makerspace in the KARA TEPE Refugee Camp in Lesvos-Greece, where it implements innovation-led, open source projects directly on the field, and advocates for the equitable development of beneficiaries through STEM learning programs. In this case, in response to refugee requests for opportunities to ‘get their stories out’, we propose several workshops in motion capture technology, somatic movement, and storytelling either at the KaraTepe Camp or off-site in the Mytilene area with community members whose identities, professional and life experiences overlap or include: refugee, aid-worker, local, dancer, designer/engineers, educator. The workshops will result in co-created, community controlled data/movement sets to visualize and tell stories of contemporary diaspora and migration. These workshops, scheduled for summer/fall 2018, will be one among multiple nodes of this larger project on movement capture and migration to develop methods and best practices.

The second invitation is a longer-term residence as a guest lecturer at the University of Pretoria, South Africa. In this case, we approach the legacy of Apartheid’s embodied identitarian regimes, rich history of migration, and dense cultural diversity. In conversation with scholars at the university, we will broach questions such as: how does the history of forced removal such as that of District Six in the Cape - or the creation of group areas that forced people into certain racial areas and out of others influence the ways bodies move in the present? We will explore specific issues of internal movement in South Africa, such as the necessity of South Africans to travel long distances for work every day, resulting in a hundred thousand minibus taxis taking more than ten million predominantly black South Africans to work. Or, the influx of ‘foreigners’ or other Africans into South Africa, how they are segregated, and how their movement around the major cities differs from others.

Through accessing specific movement patterns and investigating how specific patterns of movement —such as labor, dance, sickness, traveling— are embodied - and how that embodiment changes over time in contemporary post- Apartheid South Africa. The research looks to incorporate a nuanced history of movement that is creative, voluntary, coerced, and create an open-source digital movement repository built in collaboration with diverse communities who will develop the processes of data collection and determine its use.

This work is being planned in collaboration with faculty and artists at University of Pretoria, providing an excellent setting for this research. The pedagogy and current research there covers similar conceptual terrain, with multiple opportunities for equipment and resources, shared communities, and curriculum development. These networks will be extended while in Durban, as we have begun conversation with several colleagues at UKZN, Durban.

A Cultural Database of Motion / Prior Work

Our capacity to recognize and distinguish amongst different

kinds of movement —animal, human, animate, inanimate— is a crucial evolutionary ability as basic as breathing. It is rarely reflected upon unless that ability is damaged or lost. Relatively new, powerful, and robust motion capture systems are continuously advancing to demonstrate more and more ability to capture the density of details that render data to reflect the holistic aspects of human movement, along with the unique signature that each one of us carries like a fingerprint. As such, cultural specificities can be said to be emergent of movement, and can be clearly and immediately recognized from extraordinarily small and sparse sets of data.

The collection of digital data will result in quantitative and qualitative information. While databases of gesture, pedestrian movement and dance certainly exist [3,4,5], technological procedures in current motion capture data are, by definition, processes of reduction and separation. Instead, this project proposes to document the deeply embedded, unique, non-neutral movement vernaculars with recognizable cultural personalities. Certain formal styles of dance are culturally connected to concepts of frame and plane in a Cartesian grid space—e.g. Western European classical forms and the notation systems that ensued. Complex movement forms emerging from non- European vernaculars and other ‘new world’ diasporic movement traditions have often defied written notation for a host of technical, socio-political and cultural reasons.

There currently are several examples of open source libraries of motion capture data [2,3,4], as well as many projects utilizing motion capture data [5,6]. The open source libraries tend to focus on either simple useful gestures (running, walking, jumping) intended for use by game developers, or on the motion capture of complex choreographed forms of professional dance. Motion capture data sets such as the Open Motion Project at ACCAD [3] set out to make available motion data for video games, animation, etc. On the other side of the spectrum, Motionbank.org [4] has created a “. . . network of choreographers, dancers and researchers interested in using MoSys for their own purposes”, with the stated aim of “translating choreography and dance into new digital forms.” A notably different project leveraging motion data is White Glove Tracking Project [10] that creates an experimental framework for not only annotating the motion in a video, but also providing a community platform to highlight community responses in which crowd sourcing data sets relate to human movement and dance. Another repository that provides storage, visual browsing and annotation of motion capture data set is the RepoVizz [1]. It has a highly-advanced user interface that allows easy navigation through a tree-based structure of multi-modal data streams and rendering of real-time data through WebGL.

Our project aims to direct the power of community analysis not at a single highly choreographed performance

or online video, but rather at a wide range of movement vernaculars that we encounter around the world in our everyday lives.

Figure 1. Example Motion Capture content visualized by Jennifer Wieler.

As one of the goals for this research is the development of an open-source digital movement repository built in collaboration with a diversity of communities, the methods of both data collection and use is of utmost importance. Issues of cultural and physical appropriation are rampant and complex in the histories of continental Africa and throughout the Diaspora, and digital technologies can easily be used to propagate these injustices. At the same time, we seek to record people and patterns who are typically not documented, to find alternative ways to embody those who have not been seen; are there patterns of movement in specific groups such as black domestic workers, dancers, miners, college students?

Addressing the field of contemporary human geography, Derek McCormack asks; “Where might a discussion of the relation between dance and geography begin?” [11]. In the study of human movement and migration, this research asks; What might be revealed in recording, mining and comparing culturally distinct movement lexicons? What could a platform for the development of a linguistics of movement that considers form and context provide?

Approach

In our system, movement sequences are recorded as stretches of improvised/indeterminate movement, so that the development and dynamics of the movement vocabulary do not suggest a finite beginning and ending, as in a gesture library, and create a rich capture space to be observed and analyzed across multiple dimensions. This approach suggests that movement is expressed as decisions over time, and that the critical, somatic, physical principles display compositional intention significant to the meaning of the movements overall.

Generally speaking, somatic education emphasizes creating conditions for more efficient, functional movement patterns to emerge [8,9]. Yet it is also understood that patterns of habitual movement are, at any given moment in a person’s life, an accumulation of who, where, and what the person has done and where they have been in their lives [10]. Our research shows that an individual’s repetitive patterns of movement are often extraordinarily ‘high fidelity’ – in that the digital representation of that movement can be successfully recognized with small amounts of data.

Sourcing movement material will begin with local dance practitioners. We will create prompts with interpretable parameters around axes of rhythm, space, and duration with subjects across a diversity of styles and approaches. In sourcing movement material, of particular interest are forms that emerge ‘spontaneously’ from collective, informal, situations such as social dancing at clubs or parties, skateboard parks, community sports, and so on.

Fig 2. Comparing Vernaculars from “ Motion, Captured: An Open Repository for Comparative Movement Studies,"

It is a great challenge to collect and evaluate movement data without dissolving into reductionist methods. Much work has already been completed in this field to establish a methodology that uses advanced motion capture techniques and data analysis to clearly show complex and compelling individual movement patterns (Iyengar, V., et.al. 2016). In this work, a motion capture system was used to track movements of dancers from a variety of cultural backgrounds. Data was analyzed and simple, graphical/video models were constructed to show patterns of movement. This information was shared and discussed with participants who gained insights into how their personal cultural histories informed their movement patterns.

Data can be analyzed using existing MATLAB tools in development through a joint project between ASU's School of Arts, Media and Engineering, and the Fulton

Schools of Engineering, that utilizes streaming variants of k-means to detect motion clustering patterns. This process has been demonstrated in the initial research project. (see Figure 2).

Ultimately, the motion tracking will be conducted using simple, customizable on-body electronics, possibly encased in culturally sensitive jewelry or garments that engage the participants in the process, enabling them to become invested in their own movement patterns and data.

System Hardware

The next level of ‘field’ prototyping will be conducted using the Notch (https://wearnotch.com/), a 3-dimensional, modular motion tracking system incorporating accelerometers, GPS positional sensors, and Bluetooth LE (BLE) transmission capabilities. The modules, weighing less than 8 ounces each, are attached to the body via special straps. They are also waterproof, which is a key consideration in on- body electronics. This modular BLE system provides a new level of flexibility in how the movements may be captured, and does not require the use of a specially facilitated dance research venue.

The sensors track movement and can be used to evaluate composite, complex movements through the Notch SDK developer's toolkit. For example, a minimal movement dataset could consist of positional and 3D movement data from the chest, hip, right upper arm, left upper arm, right thigh and left thigh. Additional modules can be incorporated. (see Figure 3)

Fig 3. Sample image from data collection using the Notch devices as used in ASU Arts, Media and Engineering AME 494 Somatic Prototyping class, April 2018.

Outcomes and Dissemination

The primary outcome will be an online open source repository as outlined above. It will be freely accessible resource for scholars, artists, dancers and performers to understand, and represent the cultural and contextual components of movement. Using this as a resource, the repository will be used to create further opportunities for

public engagement and scholarship around the work, namely: a touring exhibition of the work and scholarly publications.

We envisage individual movement patterns as a kind of fingerprint that is unique for each person, having ties to movement habits that develop over the course of a lifetime and are heavily influenced by cultural context and geography; a kind of ontogenesis of movement patterns. The repository will become increasingly important to create connections between individuals and groups who have been displaced through political, environmental and economic forces at play in the 21st century. South Africa is selected as a starting point for this work because of the dense and diverse cultures and movement patterns that permit rapid development and deployment of the repository. Further work will be possible in global regions such as the Eastern Mediterranean and Oceania where significant population movements are increasingly driven by political and environmental factors.

Exhibition: Digital movement portraits will be envisioned as a multi-media installation work; a sonic and visual multi-channel media installation created with a series of data movement portraits. The Center for African Contemporary Arts and Culture, The Academy of Advanced African Studies [University of Bayreuth, Germany] and Goldsmiths University of London seek to curate and support the work that evolves from this project, while seeking other relevant art-science-technology venues. The exhibited digital movement portraits will be developed through a collaborative activity encompassing visual and sound artists and developed using interactive media tools such as Cycling’74 Max/MSP.

Acknowledgments

We thank Assegid Kidane, students of AME 494 Somatic Prototyping, and the faculty, staff and students of the School of Arts, Media and Engineering and School of Dance.

References

  1. Iyengar, V., Coleman, G. et.al., "Motion, Captured: An Open Repository for Comparative Movement Studies," MOCO'16 July 5-6, 2016.

  2. LATRA https://www.latra.gr

  3. The Ohio State University. ACCAD - Motion Capture Lab - Data and Downloads. "Open Motion Data Project”. http:

//accad.osu.edu/research/mocap/mocap_data.htm.

  1. Motion Bank. http://motionbank.org/de.

  2. CMU Graphics Lab Motion Capture Database.

http://mocap.cs.cmu.edu/.

  1. Panopticon: http://www.cs.cmu.edu/~hanbyulj/panoptic- studio/ Panoptic Studio: A Massively Multiview System for Social Motion Capture Hanbyul Joo, Hao Liu, Lei Tan, Lin Gui, Bart Nabbe, Iain Matthews, Takeo Kanade, Shohei Nobuhara, Yaser Sheikh, ICCV 2015.

  2. DCU Tennis Sensors (using fast portable IMU’s)

    1. https://www.irishtimes.com/news/science/technology- pros-serve-tennis-ace-1.945491

    2. https://pdfs.semanticscholar.org/67d7/7d699a384a4a 6912685ed579928daf4ad416.pdf

    3. http://ieeexplore.ieee.org/document/6127084/

    4. Project Tango -https://get.google.com/tango/ - for Mobile Based Kinetc

    5. Fit3d and others offer portable 3d scanning

- https://www.fit3d.com

  1. Daniel Vlasic, Rolf Adelsberger, Giovanni Vannucci, John Barnwell, Markus Gross, Wojciech Matusik, and Jovan Popović. 2007. Practical motion capture in everyday surroundings. ACM Trans. Graph. 26, 3, Article 35 (July 2007). DOI:https://doi.org/10.1145/1276377.1276421

  2. Ronit Slyper and Jessica K. Hodgins. 2008. Action capture with accelerometers. In Proceedings of the 2008 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA '08). Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, 193-199.

  3. Wang, Q., Turaga, P., Coleman, G., and Ingalls, T. Somatech: an exploratory interface for altering movement habits. In CHI’14 Extended Abstracts on Human Factors in Computing Systems, ACM (2014), 1765–1770.

  4. Feldenkrais, M. 2010. Embodied Wisdom: The Collected Papers of Moshe Feldenkrais. North Atlantic Books, USA

  5. Buchanan, P. A., and Ulrich, B. D. The feldenkrais method

R: A dynamic approach to changing motor behavior. Research Quarterly for Exercise and Sport 72, 4 (2001), 315–323.

  1. Evan Roth, Ben Engebreth. http://whiteglovetracking.com/

  2. McCormack, Derek P. (2008) Geographies for Moving Bodies: Thinking, Dancing, Spaces. Geography Compass, Blackwell Publishing Ltd. 2/6 1822–1836

  3. Repovizz: A multimodal on-line database and browsing tool for music performance research., 2010.

  4. Illich, I. Vernacular values. Philosophica 26 (1980).

Bibliography

  1. McCormack, Derek P. (2008) Geographies for Moving Bodies: Thinking, Dancing, Spaces. Geography Compass, Blackwell Publishing Ltd. 2/6 1822–1836

  2. Johannesburg: the elusive metropolis, Editor(s): Sarah Nuttall, Achille Mbembe, [Transformation: Critical Perspectives on Southern Africa, Number 71]

  3. The Ohio State University, Advanced Center for Computation and Design [ACCAD] - Motion Capture Lab - Data and Downloads. "Open Motion Data Project”

  4. Motion Bank. http://motionlab.deakin.edu.au/dr-scott- delahunta/, Centre for Dance Research, Coventry University (UK)

  5. Carnegie Mellon University, Motion Capture Database, http://mocap.cs.cmu.edu/

  6. Smigel, Libby, and Martha Goldstein. Documenting Dance, a Practical Guide. Washington, D.C.: Dance Heritage Coalition, 2006

  7. Guest, A. H. (1977). Labanotation: Or, Kinetography Laban : the system of analyzing and recording movement (Third edition, revised.). New York: Theatre Arts Books.

  8. C. Desmond, Jane. Embodying Difference: Issues in Dance and Cultural Studies. (Winter, 1993-1994) ed. Vol. No. 26. University of Minnesota Press. Pp. 33-63.

  9. Low, Setha M. (2003) Embodied Space(s): Anthropological

Theories of Body, Space, and Culture. space & culture vol. 6 no. 1, February 2003 9-18, Sage Publications

  1. DeFrantz, Thomas. Exhausting Dance: Performance and the Politics of Movement. TDR: The Drama Review, Volume 51, Number 3 (T 195), Fall 2007, pp. 189-191 Published by The MIT Press

  2. Anderson, R.E. Social impacts of computing: Codes of professional ethics. Social Science Computing Review 10, 2 (1992), 453-469.

  3. Zellweger, P.T., Bouvin, N.O., Jehøj, H., and Mackinlay, J.D. Fluid Annotations in an Open World. Proc. Hypertext 2001, ACM Press (2001), 9-18.

  4. McCormack, Derek P., Refrains for Moving Bodies. Duke University Press Books, 2013

  5. Nash, Catherine (2000) Performativity in practice: some recent work in cultural geography. Progress in Human Geography, Volume: 24 issue: 4, page(s): 653-664

  6. Thrift, Nigel. (2008). Non-Representational Theory. Routledge.

  7. Csordas, T. (1994). Embodiment and experience. Cambridge, UK: Cambridge University Press.

  8. Africa Arts Consortium http://africaartsconsortium.org/

  9. Phantasmal Media : An Approach to Imagination, Computation, and Expression, AUTHOR, D. Fox Harrell, PUBLISHER MIT Press, 2013-11-08

  10. Kitchin, R. Big Data, new epistemologies and paradigm shifts,

  11. Dourish, P. 2001. Where the Action is: The Foundations of Embodied Interaction. MIT Press, Cambridge, MA, USA.

  12. Feldenkrais, M. 2010. Embodied Wisdom: The Collected Papers of Moshe Feldenkrais. North Atlantic Books, USA

  13. Biped (1999), Retrieved from http://openendedgroup.com/artworks/biped.html

  14. Sha, X.W., Freed, A., Navab, N. 2013. Sound design as human matter interaction. In CHI '13 Extended Abstracts on Human Factors in Computing Systems (CHI EA '13). ACM, New York, NY, USA, 2009-2018.

  15. Corness, G., Schiphorst, T. (2013). Performing with a system's intention: embodied cues in performer-system interaction. In Proc. Creativity & Cognition (C&C '13) ACM, New York, NY, USA, 156-164.

  16. Rowe, R. (1996). Incrementally Improving Interactive Music Systems, Contemporary Music Review, 13:2, p.47--62.

  17. Schiphorst, T., Andersen, K. (2004) Between Bodies: using Experience Modeling to Create Gestural Protocols for Physiological Data Transfer. In: altCHI, CHI 2004, pp. 1-8.

  18. Shamma, D., Scheible, J, Sheppard, R. (2009). Graffiti dance: interaction of light, information, and environment. In Proc. Creativity and cognition 2009. ACM, NY. 479-480.

  19. Sheppard, R., Nahrstedt, K. 2009. Merging research modalities: TED (tele-immersive dance) collaboration offers a model for performance-based research and creative development. In Proc. Computational Creativity Support Workshop at ACM CHI 2009.

  20. Schiphorst, T, Sheppard, R., Loke, L., Lin, C-C. (2013). Beautiful dance moves: mapping movement, technology & computation. In Proc. Creativity & Cognition (C&C '13), ACM, New York, NY, USA, 429-433.

  21. Ghostcatching (1999), Retrieved from http://openendedgroup.com/artworks/gc.html

  22. Codd, E.F. (1968). Cellular Automata. Academic Press, Inc., Orlando, FL, USA.

  23. Reynolds, C. (1987) Flocks, herds and schools: A distributed behavioral model., In Proc. SIGGRAPH '87, 25–34

  24. Coleman, G. 2013. Listening as the Land Talks Back: Ecology, Embodiment and Information in the Science Fictions of echo::system. Leonardo Journal. 46(3), 204-210

  25. Birringer, J. 2008. After Choreography, Performance Research 13:1, 118-22.

  26. Rowe, R. 1999. The Aesthetics of Interactive Music Systems. Contemporary Music Review, Vol. 18, Part 3, pp. 83-87.

  27. Downie, M. 2005. Choreographing the Extended Agent: performance graphics for dance theater. PhD Thesis. MIT.

Authors Biographies

Grisha Coleman is an Associate Professor of Movement, Computation and Digital Media, School of Arts, Media and Engineering and The School of Dance at Arizona State University. Composer and choreographer in performance and experiential media, her work explores relationships among our physiological, technological and ecological systems. Her recent art and scholarly work echo::system http://www.echo-system.net is a springboard for re-imagining the environment, environmental change, and environmental justice.

Dr. Brenda McCaffrey is an electrical engineer and inventor with 30 years of experience in materials, sensors and electronic hardware currently engaged in research and design involving nano-energy capture through human movement. Dr. McCaffrey has extensive experience in wearable technologies, specializing in embedding electronics in garments and jewelry, and received a patent in 2017 for this work. She has been involved in numerous startup companies including efforts to commercialize non-invasive stimulation technologies for pernicious bone fractures. Dr. McCaffrey has a Ph.D. in Electrical Engineering from Arizona State University. She is also a jewelry designer, hula dancer and Pacific Islander with a passion for celebrating culture and movement.

Annotatable Readings
© Grisha Coleman & Brenda McCaffrey.
Powered by Manifold Scholarship. Learn more at manifoldapp.org