Human movement is difficult for computation because it is continuous, relational, variable, and specific to the body performing it. Dance makes that difficulty especially visible. The African and African diaspora traditions at the center of this research carry complex changes in weight, rhythm, articulation, coordination, spatial organization, and cultural reference that conventional movement-analysis systems often fail to describe well. Tools developed around narrower movement assumptions captured where joints traveled while missing much of what made the movement intelligible as a practice.

That failure became the starting point for this research. We began with dance, and specifically with African dance forms, because they give the model some of the most demanding human movement it could be asked to read. If the system can learn from movement at that level of complexity, the same architecture can be extended to other forms of human motion. Dance is therefore the proving ground for the claim.

The SEE Lab’s Movement Language Model sits at the center of the system. Around it are analytical primitives that represent different conditions arriving together in a moving body: visible movement, internal sensation, sound, geography, inherited and made material culture, facial expression, and the physical traces movement leaves behind. The family is still growing as the research identifies conditions that need their own representation.

These conditions are separated analytically because computation needs something it can read, though they are never assumed to exist separately in lived movement. A person does not move first as a skeleton, then as a cultural body, then as a sensing body, and then as a body in a place. All of those conditions arrive together. The research therefore works first by giving each condition a structured representation and then by studying how they converge again in the embodied event.

Movement enters the system on consented terms. The resulting representations carry provenance with them, so the model keeps the relationship between the data and the person, tradition, place, or context it came from. The right to refuse also travels through the system. Much of contemporary machine learning has been built through extraction. This system makes embodied knowledge computationally legible while ownership of that knowledge stays where it already sits.

The research develops in two directions at once. One asks what dance can contribute to computation when movement is treated as a serious form of knowledge in its own right. The other asks what computation can contribute back to dance once it is able to engage movement with enough depth to be useful to artists, researchers, teachers, and communities. That second direction includes analysis, preservation, publication, comparison, replication, and immersive forms through which movement can continue to circulate as living material.

The corpus is currently 163 recordings, reaching 75 traditions across 33 countries. Each recording is read into a form the system can compare, search and return, and published movement enters the scholarly record through the Movement Journal.

The corpus is small beside the reference datasets in this field, and deliberately so at this stage. Mahmood et al. (2019) unify fifteen optical marker-based motion-capture datasets into AMASS, over 11,000 motions and more than 40 hours. Guo et al. (2022) build HumanML3D on that foundation, 14,616 motion clips and 44,970 English text descriptions across 28.59 hours. Li et al. (2021) assemble AIST++, the closest comparison because it is dance, 1,408 sequences and 5.2 hours across ten street-dance genres recorded in a single laboratory. Lin et al. (2023) go further again with Motion-X, 81.1K sequences and 15.6 million pose annotations. SEE Lab has a fraction of that volume, and closing the gap is the work ahead.

In place of volume, the corpus offers coverage and provenance. Seventy-five traditions across thirty-three countries, against ten genres in one laboratory. Every recording enters on consented terms with the conditions of its capture recorded beside it. None of the datasets above records the tradition a movement belongs to, the place it comes from, the lineage it was learned through, or the terms under which it was given.

SEE Lab, the Spatial Experiential Embodiment Laboratory, is based in the Department of Theatre and Dance, College of Fine Arts, at the University of Texas at Austin. It is directed by Sinclair Emoghene, who leads the research. Dr. Lillian Chin, Assistant Professor of Electrical and Computer Engineering at the University of Texas at Austin and a member of the Texas Robotics Consortium, leads the multimodal sensing work, including the fluidic innervation sensors used to capture weight and pressure beyond what a camera can record. The model is trained at the Texas Advanced Computing Center.