Toru NAKATA, Digital Human Research Center, AIST, JAPAN. email@example.com
Updated: March 10, 2005.
At the forefront of robotics applications, some trial experiments in psychological therapies have been conducted using animal-like robots . These trials are called Robot-Assisted Therapies (RAT) or Robot-Assisted Activities (RAA). The concept of RAT/RAA is derived from Animal-Assisted Therapies (AAT) or Animal-Assisted Activities (AAA).
However, the therapeutic abilities of animals have not been researched thoroughly, so AAT/AAA has been practiced in the absence a thorough methodology necessary to produce the desirable psychological effects on patients. Similarly, current theories of RAA/RAT are not yet sufficiently rigorous. As a result, there is an absence of a cohesive strategy for designing such robot's hardware and software.
Figure 1: Strategy of Dance Therapy
Interactions between patients and therapeutic animals are mostly conducted using non-verbal communication, especially bodily communications. Dance therapy is the most explicit method of such bodily communications that can effect patients' psychological states. Therefore it is argued that the theories and the methodologies of dance therapy provide the most suitable strategy for the design of therapeutic robots' hardware and software.
Figure 1 shows the dance therapy paradigm as a loop of understanding and effectuation between the patient and therapist. The loop, body movements of both of the patient and therapist play two roles; to represent a psychological state and to effect the patients on psychological state. A well-executed body movement that unambiguously expresses a psychological state can produce a sympathetic reaction in the patient. Thus, sympathy thorough expressive bodily interaction is the key strategy of dance therapy, and furthermore, of RAA/ RAT.
This strategy necessitates the ability of body expression in therapeutic robots.
From the several existing schools of dance therapy the more explicit and quantitative theories have been selected. This is due to their ease of implementation in therapeutic robots. This paper explains such dance therapy theories and implements them in software that can generate expressive body movement of lifelike artifacts in a computer graphic environment.
This technology of automatic generation of expressive body movement is anticipated to become important and can be utilized in entertaiment robots such as pet robots.
Dance therapy researchers have developed theories on the relationship between psychological states and body movement. Body movement has two aspects, namely temporal aspect and geometrical aspect. This section introduces Kestenberg and Hunt theories in terms of the temporal aspect. Also Cohen's theory is explained in terms of geometrical theory.
Comparing body movement patterns of animals, Cohen[2, 3] found the relationship between evolutionary state and mechanical constraints on body movement (shown in Table 1).
Table 1: Cohen's Evolutional Patterning of Body Movement Pattern Name Classification in animals Order on body movement Disorder No nervous connection No synchronism Breath Multicellulars Synchronism Core-distal Diploblastica (Radial symmetric bodied animals. Starfishes, etc.) Radial symmetric movement Head-tail Triploblastica (fishes, insects, etc.) Tension propagation along nervous cord Upper-lower Amphibians Tension separation into upper/lower half bodies Homo-lateral Reptiles Tension separation into left/right half bodies Contra-lateral Mammals Left-right crossed tension separation
Frogs, for instance, control their body by the division of muscular tension control into upper and lower half bodies. Horses are relatively more complex. They can control their body in the same way as frogs do when they run very fast. This running style is called gallop.
Moreover horses are capable of more complicated medes of control. In `homo-lateral' (or `ipsilateral' in biological terms) control mode, tension control is devided between right and left half bodies. As a result, the left forelimb and left hind leg moves simultaneously, as does, the right forelimb and right hind leg. This motion in homo-lateral control mode is termed walk.
When a horse walks slightly faster (i.e. trot), its opposing forelimbs and hind legs move together. This right-left crossed tension control is named contra-lateral mode. As shown in Table 1, there are 7 stages of body constraint patterns depending on the evolutionary complexity.
Table 2: Hunt's stereotypes of muscular tension rhythm Rhythm Name Periodicity Acceleration Undulate Strong Low Sustained Weak Low Restrained Weak High Burst Strong High
Table 3: Kestenberg's interpretations of body movement rhythms Rhythm Name Darwinian classification Temporal pattern of muscular tension Psychological state When appears on human infant Sucking Indulging Small sine wave Absorbing affection Birth -- Snapping/Biting Fighting Small triangle/trapezoidal wave Attention fixed on something 0.5 year old -- Twisting Indulging Small sine + drift Intention of locomotion 9 months old -- Strain & Release Fighting Large long bang-bang wave Concentration and relinquish 1 year old -- Runnging & Drifting Indulging Drift Controlling continuity of movement 2 years old -- Starting & Stopping Fighting Bouts and intervals Decision to start then stop 2.5 years old -- Swaying Indulging Small long sine wave Enjoyment/pleasure 3 years old -- Surging & Birthing Fighting Large long sine wave Enduring pain, fleeing from pain 3.5 years old -- Jumping Indulging Large short sine wave Excitation 4 years old -- Spurting & Ramming Fighting Large short triangular wave Violent intention, hostility. 5 years old --
Resulting observations of movement in animals and humans showed that the number of tension patterns made by the nervous system said to be Central Pattern Generator (CPG) is limited. Furthermore stereotypes of CPG patterns have been shown to exist.
The hypothesis made by Valerie Hunt [4, 5] describes 4 stereotypical CPG rhythm patterns as shown in Figure 2. The patterns can be interpreted mathematically as demonstrated in Table 2. The defining keys of Hunt's categorization are periodicity and acceleration (or differentiability) of patterns.
Figure 2: Temporal patterns of Hunt tension stereotypes
Another hypothesis which describes a more explicit categorisation was made by the observing body movement of babies. Judith Kestenberg found that the oscillation patterns of CPG signals can be classified into 10 stereotypes shown in Figure 3. Each pattern has a corresponding relationship between the psychological state and developmental state as shown in Table 3. While growing up, a baby gradually acquires more muscular tensions pattern that reflect its increasingly complicated psychological state.
Figure 3: Temporal patterns of Kestenberg tension stereotypes
In order to interpret these theories from an engineering perspective and also to realize a choreography algorithm based on the theories, an example implementation is reported in this section.
This implementation uses 3-dimensional computer graphics as a suitable method of displaying the movement. The computer graphics software is developed using OpenGL libraries.
The computer model shown in Figure 4 takes a lifelike form. The body's 12 joints have 2 Degrees of Freedom (DoFs). Therefore the body has in total 24 DoFs.
Another more simplified body is also created as shown in Figure 5. This body has 14 DoFs.
The movement generation algorithm is demonstrated using both of the two body models.
Figure 4: Kinematic structure of the computer-graphic model with bendable limbs with 12 joints and 24 DoF
Figure 5: Kinematic structure of a simplified computer-graphic model with a fixed pelvis and 7 free joints. The total DoF is 14.
Figure 6 shows that when given the evolutionary or developmental state and psychological states of the agent, proper muscular tension rhythms and body movement constraints can be determined according to either the hypotheses of Cohen and Hunt or Kestenberg. The algorithm can then distribute the tension signal to each muscle.
Figure 6: Mapping from emotional and intensional state to body movements
This methodology of movement generation is determined by evolutionaray, developmental and psychological relationships as described in the previous section. Therefore, the generated movement will reflect the character's evolutionary, developmental and psychological states.
The implementation is simply a matter of distributing tension signals. Figure 7 shows the control block diagrams describing the implementation.
Figure 7: Control diagrams to realize Cohen's tension signal distributions
The Hunt and Kestenberg tension rhythm stereotypes shown in Figure 2 and Figure 3 are interpreted into mathematical terminologies for implementation. Hunt's stereotype can be interpreted using waves with periodic terms such as sinusoidal waves and stochastic drift terms. Table 3 contains examples of mathematical interpretations of Kestenberg's stereotypes.
Figure 8 depicts the automatically generated body movement in Cohen's 6 body constraint modes. The experimenter selects evolutionary/developmental state and emotional state of the character, and inputs them to the system. The system then generates the body movements as shown in Figure 6.
Figure 8: Cohen evolutional phases of body movement pattern Breath Core-distal Head-tail Upper-lower Homo-lateral Contra-lateral
This section reports results of expressions of typical body movements that are compatible with Hunt's tension rhythm stereotypes.
The experimental hypothesis is the following:the expression of movement in animated characters can convey emotions by selecting the desired tension rhythms and patterns as desired by somatic modelling theories.
The effectiveness of automatic movement generation method can be considered as effectiveness of expression. This effectiveness can be guaged by temporal aspect and by geometrical aspect. The temporal aspect relates to plausibility and expressiveness of rhythms and timing of the movement. Likewise, the geometrical aspect relates to plausibility and expressiveness of postures and paths of the movement.
The experiment was conducted as follows:
1) A subject is shown the computer-graphic scene as in Figure 9. There are 4 objects on the left of screen. These are to be fet to the character. At this stage, the character is immobile.
Figure 9: Screen shot of the initial image of the experiment
2) Each subject is told to select a food by clicking on it. After clicking, a feeding animation is activated as shown in Figure 10. The wall disappears, and the selected food fades away into the character's mouth to suggest that it has eaten the food.
Figure 10: Animation of feeding scene and one of the 4 reactions
3) The character ten animates in reaction. A unique reactive animation is triggered by each kind of food.
To reduce the number of comparisons in the experiment, only 4 stereotypes of Hunt's classification are used.
The reactive movements are driven by core-distal type body constraints and Hunt's tension stereotypes. Eating the food in the upper-most position triggers a reaction with an undulate rhythm. Eating the 2nd upper-most food causes a burst rhythm, and in the 3rd food a restrained rhythm. Lastly, the food in the lower-most position causes sustained rhythm.
The order of selection is unforced, and repetition of selection is allowed.
4) Each subject is asked to observe the reactive movement for 5 seconds, and then record their impression of the movement in the questionnaire provided (Figure 11). The questionnaire has 2 scales of impressions used to quantify answers. The first scale has 4 degrees to choose from regarding the amount of excitation in the movement. The other scale also has 4 degrees to choose from to describe whether the movement is done with pleasure or not.
Figure 11: Experimental Questionnaire Sheet
5) Mann-Whitney's U-test is then applied to the distributions of the answers. The level of significance is set to 5%.
The number of subjects was 11.
Figure 12 and Figure 13 show the distribution of all answers. Significant tendencies are marked with *.
As shown in Figure 12, Burst rhythms and Restrained rhythms produced impressions of excitation. This means that movements which have a large acceleration produce impressions of hastiness. In contrast, Sustained rhythms and Undulate rhythms produced calm impressions in the test subjects.
Figure 13 shows that Burst rhythm and Undulate rhythms produced strong impressions that the character moves with pleasure. Thus smooth and periodic body movements are interpreted as signs of pleasure.
Figure 14 summarizes the results. Each of Hunt's 4 stereotypes of tension rhythms produces unique pairs of calmness and pleasure impressions. This implies the possibility of emotional expression by showing body movement with the appropriate tension rhythm.
Figure 12: Answer distributions of `calm' vs. `excited' impressions
Figure 13: Answer distributions on `feeling pleasure' vs `displeasure' impression
Figure 14: Medians of impression answer distributions of 4 rhythms
The methodology on algorithmic generation of proper and plausible body movements is described using an implementation example and an experiment on the performance of rhythmic body expression.
This methodology is based on Cohen's and Kestenberg's theories which describe the relationship between evolutionary, developmental and psychological states and body movement. Therefore this method can be regarded as a theoretical way of generating typical body motion that produces lifelike impressions and emotional impressions.
This methodology is implemented andthe key concepts of the theories are interpreted into engineering terminologies. A scheme of choreography algorithm is also described using block diagrams.
In the experiment, movement expressions successful in conveying emotional impressions by selecting proper tension rhythm from Hunt's stereotypes.
In future work, evaluation of accuracy of expression, harmonization with other modalities such as voice, facial expression and harmonization with context, should be studied.
 K. Wada, T. Shibata, T. Saito & K. Tanie, ``Robot assisted activity for elderly people and nurses at a day service center," Proc. of ICRA 2002, 2002.
 Bonnie Bainbridge Cohen, Sensing, Feeling, and Action -- The Experimental Anatomy of Body-Mind Centering, Contact Editions, 1993.
 Peggy Hackney, Making connections -- Total body integraton through Bartenieff Fundamentals, Gordon & Breach Publishers, 1998.
 Sally Sevey Fitt, Dance Linesiology, Schirmer, 1996.
 Valerie Hunt, ``The Biological Organization of Man to Move'', Impulse, 1968.
 Judith Kestenberg et al., The meaning of movement, Gordon & Breach Publishers, 1999.
This program requires glut32.dll of OpenGL Utility Tool Kit (GLUT).
Download it. Then Install glut32.dll onto C:\Windows\System.
(If you have MSVC, install glut32.lib onto MSVC\VC98\LIB, and glut.h onto MSVC\VC98\Include\GL.)
Click on the map.
Copyright(c) Toru NAKATA.