An internal model approach for motor behavior

Special issue: 15 years of Brazilian Journal of Motor Behavior BJMB Brazilian Journal of Motor Behavior An internal model approach for motor behavior CLÁUDIO M. F. LEITE1 | CARLOS E. CAMPOS2 | CRISLAINE R. COUTO3 | HERBERT UGRINOWITSCH4 1 Departamento of Science of Physical Education and Health, Federal University of São João del Rei, São João del Rei, MG, Brazil. Universidade de Itaúna, Itaúna, MG, Brazil. 3 Centro Universitário Metodista Izabela Hendrix, Belo Horizonte, MG, Brazil. 4 Department of Sports, Universidade Federal de Minas Gerais (UFMG), Belo Horizonte, MG, Brazil. 2 Correspondence to: Herbert Ugrinowitsch. Department of Sports, Universidade Federal de Minas Gerais, Av. Antônio Carlos, 6627, Pampulha, Belo Horizonte, Minas Gerais – Brazil, Zip code: 31270-901. +55 31 98463-5890 email: [email protected] https://doi.org/10.20338/bjmb.v15i5.273 HIGHLIGHTS • Sensory inputs are slow, noisy, and fragmentary and require neural mechanisms to organize them and allow for action. • Internal Models are predictive mechanisms within the CNS that simulates the real world and organize sensory inputs to produce motor commands. • Internal Models are central for the control, learning and adaptation of motor skills. • There are two functionaly distinct but interconnected models: the Inverse and the Forward Models. • Internal Models approach present substantial internal coherence and has a large and growing body of empirical evidences. ABSTRACT Interacting with the environment requires a remarkable ability to control, learn, and adapt motor skills to everchanging conditions. The intriguing complexity involved in the process of controlling, learning, and adapting motor skills has led to the development of many theoretical approaches to explain and investigate motor behavior. This paper will present a theoretical approach built upon the top-down mode of motor control that shows substantial internal coherence and has a large and growing body of empirical evidence: The Internal Models. The Internal Models are representations of the external world within the CNS, which learn to predict this external world, simulate behaviors based on sensory inputs, and transform these predictions into motor actions. We present the Internal Models’ background based on two main structures, Inverse and Forward models, explain how they work, and present some applicability. KEYWORDS: Forward model | Inverse model | Motor control | Motor learning | Motor adaptation ABBREVIATIONS BG Basal Ganglia CNS Central nervous system FM Forward Models GC Granule cells IvM Inverse Model MC Motor Cortex M1 Primary motor cortex g Gravity m Mass t Time α Acceleration Ø Angular distance ω Angular velocity τ∫∫(b-c) Inverse kinetics α∫∫(b-c) Processed inverse kinematics PUBLICATION DATA Received 10 11 2021 Accepted 30 11 2021 Published 01 12 2021 INTRODUCTION Human beings interact with the environment, which requires a remarkable ability to control, learn, and adapt motor skills to changing environmental conditions. The intriguing Leite et al. 2021 VOL.15 N.5 https://doi.org/10.20338/bjmb.v15i5.273 356 of 371 BJMB Brazilian Journal of Motor Behavior Special issue: 15 years of Brazilian Journal of Motor Behavior complexity involved in this ability has instigated many studies under different backgrounds. For instance, some backgrounds propose a bottom-up mode of control, while others propose motor skills are controlled top-down. This paper will present a theoretical approach built upon the top-down mode of motor control that shows substantial internal coherence and has a large body of empirical evidence to explain the control, learning, and adaptation of motor actions: The Internal Models. The concept of Internal Models was firstly introduced by Kenneth Craik (1943) 1 in his work The Nature of Explanation, and researchers further developed this background using Biological2 and Math methods.3,4 Internal Models are neural representations of the external world5, which, therefore, learn to predict the external world (e.g., predictive models), simulate behaviors based on sensory inputs, and transform them into motor actions.6 The main rationale for the Internal Models proposal in biological systems is related to the capacity to simulate the dynamics of specific aspects from the environment. For example, a person who, at the same time, bounces and looks at the ball and sees when the ball touches the hand that produces a sound, a pressure encoding senses information as different inputs. All of the information is simultaneously generated, but each input travels at a particular speed activating specific cortical areas on the central nervous system (CNS). Besides, since these signals are noisy and reach the CNS belated, the sensory inputs related to the same phenomena are fragmented and present different natures. Such characteristics pose a wide variety of problems for the person to move and interact with the environment. The sensorimotor system needs to integrate these inputs to allow proper moving. Furthermore, it might somehow use the information provided by these stimuli in a predictive fashion to estimate a future state of the environment and the body. All these conditions lead to the proposal of Internal Models for motor control, which posits that the biological systems do not use direct information of the environment and the body to act but do so using Internal Models of reality. Generally, Internal Model theories propose two kinds of Internal Models working together, the Inverse Model and the Forward Model.7 The inverse model inverts the causal relationship of movement production (i.e., inverse dynamics) by using sensory information about the desired end-state and transforming it into motor commands.8,9 For example, to perform a shot in a team handball game, the sensorimotor system uses visual inputs from the context (e.g., distance) to activate an Inverse Model that estimates the velocity and acceleration profiles appropriate for shooting and transforms them into motor commands. However, such a handball shot requires fast and accurate movements, and the command the inverse model produces may not be so precise, which requires some correction. In this case, the Forward Model uses afferent and efferent information to predict the outcome based on environmental and body conditions 10 and update the motor commands. When there is a discrepancy between the intended action and the predictions of the forward model, it adjusts the commands before they leave the brain.11 Consequently, the Internal Models background can explain motor actions' control, learning, and adaptation, including fast actions. In this paper, we present Internal Models’ background based on Inverse and Forward models, explain how they produce motor actions, and present some applicability. Leite et al. 2021 VOL.15 N.5 https://doi.org/10.20338/bjmb.v15i5.273 357 of 371 BJMB Brazilian Journal of Motor Behavior Special issue: 15 years of Brazilian Journal of Motor Behavior INVERSE MODEL Consider a team handball player shooting to the goal. There is a variety of possible spots and shot types to choose. After choosing the desired spot, how will this player perform the shot? How is the desired action actually transformed into real action? To answer these questions, behavioral12 and neurophysiological findings13,14 suggest the existence of a neural mechanism named Inverse Model (IvM), which is responsible for carrying out the processes required to transform the desired action into motor commands (Figure 1-A). Moreover, these processes have been extensively investigated in computational, behavioral, cognitive, and neurophysiological studies.15-17 Three main approaches point out the existence of Inverse Models for motor control, the Direct Inverse Modeling,18 the Feedback Error Learning,19,20 and the Supervised Learning.15 In which concerns motor control, these three approaches propose that the IvM triggers a feedforward motor command to produce the motor action (Figure 1). However, they differ in the way they explain how an actual movement is controlled and how the IvM is formed and/or updated (for a general overview, see Jordan, 199621). Generally, Inverse Models work in parallel with Forward Models (FM – discussed later in this paper), as Figure 1 illustrates. The IvM produces a motor command based on the desired action, information about the body, and the environment (initial conditions). Since the IvM does not directly receive feedback input about the ongoing action and its predictive capacity may not be fine-tuned to the context, the motor command might not be appropriate. Therefore, the FM updates the motor command and the IvM itself (Figure 1-C). The updating signal from the FM works as a training to the IvM.22 In short, the IvM is responsible for triggering the motor command, and the FM supervises the activity and trains the IvM. The following section will present the roles and functions of the IvM. Figure 1. Inverse Models work in parallel with Forward Models. Functioning and roles of the IvM As previously mentioned, IvM inverts the causal relations of movement production that would be done (i.e., inverse dynamics). In this process, IvM uses sensory information from the context and initial conditions (Figure 1-A) as inputs to transform the desired endstate into motor commands.8,9 To explain the IvM involvements in movement control, it is Leite et al. 2021 VOL.15 N.5 https://doi.org/10.20338/bjmb.v15i5.273 358 of 371 BJMB Brazilian Journal of Motor Behavior Special issue: 15 years of Brazilian Journal of Motor Behavior important to understand how the sensorimotor system may provide at least three motor steps on performing voluntary motor actions, also referred to as computations.15,19,23 For example, in the handball shooting previously presented, the first step is to define the appropriate kinematics of the desirable trajectory coordinates (e.g., the chosen shoot) according to the environment information.12,24 According to both the type of shot and spot chosen, the second motor step is the transformation to the trajectory in actual limb coordinates.23,25 The third problem is determining to generate the motor command according to the specified dynamics (kinematics and kinetics). This third step is not yet explained clearly in how the sensorimotor system maintains the final movement stability and precision.24,26 On one hand, it is hard to explain how the nervous system deals with variability caused by the huge number of degrees of freedom (redundancy) available in the motor system.27 On the other hand, it has been demonstrated that the training of the IvM can reduce the variability but does not eliminate it.26,28 Notice that all the described dynamical transformations are in an inverse logic: The above illustrative equation shows that the output of motor command u is the result of the estimated motor command and the desired action y*. That is why the neural mechanism responsible for carrying out this process is named an Inverse Model. These three transformations have been studied and expressed in physical terms, as illustrated in Figure 2. In this illustration, an individual throws a ball to one of two possible targets (a condition that reminds the shooting example used at the beginning of this session). The desired trajectory is specified as a function of spatial coordinates such as the distance of the targets I and II (Figure 2-A), which will require two different angles corresponding to the actual trajectories of the limbs a-b and b-c. Figure 2-B presents the computations the IvM performs and also demonstrates specific variables processed in each computation such as time (t), angular distance (Ø), angular velocity (ω), and acceleration (α) as the processed inverse kinematics (α∫∫(b-c)), and the mass of the ball and the arm (m), and gravity (g) as the inverse kinetics (τ∫∫(b-c)). Once these dynamics are computed, the model specifies the best arm configuration (trajectories of the limbs) and environment information to achieve the target, which will result in the motor command. Results indicate the influence of the context (e.g., target distances) on the velocity and acceleration profiles in throwing actions29 that correspond to the computations proposed to the IvM.12 Figure 2. Three transformations expressed in physical terms. Leite et al. 2021 VOL.15 N.5 https://doi.org/10.20338/bjmb.v15i5.273 359 of 371 BJMB Brazilian Journal of Motor Behavior Special issue: 15 years of Brazilian Journal of Motor Behavior The need for motor efficiency to match the desired action can impose different demands on the IvM. When the environment condition is unknown and the task demands time precision, the IvM works via open-loop control.30 This kind of control is important in ballistic motor actions, performed in less than 200ms, requiring accuracy (i.e., an intercepting moving target) because these actions cannot rely on the online feedback control. Once performed in unfamiliar situations and requiring precision from the control system, these tasks require a well-calibrated IvM31,32 since there is not enough time for corrections during the movement.33,34 In an environment with constant conditions and without time precision demand, the IvM could be controlled using sensory feedback to start or correct an ongoing action. In this case, an IvM can use the feedback control before the movement onset when too welltrained IvM (e.g., an expert) matched a signal in prediction and in desiring future output.21,35 Another possibility is when the forward model is accurate enough, and the internal loop is equivalent to its pair of IvM.15 In this case, the IvM acts as a controller in an open-loop feedforward because there is no feedback from the actual command.15,21 Beyond the behavioral and physical evidence for IvM, there have also been reported neurophysiological results proving its existence, functioning in motor control and learning. Neural circuitries incorporate and combine visuospatial and proprioceptive information about physical aspects of the environment and the body (e.g., gravity). These processes result in Inverse Models.12,36,37 Neural inputs specify the desired action by encoding acquired information and new contextual input. Also, the desired output can provide information sources and be used to train the IvM as well as the Models involved in the same network.38,39 The encoding and processing of this information depend on specific cortical areas and will contribute to the selection of the IvM.34,40,41 The basal ganglia (BG) and the motor cortex (MC) are pointed as the main areas involved in planning and executing the desired action.38,42 The BG is responsible for the cognitive aspects such as planning and modulating the MC, whereas the MC, particularly the primary motor cortex (area M1), is responsible for triggering feedforward motor commands.41,42 Inverse models are only one part of a complex neural circuitry for controlling the movements. As presented in this text so far, aspects such as selecting an appropriate IvM, online feedback control, and the updating of IvM are phenomena beyond the functions of the inverse models themselves and depend upon other control mechanisms. The next session will present the forward model, which works pairwise with the IvM, and contributes distinctively to the control and learning of motor actions. FORWARD MODEL Let us still look at the team handball player shooting to the goal. The player jumps at the edge of the area and notices that the left low corner will be the right place to shoot. He does as he planned and shoots a fast ball exactly to that spot, impossible for the goalkeeper to intercept it. How did he do that? Consider the size of the ball and the specific area available to shoot. Still, consider the high speed of the movement and the small amount of time available during the flight to execute the action. How did the player accomplish this task so precisely and fast? Moreover, the goalkeeper might change Leite et al. 2021 VOL.15 N.5 https://doi.org/10.20338/bjmb.v15i5.273 360 of 371 BJMB Brazilian Journal of Motor Behavior Special issue: 15 years of Brazilian Journal of Motor Behavior position at any time. How did the player flexibly adapt to that specific demand? One possibility is that he had used a well-calibrated IvM to trigger a feedforward command that would activate the right effectors exactly as demanded. However, the IvM does not receive afferent input. So then, even if it were well-calibrated, it is very likely that the motor command would not exactly correspond to the task’s demands but would only approximate them. Moreover, even though sensory feedback was available to allow for online adjustments in the motor command, it is slow and noisy,43 which would not elicit fast, precise and flexible performances. Despite these difficulties, one can easily observe that actions as accurate as this are commonly performed in sports44 and daily life, for example, when we pick up a falling object. The Internal Models can explain. Behavioral45 and neurophysiological findings46 suggest the existence of a predictive neural mechanism named Forward Model (FM), which uses afferent and efferent input to predict movement outcomes (Figure 1-B) and enables the action to precisely occur regardless of the feedforward and sensory feedback limitations. The proposition of an FM comes from the middle 19th century by Hermann von Helmholtz.47 They observed that when the human eye is passively moved, it causes the impression that the environment is moving instead of the eye. This differs completely from the impression caused by an active movement of the eye, which indicates the eye is moving. To explain this recognition of a self-movement when a movement is actively produced, von Helmholtz proposed that when the sensorimotor system triggers a motor command to move the eyes, some region of the central nervous system (CNS) receives and processes a copy of this command (efferent copy). This copy allows for the prediction of the consequences of the ocular movements. According to the predictions, the resulting sensations are recognized as self-movements, and the remaining sensations are attributed to displacements of the surroundings. Such thinking about the use of efferent copies was further developed by Erich von Holst and Horst Mittelstaedt48 and by Roger Sperry49. At this point, the idea of predicting the effects of the motor command using an efferent copy was already clear, and the matter becomes not only to distinguish whether the movement is self or not but to understand the process underlying movement predictions and production. In other words, movements present magnitudes such as amount, duration, and intensity, which requires the sensorimotor system to extract (or simulate) all these magnitudes and provide all the predictions according to them. Although ocular movements are very simple, it is easy to perceive the computational complexity involved in their production. Such complexity becomes even larger in movements produced by body segments due to the number of degrees of freedom27 available and the possibilities of interactions with the environment. Accordingly, the FM is part of a robust and specialized predictive mechanism that simulates the dynamic behaviors of our body and the environment and elicits the production of efficient, accurate, and flexible actions. Functioning and roles of the FM Both Internal Models, Inverse and Forward, are predictive mechanisms. However, their predictive functions are different and complementary. The FM analyses the efferent copy of the motor command produced by the IvM and predicts its effects as illustrated in the red circuitry in Figure 1. That is the reason for the name “Forward” (to the front, to the future): its function corresponds to a causal relationship in which a motor command Leite et al. 2021 VOL.15 N.5 https://doi.org/10.20338/bjmb.v15i5.273 361 of 371 BJMB Brazilian Journal of Motor Behavior Special issue: 15 years of Brazilian Journal of Motor Behavior (efferent copy) produces an effect (predicted action). Besides the efferent copy, the FM receives afferent input from the sensory organs (e.g., muscle spindles, eyes), which signals the movement’s initial condition and the body and environmental changes during the movement (sensory feedback). These afferent signals allow the FM to estimate motor consequences according to the body and environmental conditions.10,50 Because efferent and afferent inputs differ in nature (e.g., signals for muscle contraction vs. signals about joint coordinates), they cannot be directly compared. So then, the predictive role of the FM begins by translating the information of the efferent copy into a predicted action. When this translation occurs, the information of the efferent copy is transformed into predicted sensory consequences and becomes compatible with the actual sensory consequences (blue circuitry in Figure 1). This process is fundamental to the functioning of the FM and allows it to accomplish three particular roles: to analyze and make predictions about one’s own body (a state estimator); to analyze and make predictions about the environment (a context estimator); to play a central role in motor learning by updating the IvM (a remote teacher). These roles are detailed below. State estimator Let us return to the team handball player. He performs exactly as he planned and gives the goalkeeper no chance to intercept the ball. How did he attain such a quality of action? Even if one simplifies the analysis and considers only the movement of the throwing arm, the IvM might not be appropriately calibrated to send the required motor command.51 Besides, online corrections via sensory feedback might not be possible or even functional.43,52 These limitations make it difficult, if not impossible, to know the actual state of the body at a given moment (e.g., position in space, joint position, speed of body segments), but this information is still necessary to properly perform a motor action. The FM supplies this need by estimating the state of the body when it integrates the efferent copy and the sensory input. That is, it projects the condition of the body in the future, which compensates for delays and reduces the uncertainty (fluctuations) that arises from noise intrinsic to the sensory and motor signals47,53,54 and enables the performance of accurate and fast actions according to the demands. One of the tasks of the FM while acting as a state estimator is to correct the motor command before it reaches the effector (i.e., the muscles) – also referred to as motor plant – employing an internal feedback circuitry/loop simulated in the brain51 as illustrated in Figure 1-B and 1-C. Because the FM receives sensory inputs, it is constantly updated/calibrated and can precisely estimate a future state. Thus, when it receives an efferent copy, it may predict whether the effects of the motor command (i.e., the motor action) are the expected ones. If there is any discrepancy between the predicted effects of the efference copy and the desired action, the FM sends a corrective signal to the motor command. Because this circuitry is within the brain,46 it takes no more than 30 ms to update the motor command.55 Thus, this internal feedback loop explains the performance of fast and accurate motor actions even though the sensory inputs are slow and noisy (Figure 3-A and 3-B). While the movements of the arms are central to performing a handball shot, the body works as a whole. Thus, there is a need for many body adjustments and segmental integration. Take as an example the control of the ball grip. When the arm begins the Leite et al. 2021 VOL.15 N.5 https://doi.org/10.20338/bjmb.v15i5.273 362 of 371 BJMB Brazilian Journal of Motor Behavior Special issue: 15 years of Brazilian Journal of Motor Behavior throwing swing, some forces are impressed into the ball. Due to its inertia, the ball tends to slip off the hands. Thus, it requires grip adjustments according to the swing velocity; otherwise, the ball will escape the hand when the arm accelerates. When the ball’s inertia is overcome, the increased grip force is no longer required, then it will reduce again. These refined adjustments in grip force require a sophisticated and harmonious arm-hand integration. One possibility to explain these adjustments is tactile feedback (the ball slipping off the hand). However, grip force adjustments occur simultaneously to the corresponding changes in the arm swing, or even slightly before these changes,56,57 which rules out the use of sensory feedback and consequently of online control. Another possibility is the use of a single feedforward motor command to the arm and hand. However, experimental results show that the coupling between arm force and grip force is learned before the arm swing,56,57 indicating that the arms and the hands receive different motor commands. An elegant solution to this issue considers the state estimator role of the FM.56-63 The FM receives sensory inputs related to the kinetics of the ball (e.g., its weight). Then, when the FM receives the copy of the motor command and estimates its effects on the body (e.g., acceleration), it can also estimate the future state of the ball in the hands and inform the exact prehension time and force required57,64 (Figure 1-C). Such a coupling role also integrates the movement of other segments of the body. Some findings show this same anticipatory/predictive behavior for eye movements65,66 and for movements of the trunk and the legs,54,67-69 which allows for motor adjustments as a function of other movements such as in throwing or reaching. These findings strongly indicate that the state estimator role of the FM is the centerpiece for the harmonious integration of the segments of the body during motor actions. Neurophysiological research has also supported this coupling function of the FM and points to the cerebellum as one of the sites for “allocating” Forward Models.54,63,70 Because the state estimator predicts the afferent inputs that will result from the motor command, it can also confirm and cancel the predicted inputs. Such a function is important for two reasons. First, it aligns with the use of the efferent copy to recognize selfmotion.48,49 In this case, the FM allows for the distinction between the effects of the motor commands and the effects from other sources. Second, confirmation and cancelation of sensory inputs also allow for distinguishing between more or less important signals. For example, a well-documented phenomenon is an inefficiency to tickle ourselves.71 Blakemore et al.72-75 investigated the tickling effects of different stimuli, which corresponded to self and external stimulation. Even when self-stimulation causes tickling, its magnitude is far lower than when tickling arises from an external agent (e.g., somebody else). Because sensory inputs related to the motor commands were already predicted, they are attenuated (confirmed and canceled out). Such an attenuation permits nonpredicted (newer) signals, mostly related to environmental agents, to be emphasized. Leite et al. 2021 VOL.15 N.5 https://doi.org/10.20338/bjmb.v15i5.273 363 of 371 BJMB Brazilian Journal of Motor Behavior Special issue: 15 years of Brazilian Journal of Motor Behavior Figure 3. Internal feedback loop. Context estimator Besides estimating the states of the body during a motor action, the FM also estimates the environmental/contextual consequences of the motor commands.10,47,55 Such an estimation becomes evident when we consider that a person performs accurately and properly in a variety of contextual conditions, even in unpredictable ones. The performance of a motor action will always depend on context estimations based on the combination of two different types of sensory inputs: signals previous to the action and the sensory feedback provided during or after the action.10,76 For example, consider a situation in which a person holds and lifts a carton of milk. This person may access the dimensions and the material of the carton but not the amount of milk. Thus, it is highly probable that the motor commands will not fit the condition properly because the cues indicating the weight of the carton are missing. Suppose that the person considered that the carton was full, and when the movement began, the carton lifted too fast because it was only a quarter full. In this case, the motor command would need an update based on estimations that would require the sensory feedback available after the action had begun. Consider also that another person would use this same carton and saw that first one pouring milk out of it. In this condition, having seen somebody else manipulate the carton would have fed the FM with cues that would allow estimating the amount of force required to act more precisely and correct the motor command before it had reached the muscles by using the internal feedback loop. Some theoretical proposals suggest that these motor adjustments occur due to the existence of multiple Forward and Inverse Models within the CNS that work interconnectedly and modularly and correspond to specific body segments and tools.10,16,17,46 This modularity allows flexible motor actions because the sensorimotor system switches the modules on and off and also combines them to accommodate the environmental demands. In addition, behavioral77,78 and neural findings46,53,79,80 support modularity, although it has not been demonstrated how the CNS combines and switches the combination of the modules.10,76 Remote teacher Leite et al. 2021 VOL.15 N.5 https://doi.org/10.20338/bjmb.v15i5.273 364 of 371 BJMB Brazilian Journal of Motor Behavior Special issue: 15 years of Brazilian Journal of Motor Behavior Motor actions occur in many different and varied contexts. For example, tools, equipment, and clothes present different types and characteristics (e.g., shape and weight); even our bodies change over time. Such variability requires both IvM and FM models to change accordingly, which the sensorimotor system provides by updating preexisting models and building new ones in a learning process.16 The Internal Models are constantly updated, but the FM is central to the learning process because it is directly updated and indirectly updates the IvM in an error-based learning process45,51 represented in the circuitry in Figure 1-B. As presented earlier, the motor command from the IvM and the sensory input cannot be directly compared. Thus, the IvM cannot be directly updated. However, the FM can because it translates the information of the motor command into predicted sensory consequences or an estimate of the sensory feedback. Comparisons between the predicted sensory feedback and the actual sensory feedback are responsible for updating the FM and refining its predictive function. Furthermore, because the FM receives the copy of the motor command, it can also translate the sensory feedback into motor coordinates and indirectly update the IvM as indicated by the dotted arrow in Figure 1-D. This function of updating the IvM is referred to as “distal teacher”. Some behavioral investigations indicate that the FM is formed previously to the 56,81 IvM, and some neural circuitry supports the functions of the FM and also its calibration.5,46,63 The Cerebro-cerebellum, the two lateral regions of the cerebellar hemisphere that communicate to the cerebral cortex, is a region for “allocating” Forward Models46 because of its characteristics and functioning as follows. The mossy fibers in the cerebellum communicate directly to the motor cortex (area M1). They activate about 10 ms after the motor cortex triggers the motor command and the modulation of its activities precede the movement in about 80 ms, which resembles the internal feedback loop of the FM. The cerebellum also receives and is highly sensitive to muscle and cutaneous sensory input, which allows it to monitor the actual state of the motor system with short time delays (about 6 ms). The granule cells (GC) of the cerebellum receive efferent and afferent input, making them very likely to integrate information. Additionally, considering that the large number of GC allows a wide combination of efferent and afferent signals, and the combination of these signals is the functional base of the FM, it can explain the movement control and the acquisition of new Forward models by new synaptic formation. Moreover, anatomical and physiological data indicate that the neural circuitry in the cerebellum is modularly organized and that each module corresponds to small body parts (e.g., parts of the arm), which reinforces that the cerebellum is very likely a region for “allocating” FM. CONCLUDING REMARKS This paper presented an overview of the theoretical proposals of Internal Models for motor control. These proposals provide robust evidence-based explanations for the production, learning, and adaptation of motor skills. Mainly, it considers that our sensory inputs present serious limitations, which hinder the possibility of controlling the body and directly interacting with the environment. Thus, the Internal Models approach proposes that the CNS simulates (models) the reality so that the organism can move and interact with the environment based on predictions and estimations. The basic elements for these predictions are the Inverse and the Forward Models, which produce a series of Leite et al. 2021 VOL.15 N.5 https://doi.org/10.20338/bjmb.v15i5.273 365 of 371 BJMB Brazilian Journal of Motor Behavior Special issue: 15 years of Brazilian Journal of Motor Behavior computations (expressed in math and physical terms), and play very specific roles. Here we summarized and generalized the functioning and roles of the Inverse and Forward Models. We notice, however, that the way these mechanisms are considered differs according to particular approaches in Internal Models, such as the Direct Inverse Modeling, the Feedback Error Learning, and the Supervised Learning approach considered at the beginning of this paper. Moreover, other mechanisms of control, not in the scope of this overview, seem to participate in the circuitry of motor production, indicating that the Internal Models theory, although consistent, is still a growing field and a place for debate. Nevertheless, undoubtedly, these theories already provide important and robust contributions to the understanding of motor behavior. REFERENCES 1. Craik KJW. The nature of explanation. Cambridge: Cambridge University Press. 1943.126 p. 2. Ito, M. Neurophysiological aspects of the cerebellar motor control system. Int J Neurol. 1970; 7(2):162–176. PMID: 5499516. 3. Francis BA, Wonham WM, The internal model principle for linear multivariable regulators. J Appl Maths Optimization. 1975; (2):170-194. doi.org/10.1007/BF01447855. 4. Francis BA, Wonham WM. The internal model principle of control theory. Automat. 1976; 12(5):457-465. doi.org/10.1016/0005-1098(76)90006-6. 5. Ito M. Control of mental activities by internal models in the cerebellum. Nat Rev Neurosci. 2008; 9(4):304-313. doi.org/10.1038/nrn2332. 6. Wolpert DM, Ghahramani Z, Jordan ML. An internal model for sensorimotor integration. Science. 1995; 269(5232):1880-1882. doi: 10.1126/science.7569931. 7. Wolpert DM, Ghahramani Z. Computational principles of movement neuroscience. Nat Neurosci. 2000; 3(Suppl):1212-1217. doi: 10.1038/81497. 8. Houk JC, Buckingham JT, Barto AG. Models of the cerebellum and motor learning. Behavi and Brain Scien. 1996; 19(3):368-383. doi.org/10.1017/S0140525X00081474. 9. Wolpert DM, Miall RC, Kawato M. Internal models in the cerebellum. Trends Cogn Sci. 1998; 2(9):338-347. doi: 10.1016/s1364-6613(98)01221-2. 10. Imamizu H. Prediction of sensorimotor feedback from the efference copy of motor commands - a review. Jpn Psychol Res. 2010;52(2):107–120. doi: 10.1111/j.14685884.2010.00428.x. 11. Maill RC, Wolpert DM. Forward Models for Physiological Motor Control. Neural Netw. 1996;8(9):1265-1279. doi: 10.1016/S0893-6080(96)00035-4 12. Venkadesan M, Mahadevan L. Optimal strategies for throwing accurately. R Soc Open Sci. 2017; 4(4): 170136. doi.org/10.1098/rsos.170136. 13. Cortese A, Yamamoto A, Hashemzadeh M, Sepulveda P, Kawato M, De Martino B. Value signals guide abstraction during learning. eLife. 2021;10:e68943. doi: 10.7554/eLife.68943. Leite et al. 2021 VOL.15 N.5 https://doi.org/10.20338/bjmb.v15i5.273 366 of 371 BJMB Brazilian Journal of Motor Behavior Special issue: 15 years of Brazilian Journal of Motor Behavior 14. Imamizu H, Miyauchi S, Tamada T, Sasaki Y, Takino R, Pütz B, Yoshioka T, Kawato M. Human cerebellar activity reflecting an acquired internal model of a new tool. Nature. 2000; 403(6766):192-5. doi: 10.1038/35003194. 15. Jordan MI, Rumelhart DE. Forward Models: Supervised Learning with a Distal Teacher. Cogn Sci. 1992;16(3):307-354. doi.org/10.1016/0364-0213(92)90036-T. 16. Haruno M, Wolpert DM, Kawato M. MOSAIC model for sensorimotor learning and control. Neural Comput. 2001;13(10): 2201-2220. doi: 10.1162/089976601750541778. 17. Imamizu H, Sugimoto N, Osu R, Tsutsui K, Sugiyama K, Wada Y, Kawato M. Explicit contextual information selectively contributes to predictive switching of internal models. Exp Brain Res. 2007;181(3):395–408. doi: 10.1007/s00221-007-0940-1. 18. Miller W. Sensor-based control of robotic manipulators using a general learning algorithm. IEEE Journal on Robotics and Automation. 1987;3(2):157-165. doi: 10.1109/JRA.1987.1087081. 19. Kawato M, Furukawa K, Suzuki R. A hierarchical neural-network model for control and learning of voluntary movement. Bio Cybern. 1987;57:169-185. doi: 10.1007/BF00364149 20. Kawato M. Computational schemes and neural network models for formation and control of multijoint arm trajectory. In W. T. Miller III, R. S. Sutton, & P. J. Werbos (Eds.), Neural networks for control. 1990:197–228. The MIT Press. 21. Jordan MI. Computational aspects of motor control and motor learning. In: Handbook of Perception and Action. 1996; 2(C):71-120 22. Jordan MI, Jacobs RA. Learning to control an unstable system with forward modeling. In Proceedings of the 2nd International Conference on Neural Information Processing Systems. 1989; (NIPS'89): 324–331. 23. Miyamoto H, Kawato M, Setoyama T, Suzuki R. Feedback-error-learning neural network for trajectory control of a robotic manipulator. Neural Netw. 1988;1(3):251-265. doi.org/10.1016/0893-6080(88)90030-5. 24. Ishida K, Murata M, Hirano Y. Baseball. Sports Biomech. 2006;5(2):183-196. doi: 10.1177/1941738109338546. 25. Kobayashi Y, Ae M, Miyazaki A, Fujii N, Iiboshi A, Nakatani H. Kinetics of throwing arm joints and the trunk motion during an overarm distance throw by skilled Japanese elementary school boys. Sports Biomech. 2016;15(3):314-28. doi: 10.1152/jn.00096.2015. 26. Crevecoeur F, Thonnardb Jl, Lefèvre P. Forward models of inertial loads in weightlessness. Neurosc. 2009;161(2):589-598. doi: 10.1016/j.neuroscience.2009.03.025. 27. Bernstein N. The co-ordination and regulation of movements. Pergamon, London (1967). 28. Hirashima M, Kudo K, Watarai K, Ohtsuki T. Control of 3D limb dynamics in unconstrained overarm throws of different speeds performed by skilled baseball players. J Neurophysiol. 2007;97(1):680-91. doi: 10.1152/jn.00348.2006. 29. Campos CE, Lage GM, Andrade AGP, Couto CR, Santos SP, Profeta VLS et al. Changes on movement control of dart throwing under distance and target weight constraints. J Human Sport and Exerc. 2019;14(4):925-933. doi:10.14198/jhse.2019.144.20. Leite et al. 2021 VOL.15 N.5 https://doi.org/10.20338/bjmb.v15i5.273 367 of 371 BJMB Brazilian Journal of Motor Behavior Special issue: 15 years of Brazilian Journal of Motor Behavior 30. Jordan MI, Wolpert DM. Computational motor control. Cogn Neurosci. 1999:601-620. 31. Kasuga S, Telgen S, Ushiba J, Nozaki D, Diedrichsen J. Learning feedback and feedforward control in a mirror-reversed visual environment. J Neurophysiol. 2015;114(4):2187-93. doi: 10.1152/jn.00096.2015 32. Telgen S, Parvin D, Diedrichsen J. Mirror reversal and visual rotation are learned and consolidated via separate mechanisms: recalibrating or learning? J Neurosci. 2014; 34(41):13768-79. doi: 10.1523/JNEUROSCI.5306-13.2014. 33. Pruszynski JA. Johansson RS. Flanagan JR. A Rapid Tactile-Motor Reflex Automatically Guides Reaching toward Handheld Objects. Current Biology. 2016;26(6):788-792. doi: 10.1016/j.cub.2016.01.027. 34. Scott SH. A Functional Taxonomy of Bottom-Up Sensory Feedback Processing for Motor Actions. Trends Neurosci. 2016;39(8):512-526. doi: 10.1016/j.tins.2016.06.001. 35. Gomi H, Kawato M. Neural network control for a closed-loop System using Feedbackerror-learning. Neural Netw. 1992;6(7):933-946. doi: 10.1016/S0893-6080(09)80004-X 36. Moscatelli A, Lacquaniti F. The weight of time: gravitational force enhances discrimination of visual motion duration. J Vis. 2011; 11(4):1-5. doi.org/10.1167/11.4.5. 37. Zago M, Mcintyre J, Senot P, Lacquaniti F. Visuo-motor coordination and internal models for object interception. Exp Brain Res. 2009;192(4):571–604. doi: 10.1007/s00221-0081691-3. 38. Grillner S, Robertson B. The basal ganglia downstream control of brainstem motor centres-an evolutionarily conserved strategy. Curr Opin Neurobiol. 2015;(33):47-52. doi: 10.1016/j.conb.2015.01.019. 39. Sanger TD, Yamashita O, Kawato M. Expansion coding and computation in the cerebellum: 50 years after the Marr-Albus codon theory. J Physiol. 2020; 598(5):913-928. doi: 10.1113/JP278745. 40. Azim E, Alstermark B. Skilled forelimb movements and internal copy motor circuits. Current Opinion of Neurobiology. 2015; (33):16-24. doi: 10.1016/j.conb.2014.12.009. 41. Shadmehr R, Krakauer JW. A computational neuroanatomy for motor control. Exp Brain Res. 2008; 185(3):359-81. doi: 10.1007/s00221-008-1280-5 42. Hikosaka O, Takikawa Y, Kawagoe R. Role of the basal ganglia in the control of purposive saccadic eye movements. Physiol Rev. 2000;80(3):953-78. doi: 10.1152/physrev.2000.80.3.953. 43. Faisal AA, Selen LPJ, Wolpert DM. Noise in the nervous system. Nat. Rev. Neurosci. 2008; 9:292-303. doi.org/10.1038/nrn2258. 44. Yarrow K, Brown P, Krakauer JW. Inside the brain of an elite athlete: the neural processes that support high achievement in sports. Nat Rev Neurosc. 2009;10:585-596. doi: 10.1038/nrn2672 45. Haruno M, Wolpert DM, Kawato M. Hierarchical MOSAIC for movement generation.Int Congr. 2003;1250:575–590. doi: 10.1016/S0531-5131(03)00190-0. Leite et al. 2021 VOL.15 N.5 https://doi.org/10.20338/bjmb.v15i5.273 368 of 371 BJMB Brazilian Journal of Motor Behavior Special issue: 15 years of Brazilian Journal of Motor Behavior 46. Ishikawa R, Fukushima H, Frankland P, Kida S. Hippocampal neurogenesis enhancers promote forgetting of remote fear memory after hippocampal reactivation by retrieval. Elife. 2016;26(5):1-17. doi: 10.7554/eLife.17464. 47. Wolpert DM, Flanagan JR. Motor prediction. Curr Biol. 2001; 11(18):729-732. doi: 10.1016/s0960-9822(01)00432-8 48. von Holst E, Mittelstaedt H. Das reafferenzprinzip [The reafference principle]. Sci. Nat. 1950;37:464-476 49. Sperry RW. Neural basis of the spontaneous optokinetic response produced by visual inversion. J Comp Physiol Psychol. 1950; 43:482-489. doi.org/10.1037/h0055479 50. Wolpert DM, Kawato M. Multiple paired forward and inverse models for motor control: an adaptation study. Neural Netw. 1998; 11(7-8):1317–1329. doi: 10.1016/s08936080(98)00066-5 51. Kawato M. Internal models of motor control and trajectory planning. Curr Opin Neurobiol. 1999;9(6):718-727. doi: 10.1016/s0959-4388(99)00028-8 52. Miall RC, Weir DJ, Wolpert DM, Stein JF. Is the Cerebellum a Smith Predictor? J Mot Behav. 1993;25(3):203-216. doi: 10.1080/00222895.1993.9942050 53. Tanaka H, Ishikawa T, Kakei S. Neural Evidence of the Cerebellum as a State Predictor. Cerebellum. 2019;18, 349–371. doi: 10.1007/s12311-018-0996-4 54. Tanaka H, Ishikawa T, Lee J, Kakei S. The cerebro-cerebellum as a locus of forward model: a review. Frontiers Systems Neurosc. 2020;14(19):1-16. doi: 10.3389/fnsys.2020.00019 55. Miall RC, Wolpert D. Forward Models for Physiological Motor Control. Neural Netw. 1996; 9(8):1265-1279. doi: 10.1016/s0893-6080(96)00035-4. 56. Flanagan JR, Wing AM. The role of internal models in motion planning and control: evidence from grip force adjustments during movements of hand-held loads. J Neurosci. 1997; 17(4):1519-1528. doi.org/10.1523/JNEUROSCI.17-04-01519.1997. 57. Ugata M, Kuroda T, Imamizu H, Yoshioka T, Wada Y, Kawatao M. The possibility of using forward models for multi-lim coordination: examination of models for grip-load force coupling in humans. Electronics and communications in Japan. 2004;Part 3; 87(11). Translated from Deshi Joho Tsushin Gakkai Ronbunshi. 2003;J86-D-II(5):715-726. 58. Johansson RS, Westling G. Roles of glabrous skin receptors and sensorimotor memory in automatic control of precision grip when lifting rougher or more slippery objects. Exp Brain Res. 1984;56:550-564. doi: 10.1007/bf00237997 59. Johansson RS, Westling G. Tactile afferent signals in the control of precision grip. In: Jeannerod M. (Ed.), Attention and performance XIII. Hillsdale NJ: Erlbaum; 1990. 677-713. 60. Flanagan JR, Wing AM. Modulation of grip force with load force during point-to-point movements. Exp Brain Res. 1993;95(1):131–143. doi: 10.1007/BF00229662 61. Flanagan JR, Wing AM. The stability of precision grip force during cyclic arm movements with a hand-held load. Exp Brain Res. 1995;105(3):455–464. doi: 10.1007/BF00233045 Leite et al. 2021 VOL.15 N.5 https://doi.org/10.20338/bjmb.v15i5.273 369 of 371 BJMB Brazilian Journal of Motor Behavior Special issue: 15 years of Brazilian Journal of Motor Behavior 62. Flanagan JR, Tresilian JR. Grip-load force coupling: a general control strategy for transporting objects. J Exp Psychol Hum Percept Perform. 1994;20(5):944–957. doi.org/10.1037/0096-1523.20.5.944. 63. Kawato M, Kuroda T, Imamizu H, Nakano E, Miyauchi S, Yoshioka T. Internal forward models in the cerebellum: fMRI study on grip force and load force coupling. Prog Brain Res. 2003;42:171–188. doi: 10.1016/S0079-6123(03)42013-X. 64. Flanagan JR, Wing AM. The Role of Internal Models in Motion Planning and Control: Evidence from Grip Force Adjustments during Movements of Hand-Held Loads. J Neurosc. 1997;17(4):1519-1528. 65. Mathew J, Sarlegna FR, Bernier PM, Danion FR. Handedness matters for motor control but not for prediction. eNeuro. 2019; 6(3):1-13. doi: 10.1523/ENEURO.0136-19.2019. 66. Miall RC, Reckess GZ, Imamizu H. The cerebellum coordinates eye and hand tracking movements. Nat Neurosci. 2001;4(6):638–644. doi: 10.1038/88465. 67. Dakin CJ, Bolton DAE. Forecast or Fall: Prediction’s Importance to Postural Control. Front Neurol. 2018;9(924):1-10. doi.org/10.3389/fneur.2018.00924. 68. Wing AM, Flanagan JR, Richardson J. Anticipatory postural adjustments in stance and grip. Exp Brain Res. 1997;116(1):122-130. doi: 10.1007/pl00005732. 69. Yiou E, Artico R, Teyssedre CA, Labaune O, Fourcade P. Anticipatory Postural Control of Stability during Gait Initiation Over Obstacles of Different Height and Distance Made Under Reaction-Time and Self-Initiated Instructions. Front Hum Neurosci. 2016; 10(449):1-16. doi.org/10.3389/fnhum.2016.00449. 70. Tamada T, Miyauchi S, Imamizu H, Yoshioka T, Kawato, M. Cerebro-cerebellar functional connectivity revealed by the laterality index in tool-use learning. Neuroreport.1999; 10(2):325–331. doi: 10.1097/00001756-199902050-00022. 71. Blakemorel SJ and Wolpert DM, Frith CD. (1998) Why can't we tickle ourselves? An fMRI study. NeuroImage. 1998a;7(4):S418. doi.org/10.1016/S10538119(18)31251-5 72. Blakemore SJ, Wolpert DM, Frith CD. Central cancellation of self-produced tickle sensation. Nat Neurosci. 1998a;1(7):635-40. https://doi.org/10.1038/2870 73. Blakemore SJ, Wolpert DM, Frith CD. Central cancellation of self-produced tickle sensation. Nat Neurosci. 1998b;1(7):635-640. doi: 10.1038/2870. 74. Blakemore SJ, Frith CD, Wolpert DM. Spatio-temporal prediction modulates the perception of self-produced stimuli. J Cogn Neurosci. 1999;11(5):551-559. doi: 10.1162/089892999563607. 75. Blakemore SJ, Wolpert DM, Frith CD. Why can’t we tickle ourselves? Neuroreport. 2000; 11(3):R11-R16. doi: 10.1097/00001756-200008030-00002. 76. Shang C, Wang, Y, Du,J. Information integration for motor generation. Current Opinion Psych. (2019);8:116-120 doi: 10.1016/j.cophys.2019.01.012 77. Ghahramani Z, Wolpert DM. Modular decomposition in visuomotor learning. Nature. 1997;386(6623):392-395. doi: 10.1038/386392a0 Leite et al. 2021 VOL.15 N.5 https://doi.org/10.20338/bjmb.v15i5.273 370 of 371 BJMB Brazilian Journal of Motor Behavior Special issue: 15 years of Brazilian Journal of Motor Behavior 78. Imamizu H, Kuroda T, Miyauchi S, Yoshioka T, Kawato M. Modular organization of internal models of tools in the human cerebellum. Proc NatL Acad Sci U S A. 2003;100(9):5461– 5466. doi.org/10.1073/pnas.0835746100 79. Nishimura T, Hara A, Miyamoto, H, Furukawa, M, Maeda T. Mutual prediction model for predicting information for human motion generation. Proceedings of the 2020 IEEE/SICE. International Symposium on System Integration. (2020);12-15. 80. Ugata M, Kuroda T, Imamizu H, Yoshioka T, Wada Y, Kawato M. The possibility of using forward models for multi-lim coordination: examination of models for grip-load force coupling in humans. Electronics and communications in Japan, part 3, v. 87, n. 11, 2004. (Translated from Deshi Joho Tsushin Gakkai Ronbunshi, v. J86-D-II, n. 5, May 2003, pp. 715-726). Citation: Leite CMF, Campos CE, Couto CR, Ugrinowitsch H. (2021). An Internal Model Approach for Motor Behavior. Brazilian Journal of Motor Behavior, 15(5):356-371. Editors: Dr Fabio Augusto Barbieri - São Paulo State University (UNESP), Bauru, SP, Brazil; Dr José Angelo Barela São Paulo State University (UNESP), Rio Claro, SP, Brazil; Dr Natalia Madalena Rinaldi - Federal University of Espírito Santo (UFES), Vitória, ES, Brazil. Copyright:© 2021 Leite, Campos, Couto and Ugrinowitsch and BJMB. This is an open-access article distributed under the terms of the Creative Commons Attribution-Non Commercial-No Derivatives 4.0 International License which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Funding: This research did not receive any specific grant from funding agencies in the public, commercial, or not-forprofit sectors. Competing interests: The authors have declared that no competing interests exist. DOI: https://doi.org/10.20338/bjmb.v15i5.273 Leite et al. 2021 VOL.15 N.5 https://doi.org/10.20338/bjmb.v15i5.273 371 of 371