I recently finished reading ‘Language & Action in Cognitive Neuroscience’ (Coello & Bartolo, 2013). It’s one of those edited books with every chapter from different authors, which allows debates and tensions within a field to emerge. It’s also a treasure trove of academic delights, exploring how language and action are more tightly integrated than might previously have been thought.
All the papers draw on an extensive body of knowledge and, broadly, situate themselves within the ‘embodied cognition’ tradition. There’s lots of good stuff in the book so I’m just going to summarise the bits that struck me as relevant / interesting. (Everything that follows is drawn from the book, unless stated otherwise.) Since I’ve got to start somewhere, I’ll start with our beginnings in infancy.
Verbal communication is a specialised form of motor action – muscles, tendons and bones are all carefully coordinated in order to create sounds that (eventually) mean something. Alongside verbal communication there is non-verbal communication, gesture. Evidence from studies on infants indicates that verbal and gestural communication develop in close synchrony during early infancy. Rhythms in gesture match rhythms in sound making. Eventually there is a fractionation of the two, verbal communication begins to outstrip gestural communication, perhaps as infants discover the potential of talk. Nonetheless, speech and action are never completely disassociated.
Learning to speak, at some point is accompanied by learning to write. Learning to write is about learning movements of fingers, hands and wrists to achieve certain (parts of) letter-shapes. Through this hand motor control, control of the mouth (to sound the letter) and the visual representation of the letter are all integrated. Research, with infants in different language communities, has shown that memory for letters is also memory for action. There is, however, an important difference between learning to write and learning to type. In writing, there is a necessary relationship between the motor actions and the letter-shape; this doesn’t hold in typing, where the lay-out of the keys is arbitrary. Thus children who learn letters through writing show a better memory for letter shape than those who learn through typing.
The relationship between language and action is clearly two-way. One of the chapters cites work done with English-speakers on the use of ‘this’ and ‘that’. Generally, people have a sense of the field of space that they can act on. The research found that English-speakers would refer to objects within that field of action as ‘this object’ and those outside the field as ‘that object’. However, if the same speakers were given tools that extended their reach in some way then some of then participants unconsciously changed their sense of what was in their reach and changed the labelling of objects accordingly. Presumably, as we develop and age our sense of the space we can act on changes and the balance between this and that will shift.
However, mostly of the work in the book looked in another direction. A lot of attention focused on the role of action within language.
To go back a step, alongside mirror neurons (or the mirroring response, wherein neurons involved in the control of action fire in response to observing someone carrying out an action), it seems there may also be ‘canonical neurons’ wherein the neurons controlling graps fire in response to seeing an object that is to be grasped. The function of these neurons is clearly a subject of on-going debate but part of their function seems to be to prepare the observer for action.
So, a lot of the work reported in the book looked at how language also evokes similar rehearsals of action. So, in the phrase, ‘The student pushed the textbook away from her’, the worked ‘pushed’ evokes activity in the neurons used to control the action of pushing. Likewise for other action-verbs, the word evokes the relevant neural response but not enough to make our limbs twitch.
There are all sort of nuances and debates that are covered in the book but which I am skimming over. For instance, how far can you move into the metaphorical domain (away from the concrete action) before the response no longer occurs. For example, ‘He’s pushing drugs.’ – is that too much of a metaphorical use of the verb? Researchers are exploring this.
For me, a more significant debate, which the editors drew out, was the question of whether this rehearsal of action is really part of the mental representation of the word. I noted in an earlier blog, how research indicates that semantic representations are distributed and multi-modal. So it is entirely reasonably to suppose that these motor actions are an intrinsic part of the word. Some researchers clearly argue for this, others argue for a different interpretation. The alternative is that the action-rehearsal is not part of the representation of the word but that the representation is linked to other neural networks that, in turn, link to the pre-motor cortices. (There is some evidence indicating that when we hear a word it evokes activity in regions just anterior of the regions in the pre-motor cortex that are responsible for action. However, the pre-motor cortex is, apparently, very complex and a subject of on-going research.) On the other side, lesion studies (i.e. studies of people who have, through some mishap, suffered damage to specific parts of the brain) show a much more direct link between the two. Some studies have shown that people who have damage to parts of the pre-motor cortex are impaired on certain types of word comprehension test!
Whichever way the argument goes, the function of this link will not be changed. Only one of the authors really explored this and, as noted for mirror and canonical neurons (above), they suggest that the outcome of this link might be that it prepares us for action on the environment. Hearing the word may make us a little bit more ready to do the action. A side-effect of the action-language link is that it means that our understanding of an event becomes considerably richer as the mental representation involves more and more modalities. One paper talks about how the brains of dancers are considerably more active in more regions of the brain (compared to non-dancers) when they are watching a routine they have practiced themselves.
2 questions have come to mind through reading the book:
(1) Baddeley’s model of working memory (which I’ve not written about in this blog but which I’ve been studying these last few months) splits working memory into a central executive, which manipulates information, and 2 modality specific slave systems for short-term storage (one which is phonological-lexical and one which is visuo-spatial). At quite a late stage in the model’s development, they shoe-horned most of the other modalities, including tactile and kinaesthetic, as inputs into the visuo-spatial slave system. Given the wealth of evidence presented in this book on the integration of action and language, it seems to me that it may have been more sensible (if slightly counter-intuitive) to posit tactile and kinaesthetic modalities as inputs into the phonological slave sytem. I wonder why he and his co-workers didn’t do that?
(2) Given results from the lesion studies, I wonder what this implies for people in their fourth age. Oftentimes, people’s physical abilities begin to atrophy much later in life. This can be because of things like arthritis but it can also be a consequence of the regimens of care that inhibit people from exercising. I observed this with my own father-in-law, whose ability to walk diminished rapidly in care. Does physical atrophy lead to neural atrophy? If so, given that lesions to the pre-motor cortex have a negative effect on comprehension, would physical atrophy have a similar effect?
Coello, Y. & Bartolo, A. (Eds.) (2013). Language and Action in Cognitive Neuroscience. London & New York: Psychology Press