Brain hardwired to link images and sounds, baby study finds


creative commons licensed (BY-NC) flickr photo by I_am_Allan

A study of the electrical brain activity of baby infants suggests that we are biologically predisposed to link images and sounds to create language.

An international team, including researchers at the University of Warwick, examined the electrical activity of the brains of 11 month-olds in the initial stages of word learning.

Using fictitious words, for example ‘kipi’ to refer to pictures of spiky shapes or ‘moma’ to refer to rounded shapes, the researchers were able to study the affect of unknown words with images or without images on the human brain. The babies were seen to very quickly begin to match the word to the image. The electical brain activity backed up this observation with brain activity increasing when images and words matched.

Dr Sotaro Kita from the University of Warwick explains: ‘The oscillatory activity of the infant brainincreased when the word they heard matched the shape they were shown, compared to when it did not. This suggests that the infant brain spontaneously engages in matching visual and auditory input.’

Brain jam

‘Traffic’ in the brain also exhibited different patterns when images were used in addition to word learning. When the word matched the shape, the brain was ‘light’, according to Dr Kita. She added that when the word did not match the shape, the traffic became heavy, especially in the left hemisphere where Dr Kita points out, language is typically processed.

Dr Kita said: The left-hemisphere had to work harder to associate visual and auditory input when they are not a natural match.’

The N400 response – a term referring to the brain’s normal processing of words and meaning – was higher for mismatching word-image pairs. This indicates that the babies were trying to work out the meaning of the novel words.

Sound symbolism

The findings suggest that sound symbolism allows babies to spontaneously bind the speech sound with the image that represents the word. Researchers believe that this spontaneous binding also provides babies with an insight that spoken words refer to objects you can see in the world.

Dr Kita: ‘It is this cross-modal mapping between sound and image that plays a key role in the origin and development of language-learning.’

University of Warwick press release

Sound symbolism scaffolds language development in preverbal infants