Synesthesia

Signs and characteristics

Some synesthetes often report that they were unaware their experiences were unusual until they realized other people did not have them, while others report feeling as if they had been keeping a secret their entire lives. The automatic and ineffable nature of a synesthetic experience means that the pairing may not seem out of the ordinary. This involuntary and consistent nature helps define synesthesia as a real experience. Most synesthetes report that their experiences are pleasant or neutral, although, in rare cases, synesthetes report that their experiences can lead to a degree of sensory overload.

Though often stereotyped in the popular media as a medical condition or neurological aberration, many synesthetes themselves do not perceive their synesthetic experiences as a handicap. On the contrary, some report it as a gift – an additional "hidden" sense – something they would not want to miss. Most synesthetes become aware of their distinctive mode of perception in their childhood. Some have learned how to apply their ability in daily life and work. Synesthetes have used their abilities in memorization of names and telephone numbers, mental arithmetic, and more complex creative activities like producing visual art, music, and theater.

Despite the commonalities which permit the definition of the broad phenomenon of synesthesia, individual experiences vary in numerous ways. This variability was first noticed early in synesthesia research. Some synesthetes report that vowels are more strongly colored, while for others consonants are more strongly colored. Self-reports, interviews, and autobiographical notes by synesthetes demonstrate a great degree of variety in types of synesthesia, the intensity of synesthetic perceptions, awareness of the perceptual discrepancies between synesthetes and non-synesthetes, and the ways synesthesia is used in work, creative processes, and daily life.

Synesthetes are very likely to participate in creative activities. It has been suggested that individual development of perceptual and cognitive skills, in addition to one's cultural environment, produces the variety in awareness and practical use of synesthetic phenomena. Synesthesia may also give a memory advantage. In one study, conducted by Julia Simner of the University of Edinburgh, it was found that spatial sequence synesthetes have a built-in and automatic mnemonic reference. Whereas a non-synesthete will need to create a mnemonic device to remember a sequence (like dates in a diary), a synesthete can simply reference their spatial visualizations.

Mechanism

As of 2015, the neurological correlates of synesthesia had not been established.

Dedicated regions of the brain are specialized for given functions. Increased cross-talk between regions specialized for different functions may account for the many types of synesthesia. For example, the additive experience of seeing color when looking at graphemes might be due to cross-activation of the grapheme-recognition area and the color area called V4 (see figure). This is supported by the fact that grapheme–color synesthetes can identify the color of a grapheme in their peripheral vision even when they cannot consciously identify the shape of the grapheme.

An alternative possibility is disinhibited feedback or a reduction in the amount of inhibition along normally existing feedback pathways. Normally, excitation and inhibition are balanced. However, if normal feedback was not inhibited as usual, then signals feeding back from late stages of multi-sensory processing might influence earlier stages such that tones could activate vision. Cytowic and Eagleman find support for the disinhibition idea in the acquired forms of synesthesia that occur in non-synesthetes under certain conditions: temporal lobe epilepsy, head trauma, stroke, and brain tumors. They also note that it can likewise occur during stages of meditation, deep concentration, sensory deprivation, or with the use of psychedelics such as LSD or mescaline, and even, in some cases, marijuana. However, synesthetes report that common stimulants, like caffeine and cigarettes do not affect the strength of their synesthesia, nor does alcohol.

A very different theoretical approach to synesthesia is that based on ideasthesia. According to this account, synesthesia is a phenomenon mediated by the extraction of the meaning of the inducing stimulus. Thus, synesthesia may be fundamentally a semantic phenomenon. Therefore, to understand neural mechanisms of synesthesia the mechanisms of semantics and the extraction of meaning need to be understood better. This is a non-trivial issue because it is not only a question of a location in the brain at which meaning is "processed" but pertains also to the question of understanding – epitomized in e.g., the Chinese room problem. Thus, the question of the neural basis of synesthesia is deeply entrenched into the general mind–body problem and the problem of the explanatory gap.

Genetics

Due to the prevalence of synesthesia among the first-degree relatives of people affected, there may be a genetic basis, as indicated by the monozygotic twins studies showing an epigenetic component. Synesthesia might also be an oligogenic condition, with locus heterogeneity, multiple forms of inheritance, and continuous variation in gene expression. While the exact genetic loci for this trait haven't been identified, research indicates that the genetic constructs underlying synesthesia are most likely more complex than the simple X-linked mode of inheritance that early researchers believed it to be. Further, it remains uncertain as to whether synesthesia perseveres in the genetic pool because it provides a selective advantage, or because it has become a byproduct of some other useful selected trait.

Although often termed a "neurological condition", synesthesia is not listed in either the Diagnostic and Statistical Manual of Mental Disorders IV (1994) or the International Classification of Diseases (ICD) since it usually does not interfere with normal daily functioning. Indeed, most synesthetes report that their experiences are neutral or even pleasant. Like perfect pitch, synesthesia is simply a difference in perceptual experience.

The simplest approach is test-retest reliability over long periods of time, using stimuli of color names, color chips, or a computer-screen color picker providing 16.7 million choices. Synesthetes consistently score around 90% on the reliability of associations, even with years between tests. In contrast, non-synesthetes score just 30–40%, even with only a few weeks between tests and a warning that they would be retested.

Many tests exist for synesthesia. Each common type has a specific test. When testing for grapheme–color synesthesia, a visual test is given. The person is shown a picture that includes black letters and numbers. A synesthete will associate the letters and numbers with a specific color. An auditory test is another way to test for synesthesia. A sound is turned on and one will either identify it with a taste or envision shapes. The audio test correlates with chromesthesia (sounds with colors). Since people question whether or not synesthesia is tied to memory, the "retest" is given. One is given a set of objects and is asked to assign colors, tastes, personalities, or more. After some time, the same objects are presented and the person is asked again to do the same task. The synesthete can assign the same characteristics because that person has permanent neural associations in the brain, rather than memories of a certain object.

Grapheme–color synesthetes, as a group, share significant preferences for the color of each letter (e.g., A tends to be red; O tends to be white or black; S tends to be yellow, etc.) Nonetheless, there is a great variety in types of synesthesia, and within each type, individuals report differing triggers for their sensations and differing intensities of experiences. This variety means that defining synesthesia in an individual is difficult, and the majority of synesthetes are completely unaware that their experiences have a name.

Neurologist Richard Cytowic identifies the following diagnostic criteria for synesthesia in his first edition book. However, the criteria are different in the second book:

1. Synesthesia is involuntary and automatic
2. Synesthetic perceptions are spatially extended, meaning they often have a sense of "location". For example, synesthetes speak of "looking at" or "going to" a particular place to attend to the experience
3. Synesthetic percepts are consistent and generic (i.e., simple rather than pictorial)
4. Synesthesia is highly memorable
5. Synesthesia is laden with affect

Cytowic's early cases mainly included individuals whose synesthesia was frankly projected outside the body (e.g., on a "screen" in front of one's face). Later research showed that such stark externalization occurs in a minority of synesthetes. Refining this concept, Cytowic and Eagleman differentiated between "localizers" and "non-localizers" to distinguish those synesthetes whose perceptions have a definite sense of spatial quality from those whose perceptions do not.

Prevalence

Estimates of the prevalence of synesthesia have ranged widely, from 1 in 4 to 1 in 25,000–100,000. However, most studies have relied on synesthetes reporting themselves, introducing self-referral bias. In what is cited as the most accurate prevalence study so far, self-referral bias was avoided by studying 500 people recruited from the communities of Edinburgh and Glasgow Universities; it showed a prevalence of 4.4%, with 9 different variations of synesthesia. This study also concluded that one common form of synesthesia – grapheme–color synesthesia (colored letters and numbers) – is found in more than one percent of the population, and this latter prevalence of graphemes–color synesthesia has since been independently verified in a sample of nearly 3,000 people in the University of Edinburgh.

The most common forms of synesthesia are those that trigger colors, and the most prevalent of all is day–color. Also relatively common is grapheme–color synesthesia. Prevalence refers to both how common is synesthesia (or different forms of synesthesia) within the population, or how common are different forms of synesthesia within synesthetes. So within synesthetes, forms of synesthesia that trigger color also appear to be the most common forms of synesthesia with a prevalence rate of 86% within synesthetes. In another study, music–color is also prevalent at 18–41%. Some of the rarest are reported to be auditory–tactile, mirror-touch, and lexical–gustatory.

There is also disagreement regarding synesthesia's apparent prevalence in females over males, with the commonly cited 1966 study by Baron-Cohen and others putting the female-male prevalence ratio at 5.5 to 1, which was followed by a second study proposing a female bias of around 6 to 1. However, several later studies refuted the claims of female bias in synesthesia, given that the studies were carried out with the same self-referral process that they argue disproportionately increases the female count as females are known to be more likely than males to come forward to report atypical experiences. One such study put the female-male ratio at 1.1 to 1 instead in a population of 500, and 0.9 to 1 in a population of 1190 when considering grapheme-colour synaesthesia. Conversely, a subsequent study pointed out that the sex-related differences in self-referrals only accounted for a 10% change, and evaluated data they stated was uncontaminated by the self-referral confound, although this has been debated along with their methodology for identifying genuine synesthetes, due to 53 of the 93 synesthetes identified self-referred to the university in response to a media advert, perpetrating female bias. In addition, the study appeared to directly compare the 78 female synesthetes against 14 male synesthetes, leading to a female prevalent ratio of 5.57 to 1, yet Barnett et al. of the study evaluated twice as many females than males, with 118 females and 61 males, which would mean 66% prevalence among females compared to 23% for males, when the average prevalence is 4.4%

Some studies suggest that the likelihood of having synesthesia is greater in autism. Another study claims that synesthesia is specifically more common in the subset of autistic people who self-report unusual skills not ordinarily found in the general population, which study authors describe in terms of savant syndrome. The relationship between autistic traits and synesthesia appears to be primarily driven by shared genetic underpinnings.

Prior research also reports that sensory hyperresponsiveness, and according to one study also sensory hyporesponsiveness, are more common in people with synesthesia (as well as autistic people) than in control participants. Researchers have also suggested that synesthesia may be related to misophonia, a specific type of heightened responsiveness to sensory stimulation in which certain trigger sounds can evoke distressing emotional responses. A study exploring whether individuals with misophonia reported chromesthesia, grapheme–colour synesthesia, number form synesthesia, or other kinds of synesthesia reported 9–17% experienced each type; although the study did not include a comparison group of people without misophonia, the study authors noted this prevalence would be higher than previously found in the general population. However, research exploring whether synesthesia and misophonia are linked remains very limited.

History

The interest in colored hearing dates back to Greek antiquity when some theorists wondered whether the color (chroia, what we now call timbre) of music was a quantifiable quality of sound, together with pitch and duration. Additionally, one kind of musical scale (genos) introduced by Plato's friend Archytas of Tarentum in the fourth century BC was named chromatic. The late sixth century BC kitharist Lysander of Sicyon was said to have introduced a more 'colorful' style, even before the development of the chromatic scale itself. In Plato's time, the description of melody as 'colored' had become part of professional jargon, while the musical terms 'tone' and 'harmony' soon became integrated into the vocabulary of color in visual art. Isaac Newton proposed that musical tones and color tones shared common frequencies, as did Goethe in his book Theory of Colours. There is a long history of building color organs such as the clavier à lumières on which to perform colored music in concert halls.

The first medical description of "colored hearing" is in an 1812 thesis by the German physician Georg Tobias Ludwig Sachs. The "father of psychophysics", Gustav Fechner, reported the first empirical survey of colored letter photisms among 73 synesthetes in 1876, followed in the 1880s by Francis Galton. Carl Jung refers to "color hearing" in his Symbols of Transformation in 1912.

In the early 1920s, the Bauhaus teacher and musician Gertrud Grunow researched the relationships between sound, color, and movement and developed a 'twelve-tone circle of colour' which was analogous with the twelve-tone music of the Austrian composer Arnold Schönberg (1874–1951). She was a participant in at least one of the Congresses for Colour–Sound Research (German:Kongreß für Farbe-Ton-Forschung) held in Hamburg in the late 1920s and early 1930s.

Research into synesthesia proceeded briskly in several countries, but due to the difficulties in measuring subjective experiences and the rise of behaviorism, which made the study of any subjective experience taboo, synesthesia faded into scientific oblivion between 1930 and 1980.

As the 1980s cognitive revolution made inquiry into internal subjective states respectable again, scientists returned to studying synesthesia. Led in the United States by Larry Marks and Richard Cytowic, and later in England by Simon Baron-Cohen and Jeffrey Gray, researchers explored the reality, consistency, and frequency of synesthetic experiences. In the late 1990s, the focus settled on grapheme → color synesthesia, one of the most common and easily studied types. Psychologists and neuroscientists study synesthesia not only for its inherent appeal but also for the insights it may give into cognitive and perceptual processes that occur in synesthetes and non-synesthetes alike. Synesthesia is now the topic of scientific books and papers, Ph.D. theses, documentary films, and even novels.

Since the rise of the Internet in the 1990s, synesthetes began contacting one another and creating websites devoted to the condition. These rapidly grew into international organizations such as the American Synesthesia Association, the UK Synaesthesia Association, the Belgian Synesthesia Association, the Canadian Synesthesia Association, the German Synesthesia Association, and the Netherlands Synesthesia Web Community.

Society and culture

Notable cases

Solomon Shereshevsky, a newspaper reporter turned mnemonist, was discovered by Russian neuropsychologist Alexander Luria to have a rare fivefold form of synesthesia, of which he is the only known case. Words and text were not only associated with highly vivid visuospatial imagery but also sound, taste, color, and sensation. Shereshevsky could recount endless details of many things without form, from lists of names to decades-old conversations, but he had great difficulty grasping abstract concepts. The automatic, and nearly permanent, retention of every detail due to synesthesia greatly inhibited Shereshevsky's ability to understand what he read or heard.

Neuroscientist and author V. S. Ramachandran studied the case of a grapheme–color synesthete who was also color blind. While he couldn't see certain colors with his eyes, he could still "see" those colors when looking at certain letters. Because he did not have a name for those colors, he called them "Martian colors".

Art

Other notable synesthetes come particularly from artistic professions and backgrounds. Synesthetic art historically refers to multi-sensory experiments in the genres of visual music, music visualization, audiovisual art, abstract film, and intermedia. Distinct from neuroscience, the concept of synesthesia in the arts is regarded as the simultaneous perception of multiple stimuli in one gestalt experience. Neurological synesthesia has been a source of inspiration for artists, composers, poets, novelists, and digital artists.

Writers

Vladimir Nabokov wrote explicitly about synesthesia in several novels. Nabokov described his grapheme–color synesthesia at length in his memoir, Speak, Memory:

I present a fine case of colored hearing. Perhaps "hearing" is not quite accurate, since the color sensations seem to be produced by the very act of my orally forming a given letter while I imagine its outline. The long a of the English alphabet (and it is this alphabet I have in mind farther on unless otherwise stated) has for me the tint of weathered wood, but the French a evokes polished ebony. This black group also includes hard g (vulcanized rubber) and r (a sooty rag being ripped). Oatmeal n, noodle-limp l, and the ivory-backed hand mirror of o take care of the whites. I am puzzled by my French on which I see as the brimming tension-surface of alcohol in a small glass. Passing on to the blue group, there is steely x, thundercloud z, and huckleberry k. Since a subtle interaction exists between sound and shape, I see q as browner than k, while s is not the light blue of c, but a curious mixture of azure and mother-of-pearl.

Daniel Tammet wrote a book on his experiences with synesthesia called Born on a Blue Day. Joanne Harris, author of Chocolat, is a synesthete who says she experiences colors as scents. Her novel Blueeyedboy features various characters with synesthesia.

Painters and photographers

Wassily Kandinsky (a synesthete) and Piet Mondrian (not a synesthete) both experimented with image–music congruence in their paintings. Georgia O'Keeffe gave titles to several of her paintings, e.g. Blue and Green Music (1919–1921), using music–color associations. Contemporary artists with synesthesia, such as Carol Steen (a photographer who waits until she gets a synesthetic response from what she sees and then takes the picture), use their synesthesia to create their artwork. Linda Anderson, according to NPR considered "one of the foremost living memory painters", creates with oil crayons on fine-grain sandpaper representations of the auditory-visual synaesthesia she experiences during severe migraine attacks. Brandy Gale, a Canadian visual artist, experiences an involuntary joining or crossing of any of her senses – hearing, vision, taste, touch, smell and movement. Gale paints from life rather than from photographs and by exploring the sensory panorama of each locale attempts to capture, select, and transmit these personal experiences. David Hockney perceived music as color, shape, and configuration and used these perceptions when painting opera stage sets (though not while creating his other artworks). Kandinsky combined four senses: color, hearing, touch, and smell.

Composers

Several composers had experienced synesthesia.

Mikalojus Konstantinas Čiurlionis, a Lithuanian painter, composer, and writer, perceived colors and music simultaneously. Many of his paintings bear the names of matching musical pieces: sonatas, fugues, and preludes.

Alexander Scriabin composed colored music that was deliberately contrived and based on the circle of fifths, whereas Olivier Messiaen invented a new method of composition (the modes of limited transposition) specifically to render his bi-directional sound–color synesthesia. For example, the red rocks of Bryce Canyon are depicted in his symphony Des canyons aux étoiles... ("From the Canyons to the Stars"). New art movements such as literary symbolism, non-figurative art, and visual music have profited from experiments with synesthetic perception and contributed to the public awareness of synesthetic and multi-sensory ways of perceiving. Other composers who reported synesthesia include Duke Ellington, Nikolai Rimsky-Korsakov, and Jean Sibelius.

Several contemporary composers with synesthesia are Michael Torke, and Ramin Djawadi, best known for his work on composing the theme songs and scores for such TV series as Game of Thrones, Westworld and for the Iron Man movie. He says he tends to "associate colors with music, or music with colors".

British composer Daniel Liam Glyn created the classical-contemporary music project Changing Stations using grapheme–colour synesthesia. Based on the 11 main lines of the London Underground, the eleven tracks featured on the album represent the eleven main tube line colours. Each track focuses heavily on the different speeds, sounds, and mood of each line, and are composed in the key signature synaesthetically assigned by Glyn with reference to the colour of the tube line on the map.

Musicians

The producer, rapper, and fashion designer Kanye West is a prominent interdisciplinary case. In an impromptu speech he gave during an Ellen interview, he described his condition, saying that he sees sounds and that everything he sonically makes is a painting. Other notable synesthetes include musicians Thom Yorke, Billy Joel, Andy Partridge, Itzhak Perlman, Lorde, Billie Eilish, Charli xcx, Brendon Urie, Ida Maria, Brian Chase, and classical pianist Hélène Grimaud. Musician Kristin Hersh sees music in colors. Drummer Mickey Hart of the Grateful Dead wrote about his experiences with synaesthesia in his autobiography Drumming at the Edge of Magic. John Frusciante, guitarist of the Red Hot Chili Peppers, talks about his experiences with synesthesia in a podcast with Rick Rubin. Pharrell Williams, of the groups The Neptunes and N.E.R.D., also experiences synesthesia and used it as the basis of the album Seeing Sounds. Singer/songwriter Marina and the Diamonds experiences music → color synesthesia and reports colored days of the week. Awsten Knight from Waterparks has chromesthesia, which influences many of the band's artistic choices.

Artists without synesthesia

Some artists frequently mentioned as synesthetes did not, in fact, have the neurological condition. Scriabin's 1911 Prometheus: The Poem of Fire, for example, is a deliberate contrivance whose color choices are based on the circle of fifths and appear to have been taken from Madame Blavatsky. The musical score has a separate staff marked luce whose "notes" are played on a color organ. Technical reviews appear in period volumes of Scientific American. On the other hand, his older colleague Rimsky-Korsakov (who was perceived as a fairly conservative composer) was, in fact, a synesthete.

French poets Arthur Rimbaud and Charles Baudelaire wrote of synesthetic experiences, but there is no evidence they were synesthetes themselves. Baudelaire's 1857 Correspondances introduced the notion that the senses can and should intermingle. Baudelaire participated in a hashish experiment by psychiatrist Jacques-Joseph Moreau and became interested in how the senses might affect each other. Rimbaud later wrote Voyelles (1871), which was perhaps more important than Correspondances in popularizing synesthesia. He later boasted "J'inventais la couleur des voyelles!" (I invented the colors of the vowels!).

Science

Some technologists, like inventor Nikola Tesla, and scientists also reported being synesthetic. Physicist Richard Feynman describes his colored equations in his autobiography, What Do You Care What Other People Think?: "When I see equations, I see the letters in colors. I don't know why. I see vague pictures of Bessel functions with light-tan j's, slightly violet-bluish n's, and dark brown x's flying around."

Film

In the 2024 Canadian film Magnetosphere, the central character is a 13-year-old girl with synesthesia.

The Colors Within, a film released in 2024, centers around a high-school student with person-color synesthesia.

Literature

Synesthesia is sometimes used as a plot device or a way of developing a character's inner life. Author and synesthete Patricia Lynne Duffy describes four ways in which synesthetic characters have been used in modern fiction.

• Synesthesia as Romantic ideal: in which the condition illustrates the Romantic ideal of transcending one's experience of the world. Books in this category include The Gift by Vladimir Nabokov.
• Synesthesia as pathology: in which the trait is pathological. Books in this category include The Whole World Over by Julia Glass.
• Synesthesia as Romantic pathology: in which synesthesia is pathological but also provides an avenue to the Romantic ideal of transcending quotidian experience. Books in this category include Holly Payne's The Sound of Blue (2004) and Anna Ferrara's The Woman Who Tried to Be Normal (2018)
• Synesthesia as psychological health and balance: Painting Ruby Tuesday by Jane Yardley, and A Mango-Shaped Space by Wendy Mass.

Literary depictions of synesthesia are criticized as often being more of a reflection of an author's interpretation of synesthesia than of the phenomenon itself.

Research

Research on synesthesia raises questions about how the brain combines information from different sensory modalities, referred to as crossmodal perception or multisensory integration.

An example of this is the bouba/kiki effect. In an experiment first designed by Wolfgang Köhler, people are asked to choose which of two shapes is named bouba and which kiki. The angular shape, kiki, is chosen by 95–98% and bouba for the rounded one. Individuals on the island of Tenerife showed a similar preference between shapes called takete and maluma. Even 2.5-year-old children (too young to speak) show this effect. Research indicated that in the background of this effect may operate a form of ideasthesia.

Researchers hope that the study of synesthesia will provide a better understanding of consciousness and its neural correlates. In particular, synesthesia might be relevant to the philosophical problem of qualia, given that synesthetes experience extra qualia (e.g., colored sound). An important insight for qualia research may come from the findings that synesthesia has the properties of ideasthesia, which then suggest a crucial role of conceptualization processes in generating qualia. Some philosophers argue that synesthesia creates problems for theories that say our conscious experiences are determined by what our minds represent about the world. For example, when both a synesthete and a non-synesthete feel pain in the same way, they might be representing the same bodily damage, but the synesthete also experiences colors while the non-synesthete does not. This suggests that identical mental representations can lead to different conscious experiences.

Technological applications

Synesthesia also has several practical applications, including 'intentional synesthesia' in technology, and sensory prosthetics.

The Voice (vOICe)

Peter Meijer developed a sensory substitution device for the visually impaired called The vOICe (the capital letters "O", "I", and "C" in "vOICe" are intended to evoke the expression "Oh I see"). The vOICe is a privately owned research project, running without venture capital, that was first implemented using low-cost hardware in 1991. The vOICe is a visual-to-auditory sensory substitution device (SSD) preserving visual detail at high resolution (up to 25,344 pixels). The device consists of a laptop, head-mounted camera or computer camera, and headphones. The vOICe converts visual stimuli of the surroundings captured by the camera into corresponding aural representations (soundscapes) delivered to the user through headphones at a default rate of one soundscape per second. Each soundscape is a left-to-right scan, with height represented by pitch, and brightness by loudness. The vOICe compensates for the loss of vision by converting information from the lost sensory modality into stimuli in a remaining modality.