The visual system controls sight and visual processing, making the system simple to understand even though it handles complicated tasks.
It is a highly complex series of areas that work toward the common goal of turning light into something recognizable. Though at first it appears to be a single region of the brain, the visual system is made up of multiple, neighboring functional areas that work collectively for cohesive visual processing.
Engage in visual processing
Recognize patterns, textures, and faces
Identify the location of objects, buildings, or people in space
Determining the function and permanence of objects
Aid in problem solving
Reinforce visually learned behaviors
Today, modern neuroscience continues to explore how the visual system makes sense of the world around us.
Yet, there are many capabilities still to be studied, as our understanding of this complex system continually expands.
The visual system’s complex roles in the brain
The visual system can quickly identify visual changes, consistencies, or inconsistencies that aid in problem solving and evaluating the world around us.
To break this down, take a look at the following pattern-based question. Which image comes next in the sequence?
This problem could be solved mathematically by assigning each vertex of the pentagon a numerical value and determining the relationship between vertex 1 and 4, 2 and 5, 3 and 1, etc.
Without mathematics, the problem can be visually solved by recognizing that the arrow moves one vertex clockwise in each sequential image. This indicates that the next image would be image B.
The visual system is responsible for making sense of what we see, but it can also be tricked based on what it thinks it perceives — giving us optical illusions.
This optical illusion plays with the visual system’s ability to process light, dark, and linear elements.
At first glance, the grey lines in this optical illusion appear to be slanted, but when you look more closely, you can see that the grey lines are parallel and the squares are equal size. (If your eyes won’t stop playing tricks on you, cover all but the top row with a piece of paper and compare the size of the squares.)
With the squares shifted off-center to the left or right, they give a warped perception that the grey lines are changing in perspective or growing in size. This is due to our visual system’s ability to process depth perception1 using size and perspective cues.2 In these examples, and in most of our daily visual processing, the visual system analyzes images and movement to make sense of what we see. This can be applied to learning movement behaviors, understanding maps, reading written text,3 and the recognition of facial expressions.4 Each of these functions is based on patterns that can be identified in a single moment or over the course of an individual's learned experience.
The visual system’s pattern recognition capabilities have also become a key basis for creating computer-based neural networks. Developers model machine learning algorithms after human visual processes to design computer-based image recognition or object identification (more on this later).
To understand how the visual system can be leveraged for computer technology, it’s important to first look at how the system transfers information from the eyes throughout the occipital lobe.
Discovery of the visual system and its cortical organization
The visual system is one of the older brain networks, having been studied since before the turn of the 20th century. German physiologist, Hermann Munk, identified the occipital cortex as central to vision capabilities in 1881 through experiments on monkeys and dogs,5 and he first determined a connection between the eyes and the rear of the brain.
The Russo-Japanese War and World War I later led to increased awareness of the visual cortex as military doctors studied visual impairments — including blindness — in soldiers who had suffered rear-brain injuries.6 Japanese physician, Tatsuji Inouye, was the first to develop retinotopic maps of the visual cortex,7 which were reinforced by subsequent wartime studies.8,9
Using this foundation, neuroscience has continued to map the relationship between the eyes, the visual pathway, and the visual cortex. The visual pathway extends from the retina through the optic nerves10 and the lateral geniculate nuclei (LGN) to the visual cortex. Within the visual cortex, information is then processed through a series of closely connected — but functionally selective11 — nodes in the brain.
Each of these nodes are functionally and structurally connected (see Connectomics) as distributed networks that make up the dorsal and ventral pathways.
Hierarchy of functional areas in the dorsal and ventral visual pathways
The functional nodes of the visual system have a hierarchical organization for how they distribute and process information through the dorsal and ventral visual pathways.
Upon receiving visual information from the retina, imagery is initially processed in the primary node of the visual cortex, V1. It is then passed through the connected nodes V2 and V3, and through the dorsal or ventral visual pathways where task segregation occurs in highly specialized areas12 of the visual cortex.
The dorsal visual pathway is responsible for the perception of objects,13 global motion,14 and processing visual stimuli15 for a coordinated response. At the same time, the ventral visual pathway is thought to be involved in the recognition of faces, places,16 bodies,17 shapes, letters,18 and words.19
The following chart outlines how visual information can be initially processed in the primary nodes of the visual cortex, before being deployed through the dorsal and ventral pathways to highly specialized areas of the visual system.12
As an example, when you look at a person’s face, both the dorsal and ventral visual pathways are thought to be involved in the initial stages of processing.21 The specialized recognition of facial features through spatial relationships then occurs later in the dorsal visual pathway with strong functional connectivity to the fusiform face area (FFA).22
The hierarchy of the visual system shows us that visual neural networks are closely connected, highly specialized, and synchronized.
Yet, just because the visual cortex appears relatively self-contained anatomically — unlike the default mode network (DMN), for example, which can be found in multiple separate areas in each hemisphere — it still interacts frequently with other networks.
Network integration and impact on visual and mental health
The visual system works closely with many of the brain’s networks:
Limbic system for visual memory and processing threat-based stimuli noticed in an individual’s peripheral vision4
Salience network for self-control when presented with tempting imagery,27 as well as emotional empathy28
These are just some of the ways the visual system primarily partners with other networks to carry out complex functions. However, there are also many occasions where this coordination breaks down causing visual impairments or mental health issues.
Many individuals find it difficult to stay focused while reading, and this may correlate with a decoupling of the visual system and the default mode network.29 It has been suggested that stronger connectivity with the DMN is linked to better text processing,29 otherwise it is easy for our minds to wander when we are reading.
Likewise, increased asynchrony between the visual system and the sensorimotor network has been shown to correlate with mild to severe impairment in children with autism spectrum disorder (ASD).30 This impairment likely impacts motor and social skill acquisition. Evaluating brain biomarkers based on the overconnectivity or underconnectivity31 of networks may present an opportunity for identifying and diagnosing ASD.32
Decreased connectivity in the visual system can also be present in individuals with schizophrenia. Diminished connectivity has been shown to correlate with the severity of psychotic symptoms,33 and it may lead to impaired visual object perception.33
Studying changes in functional connectivity of the visual system is also relevant to vision disorders. In patients with primary open angle glaucoma (POAG) which causes blindness, there are not only structural abnormalities34 within the eyes. Atrophy also occurs in the visual cortex32 as well distant gray matter areas of the brain — including decreased functional connectivity in working memory and the dorsal attention network (DAN).34 This degradation suggests that POAG could be considered a neurodegenerative condition.
While studies of the visual system are providing ongoing insight into mental illnesses and neurological conditions, understanding the system’s functional connectivity has also led to advances in computer technology.
From the human brain to the neural network
Moving beyond our understanding of the brain, the visual system’s functional connectivity has become a model for computer-based neural networks. A neural network is a series of algorithms that can be trained to look for visual patterns in large datasets.35 Using computer algorithms, we can recreate cognitive processes36 like visual perception because neural networks mimic the hierarchical organization that occurs in the visual pathways.
Furthermore, the dorsal and ventral pathways present a useful model for close-proximity, small-world networks.37 When developing computer-based information processing systems, the anatomically localized connections between the primary visual nodes and the highly specialized areas of the visual system are a key example for short-range connections in neural networks.
Deep learning, neural networks can then be leveraged as predictive models for image recognition38 by comparing objects, faces, places, and other key tasks. Models can also be further trained to recognize patterns and movement.39
Computationally modeling neural networks poses two unique opportunities:
It gives the ability to use machine learning algorithms to process information in a similar method as a human viewer
It offers better possibilities for understanding the human brain.
If we can correctly model the brain’s intricate functions, we can use computer technology to dive deeper into how the brain operates when it is healthy versus when it experiences functional problems. As neuroscience continues to study the abnormalities resulting in physical or mental health disorders, we are also paving the way for discovering possible brain-focused treatments.
The visual system has led to incredible technological advances, but other networks have also posed new opportunities for the field of neuroscience.
The medial areas in the visual network include: V1 for the initial detection of motion and possibly directional and spatial integration of global motion. V2 to partition the visual field into two discontinuous, inverted, contralateral, quarterfield maps, and it perceives visual stimuli at greater angles from the fovea than V1. V3 is similar to V2 with still wider degrees of visual field, and it plays a role in perception and integration of global motion. It also transmits information to V6. V4 encodes texture, patterns, form, and color perception, and it is integral to object and pattern recognition.
In the dorsal stream, functional areas V3A, V3B, V6 are implicated in processing visual motion. V6A and V7 show response to visual stimuli, and V6A evaluates spatial information for coordinated hand-eye response. Area IPS1 also occurs in the dorsal stream for visualizing objects and determining their permanence and function.
The ventral stream includes area FFC which recognizes facial features, identity, and emotional expression. Areas PIT, V8, and VVC are implicated in color analysis and perception, while areas VMV1, VMV2, and VMV3 are more recent discoveries suggested to be important for the integration of color, texture and form information. VMV1 and VMV2 are thought to focus on places, while VMV3 is considered related to object recognition.
Areas in the lateral stream include FST, MT, and MST for the perception of motion, self-motion, and spatial reference. Area MT integrates one-dimensional visual signals into a two-dimensional visual motion pattern, before sending the information to area MST. Areas LO1, LO2, and LO3 receive information from the dorsal and ventral streams for the processing of object recognition, boundaries, motion, and shape. Additionally, area PH in the lateral stream is implicated in the formation of spatial maps for place recognition; area V3CD integrates object detail information; and area V4t is responsible for global motion processing and shape-sensitive information.
Expand for full list of references
Poggio GF, Gonzalez F, Krause F. Stereoscopic mechanisms in monkey visual cortex: binocular correlation and disparity selectivity. J Neurosci. 1988;8(12):4531-4550. doi:10.1523/JNEUROSCI.08-12-04531.1988
Kalloniatis M, Luu C. The perception of depth. In: Kolb H, Fernandez E, Nelson R, eds. Webvision: The Organization of the Retina and Visual System [Internet]. Salt Lake City, Utah: University of Utah Health Sciences Center; 1995. https://www.ncbi.nlm.nih.gov/books/NBK11512/. Published May 1, 2005. Updated June 6, 2007. Accessed January 15, 2019.
Sajedinia Z, Pizlo Z, Hélie S. Investigating the role of the visual system in solving the traveling salesperson problem. Proceedings of the 41st Annual Meeting of the Cognitive Science Society. Retrieved from https://par.nsf.gov/biblio/10177542. Accessed January 14, 2021.
Bayet, Laurie. The development of facial expressions perception. Psychology. Université Grenoble Alpes; 2015. English. NNT : 2015GREAS049ff. Tel-01475668v2f.
Munk H. Uber die funktionen der grosshirnrinde. A. Hirschwald, Berlin. 1881.
Lanska DJ. Elaboration of the Visual Pathways from the Study of War-Related Cranial Injuries: The Period from the Russo-Japanese War to World War I. In: Tatu L, Bogousslavsky J, eds. War Neurology. Front Neurol Neurosci. Basel, Switzerland: Karger; 2016;38:31-42. doi.org/10.1159/000442567
Glickstein M. The discovery of the visual cortex. Sci Am. 1988;259(3):118-127. http://www.jstor.org/stable/24989235. Accessed January 4, 2021.
Holmes G. DISTURBANCES OF VISION BY CEREBRAL LESIONS. Br J Ophthalmol. 1918;2(7):353-384. doi.org/10.1136/bjo.2.7.353
Benson NC, Butt OH, Datta R, Radoeva PD, Brainard DH, Aguirre GK. The retinotopic organization of striate cortex is well predicted by surface topology. Curr Biol. 2012;22(21):2081-2085. doi.org/10.1016/j.cub.2012.09.014
Crumble L. Visual pathway. Salvador F, ed. Kenhub website.https://www.kenhub.com/en/library/anatomy/the-visual-pathway. Updated November 13, 2020. Accessed January 14, 2021.
Yamasaki T, Tobimatsu S. Parallel visual pathways in humans. In: Driving ability in the Alzheimer disease spectrum: Neural basis, assessment, and potential use of optic flow event-related potentials. Front Neurol. 2018;9(750). doi:10.3389/fneur.2018.00750
Kanwisher N. Functional imaging of the human brain: A window into the architecture of the mind. Lecture presented at CNS 2020 Virtual; May 5, 2020.
Freud E, Plaut DC, Behrmann M. ‘What’ is happening in the dorsal visual pathway. Trends Cogn Sci. 2016;20(10):773-784. doi.org/10.1016/j.tics.2016.08.003
Pellicano E, Gibson LY Investigating the functional integrity of the dorsal visual pathway in autism and dyslexia. Neuropsychologia. 2008;46(10):2593-2596. doi.org/10.1016/j.neuropsychologia.2008.04.008
Siegel M, Donner TH, Oostenveld R, Fries P, Engel AK. Neuronal synchronization along the dorsal visual pathway reflects the focus of spatial attention. Neuron. 2008;60(4):709-719. doi.org/10.1016/j.neuron.2008.09.010
Kanwisher N. Functional specificity in the human brain: A window into the functional architecture of the mind. Proc Natl Acad Sci U S A. 2010;107(25):11163-11170. doi:10.1073/pnas.1005062107
Schwarzlose RF, Baker CI, Kanwisher N. Separate face and body selectivity on the fusiform gyrus. J Neurosci. 2005;25(47):11055-11059. doi:10.1523/JNEUROSCI.2621-05.2005
Kanwisher N, Dilks D. The Functional Organization of the Ventral Visual Pathway in Humans. In: Chalupa L, Werner J, eds. The New Visual Neurosciences. Cambridge, Mass: MIT Press; 2013. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.644.5503&rep=rep1&type=pdf. Accessed January 27, 2019.
Ishai A, Ungerleider LG, Martin A, Schouten JL, Haxby JV. Distributed representation of objects in the human ventral visual pathway. Proc Natl Acad Sci U S A. 1999;96(16):9379-9384. doi.org/10.1073/pnas.96.16.9379
Perry C, Fallah, M. Hierarchy of visual processing in ventral and dorsal streams. In: Feature integration and object representations along the dorsal stream visual hierarchy. Front Comput Neurosci. 2014;8(84). doi:10.3389/fncom.2014.00084
Rosazza C, Cai Q, Minati L, Paulignan Y, Nazir TA. Early involvement of dorsal and ventral pathways in visual word recognition: An ERP study. Brain Res. 2009;1272:32-44. doi:10.1016/j.brainres.2009.03.033
Zachariou V, Nikas CV, Safiullah ZN, Gotts SJ, Ungerleider LG. Spatial mechanisms within the dorsal visual pathway contribute to the configural processing of faces. Cereb Cortex. 2017;27(8):4124-4138. doi:10.1093/cercor/bhw224
Vogel AC, Miezin FM, Petersen SE, Schlaggar BL. The putative visual word form area is functionally connected to the dorsal attention network. Cereb Cortex. 2012;22(3):537-549. doi.org/10.1093/cercor/bhr100
Vogel A, Petersen S, Schlaggar B. The VWFA: it's not just for words anymore. Front Hum Neurosci. 2014;8:88. doi:10.3389/fnhum.2014.00088
Sale A, Putignano E, Cancedda L, et al. Enriched environment and acceleration of visual system development. Neuropharmacology. 2004;47(5):649-660. doi.org/10.1016/j.neuropharm.2004.07.008
Iacoboni M. Neurobiology of imitation. Curr Opin Neurobiol. 2009;19(6):661-665. doi.org/10.1016/j.conb.2009.09.008
Steimke R, Nomi JS, Calhoun VD et al. Salience network dynamics underlying successful resistance of temptation. Soc Cogn Affect Neurosci. 2017;12(12):1928-1939. doi.org/10.1093/scan/nsx123
Bilevicius E, Kolesar TA, Smith SD, Trapnell PD, Kornelsen J. Trait emotional empathy and resting state functional connectivity in default mode, salience, and central executive networks. Brain Sci. 2018; 8(7):128. https://doi.org/10.3390/brainsci8070128
Zhang M, Savill N, Margulies DS. et al. Distinct individual differences in default mode network connectivity relate to off-task thought and text memory during reading. Sci Rep. 2019;9:16220. doi.org/10.1038/s41598-019-52674-9
Nebel MB, Eloyan A, Nettles CA. Intrinsic visual-motor synchrony correlates with social deficits in autism. Biol Psychiatry. 2016;79(8):633-641. doi:10.1016/j.biopsych.2015.08.029
Keown CL, Shih P, Nair A, Peterson N, Mulvey ME, Müller R-A. Local functional overconnectivity in posterior brain regions is associated with symptom severity in autism spectrum disorders. Cell Rep. 2013;5(3):567-572. doi.org/10.1016/j.celrep.2013.10.003
Chen CP, Keown CL, Jahedi A, et al. Diagnostic classification of intrinsic functional connectivity highlights somatosensory, default mode, and visual regions in autism, Neuroimage Clin. 2015;8:238-245. doi.org/10.1016/j.nicl.2015.04.002
van de Ven V, Jagiela AR, Oertel-Knöchel V, Linden DEJ. Reduced intrinsic visual cortical connectivity is associated with impaired perceptual closure in schizophrenia. Neuroimage Clin. 2017;15:45-52. doi.org/10.1016/j.nicl.2017.04.012
Frezzotti P, Giorgio A, Motolese I. Structural and functional brain changes beyond visual system in patients with advanced glaucoma. PLoS One. 2014. doi.org/10.1371/journal.pone.0105931
Hardesty L. Explained: Neural networks. MIT News website. https://news.mit.edu/2017/explained-neural-networks-deep-learning-0414. Published April 14, 2017. Accessed January 21, 2021
Yang YL, Deng HX, Xing GY, Xia XL, Li HF. Brain functional network connectivity based on a visual task: visual information processing-related brain regions are significantly activated in the task state. Neural Regen Res. 2015;10(2):298-307. doi:10.4103/1673-5374.152386
Bassett DS, Bullmore E. Small-World Brain Networks. Neuroscientist. 2006;12(6):512-523. doi:10.1177/1073858406293182
Eickenberg M, Gramfort A, Varoquaux G, Thirion B. Seeing it all: Convolutional network layers map the function of the human visual system. Neuroimage. 2017;152:184-194. doi.org/10.1016/j.neuroimage.2016.10.001
George D, Hawkins J. A hierarchical Bayesian model of invariant pattern recognition in the visual cortex. Proc Int Jt Conf Neural Netw. 2005;3:1812-1817. doi:10.1109/IJCNN.2005.1556155