AI can now 'see' optical illusions. What does it tell us about our own brains? - BBC
Our eyes can frequently play tricks on us, but scientists have discovered that some artificial intelligence can fall for the same illusions. And it is changing what we know about our brains. When we look up at the Moon, it seems larger when it is close to the horizon compared to when it is higher up in the sky even though its size, and the distance between the Earth and the Moon, remains much the same during the course of a night. Optical illusions such as these show that we don't always perceive reality the way we should. They are often considered to be mistakes made by our visual system. But illusions also reveal the clever shortcuts our brains use to extract the most important details of our surroundings. In truth, our brains only accept a sip of the world around us – it would be too much to process every detail of our busy visual environments, so instead they pick out only the details we need. But what happens when you give a synthetic mind – a machine vision system powered by artificial intelligence – an optical illusion? These systems excel at detail. They are designed to spot the patterns and the blemishes we do not. It is why they have been so effective at spotting the early signs of diseases in medical scans, for example. Yet, some deep neural networks (DNNs) – the technology that underpins many of today's advanced AI algorithms – are susceptible to some of the same visual tricks as us humans. And it is providing new insights into how our own brains work. "Using DNNs in illusion research allows us to simulate and analyse how the brain processes information and generates illusions," says Eiji Watanabe, an associate professor of neurophysiology at the National Institute for Basic Biology in Japan. "Conducting experimental manipulations on the human brain raises serious ethical concerns but no such restrictions apply to artificial models." Although there are many theories about why we perceive different optical illusions, in most cases there still isn't a decisive explanation. Studying people who don't experience optical illusions has provided some clues. In one case, a person who lost his sight as a young child and had it restored in his 40s was not tricked by illusions involving shapes, such as the famous Kanizsa square, where four strategically positioned circular fragments create illusory contours that evoke a square. He could, however, perceive illusions of motion such as the barber pole, where stripes appear to move upwards even though the pole is simply turning on a vertical axis. Studies on cases such as this seem to suggest that our ability to perceive motion is more robust to sensory deprivation compared to making sense of shapes. This could be because we learn to process motion earlier on as infants. Alternatively, the way we process shapes could simply be more plastic and primed to recognise shapes we are exposed to the most. Brain-imaging studies using functional magnetic resonance imaging (fMRI) have also provided insight into what parts of the brain are active when we see different illusions and how they interact. Our perception of optical illusions is subjective, however, and can differ between individuals, as illustrated by a photo of a striped dress that went viral online in 2015, where viewers couldn't agree on whether it was blue and black or white and gold. This makes it hard to study them objectively since researchers typically depend on participants describing what they see. Now, AI is offering a new way of understanding what is going on in our brains when we look at optical illusions. Many of the AI algorithms in use today – including chatbots such as ChatGPT – are powered by deep neural networks, which are models made up of artificial neurons that attempt to mimic how our brain processes information. In recent work, Watanabe and his colleagues wanted to see whether a deep neural network could replicate what happens in our own brains when we look at illusions involving motion, such as the rotating snakes illusion. This uses a trippy pattern of colourful circles in a static image but appears to be rotating when we stare at it. Watanabe and his team used a deep neural network called PredNet, which was designed based on a leading theory about how our brains deal with visual information called predictive coding. It suggests that our visual system doesn't just passively process features in our surroundings when we look around. Instead, it first predicts what it expects to see by drawing on past experience before it processes discrepancies in the input from our eyes. This allows us to see more quickly. Similarly, PredNet predicts future frames in a video based on knowledge it acquires from previous frames it has seen. For his experiment, Watanabe trained the AI using videos of natural landscapes captured by head-mounted cameras which were similar to what humans might see when looking around. The system was never exposed to any optical illusions. By showing it certain frames it hadn't seen, it was designed to make its prediction match the frame as closely as possible. "After processing around a million frames, PredNet learns certain rules of the visual world," says Watanabe. "It extracts and remembers the essential rules and among these, it may have also learned characteristics of moving objects." Watanabe then presented the AI model with a few variations of the rotating snakes illusion and an altered version that human brains are not fooled by, and so perceive as static. He found that the AI was tricked by the same images as humans. Watanabe thinks it supports the theory that our brains use predictive coding. In this case there are aspects of the images that are indicative of moving objects that trigger our brain's prediction system into assuming the multicoloured snakes are in motion. "I think PredNet's perception is similar to human perception," he says. However, Watanabe and his team also found differences between how the AI and humans perceive the illusion. When we fix our gaze on one of the rotating circles, for example, it seems to stop turning whereas the other discs in our peripheral vision continue to spin. PredNet, however, always perceives all the circles moving at the same time. "This is likely because PredNet lacks an attention mechanism," says Watanabe. This means it is unable to focus on a specific spot on the image, but processes it in its entirety. Although AI systems and robots may be able to mimic certain aspects of our visual system, they are still far off from being able to see the world as we do. So far, there is no deep neural network that can experience all the illusions that humans do, says Watanabe. In some ways this shouldn't surprise us. "ChatGPT, for example, might seem to converse like a human but its underlying DNN functions very differently from the human brain," says Watanabe. "The key similarity is that both systems use [some type of] neurons but how they are structured and applied can be vastly different." More like this: • What puzzles reveal about our own minds • We built a nasty game to test AI's ability to apologise • What happens when an AI takes an inkblot test? Some researchers are trying to combine AI with the weirdness of quantum mechanics to better simulate how humans perceive certain illusions. Previously, researchers have used concepts from quantum mechanics to explain our perception of the Necker cube, a famous ambiguous figure illusion where a cube appears to randomly switch between two different orientations. Classical theories of physics would predict that the cube should be perceived in one way or another. But in quantum mechanics, the cube could be in two states at once until our brain chose to perceive one. Think of the famous Schrödinger's cat thought experiment, where a cat trapped in a box with a mechanism that could kill it is both dead and alive until someone looks inside. Inspired by this work, Ivan Maksymov, a research fellow at Charles Sturt University's Artificial Intelligence and Cyber Futures Institute in Bathurst, Australia, developed a model that combined quantum physics with AI to see if it could simulate the way we perceive the Necker cube and a similar illusion called Rubin vase, where we see either a vase or two faces in profile. He designed a deep neural network that processes information using a phenomenon called quantum tunnelling. The system was then trained to recognise the two illusions. When one of the illusions was input into the system, it would generate one of the two interpretations. Maksymov found that the AI would regularly switch between each of them over time – much as humans do. The time intervals of these switches were similar too. "It's quite close to what people see in tests," he says. Maksymov doesn't think this suggests that our brain has quantum properties, although it is an active field of research. Instead, he thinks that it shows that certain aspects of human thought, such as how we make decisions, can be better modelled by using quantum theory, the basis of a field called quantum cognition. With the illusions, our brain is choosing one version or the other, for example. Such a system could also be used to simulate how our visual perception may change in space under different gravitational conditions. Researchers have previously studied how astronauts who have spent time on the International Space Station (ISS) experience changes in the way they see optical illusions. They found that astronauts will see illusions such as the Necker Cube more often in one of its two perspectives when on Earth. But after three months in orbit, they saw each one perspective equally as often. Scientists believe this may occur because some of how we judge depth relies upon gravity. When free-floating in orbit, an astronaut cannot estimate distance based on how high their eyes are from the ground when sitting or standing upright. "While it's a narrow field of research, it's quite important because humans want to go to space," says Maksymov. And with all the wonders the Universe holds, those space travellers will definitely want to know they can believe their eyes. -- For more technology news and insights, sign up to our Tech Decoded newsletter, while The Essential List delivers a handpicked selection of features and insights to your inbox twice a week. For more science, technology, environment and health stories from the BBC, follow us on Facebook and Instagram.