In this news, we discuss the Intelligent cameras that can learn by viewing.
Smart cameras could go one step further thanks to a research collaboration between the universities of Bristol and Manchester, which have developed cameras capable of learning and understanding what they see.
Roboticists and artificial intelligence (AI) researchers know there is a problem with the way today’s systems perceive and treat the world. Currently, they still combine sensors, such as digital cameras designed to record images, with computing devices such as graphics processing units (GPUs) designed to speed up graphics for video games.
This means that AI systems only perceive the world after recording and transmitting visual information between sensors and processors. But a lot of the things that can be seen are often irrelevant to the task at hand, such as the detail of leaves on trees by the side of the road as an autonomous car passes by. However, for now, all of this information is captured by sensors in meticulous detail and sent out, clogging the system with irrelevant data, consuming power and taking processing time. A different approach is needed to enable an effective view of intelligent machines.
Two articles from the Bristol and Manchester collaboration showed how sensing and learning can be combined to create new cameras for AI systems.
Walterio Mayol-Cuevas, Professor of Robotics, Computer Vision and Mobile Systems at the University of Bristol and Principal Investigator, commented: “To create effective perceptual systems, we need to push the boundaries beyond the paths we have followed. until now.
“We can take inspiration from the way natural systems process the visual world – we don’t perceive everything – our eyes and our brain work together to make sense of the world and in some cases the eyes themselves perform processing. to help the brain shrink what is irrelevant. “
This is evidenced by the way the frog’s eye has detectors that spot fly-like objects, directly at the point where the images are picked up.
The articles, one edited by Dr Laurie Bose and the other by Yanan Liu in Bristol, revealed two improvements towards this goal. By implementing convolutional neural networks (CNN), a form of AI algorithm allowing visual comprehension, directly on the image plane. The CNNs the team developed can categorize images thousands of times per second, without ever having to save those images or send them through the processing pipeline. The researchers considered demonstrations of classification of handwritten numbers, hand gestures, and even the classification of plankton.
Research suggests a future with dedicated smart AI cameras – visual systems that can simply send high-level information to the rest of the system, such as the type of object or event happening in front of the camera. This approach would make the systems much more efficient and secure because no images need to be saved.
The work was made possible by the SCAMP architecture developed by Piotr Dudek, Professor of Circuits and Systems and Principal Investigator at the University of Manchester, and his team. The SCAMP is a camera processor chip that the team describes as a Pixel Processor Array (PPA). A PPA has a processor built into each pixel that can communicate with each other for processing in a truly parallel form. This is ideal for CNNs and vision algorithms.
“The integration of pixel-level sensing, processing and memory not only enables high-performance, low-latency systems, but also promises low-power and high-efficiency hardware,” Prof Dudek said.
“What’s so exciting about these cameras isn’t just the new machine learning capability, but also the speed at which they operate and the lightweight setup,” said Tom Richardson, Senior Lecturer in Mechanics flight to Bristol University.
“They are absolutely perfect for high speed, very agile aerial platforms that can literally learn on the fly!
Smart cameras that can learn by watching
Scientists Develop Cameras That Understand What They See