‘MISEL’ launched at USC’s CiTIUS, a European project aimed at creating new autonomous vision devices with built-in Artificial Intelligence
The initiative, coordinated in Spain by the centre’s researcher Víctor Brea and led by the Finnish institute VTT (Technical Research Centre of Finland), is a consortium funded with nearly 5 million euros by the European Union (EU) through the ‘Horizon 2020’ excellence framework programme.
The CiTIUS (Singular Research Centre on Intelligent Technologies of the University of Santiago de Compostela) is taking part as a partner in the MISEL project (Multispectral Intelligent Vision System with Embedded Low-Power Neural Computing), a new European consortium funded under the ‘FET-PROACT – Horizon 2020’ research support programme, aimed at backing cutting-edge proposals that seek a new technological paradigm.
MISEL was created with the purpose of bringing Artificial Intelligence (AI) to edge computing, that is, doing what is necessary to enable decisions made by intelligent systems to be carried out on the devices themselves, without the need for complex external processing systems or cloud computing. The project has received funding of around 4.96 million euros from the European Union (Horizon 2020), of which 653,000 will cover the share corresponding to CiTIUS.
The goal of MISEL is to solve problems of object and action recognition in video, such as anomaly detection (vehicles in pedestrian areas, drones near airports, animals on the roadway, etc.), or distinguishing between birds and UAVs, among other applications. The researchers hope to be able to address this type of problem at the device level itself, a specific system (a multispectral camera) that will be designed and manufactured within the framework of this project.
Three key cogs
“This is an ambitious initiative based on three pillars,” explains CiTIUS researcher Víctor Brea, who is responsible for the Spanish contribution to the project: “firstly at device level, because we are going to design and fully manufacture a specific model of multispectral camera with quantum-dot sensors.” “In addition, this device will also include purpose-specific ferroelectric memories for data storage, likewise created by MISEL’s partners.”
“The second pillar is circuitry,” he continues: “when we capture the images, we will process those signals by means of spatio-temporal neural networks, as a preliminary step to decision-making.” This is where another of MISEL’s singular features lies: “it is important to highlight that our cameras will process raw information, because they will not work with conventional images but with events, or in other words, specific situations that the system will extract automatically.” Brea points out the advantages of events over conventional images: “by working directly with events we achieve a higher dynamic range” (greater ability to capture low-light scenes) “and, above all, lower latency, something crucial to allow processing to take place at higher speed,” he argues.
Finally, the third and last key part of the project is the design of specific spatio-temporal neural networks, adapted to the small space available in the multispectral camera, with limited hardware. “The challenge is to create an integrated circuit with low weight, small size and, above all, low power consumption, below one watt, in order to make the most of the reduced computing capacity,” the researcher concludes.
Víctor Brea also wanted to convey his enthusiasm for the new project: “MISEL is, at the same time, an enormous satisfaction, a major challenge and an opportunity to grow both personally and as a research group, and that changes everything,” he says. “I am especially delighted with this project because of the opportunity to work with VTT and Kovilta Oy, both from Finland, where researchers work with whom I have a friendship forged over more than 20 years of research stays and international conferences.”
For her part, Paula López, deputy director of CiTIUS and Brea’s colleague in the Artificial Vision Group, underlined the importance of the project for the centre, stating that “this project places us in an internationally prominent position in the field of intelligent edge systems, specifically in the design of intelligent technologies embedded in micro- and nanoelectronic systems, in what is known as the ‘Artificial Intelligence of Things’ (AIoT).”
9 partners, 5 countries
In addition to CiTIUS (Spain) and the VTT Technical Research Centre (Finland), the project also involves 7 other institutions: AMO GmbH (Germany), the University of Wuppertal (Germany), the Fraunhofer Center for Applied Nanotechnology (Germany), Kovilta Oy (Finland), Lund University (Sweden), Lodz University of Technology (Poland) and the Laboratoire national de métrologie et d'Essais (France).
The CiTIUS (Singular Research Centre on Intelligent Technologies of the University of Santiago de Compostela) is taking part as a partner in the MISEL project (Multispectral Intelligent Vision System with Embedded Low-Power Neural Computing), a new European consortium funded under the ‘FET-PROACT – Horizon 2020’ research support programme, aimed at backing cutting-edge proposals that seek a new technological paradigm.
MISEL was created with the purpose of bringing Artificial Intelligence (AI) to edge computing, that is, doing what is necessary to enable decisions made by intelligent systems to be carried out on the devices themselves, without the need for complex external processing systems or cloud computing. The project has received funding of around 4.96 million euros from the European Union (Horizon 2020), of which 653,000 will cover the share corresponding to CiTIUS.
The goal of MISEL is to solve problems of object and action recognition in video, such as anomaly detection (vehicles in pedestrian areas, drones near airports, animals on the roadway, etc.), or distinguishing between birds and UAVs, among other applications. The researchers hope to be able to address this type of problem at the device level itself, a specific system (a multispectral camera) that will be designed and manufactured within the framework of this project.
Three key cogs
“This is an ambitious initiative based on three pillars,” explains CiTIUS researcher Víctor Brea, who is responsible for the Spanish contribution to the project: “firstly at device level, because we are going to design and fully manufacture a specific model of multispectral camera with quantum-dot sensors.” “In addition, this device will also include purpose-specific ferroelectric memories for data storage, likewise created by MISEL’s partners.”
“The second pillar is circuitry,” he continues: “when we capture the images, we will process those signals by means of spatio-temporal neural networks, as a preliminary step to decision-making.” This is where another of MISEL’s singular features lies: “it is important to highlight that our cameras will process raw information, because they will not work with conventional images but with events, or in other words, specific situations that the system will extract automatically.” Brea points out the advantages of events over conventional images: “by working directly with events we achieve a higher dynamic range” (greater ability to capture low-light scenes) “and, above all, lower latency, something crucial to allow processing to take place at higher speed,” he argues.
Finally, the third and last key part of the project is the design of specific spatio-temporal neural networks, adapted to the small space available in the multispectral camera, with limited hardware. “The challenge is to create an integrated circuit with low weight, small size and, above all, low power consumption, below one watt, in order to make the most of the reduced computing capacity,” the researcher concludes.
Víctor Brea also wanted to convey his enthusiasm for the new project: “MISEL is, at the same time, an enormous satisfaction, a major challenge and an opportunity to grow both personally and as a research group, and that changes everything,” he says. “I am especially delighted with this project because of the opportunity to work with VTT and Kovilta Oy, both from Finland, where researchers work with whom I have a friendship forged over more than 20 years of research stays and international conferences.”
For her part, Paula López, deputy director of CiTIUS and Brea’s colleague in the Artificial Vision Group, underlined the importance of the project for the centre, stating that “this project places us in an internationally prominent position in the field of intelligent edge systems, specifically in the design of intelligent technologies embedded in micro- and nanoelectronic systems, in what is known as the ‘Artificial Intelligence of Things’ (AIoT).”
9 partners, 5 countries
In addition to CiTIUS (Spain) and the VTT Technical Research Centre (Finland), the project also involves 7 other institutions: AMO GmbH (Germany), the University of Wuppertal (Germany), the Fraunhofer Center for Applied Nanotechnology (Germany), Kovilta Oy (Finland), Lund University (Sweden), Lodz University of Technology (Poland) and the Laboratoire national de métrologie et d'Essais (France).