Research
Projects
i-Caveats-E: Embedded vision system for image features extraction with on-chip energy harvesting ...

i-Caveats-E: Embedded vision system for image features extraction with on-chip energy harvesting and management for unmanned mobile platforms

The inclusion of embedded vision systems in mobile platforms for the intelligent transportation of people and goods would represents an important technological leap forward. It is so also in surveillance applications for security and defense. However, the implementation of a compact autonomous vision system, with low power consumption is not an easy task to accomplish. The visual stimulus contains a great deal of data. In order to process them, a considerable computational effort needs to be put in place. In practical terms, this means to be able to realize several million operations per second. Achieving this performance with a restricted power budget is very difficult. In a conventional image processing chain —where images are captured, then digitized and stored in a memory and then processed— we will most probably arrive to impossible specifications for the sensor, the analog-to-digital converter, processor and memory.
A viable alternative, as has been evidenced in our previous projects, is to take advantage of the inherent parallelism of early vision tasks. In order to do that, part of the low level processing can be conveyed to the focal plane. The distributed implementation of the processing resources implies a reduction on data transfers to and from the memory. Besides, these processing elements can be built with analog and mixed-signal circuit blocks, which can be very efficient. The main problem of this approach is that an ad hoc implementation lacks flexibility to be migrated to other application fields. In addition, computer vision experts and application developers, who work at higher abstraction levels, do not easily handle hardware programming at low level. In order to bridge this gap, an important industrial consortium has been created very recently to generate standards for the hardware acceleration of computer vision and, in general, massive sensory signal processing. OpenVX, for instance, defines, at a layer that is just above the hardware, a set of functions that can be employed by computer vision application developers who also search for a power-optimized implementation.
This project aims to the capitalization of the acquired know-how by developing a library of hardware components and architectures that follow these principles. They have to be compatible with OpenVX descriptions and aimed to reduce power consumption of mainly-software implementations. We will include low and medium-level processing blocks, new sensor abilities, like photon counting and time-of-flight estimation, and aspects more related to the system level, like energy management and interfacing with other signal processing chips. We will explore technological alternatives that may provide a more efficient implementation of OpenVX functions. These modifications will be transparent from the point of view of the application developer and the designer of computer vision algorithms.
In order to demonstrate the validity of the approach we will build a vision system on-a-chip for intelligent transportation and security applications. We will develop demonstrators to properly expose the potential of this approach to embedded vision.

Objectives

The main objectives of this sub-project are:

  1. The management and harvesting of environmental energy within the same silicon substrate where the image is acquired,
  2. The hierarchical image features extraction and 
  3. Iits application for the simultaneous visual tracking and mapping (V-SLAM) in a UAV platform.

Regarding the first objective, we will design an integrated circuit in a standard 0.18 μm CMOS technology capable of both acquiring images and collecting solar energy from the environment. We will first study the most adequate dual photosensing/harvesting structure at device simulation level. Also, a power management unit based on switched capacitor DC/DC converters and a maximum power point tracking unit and storage capacitors will be designed. The final goal will be to come up with a strategy for the optimal switching between the energy harvesting mode and the imaging mode to guarantee self-sustained operation. In this approximation power reduction and low power consumption design techniques will be needed.
The second objective deals with the design of the circuitry components of a feature extractor data path combining the FAST detector and the BRIEF descriptor with the aim of implementing the tracking of moving objects. The FAST-BRIEF combination ensures high speed operation with respect to other combinations of detectors and features descriptors. This combination implies the need of designing circuits in the focal plane for the extraction of the characteristic points of the FAST detector, possibly within the analog domain, and digital circuits
for the BRIEF descriptor and the search of correspondences between characteristic points of two images to perform tracking operations.
These circuits will comply with the OpenVX standard and will be part of a chip including more functionality through the combination of both
low and medium level operators that will be jointly designed by the three partners of the project. The methodology for the design of the
FAST-BRIEF circuitry starts with the performance evaluation using the commercial platform Parallela. The BRIEF descriptor will be then
synthesized in VHDL and finally integrated at chip level. Within this objective we plan to apply as well dynamic voltage scaling (DVS)
techniques taking as a reference the DC/DC converters designed for the energy harvesting task.
Finally, the third objective of this sub-project is related to the integration of the feature detector chip combining low and medium level
operators in a UAV platform to perform visual SLAM at a rate of at least 10 frames per second.