New algorithms to reduce the amount of processing calculation

Toshiba's newly developed technology has two main characteristics. Firstly, the image processing algorithms have been re-worked, leading to significantly less amount of calculations for image processing.

Figure 1:Spatial restriction of the visual disparity being searched

Toshiba's new algorithms are as follows. Generally, stereo vision aligns and matches images captured by two cameras, one on the left and one on the right, so that they correspond to the same single point. The matching process is usually performed by calculating the degree of similarity to the summation of absolute values of brightness difference in the target region (the search window). Toshiba, in this process, employs a method called "recursive computation of correlation measures" which utilizes calculated results that were obtained in the immediately preceding operation. As a result, a visual disparity map is to be obtained with fewer calculations.

Figure 2:Example of input images and visual disparity map

In addition to the above, Toshiba was able to reduce the amount of calculation by restricting the visual disparity search space for which visual disparity should be calculated. It works this way; the pedestrians and cars to be detected exist on the road's surface, and have a limited height. The road surface is used as the lower limit, and the height of target objects is set to the upper limit, meaning that the searched region is restricted exclusively to the area in between these 2 planar surfaces (Figure 1).

New algorithms implemented to LSI

Another feature to note about Toshiba's new technology is that the developed algorithms have been implemented to Toshiba's 'Visconti' LSI, allowing high-speed processing performance. Implementation to the processor LSI was conducted in the following way.
Stereo image processing usually adjusts the input image so that the search line conforms to the screen's scanning line. This process of adjustment or correction is known as parallelization. And there are processing 4 modules in Visconti; one is a coordinate conversion module, and the remaining three are processing modules. Toshiba has assigned the coordinate conversion module to parallelizationm, and the remaining 3 processing modules to visual disparity calculation with balancing the load of the three modules equally.

Table 1:A comparison of processing times

To evaluate the performance of the LSI, Toshiba created a visual disparity map from the input images of 320 x 240 pixel resolution, with a visual disparity range 18-pixel and a search window 7 x 7 pixel. The result was that the processing speed per frame was 15.47ms, achieving a 25% faster search than a full search