Road Sign Detection and Recognition Using Perceptual Grouping

This project presents a new method for detecting and recognizing road signs on the basis of their shape. The detection step is based on a geometrical analysis of the edges and groups extracted from an intensity image of the sign obtained from an arbitrary viewpoint. It is able to identify triangular-shaped and rectangular-shaped road signs under different viewing conditions. After, the recognition step validates and classifies the detected road sign on the basis of the shape of its inner symbols to improve the reliability and the robustness of the system. The analysis and modeling phases are based on a step of preprocessing, where each road sign model is described by a minimum set of interest points invariant in space. These interest points are obtained by (i) approximating the outer and inner road sign contours using straight line segments and circular arcs and (ii) extracting all junctions points between pairs of lines, arcs and line-arc. Different images have been successfully tested with this method. 

Introduction

 The ability to recognize objects in cluttered scenes is essential to the functionality of a partly on totally autonomous mobile system. In most techniques proposed in literature, the road signs present no occlusion. Also, no such method is able to verify recognition hypotheses under the various possible imaging conditions [1] [2]. For this purpose, the proposed approach is based on a correspondence method known as "Geometric hashing" [3]. Under this approach, several matching features are possible: points, lines and curves. In the present work, we developed a system based on point and line matching techniques to recognize road signs under arbitrary views from intensity images.

 Road signs representation

 Before recognition operations, a preprocessing of one or several models is executed. During this step, the information of each model is indexed in the reference table using a minimal number of relative features which are (i) invariants with respect to the transforms (rotation, translation and scaling) between model and scene and (ii) which make it possible to deal with occlusion. This modelling phase is realized using the same algorithm as the one used when processing a target image. An image acquisition of road signs have been realized in laboratory with an uniform background. The obtained models are made up of straight line segments and circular arcs. To facilitate the computation, a circular arc is always modelled as two straight line segments connecting the first, middle and endpoint of the arc. With this approach, the processing is uniform for any road signs type.

 Road signs detection

 We exploit organizational principles to extract the 2D structural information of the road sign contours. The perceptual grouping extraction is realized using five processing steps:

 a) preprocessing of an intensity image to extract the edges of the frame and inners symbols,

 b) linking process to build connected contours,

 c) segmentation of these contours into straight line segments and circular arcs,

 d) extraction of groups of connected segments between any pair of straight line segments and circular arcs,

 e) identification of closed contours.

 With closed contours and groups of junctions between straight line segments and circular arcs, it is possible to detect road signs in complex scene.

 Interest points considered include all junction points connecting lines and arc segments on closed contours

 Recognition

 During the recognition phase, a voting procedure is executed using the model reference table to determine the correspondence between three interest feature points of the model and those in the segmented intensity image of the scene. For this project, the model objects and the scene are described by sets of interest points, which represent the corners, points of sharp concavities and convexities. 

The notion of affine coordinates is used to compute the invariant of a point V given three non-collinear points (A,B,C). The point V is expressed as:

The affine transform approach may allow recognition of occluded objects since the point coordinates of an occluded object in the scene will have a partial overlap with coordinates of the stored model. This is true if both objects are represented using the same triplet of points as basis. However, this dependence of the representation on a specific basis triple may preclude recognition when at least one of the basis points is occluded. Hence, we represent the object points by their coordinates in all possible affine basis triplets in order to enable recognition of the objects with partial occlusion.

 Linking interest points

 This step connects all extracted interest points on a contour to form a closed polygon. This polygon is extracted to make possible the verification of the recognition. Two interest points will be linked if both are on the same segment. Also, a post-processing step was added to find missing junctions in order to complete a closed polygon.

 Determination of the pose

 The affine transform is also used for the determination of the pose, that is the position and the orientation of the road sign with respect to the camera. The precision of the computed pose depends on the image quality (model or scene). However, the pose has been computed with success for different road signs.

 Acknowledgments

 This project is realized in collaboration with Centre de Recherche Informatique de Montréal (CRIM).

 References

 [1] G. Piccioli, E. DeMicheli and M. Campani, "A Robust Method for Road Sign Detection and Recognition", Lecture Notes in Computer Science, Vol. 800, 1994.

 [2] M. Betke and N.C. Makris, "Fast Object Recognition in Noisy Images Using Simulated Annealing", Proc. of ICCV, 1995.

 [3] Y. Lamdam and H.J. Wolfson, "Geometric Hashing: a General and efficient Model-Based Recognition Scheme", Proc. of ICCV, Dec. 1988.

 [4] J.L. Arseneault, R. Bergevin and D. Laurendeau, "Grouping of Straight Line Segments and Circular Arcs for Scene Analysis", Vision Interface 94, Banff Alberta, May 1994.