Road Sign Recognition Survey

What is the Road Sign Recognition ?

The Road Sign Recognition (RSR) is a field of applied computer vision research concerned with the automatical detection and classification of traffic signs in traffic scene images acquired from a moving car. The result of RSR research effort will be the subsystem of Driver Support System (DSS) mentioned above. The aim is to provide DSS with the ability to understand its neighborhood environment and so permit advanced driver support such as collision prediction and avoidance.

Driving is a task based fully on visual information processing. The road signs and traffic signals define a visual language interpreted by drivers. Road signs carry many information necessary for successful driving - they describe current traffic situation, define right-of-way, prohibit or permit certain directions, warn about risky factors etc. Road signs also help drivers with navigation.

Two basic applications of RSR are under consideration in the research community - driver's aid (DSS) and automated surveillance of road traffic devices. It is desirable to design smart car control systems in such a way which will allow evolution of fully autonomous vehicles in the future. The RSR system is also being considered as the valuable complement of the GPS-based navigation system. The dynamical environmental map may be enriched by road sign types and positions (acquired by RSR) and so help with the precision of current vehicle position.

The Road Sign Recognition Problem

Let us firstly define the road sign recongition problem. At the first sight the goal is well defined and seem to be quite a simple one : road signs occur in standartized positions in traffic scenes, their shapes, colours and pictograms are known (because of international standards) ...

To see the problem in its whole complexness we must add aditional features that influence the recognition system design and performance : Road signs are acquired from car moving on the (often uneven) road surface by considerable speed. The traffic scene images then often suffer from vibrations; colour information is affected by varying illumination. Road signs are frequently occluded partially by other vehicles. Many objects are present in traffic scenes which make the sign detection hard (pedestrians, other vehicles, buildings and billboards may confuse the detection system by patterns similar to that of road signs). Road signs exist in hundreds of variants (see e.g. collections of road sign local abberations at often different from legally defined standard. Furthermore, the algorithms must be suitable for the real-time implementation. The hardware platform must be able to process huge amount of information in video data stream.

From above problem definition follows, that to design a successful road sign recognition engine one must provide a large number of sign examples to make the system responding correctly to real traffic scenes. This so-called statistical approach to machine learning requires acquiring of large image databases what is both expensive and time-consuming task.

Present State of the Road Sign Recognition Research

Up to now, many algorithms for the road sign detection and classification have been introduced. Road signs are often used as convenient real-world objects suitable for algorithm testing purposes. There may be found papers focusing on the presentation of successful recognition of particular road sign by some special algorithm in literature. These papers are valuable source of information about different recognition approaches.

road sign recognition articles :

There also exist research groups developing the working RSR prototypes in the world. As the RSR research has started in Japan in 1984, there exist several laboratories working on the topic further on. Aside from commonly used approaches the use of optical correlators have been also reported by Japanese researchers :

The optical correlation technology has been presented also by P.S.A. Peugeot Citroen and University of Cambridge : We must stress here the most advanced system TSR developed by Image Recognition Laboratory of Koblenz-Landau University. System exploits the advanced colour segmentation method for the sign detection and has already been implemented in parallel real-time environment (PowerPC and Transputers). The system is able to analyze three images per second. The TSR system has been developed in partnership with the Daimler-Benz (now Daimler-Chrysler). Daimler-Chrysler has been supporting the road sign recognition research extensively. Its research group also reported papers concerning colour segmentation, parallel compuation structures design and more. The RSR system being developed by Daimler-Chrysler is designed to use colour information for the sign detection. The classification stage is covered by various neural-network or nearist-neighbor classifiers. The presence of colour is substantial and the system is unable to operate on images with weak or missing colour information. The most important advantage of research groups supported by Daimler-Chrysler to the others is the library of 60000 traffic scene images used for system training and evaluation.

The research group at the University of Genoa, Italy, has published several papers related to recognition of road signs in gray-level images by employing of cross-corelation method. Further papers have described new approach employing colour information. The necessity to combine the colour segmentation results with further shape analysis has been proposed.

Successful road sign recognition algorithm implementation based on TMS320C40 DSP processor has also been published in :

The research group at the Faculty of Transportation Sciences, CTU Prague has been developing the Road Sign Recognition System (RS2) since 1995. In the first project phase various algorithms have been tested and a general framework for the general road signs recognition has emerged. Now, the algorithms are being ported to low level form (C language) to fit real-time hardware. The RS2 uses local orientations of image edges to find geometrical shapes which could correspond to the road signs. The detection algorithm has been tested successfully in the sequential form by Zikmund. It has been developed further by Libal into parallel environment of TMS320C80 DSP processor (Texas Instruments) and is capable of the real-time operation. The detection phase does not require colour images and works even on badly illuminated scenes. The subsequent classification algorithm is being developed by Paclik in the form of combination of statistical kernel classifiers. The area of interest is searched where road signs should occur most likely by employing algorithms of Kovar.

Advantages and Disadvantages of Computer Vision Approach

Employing computer vision technology into smart vehicle design calls for consideration of all its advantages and disadvantages. Firstly, vision subsystem incorporated into the DSS may exploit all the information processed by human drivers without any requirements for new traffic infrastructure devices (a very hard and expensive task). Smart cars equipped with vision based systems will be able to adapt themselves to operate in different countries (with often quite dissimilar traffic devices).

As the integration of various technologies in the field of traffic engineering has been introduced (ITS) the convenience of computer vision usage has become more obvious. We may observe this trend e.g. in proceedings of annual IEEE International Conference on Intelligent Vehicles (IVS). More then 50% of papers are focused on Image Processing and Computer Vision methods [2].

Obviously, there exist even disadvantages of the vision-based approach. Smart vehicles will operate in real traffic conditions on the road. So, the algorithms must be robust enough to give good results even under adverse illumination and weather conditions. Although this system property may seem to be solved easily it is the real challenge for the algorithm developers. For example Fridtjof Stein, main project manager of Cleopatra project (Clusters of embedded parallel time-critical applications) said [3] that "reliable optical detection is the biggest hurdle the project must overcome".

There cannot be assured absolute system reliability and the system will not be "fail-safe" because of the definition of individual transportation system. The aim is to provide a level of safety similar to or higher than that of human drivers.

From experiments follows, that 60 percent of crashes at intersections and 30 percent of head-on collisions could be avoided if the driver had an additional half-second to react. About 75 percent of vehicular crashes are caused by inattentive drivers. (cited from [1])

Future trends

We think that the road sign recognition will become a valuable part of upcoming driver support systems soon. According to [1] the first generation of individual driver information and warning systems appear on the market in five to 10 years. We think that Daimler-Chrysler will capitalize its long-termed deposits into RSR research support soon by introducing first systems in about two years.


Further information related to road sign recognition, on-line articles and more may be found at the web page of RS2 system at the Department of Applied Mathematics, Faculty of Transportation Sciences, CTU Prague :

[1] The Intelligent Vehicle Initiative - Advancing Human Centered Smart Vehicles, Cheryl Little, on-line:
[2] Image Processing in ITS, Masayoki Aoki, in proceedings of IEEE International Conference on Intelligent Vehicles, 1998
[3] CLEOPATRRA project page at PAC (Parallel Application Center), on-line:
[4] Computer Vision for Support of Road Vehicle Driver, Nagel, Institut für Algorithmen und Kognitive Systeme, Fakultät für Informatik der Universität Karlsruhe, on-line:

v 1.0
Pavel Paclik
Last modified: Sun May 16 22:20:37 CEST 1999