Recently, there exists an extensive development in the field of on-board systems of smart vehicles. The system is usually called Driver Support System and it is designed to help a human driver in vehicle guidance by monitoring current traffic situation and providing him with a valuable information source. It is supposed to help a driver in avoiding the potentially risky situations within today heavy traffic. Because of vital role of visual information for the human driver, the traffic situation should be interpreted by the computer vision methods. The recognition of traffic devices like road signs or traffic lights is necessary for such interpretation. We present here the Road Sign Recognition System, which is designed to detect and classify ideogram-based road signs in images of traffic scenes acquired form the moving car. The system uses promising edge-based detection method and tree structure of statistical classifiers employing the apriori information. The localization of the area of interest in the image abridges considerably time needed for the sign detection.

The road sign recognition system analyses the traffic scene acquired from a camera mounted in a moving car for the presence of the road signs. The result is the list of signs in the current view which may serve as a valuable source of information for the human driver. Presently, there exist several groups concerned with the road sign recognition in the world. We are presenting here an unique problem solution which is general enough to cover many road sign types and effective for the real-time implementation. In the following paragraphs we are going to introduce the road sign recognition in the context of Intelligent Transportation System (ITS). Then we will describe desired properties of the system Finally the design and implementation details of the Road Sign Recognition System (RS2) are given.

The Road Sign Recognition in the Context of ITS

The present development in the field of traffic engineering is driven by the effort to make transportation systems safer and more efficient. There are two basic parts of the traffic system - vehicles and infrastructure. The increasing of the intelligence of these two subsystems leads to different results. The intelligent infrastructure improve the efficiency of the whole traffic system at the higher level and may subsequently increase the safety. On the other hand, the intelligent vehicles increase the elementary safety at the lowest level and so may improve the overall system quality.

The progress in the field of computer hardware, robotics and artificial intelligence has enabled the developers to design first prototypes of intelligent vehicles. There have been analyzed various partial technologies like anti-collision systems, automated cruise control or driver warning systems. As the amount of such technologies has been growing it became apparent that there must be some high-level system integrating them into one cooperative entity which could offer the human driver useful information and abilities. This concept is called a smart car. Another term often used for the intelligent complement of vehicle mechanics is the Driver Support System (DSS).

The DSS is intended primarily for the improving of the road traffic safety and vehicle operation efficiency. To achieve this aim methods of artificial intelligence are used. The vehicle should know its actual state by observing its surroundings and reason about the objects it senses there. As the vision is the most important sense for the human driver, the DSS is considered to use the computer vision methods [Nagel]. . The view of the traffic scene in front of the car contains the most of information necessary for the successful driving. There exist even other useful sensors like distance meassures, too.

There are many various objects the DSS should be able of recognize in the traffic scene images like other vehicles, pedestrians, traffic devices, intersections etc. The main goal of the visual subsystem is to understand the current traffic scene properly. By understanding we mean the process of building of dynamic models for the environment the vehicle is moving through.

A short note should be made about the DSS implementation. It is consedered to be an useful idea to design the DSS as the agent process endowed by the particular goal. The DSS will plan actions in order to achieve that goal and spawn corresponding subagents to solve subproblems like obstacle detection or road sign recognition. It is obvious that parallel machines are the most convenient for such system implementation.

Road Signs as the Object of Recognition

The traffic devices like road signs, lane markings, traffic lights are the valuable source of information about the current traffic situation. The RS2 is concerned with the recognition of ideogram-based road signs (we use the term signs further on). As the traffic devices have evolved for several decades a lot of experience with human cognition and psychology has been embedded in their design. There are three basic features used for the sign meaning expression - board shape, colour combination and the ideogram. While the coarse sign meaning (e.g. warning or prohibition) is given together by the shape and the colours the exact type is specified by ideogram.

The road signs may offer many disparate information to the smart vehicle. These are primarily traffic streams priorities, speed limits, prohibitions or warnings. The DSS should use them for the anticipation of further traffic situation. The knowledge about the road signs placement in the environment may be useful also for the navigation because it may complement the digital map and allow the car to precise its actual position on the road. This application of the road sign recognition is called geomodeling and is discussed for example in [Lalonde]. The advantage in comparison to the differential GPS systems is the absence of special equipment. It rather uses the natural landmarks to identify particular place on the road.

There exist several properties the road sign recognition system should satisfy. It should operate under various lightning conditions including twilight and the darkness (with appropriate lighting). The system should not miss important road signs or to misclassify them. As a part of vehicle control system, it must work in the real-time. Furthermore, it should be easily extensible for new road sign types or variants (the real road signs often differe from international standards and appear in many alternations).

The Road Sign Recognition System

Now, let us discuss the Road Sign Recognition System (RS2) developed by the Faculty of Transportation Sciences and Faculty of Electrical Engineering of Czech Technical University in Prague from 1995. The system takes the traffic scene image acquired from the moving car and returns the list of road signs recognized there. Considering the necessity for the general solution it is a good idea to separate the detection and classification steps apart. The advantage the more general system design and its flexibility in choosing particular algorithms for both steps. It is common to use the colour segmentation method for the sign detection methods in many road sign recognition systems (Priese,Ritter ... ). RS2 uses, in contrast, the edge based detection approach. The idea is that edges (sharp brightness transitions) hold the most information necessary for the extraction of regular shapes like road sign boards [Marr]. The detection algorithm searches the input image for basic primitives like line segments and corners forming together the desired shape model.

The list of shapes found in the image (called candidate regions list) is then delivered to the road sign classifier. Its task is to label those regions either with particular road sign codes or to reject some of them as void ones. While the shape feature played the important role in the sign detection, the classification relies mostly on the colour combination and the ideogram data. The task is, in principal, that of statistical character - there are many various road sign types and we decide for the one that seems to be the most probable one. To compare various images we use numerical characteristics called features which have been chosen to be close enough for the similar signs and different for other. The apriori information about signs is used to divide the large sign groups into smaller ones and solve these more efficiently. Presently, we experiment with the group of some 40 road sign types.

Similarly to all presented algorithms, the RS2 spends the most of time in the detection phase. Therefore, it is convenient to reduce the space searched for the signs to the relevant traffic scene area only. This is the idea of the determination of the area of interest. Because of well defined places the road sign may occupy in the scene, the area of interest is based on the actual road shape in the traffic scene. The algorithm used for this purpose looks the consistent image area with the road-like surface texture.

The RS2 Implementation and Experimental Results

The RS2 has been implemented on the Win32 workstations to examine its behaviour and to learn the system on the real data. Then we have been moving it to Texas Instruments TMS320C80 digital signal processor. Presently, the detection algorithm is running there and offers very encouraging results (processing time about xxx with high recognition rate - xxx ). The main advantage is that it is able of operation even in strongly unfavourable illumination conditions and is not dependent on colour information at all.

It has been shown experimentally that the colour information is not necessary even for the separation of various road sign groups by the statistical classifier. Therefore, the whole system may use colour but does not need it for its operation. The classifier processes one candidate region in the time of tens of miliseconds on ordinary PC. The time requirements of the area of interest are very similar to that of the classifier. The classification algorithm and the area of interest searching has been already implemented in ANSI C to work on the DSP processor too. We are planning the port to the promising C60 platform soon.

To make the whole system even more reliable it would be useful to employ the contextual information in the process of sign recognition. These may be supplied by the higher levels of DSS and could include road signs detected in the past (that are likely to appear in following images). The model of the road the car is moving on may precise the groups of sign to be searched for and also those that cannot occur there at all. It is obvious that the cooperation with other DSS subsystems may further increase the overall road sign recognition system robustness.

Last modified: Wed Feb 24 15:15:42 EST 1999