3º Step

Multi target tracking (mtt) by Jorge Almeida

This algorithm consists in a global object tracker from 2D planar scans, primarily from laser scanners.

It's very good because it brings a lot of good advantages to my ideas, if one object/pedestrian is tracked, it's tagged a number and maintains it until the LASER cannot see the object any more. Using this method the pedestrian detection algorithm only needs to verify that object/pedestrian one time since the LASER started to see it, because if the object is a pedestrian, it can be "excluded" from the image, because we already know it's a pedestrian.

method:

Laser scan with mtt contribution

2º Step

Experiment the reduction in time, manual recreation of the laser purpose

To see if the LIDAR data is going to make a difference in reducing time, a simple test was made. Manual cutted images, i.e instead of giving the algorithm the entire image (640x480), only give the section of the image that has something that the LASER can detect (excluding big objects like cars, or very small objects like traffic signs [objects that the LASER only detects with one point] ), for example:

Entire image 

time to process = 82s

  ++      time to process = 0.67s

only image that was                                                               

detected pedestrians                                               

                   

The initial algorithm was modified, so that it continues to detect pedestrians even in small images. 

To speed the process, when the small images are "given" to the algorithm, the only response/publish of the algorithm is if the image contains a pedestrian(s) or not.

The same group of pedestrians were detected in the two cases.

1º Step

Comprehend last year developed pedestrian detection algorithm 

After 2 (almost 3) weeks passed, I was still struggling to run the algorithm, tried everything (even contacting the maker of the algorithm, Pedro Silva), but nothing happened, only driving me crazy.

Arranged a meeting with Pedro Silva, so that he could help me understand his algorithm and together find the problem. Two hours passed and starting to get late, he reminded me that the only thing different between my laptop and his, was that he had Ubuntu 64-bit version install and I had Ubuntu 32-bit version install. 

Than I came home and after installing Ubuntu 64-bit version, I tried to run the algorithm again and finally it worked, so here's my thanks to Pedro Silva :), 

Now, talking about the algorithm:
 
- Subscribes an image publish in ROS environment;
- Processes an 640x480 image in 70/80 sec (in my laptop);

before

after

Main Tasks

1.  Study ATLASCAR vision systems and existing lasers

Comprehend all available sensors, their localization, and all software tools who are available to be data access (ROS and already existing software modules).

2. State of the art in laser-vision pedestrian detection

Search for sensors and algorithms used, as historical pedestrian detection systems and academic/comercial solutions available. 

3. Re-parametrize last year developed pedestrian detection algorithm

Comprehend how the algorithm works and run it; modify the execution parameters so that it could be used in different conditions, turning the result more efficient and versatile.   

4. Develop application for vision and laser sensor calibration in ATLASCAR.

To be able to establish a relation through an image pixels and the correct region of targets detected by a LIDAR system, it's necessary to make a geometrics transformation between the two systems and their referentials. Saying this, an application should be created, that allows the 2D image pixels being calibrated with LIDAR data in a 3D context.

5. Combine LIDAR processed data with camera images applying the pedestrian detection algorithm "actively", in a created application.
   

This last stage is one of the main tasks and consists in merging LIDAR data with the camera image, so that it could test/verify if the object in question is definitely a pedestrian.

Main goal

Pedestrian detection algorithm adaption.

Software development for manual/semi-manual calibration between a camera and the LIDAR measuring systems.

Development of an application for active detection of pedestrians in an image using integrated forms to combine LIDAR and visual data.

My name is Rui Azevedo

I'm studying Mechanical Engineering in University of Aveiro, this is my last year ( only if the thesis work at the beginner of the summer is done :D, cross fingers everyone )