License Plate Detection with OpenCV and my mental map.

Having a mental map or processes will steer us to a direction to explore and find the right path to reach our goal. When I start on this license plate detection project, that’s what I did. I thought to myself, if I was handed an image, how did I locate the license plate and extract the text?

Here are my general mental steps:

1. Recognize the input data is an image.
2. Scan through the image to see all the different shapes, defined by its edges.
3. Given the license plate is a rectangular shape, out all the different shapes from the previous steps, locate the shapes that best match the rectangular shape.
4. Once the rectangular shape is found, the information inside this shape will be the license plate number.

I transfer these steps into computer’s perspective.

I will be using Python and the image below for this project.

Figure 1. Image name : us_license_plate.jpg

1. Recognize the input data is an image.

In order for Python to process the input data accordingly, I will import the proper libraries. I will be using ‘OpenCV (cv2) to read in the image. In addition, I want the standardize the image size to width of 512px with Imutils (I chose 512px because it is the middle ground between image size vs. image detail and Imutils library will automatically adjust its height to match its original ratio).

Figure 2. Code_1: Libraries for image processing.

After reading in the image, I converted it to gray scale. Not only converting to gray scale reduce computation complexity, it is also important for finding contours (later steps) because OpenCV find contours from white connecting objects in black backgrounds.

Figure 3. Code_2: Image data pre-processing.

The image after resized and converted to gray scale:

Figure 4. Image from Figure 1 after pre-processing.

2. Scan through the image to see all the different shapes, defined by its edges.

When we look at an object, our eyes detect the shape of an object by its edges, which has color differences to its background, surrounding, or adjacent objects. So, in order for the computer to outline all the different shapes in an image, we need to apply this concept.

This is an important step. If the computer couldn’t outline the important edges, it might not be able to find the license plate.

I will apply Canny function from OpenCV onto the pre-processed image to outline its edges or color gradient. For more information on Canny function, click here.

Before I apply Canny function, I will first apply smoothing method to the image for noise reduction. Here, I applied bilateral filter method to the image to reduce noise while keeping the edges sharp. For more information about bilateral filtering or different smoothing methods, click here.

Figure 3. Code_3. BilateralFilter and Canny function

Note: the parameters inside theses 2 functions aren’t ‘one-size-fits-all’. It needs to be tuned for your application.

Figure 4. Image after BilateralFilter and Canny function.

3. Given the license plate is a rectangular shape, out all the different shapes from the previous steps, locate the shapes that best match the rectangular shape.

When humans are given an image with a license plate on it, our eyes are able to locate the license plate out of all the others shapes because our prior knowledge told us it is a rectangular shape with connected 4 corners.

To apply this idea to Python, I will first apply OpenCV findContours function onto Figure 4 to find all the closed contours. Then, to visualize them, I applied the drawContours function to draw the contours onto the original image. For more information about findContours and drawContour, click here.

Figure 5. Code_4: findContours and drawContours.

Figure 6. Image after drawing all the contours from Figure 5

As shown in Figure 6, it has many contours and most of them are not in correct shape or doesn’t have the area to be considered a rectangular shape. Thus, I will sort the contours based on its areas and filter the contours with its areas. Lastly, I will use drawContour function again to display the filtered contours.

Figure 7. Code_5: sort and filter contours created from Figure 5

Figure 8. Image with filtered contours

Next, locate the shape that best fit to be a license plate, a rectangular shape. To do so, I will iterate through each of the remaining contours and apply arcLength and approxPolyDP function to approximate closed the contours. Once is found, I use boundingRect function to locate corner points for cropping. For more information about these functions, click here.

Figure 7. Code_6: Finding rectangular contours

Figure 8. Image of ‘lp_bw_crop’ from line 34 of Figure 7

Figure 9. Code_7. Draw ‘lp_contour’ of Figure 7 onto its original image

Figure 10. Image with the license plate contour.

4. Once the rectangular shape is found, the information inside this shape will be the license plate number.

Once it found the correct contour, I need to extract the text from this contour. To do so, I will use Pytesseract. I will also need to install Teseract and direct it to work in conjunction with Pytesseract. To download Teseract for Windows, click here.

Figure 11. Code_8. Import Pytesseract

Use ‘image_to_string’ function to extract text from the contour. Please note, ‘config’ is a varying parameter and might need to be change per application. See here.

Figure 12. Code_8. Extract text from the Figure 8

Figure 13. Final Image with text.

OpenCV and Pytesseract on this project is one of the many methods. But by having roadmap, it will give you better idea of the approach you want to take and how complex you need/want the project to be.

I hope you will find this helpful.

That’s the wrap.