Real-Time License Plate Detection Using YOLOv8 and OCR Model

Project Overview

This project takes license plate detection to the next level with YOLOv8. Yolov8 is a super-fast object detection model. The goal is to detect license plates in images and then extract the text from them using OCR models. It’s perfect for building smart systems, like parking lots or toll booths, that need to recognize vehicles automatically. Whether you're working with images or video streams, it handles both with ease. The project is designed to be fast, accurate, and easy to use. Plus, it’s scalable.

Key Features:

YOLOv8 is fast and precise. It detects plates instantly in real-world conditions.
The project doesn’t just detect plates, it reads them! OCR helps extract the plate number quickly.
With high accuracy, this system minimizes mistakes in license plate recognition, making it dependable.
Built with simplicity in mind, you can easily integrate it into any system or application.

Prerequisites

Before you dive into this fun project, here are a few things you'll need to have ready:

Ensure that Python is installed (recommended to use at least the third version).
For license plate detection, you will need to install YOLOv8 from Ultralytics.
Don’t forget to integrate an OCR library that will help to read the text of the license plate, for example, Tesseract or EasyOCR.
The OpenCV library will be associated with image processing, and it will be helpful to display the results in real-time.
Ensure that you have pip or conda to install the necessary Python packages as the installation process will be easy.
You don’t have to be a programming expert, although any background in Python programming will assist with reading and tweaking the code.
Join RoboFlow and sign up for a free account and organize, label, and annotate your data set. RoboFlow helps in simplifying the preparation of the data for YOLOv8.

Approach

Here in this project, we apply YOLOv8 to detect license plates in the images and video stream. The Yolov8 model is trained for license plate detection. This model guarantees a high level of detection and accuracy. Once the license plates are detected, the Optical Character Recognition tools such as Tesseract or EasyOCR extract the text out of those plates. For real-time image processing and visualization, OpenCV is employed to handle the input and display the results. The dataset for training and testing is managed and labeled using RoboFlow. This approach effectively combines YOLOv8 for object detection with OCR for text recognition. This enables fast and accurate detection and reading of license plates, even in challenging conditions.

Flow of Actions

The overall workflow of this project includes:

Data Collection: We start by collecting a dataset from Roboflow.
Data Labeling and Preprocessing: Before the detection of the license plate images, they were labeled using the RoboFlow.
Model Setup: Images went through a pre-trained YOLOv8 model for detecting license plates.
Detection Phase: This project used the YOLOv8 model to detect and localize license plates on the input images.
OCR Integration: Tesseract or EasyOCR was used to extract the numbers from the license plate.
Image Processing: For managing input images and displaying the detection results OpenCV was used.
Text Output: Text was extracted from the license plate and further preprocessed and exclaimed in visual form.

Data Collection

For this project, the dataset used was obtained from publicly available license plate image datasets from Roboflow. These images were then analyzed using RoboFlow for labeling and making the images fit for training YOLOv8 models. No custom data collection was performed. The project collected the dataset that was primarily developed for license plate detection and recognition challenges.

Data Preparation

The dataset involved in this project comprises vehicles with clear number plates. To label these images, the RoboFlow was used to draw the exact location of the plate. To make the dataset accurate, the images were taken under different lighting conditions, at different angles, and perspectives. After labeling, the data was split again into train and test so as to train the YOLOv8 model for license plate detection and testing the performance of the model. Such a preparation phase contributed to a model capable of identifying license plates in different scenarios in a real-world setting.

Data preparation workflow:

Dataset Upload: Users uploaded vehicle images showing their number plates to RoboFlow for further processing.
Annotation: All the images were annotated in RoboFlow by indicating various positions of the number plates for proper training of the intended model.
Data Augmentation: Increase the dataset with image variations (rotate, crop, brighten, etc.) to improve the reliability of the model.
Data Splitting: The labeled data was divided into two parts: training data to train the model and test data to evaluate the trained model.
Exporting Dataset: At last, the dataset was exported into a format that can be used with the YOLOv8 model in preparation for the training of the model.

Step-by-step guide:

STEP 1:

Connect Google Drive

You can mount your Google Drive in a Google Colab notebook with this piece of code. This makes it easy to view files saved in Google Drive. Additionally, in Colab, you can change and analyze data. You can also train models.

from google.colab import drive
drive.mount('/content/drive')

The os.chdir() function sets the current working directory to the specified path. This means that the code will now operate in this folder for any file-related tasks. The os.getcwd() function is used to print the current working directory. This confirms that the script has successfully moved to the specified directory.

import os
# Set the current directory
os.chdir("/content/drive/aionlinecourse/real_time_license_plate_detection_and_using_ocr")
# Verify the change in the current directory
os.getcwd()

STEP 2:

Install Necessary Packages.

This code installs the Ultralytics library. It provides access to the YOLOv8 model for object detection. After installation, it imports the library and runs a system check using ultralytics.checks() to ensure the environment is properly set up and all dependencies are ready for running YOLOv8.

%pip install ultralytics
import ultralytics
ultralytics.checks()

STEP 3:

Load dataset

This code trains the YOLOv8 model on a license plate dataset for 100 epochs using images sized at 640x640 pixels.

!yolo train model=yolov8n.pt data= "/content/drive/aionlinecourse/real_time_license_plate_detection_and_using_ocr/dataset/data.yaml" epochs=100 imgsz=640

STEP 4:

Install the necessary packages.

This code imports libraries required for image processing and object detection, including torch for deep learning, OpenCV for handling images, Ultralytics for using the YOLOv8 model, PIL for image manipulation, torchvision transforms for image transformations, and matplotlib for visualizing results.

import torch
import cv2
import ultralytics
from cv2 import imread
from ultralytics import YOLO
from torchvision import transforms
from PIL import Image
from matplotlib import pyplot as plt

STEP 5:

Choosing an AI Model for Real-Time License Plate Detection

For this project, YOLOv8 was chosen due to its exceptional ability. To perform real-time object detection with this model. YOLO models are known for their speed and accuracy in identifying objects within images. YOLOv8 processes number plate images in a single deep layer. It detects and identifies number plates. After that, OCR extracts the main text from number plates for license plate recognition.

By using YOLOv8, we ensured that the model aligns with the project’s goal. It provides fast, accurate, and detection. Also, it enhances the detection capabilities in self-driving applications.

Load a pretrained YOLOv8n model

This code loads a pre-trained YOLOv8n model from the specified path using Ultralytics.

# Load a pretrained YOLOv8n model
model = YOLO('/content/drive/aionlinecourse/real_time_license_plate_detection_and_using_ocr/best.pt ')

Moreover, this code uses the previously loaded YOLOv8n model to do inference on an image file ('car.png'). The results are stored in the 'results' variable.

# Run inference on '.jpg'
results = model('/content/drive/aionlinecourse/real_time_license_plate_detection_and_using_ocr/car.png')  # results list

Custom YOLOv8 Model Inference and Result Visualization

This code first loads a custom YOLOv8 model from the specified path. It then performs inference on the image 'car.png,' saving the output as specified by save=True. After inference, the results are iterated over, converting each prediction to an RGB image format for display and visualization. Using the plot() method, the predictions are plotted, then displayed, and the final image is saved as 'results.jpg'.

from ultralytics import YOLO
model = YOLO('/content/drive/aionlinecourse/real_time_license_plate_detection_and_using_ocr/best.pt')  # load a custom model
# Predict with the model
results = model('/content/drive/aionlinecourse/real_time_license_plate_detection_and_using_ocr/car.png',save=True)
for r in results:
    im_array = r.plot()  # plot a BGR numpy array of predictions
    im = Image.fromarray(im_array[..., ::-1])  # RGB PIL image
    im.show()  # show image   im.save('/content/drive/aionlinecourse/real_time_license_plate_detection_and_using_ocr/results.jpg') # predict on an image

Thirdly, the predicted image from the YOLOv8 model is loaded and shown using this code. The image is read using OpenCV's read function, and it is shown using Matplotlib. More so, the YOLOv8 model is run, and the anticipated image is stored in the supplied directory.

# Load the predicted image
predicted_img_path = '/content/drive/aionlinecourse/real_time_license_plate_detection_and_using_ocr/runs/detect/predict/car.png'
predicted_img = imread(predicted_img_path)
# Display the image
plt.figure(figsize=(8, 8))
plt.imshow(predicted_img)
plt.axis('off')  # Hide axes
plt.show()

STEP 6:

Identifying Real-Time License Plate Detection from Video

This code loads a pre-trained YOLOv8 model from the file best.pt. It was previously trained for license plate detection. It then uses this model to run detection on a video file. Then The save=True parameter ensures that the results are saved to the specified location.

from ultralytics import YOLO
model = YOLO('/content/drive/aionlinecourse/real_time_license_plate_detection_and_using_ocr/best.pt')  # Load a custom model
# Predict with the model on a video
results = model('/content/drive/aionlinecourse/real_time_license_plate_detection_and_using_ocr/sample.mp4', save=True)

Installing Dependencies for Image Processing and OCR

This code installs various libraries needed for the project. This includes easyocr for text recognition, scikit-image and opencv-python for image processing, lap and filterpy for tracking and filtering tasks, pandas for data manipulation, and numpy and scipy for numerical computations.

!pip install easyocr
!pip install scikit-image==0.17.2
!pip install lap
!pip install filterpy
!pip install pandas
!pip install opencv-python
!pip install numpy
!pip install scipy
!pip install easyocr
!pip install filterpy
!pip install utils

STEP 7:

OCR implementation

Starts an OCR (optical character recognition) reader utilizing the 'easyocr' library. The reader is set to recognize English characters, and the GPU is disabled. It means the OCR processing will run on the CPU instead of the GPU.

import string
import easyocr
# Initialize the OCR reader
reader = easyocr.Reader(['en'], gpu=False)

This code creates two dictionaries to handle character-to-number and number-to-character conversions. It is for common visual mistakes in license plate recognition. For example, 'O' is often misread as '0', and 'I' as '1'.

# Mapping dictionaries for character conversion
dict_char_to_int = {'O': '0', 'I': '1', 'J': '3', 'A': '4', 'G': '6', 'S': '5'}
dict_int_to_char = {'0': 'O', '1': 'I', '3': 'J', '4': 'A', '6': 'G', '5': 'S'}
# Print the contents of the dictionaries
print("Dictionary char_to_int:", dict_char_to_int)
print("Dictionary int_to_char:", dict_int_to_char)

The 'write_csv' Python function saves the output of a process detecting license plates and cars into a CSV file. It requires two parameters: output_path and results. It first creates a header with columns for frame number, car ID, bounding boxes for cars and license plates, detection scores, and recognized license plate text. It then loops through the results dictionary, extracting and formatting relevant details like bounding boxes and text for each detected car and license plate.

def write_csv(results, output_path):
    with open(output_path, 'w') as f:
        f.write('{},{},{},{},{},{},{}\n'.format('frame_nmr', 'car_id', 'car_bbox',
                                                'license_plate_bbox', 'license_plate_bbox_score', 'license_number',
                                                'license_number_score'))
        for frame_nmr in results.keys():
            for car_id in results[frame_nmr].keys():
                print(results[frame_nmr][car_id])
                if 'car' in results[frame_nmr][car_id].keys() and \
                   'license_plate' in results[frame_nmr][car_id].keys() and \
                   'text' in results[frame_nmr][car_id]['license_plate'].keys():
                    f.write('{},{},{},{},{},{},{}\n'.format(frame_nmr,
                                                            car_id,
                                                            '[{} {} {} {}]'.format(
                                                                results[frame_nmr][car_id]['car']['bbox'][0],
                                                                results[frame_nmr][car_id]['car']['bbox'][1],
                                                                results[frame_nmr][car_id]['car']['bbox'][2],
                                                                results[frame_nmr][car_id]['car']['bbox'][3]),
                                                            '[{} {} {} {}]'.format(
                                                                results[frame_nmr][car_id]['license_plate']['bbox'][0],
                                                                results[frame_nmr][car_id]['license_plate']['bbox'][1],
                                                                results[frame_nmr][car_id]['license_plate']['bbox'][2],
                                                                results[frame_nmr][car_id]['license_plate']['bbox'][3]),
                                                            results[frame_nmr][car_id]['license_plate']['bbox_score'],
                                                            results[frame_nmr][car_id]['license_plate']['text'],
                                                            results[frame_nmr][car_id]['license_plate']['text_score'])
                            )
        f.close()

The license_complies_format function checks if a given text string matches with a specific license plate format. It first checks if the text is exactly 7 characters long. Then, it validates that the first two and the last three characters are uppercase letters. The third and fourth positions must be digits. If all these conditions are fulfilled the function returns True otherwise it returns False.

def license_complies_format(text):
    if len(text) != 7:
        return False
    if (text[0] in string.ascii_uppercase or text[0] in dict_int_to_char.keys()) and \
       (text[1] in string.ascii_uppercase or text[1] in dict_int_to_char.keys()) and \
       (text[2] in ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9'] or text[2] in dict_char_to_int.keys()) and \
       (text[3] in ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9'] or text[3] in dict_char_to_int.keys()) and \
       (text[4] in string.ascii_uppercase or text[4] in dict_int_to_char.keys()) and \
       (text[5] in string.ascii_uppercase or text[5] in dict_int_to_char.keys()) and \
       (text[6] in string.ascii_uppercase or text[6] in dict_int_to_char.keys()):
        return True
    else:
        return False

The function 'format_license' converts a license plate text into a predefined dictionary by mapping. It uses dictionaries 'dict_int_to_char' and 'dict_char_to_int' to format the license plate. If a conversion is possible, it replaces the character; otherwise, it leaves the character unchanged. The function then returns the reformatted license plate string.

def format_license(text):
    license_plate_ = ''
    mapping = {0: dict_int_to_char, 1: dict_int_to_char, 4: dict_int_to_char, 5: dict_int_to_char, 6: dict_int_to_char,
               2: dict_char_to_int, 3: dict_char_to_int}
    for j in [0, 1, 2, 3, 4, 5, 6]:
        if text[j] in mapping[j].keys():
            license_plate_ += mapping[j][text[j]]
        else:
            license_plate_ += text[j]
    return license_plate_

The function 'read_license_plate' uses an OCR reader to extract text from a cropped license plate image. It uses the easyOCR reader to detect text in the cropped image. The function removes spaces, converts the text to uppercase, and verifies if the text follows the required format. It then checks if the text is in the required format. if it is then return the formatted plate and its detection score. If the text is not valid then return None

def read_license_plate(license_plate_crop):
    detections = reader.readtext(license_plate_crop)
    for detection in detections:
        bbox, text, score = detection
        text = text.upper().replace(' ', '')
        if license_complies_format(text):
            return format_license(text), score
    return None, None

The get_car function determines which car a license plate detected belongs to by matching the coordinates of the license plate with the vehicles. It iterates through the vehicle_track_ids, to see if this license plate bounding box fits within this vehicle bounding box entirely. If there is a match it will result in an applicable vehicle bounding box as well as that vehicle’s id. If no match is made then it returns -. It means that the corresponding vehicle was not found.

def get_car(license_plate, vehicle_track_ids):
    x1, y1, x2, y2, score, class_id = license_plate
    foundIt = False
    for j in range(len(vehicle_track_ids)):
        xcar1, ycar1, xcar2, ycar2, car_id = vehicle_track_ids[j]
        if x1 > xcar1 and y1 > ycar1 and x2 < xcar2 and y2 < ycar2:
            car_indx = j
            foundIt = True
            break
    if foundIt:
        return vehicle_track_ids[car_indx]
    return -1, -1, -1, -1, -1

This piece of code imports the SORT algorithm ('Sort'), OpenCV ('cv2'), and the YOLO from Ultralytics ('YOLO'). An empty dictionary ('results') and a SORT tracker object are initialized.

from ultralytics import YOLO
import cv2
# import util
from sort.sort import *
# from util import get_car, read_license_plate, write_csv
results = {}
mot_tracker = Sort()

# load models
coco_model = YOLO('/content/drive/aionlinecourse/real_time_license_plate_detection_and_using_ocr/yolov8n.pt')
license_plate_detector = YOLO('/content/drive/aionlinecourse/real_time_license_plate_detection_and_using_ocr/best.pt')
# load video
cap = cv2.VideoCapture('/content/drive/aionlinecourse/real_time_license_plate_detection_and_using_ocr/v.mp4')
vehicles = [2, 3, 5, 7]

STEP 8:

Store result

This code processes video frames to detect vehicles and license plates. It reads frames from the video and uses the COCO model to detect vehicles and store their bounding boxes. These detections are tracked using a multi-object tracker. The license plate detector identifies license plates, and the function get_car assigns each license plate to its corresponding vehicle. The detected license plates are cropped, processed, and converted to grayscale for easier text recognition. The easyOCR model then reads the license plate text. If valid text is found, the result is stored, including vehicle and license plate details. Finally, all results are saved into a CSV file.

# read frames
frame_nmr = -1
ret = True
while ret:
    frame_nmr += 1
    ret, frame = cap.read()
    if ret:
        results[frame_nmr] = {}
        # detect vehicles
        detections = coco_model(frame)[0]
        detections_ = []
        for detection in detections.boxes.data.tolist():
            x1, y1, x2, y2, score, class_id = detection
            if int(class_id) in vehicles:
                detections_.append([x1, y1, x2, y2, score])
        # track vehicles
        track_ids = mot_tracker.update(np.asarray(detections_))
        # detect license plates
        license_plates = license_plate_detector(frame)[0]
        for license_plate in license_plates.boxes.data.tolist():
            x1, y1, x2, y2, score, class_id = license_plate
            # assign license plate to car
            xcar1, ycar1, xcar2, ycar2, car_id = get_car(license_plate, track_ids)
            if car_id != -1:
                # crop license plate
                license_plate_crop = frame[int(y1):int(y2), int(x1): int(x2), :]
                # process license plate
                license_plate_crop_gray = cv2.cvtColor(license_plate_crop, cv2.COLOR_BGR2GRAY)
                _, license_plate_crop_thresh = cv2.threshold(license_plate_crop_gray, 64, 255, cv2.THRESH_BINARY_INV)
                # read license plate number
                license_plate_text, license_plate_text_score = read_license_plate(license_plate_crop_thresh)
                if license_plate_text is not None:
                    results[frame_nmr][car_id] = {'car': {'bbox': [xcar1, ycar1, xcar2, ycar2]},
                                                  'license_plate': {'bbox': [x1, y1, x2, y2],
                                                                    'text': license_plate_text,
                                                                    'bbox_score': score,
                                                                    'text_score': license_plate_text_score}}
# write results
write_csv(results, './test.csv')

The interpolate_bounding_boxes function fills in any missing data for vehicles and license plates between video frames. If some frames don’t have bounding box information, it estimates the missing data by smoothly interpolating between the known frames. This way, the tracking of vehicles and license plates remains continuous across the video, even when detections are missing. For the frames where data was filled in, it assigns default scores, while the original frames keep their actual detection values.

import csv
import numpy as np
from scipy.interpolate import interp1d
def interpolate_bounding_boxes(data):
    # Extract necessary data columns from input data
    frame_numbers = np.array([int(row['frame_nmr']) for row in data])
    car_ids = np.array([int(float(row['car_id'])) for row in data])
    car_bboxes = np.array([list(map(float, row['car_bbox'][1:-1].split())) for row in data])
    license_plate_bboxes = np.array([list(map(float, row['license_plate_bbox'][1:-1].split())) for row in data])
    interpolated_data = []
    unique_car_ids = np.unique(car_ids)
    for car_id in unique_car_ids:
        frame_numbers_ = [p['frame_nmr'] for p in data if int(float(p['car_id'])) == int(float(car_id))]
        print(frame_numbers_, car_id)
        # Filter data for a specific car ID
        car_mask = car_ids == car_id
        car_frame_numbers = frame_numbers[car_mask]
        car_bboxes_interpolated = []
        license_plate_bboxes_interpolated = []
        first_frame_number = car_frame_numbers[0]
        last_frame_number = car_frame_numbers[-1]
        for i in range(len(car_bboxes[car_mask])):
            frame_number = car_frame_numbers[i]
            car_bbox = car_bboxes[car_mask][i]
            license_plate_bbox = license_plate_bboxes[car_mask][i]
            if i > 0:
                prev_frame_number = car_frame_numbers[i-1]
                prev_car_bbox = car_bboxes_interpolated[-1]
                prev_license_plate_bbox = license_plate_bboxes_interpolated[-1]
                if frame_number - prev_frame_number > 1:
                    # Interpolate missing frames' bounding boxes
                    frames_gap = frame_number - prev_frame_number
                    x = np.array([prev_frame_number, frame_number])
                    x_new = np.linspace(prev_frame_number, frame_number, num=frames_gap, endpoint=False)
                    interp_func = interp1d(x, np.vstack((prev_car_bbox, car_bbox)), axis=0, kind='linear')
                    interpolated_car_bboxes = interp_func(x_new)
                    interp_func = interp1d(x, np.vstack((prev_license_plate_bbox, license_plate_bbox)), axis=0, kind='linear')
                    interpolated_license_plate_bboxes = interp_func(x_new)
                    car_bboxes_interpolated.extend(interpolated_car_bboxes[1:])
                    license_plate_bboxes_interpolated.extend(interpolated_license_plate_bboxes[1:])
            car_bboxes_interpolated.append(car_bbox)
            license_plate_bboxes_interpolated.append(license_plate_bbox)
        for i in range(len(car_bboxes_interpolated)):
            frame_number = first_frame_number + i
            row = {}
            row['frame_nmr'] = str(frame_number)
            row['car_id'] = str(car_id)
            row['car_bbox'] = ' '.join(map(str, car_bboxes_interpolated[i]))
            row['license_plate_bbox'] = ' '.join(map(str, license_plate_bboxes_interpolated[i]))
            if str(frame_number) not in frame_numbers_:
                # Imputed row, set the following fields to '0'
                row['license_plate_bbox_score'] = '0'
                row['license_number'] = '0'
                row['license_number_score'] = '0'
            else:
                # Original row, retrieve values from the input data if available
                original_row = [p for p in data if int(p['frame_nmr']) == frame_number and int(float(p['car_id'])) == int(float(car_id))][0]
                row['license_plate_bbox_score'] = original_row['license_plate_bbox_score'] if 'license_plate_bbox_score' in original_row else '0'
                row['license_number'] = original_row['license_number'] if 'license_number' in original_row else '0'
                row['license_number_score'] = original_row['license_number_score'] if 'license_number_score' in original_row else '0'
            interpolated_data.append(row)
    return interpolated_data

STEP 9:

Bounding Box Interpolation and CSV Update

This code reads bounding box data from a CSV file, interpolates it, and saves the updated information into a new CSV file. First, it opens test.csv in read mode, using csv.DictReader to load the data as dictionaries in a list called data. Next, it processes this data with the interpolate_bounding_boxes(data) function to enhance or fill in bounding box details across frames. Finally, it writes the interpolated data to a new CSV file named test_interpolated.csv, using csv.DictWriter with a defined header that includes frame numbers, car and license plate IDs, bounding boxes, license numbers, and their respective scores. This process enables refined bounding box data for further analysis or use.

# Load the CSV file
with open('/content/drive/aionlinecourse/license_plate/test.csv', 'r') as file:
    reader = csv.DictReader(file)
    data = list(reader)
# Interpolate missing data
interpolated_data = interpolate_bounding_boxes(data)
# Write updated data to a new CSV file
header = ['frame_nmr', 'car_id', 'car_bbox', 'license_plate_bbox', 'license_plate_bbox_score', 'license_number', 'license_number_score']
with open('test_interpolated.csv', 'w', newline='') as file:
    writer = csv.DictWriter(file, fieldnames=header)
    writer.writeheader()
    writer.writerows(interpolated_data)

STEP 10:

Create a bounding box to detect number plates to check Real-Time License Plate Detection

The draw_border function adds a border around a rectangle in an image. It uses the top-left and bottom-right coordinates to define the rectangle. Instead of a full box, it draws partial lines at each corner. You can change the color, thickness, and line length for both the x and y axes. The function uses OpenCV’s cv2.line() to create these lines. Once done, it returns the image with the border added.

def draw_border(img, top_left, bottom_right, color=(0, 255, 0), thickness=10, line_length_x=200, line_length_y=200):
    x1, y1 = top_left
    x2, y2 = bottom_right
    cv2.line(img, (x1, y1), (x1, y1 + line_length_y), color, thickness)  #-- top-left
    cv2.line(img, (x1, y1), (x1 + line_length_x, y1), color, thickness)
    cv2.line(img, (x1, y2), (x1, y2 - line_length_y), color, thickness)  #-- bottom-left
    cv2.line(img, (x1, y2), (x1 + line_length_x, y2), color, thickness)
    cv2.line(img, (x2, y1), (x2 - line_length_x, y1), color, thickness)  #-- top-right
    cv2.line(img, (x2, y1), (x2, y1 + line_length_y), color, thickness)
    cv2.line(img, (x2, y2), (x2, y2 - line_length_y), color, thickness)  #-- bottom-right
    cv2.line(img, (x2, y2), (x2 - line_length_x, y2), color, thickness)
    return img

The code starts by reading a CSV file into a variable called results. It loads the file test_interpolated.csv using pandas. Next, it opens a video file located at v.mp4. The video is loaded using OpenCV's cv2.VideoCapture. It sets up the video writer with the codec mp4v. The frames per second (FPS) and video dimensions are taken from the video. Finally, it prepares to write the output video as out1.mp4 using the original video’s width, height, and FPS.

results = pd.read_csv('/content/drive/aionlinecourse/real_time_license_plate_detection_and_using_ocr/test_interpolated.csv')
# load video
video_path = '/content/drive/aionlinecourse/real_time_license_plate_detection_and_using_ocr/v.mp4'
cap = cv2.VideoCapture(video_path)
fourcc = cv2.VideoWriter_fourcc(*'mp4v')  # Specify the codec
fps = cap.get(cv2.CAP_PROP_FPS)
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
out = cv2.VideoWriter('./out1.mp4', fourcc, fps, (width, height))

The code creates a dictionary called license_plate. It loops through each unique car_id in the results. For each car, it finds the highest license plate score. It stores the license plate number with the highest score in the dictionary. The video frame number for that car is set using cv2.CAP_PROP_POS_FRAMES. The code reads the frame from the video. Next, it extracts the license plate coordinates from the results. The license plate is cropped from the frame using these coordinates. The cropped image is resized for better visibility. The resized license plate crop is then saved in the license_plate dictionary. After the loop, the frame number is reset to 0 for future processing.

license_plate = {}
for car_id in np.unique(results['car_id']):
    max_ = np.amax(results[results['car_id'] == car_id]['license_number_score'])
    license_plate[car_id] = {'license_crop': None,
                             'license_plate_number': results[(results['car_id'] == car_id) &
                                                             (results['license_number_score'] == max_)]['license_number'].iloc[0]}
    cap.set(cv2.CAP_PROP_POS_FRAMES, results[(results['car_id'] == car_id) &
                                             (results['license_number_score'] == max_)]['frame_nmr'].iloc[0])
    ret, frame = cap.read()
    x1, y1, x2, y2 = ast.literal_eval(results[(results['car_id'] == car_id) &
                                              (results['license_number_score'] == max_)]['license_plate_bbox'].iloc[0].replace('[ ', '[').replace('   ', ' ').replace('  ', ' ').replace(' ', ','))
    license_crop = frame[int(y1):int(y2), int(x1):int(x2), :]
    license_crop = cv2.resize(license_crop, (int((x2 - x1) * 400 / (y2 - y1)), 400))
    license_plate[car_id]['license_crop'] = license_crop
frame_nmr = -1
cap.set(cv2.CAP_PROP_POS_FRAMES, 0)

STEP 11:

Frame Processing with Bounding Boxes and License Plate Overlay

The code adds license plate numbers to video frames, overlays bounding boxes, and creates a new video file. It's compact, reuses border drawing, and adds text to the frame for improved readability.

# read frames
ret = True
while ret:
    ret, frame = cap.read()
    frame_nmr += 1
    if ret:
        df_ = results[results['frame_nmr'] == frame_nmr]
        for row_indx in range(len(df_)):
            # draw car
            car_x1, car_y1, car_x2, car_y2 = ast.literal_eval(df_.iloc[row_indx]['car_bbox'].replace('[ ', '[').replace('   ', ' ').replace('  ', ' ').replace(' ', ','))
            draw_border(frame, (int(car_x1), int(car_y1)), (int(car_x2), int(car_y2)), (0, 255, 0), 25,
                        line_length_x=200, line_length_y=200)
            # draw license plate
            x1, y1, x2, y2 = ast.literal_eval(df_.iloc[row_indx]['license_plate_bbox'].replace('[ ', '[').replace('   ', ' ').replace('  ', ' ').replace(' ', ','))
            cv2.rectangle(frame, (int(x1), int(y1)), (int(x2), int(y2)), (0, 0, 255), 12)
            # crop license plate
            license_crop = license_plate[df_.iloc[row_indx]['car_id']]['license_crop']
            H, W, _ = license_crop.shape
            try:
                frame[int(car_y1) - H - 100:int(car_y1) - 100,
                      int((car_x2 + car_x1 - W) / 2):int((car_x2 + car_x1 + W) / 2), :] = license_crop
                frame[int(car_y1) - H - 400:int(car_y1) - H - 100,
                      int((car_x2 + car_x1 - W) / 2):int((car_x2 + car_x1 + W) / 2), :] = (255, 255, 255)
                (text_width, text_height), _ = cv2.getTextSize(
                    license_plate[df_.iloc[row_indx]['car_id']]['license_plate_number'],
                    cv2.FONT_HERSHEY_SIMPLEX,
                    4.3,
                    17)
                cv2.putText(frame,
                            license_plate[df_.iloc[row_indx]['car_id']]['license_plate_number'],
                            (int((car_x2 + car_x1 - text_width) / 2), int(car_y1 - H - 250 + (text_height / 2))),
                            cv2.FONT_HERSHEY_SIMPLEX,
                            4.3,
                            (0, 0, 0),
                            17)
            except:
                pass
        out.write(frame)
        frame = cv2.resize(frame, (1280, 720))
        # cv2.imshow('frame', frame)
        # cv2.waitKey(0)
out.release()
cap.release()

The code defines a 'show_video' function to compress and view videos in a colab Notebook, generating a data URL for display and returning the compressed video.

from IPython.display import HTML
from base64 import b64encode
import os
def show_video(input_video_path):
  # Compressed video path
  video_name = "Compressed_" + input_video_path.split("/")[-1].split(".")[0]+".mp4"
  compressed_video_path = os.path.join("/content",video_name)
  # Use FFmpeg to compress the video
  ffmpeg_command = f"ffmpeg -i '{input_video_path}' -vcodec libx264 '{compressed_video_path}'"
  os.system(ffmpeg_command)
  # Display the compressed video
  compressed_video_data = open(compressed_video_path, 'rb').read()
  data_url = "data:video/mp4;base64," + b64encode(compressed_video_data).decode()
  return data_url

The code compresses and displays a video using the'show_video' method. More so, it enables interactive viewing through an HTML \<'video'> element with playing controls.

data_url = show_video('/content/drive/aionlinecourse/real_time_license_plate_detection_and_using_ocr/out1.mp4')
HTML(f"""
<video width=400 controls>
        <source src="{data_url}" type="video/mp4">
</video>
  """)

Project Conclusion

In this project, we developed a tool that can automatically find cars and read license plates from video clips and images. We used a method called YOLOv8 to detect the cars and another technique, OCR, to read the numbers on the plates. If any frames were missing data, we filled in the blanks using interpolation to ensure accuracy. The system then linked each plate to the right car and showed the results on the video with clear borders around the vehicles and the plate numbers displayed. This tool can be helpful for things like managing traffic or automating parking systems, offering a simple and practical solution.
Whether you're a tech enthusiast or just curious, diving into this project could open a world of opportunities in AI-powered automation!

Challenges and Troubleshooting

Data Accuracy and Image Quality:
Solution: The dataset should consist of better images or image enhancement processes like sharpening, contrast, and brightness adjustments can be applied. Image noise reduction can also be utilized or the images may be cleaned up in advance.
Dealing with Missing Data for Frames:
Solution: Interpolation techniques may be used to fill in the empty data set in between frames in such a manner that the motion is smooth and unbroken. Furthermore, frame rates can also be improved or motion deblurring techniques can be used to restrain losses in detection due to blind spots.
Real-Time Processing Speed:
Solution: Make use of hardware acceleration for instance a GPU to enhance the speed during processing. As an alternative, one may optimize the model by reducing the video quality or simplifying the model.
License Plate Recognition in Various Conditions:
Solution: Train the OCR model on diverse datasets that include plates with different formats and from various angles or lighting conditions. Data augmentation techniques could also be practiced to create more diverse training samples, thus improving performance

FAQs

Question 1: What is the main purpose of the Real-Time License Plate Detection and OCR Recognition project?
Answer: The first goal is to perform real-time license plate recognition using YOLOv8 for the detection of plates. OCR models for the recognition of text. This is useful in areas such as; traffic control, toll gates, and tracking of vehicles.

Question 2: What does YOLOv8 bring to the table regarding the involvement of license plate detection and the growth of its efficiency?
Answer: YOLOv8 is an advanced object detection model that can effectively detect license plates. More so, in single or multiple scenarios, and can accurately detect low-quality or overcrowded images.

Question 3: What role does the OCR model play in detecting license plates?
Answer: The OCR model serves to recognize and decode license plates having been detected. Moreover, it converts them into formats that can be analyzed by the computational system.

Question 4: What are the challenges one might face executing license plate detection using Yolov8?
Answer: Major difficulties of running OCR on low-quality pictures to character recognition, detecting the speed of the algorithm. Moreover, Changes to the light or weather according to which the text was taken.

Question 5: Is it possible to use this system together with other traffic or vehicle observation systems?
Answer: Yes, this can indeed be easily implemented with real-time applications. Examples: traffic monitoring, parking space, automated tolling systems. Because of the scalability of the proposed system.

Ready to try by yourself!

Ready to take your AI projects to the next level?

Start exploring today, implement your own solutions, and watch how AI-driven automation can transform the way you work. Don’t wait—explore the future of license plate recognition now! To explore more you can visit our website.