Vision-Based Object Tracking with Dual PID Control

Building a Vision-Based Object Tracking System Using Dual PID Control

Project Repository


View the complete source code on GitHub

Introduction

In robotics, feedback is what makes a system intelligent. Unlike open-loop systems, closed-loop systems continuously measure their output and correct themselves in real time.

In this project, I built a vision-based closed-loop control system using a webcam and dual PID controllers. The system detects a colored object, aligns itself horizontally, and maintains a safe distance using only camera input.

This setup mimics the core logic used in autonomous vehicles, drones, and mobile robots.

Problem Statement

The objective of this project was to design a system that can track a target object using only camera input and respond to it in real time.

The system should:

• Detect a colored object from a live video stream


• Align itself horizontally with the object


• Maintain a safe and consistent distance


• Continuously adjust its motion using feedback

The challenge was to combine computer vision for perception and PID control for decision-making into a single closed-loop system.

Vision Pipeline

The perception layer of the system is responsible for extracting meaningful information from the camera feed.

Each frame from the webcam is first converted from RGB to HSV color space. HSV (Hue, Saturation, Value) is preferred over RGB for color segmentation because it separates color information (hue) from brightness. This makes object detection more robust to lighting variations.

After conversion, a color threshold is applied to isolate the blue object. This produces a binary mask where the detected object appears white, and the rest of the frame appears black.

Next, contour detection is performed on the binary mask. The largest contour is assumed to be the target object. From this contour, two important measurements are extracted:

• The horizontal center of the object (x_object)


• The bounding box height (h_object)

The horizontal center is used to compute alignment error:

Error_steer = x_object − x_frame_center

The bounding box height is used as an approximation of distance. As the object gets closer to the camera, its height in the image increases.

These measurements are then passed to the control layer for PID computation.

Control System Design

The system uses two PID controllers — one for steering and one for maintaining distance.

For steering, the controller checks how far the object is from the center of the frame:

Error_steer = x_object − x_frame_center

If the object shifts left or right, the controller adjusts the steering to bring it back to the center.

For distance control, the system uses the object's height in the frame as an estimate of how close it is. If the object appears too large, it means it is too close. If it appears small, it is far away. The controller adjusts the forward speed to maintain a safe distance.

Each PID controller works using three components:

• Proportional (reacts to current error)


• Integral (corrects accumulated past error)


• Derivative (reduces sudden oscillations)

By combining these two controllers, the system can align itself with the object and maintain distance at the same time.

Closed-Loop Architecture

The system operates as a continuous feedback loop.

1. The camera captures a frame.


2. The vision pipeline processes the image and extracts object position and size.


3. Errors for steering and distance are calculated.


4. The PID controllers compute correction outputs.


5. The system adjusts steering and speed.


6. The process repeats for the next frame.

This constant cycle allows the system to correct itself in real time.

If the object moves, the error changes. The controller reacts immediately and adjusts the motion. Unlike an open-loop system, the output directly influences the next measurement, forming a closed feedback loop.

This feedback-based approach makes the system stable and responsive, which is essential in robotics and autonomous systems.

Experimental Observations

While tuning the PID controllers, I observed how each parameter affected system behavior.

When Kp was too high, the system reacted aggressively and started oscillating around the target. When it was too low, the response became slow and sluggish.

Without the derivative term, the system showed noticeable overshoot, especially during sudden object movement. Adding derivative control helped smooth out rapid changes.

If the integral gain was too high, the system accumulated error and caused instability over time. Careful tuning was necessary to achieve stable and smooth tracking.

These observations helped me better understand how PID parameters influence real-world system behavior.

Real-World Applications

This type of vision-based closed-loop control system is widely used in robotics and autonomous systems.

Similar architectures are found in:

• Line-following robots


• Self-driving vehicles


• Drone tracking systems


• Warehouse automation robots


• Visual servoing systems in industrial robotics

Although this project is a simplified implementation, it reflects the core principles used in real autonomous platforms.

Conclusion

This project demonstrates how computer vision and classical control theory can be combined to build an intelligent feedback-driven system.

By integrating object detection with dual PID control, the system is able to align itself and maintain distance using only camera input.

Through this implementation, I gained practical experience in perception-to-control pipelines, PID tuning, and closed-loop system design — all of which are fundamental concepts in robotics and autonomous systems.