Development of an object detection, segmentation, and tracking framework using self-supervised and zero-shot machine learning models with compression and optimization

Author

Bradley Michael Killen, Mississippi State UniversityFollow

ORCID

https://orcid.org/0000-0003-2001-4277

Advisor

Bethel, Cindy L.

Committee Member

Davis, Jeremy

Committee Member

Carruth, Daniel

Committee Member

Rahimi, Shahram

Committee Member

Chen, Jingdao

Date of Degree

12-12-2025

Original embargo terms

Embargo 1 year

Document Type

Dissertation - Open Access

Major

Computer Science

Degree Name

Doctor of Philosophy (Ph.D.)

College

James Worth Bagley College of Engineering

Department

Department of Computer Science and Engineering

Abstract

This research presents the design of an object detection and tracking framework that leveraged optimized and compressed state-of-the-art models to enable near real-time processing. Recent advances in computer vision, including the Segment Anything Model (SAM), You Only Look Once version 8 (YOLOv8), and Self-Distillation with No Labels (DINO), have significantly improved performance across various tasks. However, these models remain computationally demanding, requiring substantial resources and time due to the extensive information analyzed per frame. To address these challenges, the developed framework integrated three key models: YOLOv8 for object detection and bounding box generation, DINOv2 for creating vector embeddings to enable similarity comparisons, and SAM for segmenting and masking objects based on similarity with the target. This modular design allowed the framework to detect any object of interest, regardless of its type. Each component was evaluated using standard machine learning metrics, and the entire framework was optimized and compressed to reduce computational complexity. Post-optimization, the framework was reevaluated with the same metrics to confirm that it maintained its performance while achieving greater efficiency. This research demonstrates the feasibility of deploying highperformance object detection and tracking models in real-time applications through effective model compression and optimization.

Sponsorship (Optional)

15PBJA-22-GG-00098-BRND

Recommended Citation

Killen, Bradley Michael, "Development of an object detection, segmentation, and tracking framework using self-supervised and zero-shot machine learning models with compression and optimization" (2025). Theses and Dissertations. 6753.
https://scholarsjunction.msstate.edu/td/6753