Labeling and Describing Objects in a Photogrammetry-Based 3D Environment Using Computer Vision and Artificial Intelligence

Authors

• Colin Donald Moschella, Mississippi State UniversityFollow

ORCID

https://orcid.org/0009-0006-1867-3395

MSU Affiliation

College of Integrative Studies; Data Science Academic Institute

Advisor

Dr. Jingdao Chen

Abstract

Cataloging and digitizing the objects inside a building manually is a task that is often impractical at scale. This project therefore automates the process, using a custom-made system. Using a photogrammetry-based 3D reconstruction of a room, this system is applied to sequences of 2D images used to make the 3D models. The system applies object detection, image segmentation, and image-text models to identify and describe objects, using CNN based models such as YOLO and OpenCLIP. Each analyzed object is then stored in a structured database with spatial coordinates from the 3D scanning, descriptive attributes from the image-text models, and other associated metadata produced by the system, all put together in organized csv files. Using custom-collected data from local buildings, the project evaluates model performance across rooms and objects of varying types and sizes, supporting deployment in diverse environments.

Publication Date

Spring 5-9-2026

Keywords

computer vision, data science, convolutional neural network, vision-language

Disciplines

Artificial Intelligence and Robotics | Databases and Information Systems | Data Science

Recommended Citation

Moschella, Colin Donald, "Labeling and Describing Objects in a Photogrammetry-Based 3D Environment Using Computer Vision and Artificial Intelligence" (2026). Capstone Projects. 5.
https://scholarsjunction.msstate.edu/data-capstone/5