Grab and Go: Leveraging Computer Vision for Intelligent Shopping

Abstract

In an effort to address the challenges posed by long queues and slow payment processes in shopping, new technologies like Amazon's "Just Walk Out"have emerged to enable checkout-free purchases. Computer vision methods, specifically the application of On-Shelf Availability (OSA) monitoring, have been highlighted as crucial for efficient inventory management in grocery stores, using semi-supervised learning and YOLO deep learning architecture. However, the high cost associated with implementing such comprehensive smart market solutions, including the need for RFID sensors on every item, may hinder adoption in developing countries. This paper aims to provide a more affordable alternative by leveraging computer vision techniques, replacing sensors with cameras, and utilizing deep learning models. The proposed approach involved training YOLO V5 and various versions of YOLO V8 models, incorporating annotations, data augmentations, and exploring different techniques. The achieved results demonstrated a mean Average Precision (mAP) of 78.3%, precision of 64.2%, and recall of 61.4% using YOLO V8 with augmentations such as Mosaic, Horizontal Flip, Blur, etc. While the results were considered acceptable even with limited access to natural images, the performance is expected to improve further with increased access to a wider range of training images, especially natural ones.