SPATIO-TEMPORAL SHOT DETECTION IN COMPRESSED VIDEO STREAMS

Abstract

The objective of shot detection is to determine the beginning of each shot in the video streams. It is important task for many video applications such as: 1) video browsing, indexing and retrieving, 2) video abstracting, and 3) semantic video analysis. Moreover it could be utilized for further video compression. The basic problem addressed in this thesis is the detection of shots in compressed MPEG video streams with partial decoding and utilizing the predefined information available in the compressed data. The detection is explored by detecting transitions between consecutive shots, which could be abrupt or gradual. Each was detected by utilizing the spatial and temporal information in the video streams through two phases: macro-block type's analysis in B-frames, and on-demand intensity information analysis. The abrupt transition detection is explored by first examining the number of forward and backward predicted macro-blocks (p- and b-MBs) in consecutive B-frames. Then an intensity histogram comparison is applied in a novel appropriate way to confirm detected transitions, Adaptive thresholds are used to accomplish the detection process based on the temporal sliding window technique. The gradual transition is detected first by examining the intra-coded predicted macro-blocks (i-MBs) within successive B-frames. Then the detection is confirmed by checking the parabolic shape of the frame variances of the candidate sequence. The macro-block types' analysis of the abrupt shot boundary detection saves 95% of the time needed to compare intensity histograms of each consecutive two frames. On the other hand the novel method introduced in the presented study tracking i-MBsin successive B-frames as well as the frame variance analysis -to detect gradual transitions- do not need any thresholds. Moreover more than 80% of the frames composing gradual transitions could be detected. The developed shot detection system was tested on numerous sequences containing a large variety of different types of shot boundaries. Results on a test set representing a total of ?OO cuts and 200 gradual transitions in almost one hour video data are remarkable with overall recall and precision of approximately 83% and 69%, respectively.