A shot is a sequence frame in an edited video taken by a single camera. Shot Segmentation is the process of splitting video and finding the boundaries of video data. In this paper, we study the method in content-based video retrieval which uses object detection and tracking for video segmentation. Data collected via segmenting can be categorized in a hierarchy manner as scene layer, camera shot layer, and the frame in their accordance. The data collected is used for segmentation. YOLOv4 is used to enhance the accuracy and the process of tracking and detection much faster.