Video Key Frame Extraction With katna

Let us do this exercise — Watch this video and find out which frames are concise representation of the full video.

While watching the entire video, scrubbing frame by frame or pausing at right moment to get an unblurry frame is OK for few videos, the process becomes unimaginable at scale.

How can we automate the repetitive tasks?

Key Frame extraction is the process of extracting frame or set of frames that have a good representation of a shot. It must preserve the salient feature of the shot, while removes most of the repeated frames.

Video is a massive data object and its analysis plays an important role in many fields such as video summarization, video browsing, compression, marketing and so on.

Katna is open source tool that automates the boring, error prone task of videos key/best frames extraction. The extracted frames are the most accurate and compact summary of the video content. Katna video module has been written in python 3 and has been successfully tested on mp4,.mov and .avi video formats. Technically, it should work well with other video formats since we have used openCV VideoCapture function. Katna has been written in an API style so that developers with no computer vision experience can use it as well.

The best way to learn about Katna is to actually use it.

Installation

Katna can be installed either via PyPI or directly from source.

PyPI — Installation via PyPI is pretty straight forward

• Install python 3
• Install with pip

pip install katna

Install from source — Follow the steps below

• Install git.
• Install python 3.
• Clone the git repo.

git clone https://github.com/keplerlab/Katna.git

• Change the current working directory to the folder where you have cloned Katna repo.

cd <<path_to_the_folder_repo_cloned>>

• If you use the anaconda python distribution then create a new conda environment. Keeping environment separate is a good practice.

conda create --name katna python=3
source activate katna

• Run the setup

python setup.py install

How to use katna

• Import the video module from the katna library

from Katna.video import Video

Instantiate the video class.

vd = Video()

• Call the extract_frames_as_images method of video object. The method accepts two parameters and returns a list of numpy 2D array which are images. The method accepts two parameters

1. no_of_frames: Number of key frames to be extracted
2. file_path: Video file path.

imgs = vd.extract_frames_as_images(no_of_frames = 10, file_path= <<File Path>>)

• Incase you want to persist the extracted key frames then call the save_frame_to_disk method of video object.. The method accepts three parameters and returns nothing.

1. frame: In-menory images generated by extract_frames_as_images method.
2. file_name: File name pattern for the persisted image.
3. file_path: Folder location where files needs to be saved
4. file_ext: File extension indicating the file type for example — ‘.jpg’

vd.save_frame_to_disk(img, file_path=output_folder_video_image, \
     file_name="test_"+str(counter), file_ext=".jpeg"

That’s it!! You can find a complete application here.

Frame extraction and selection criteria

• Frame that are sufficiently different from previous ones using absolute differences in LUV color space
• Brightness score filtering of extracted frames
• Entropy/contrast score filtering of extracted frames
• K-Means Clustering of frames using image histogram
• Selection of best frame from clusters based on and variance of laplacian (image blur detection)

High level architecture

What’s next

We are looking at new filters to add. We would like to make the filter addition configurable so that community can add their own or integrate commercial filters.

Wrap up

We have used katna for hero image extractions from videos and has already saved more than 1000 person hours so far. We are thankful to the open source community and especially this project which helped us get started.

Needless to say, you feedback and suggestions are highly appreciated.