Compression & Codec Basics

Post Production 12/01/2019 4 min read

Video output is a complex beast. Especially in the digital and internet era. In the old days, uncompressed video files could be laid to tape and then aired on TV stations. But today, videos are output for YouTube, file sharing, TV, and any other number of uses.

Anyone who has ever exported a video from After Effects, Adobe Premiere Pro, or any editing software knows that there are a ton of codec options, bit rates, resolutions, and other variables that will alter the file size and also the video quality.

You may need to use a variety of codecs and compression setups for your delivery needs, but today we’ll go over some of the basics of which codecs are most popular and how compression works, so you can make better choices when outputting your videos.

What is compression anyway?

Well, on the basic level, compression is what it sounds like. When we add compression to a video file, we are squeezing the file size down. We do this by deleting pixel data from the video. Codecs, which we’ll explain shortly, exist to find the best ways of deleting data and compressing file size while maintaining the best overall video quality.

As an example, here is a single image at a higher quality and the same image with compression.

At first glance, the images may look very similar, but upon closer inspection, you can see that some of the detail is lost in the image on the right, and there is some pixelation in a few spots.

The more you compress an image or video, the lower quality result you will get. But it’s often important for us to get file sizes down for internet transfer or for web streaming.

This is where codecs come in.

Okay, then what’s a codec? Glad you asked.

First of all, a codec is not to be confused with Kodak (the film company) or a codex (an ancient manuscript or text). Codec is actually an abbreviation of two words coder-decoder or compressor-decompresser. Dealer’s choice. Either way, it’s a piece of software that analyzes a video, image, or piece of audio and uses complex algorithms to figure out which bits it can delete with minimal impact to quality.

Various codecs use different algorithm types for different results. Some are great at getting file sizes down. Some are great at maintaining quality. The best ones do both well.

When working with codecs, it’s also important to make sure that you are using a codec that your delivery system can also use. Otherwise, you won’t be able to “decode” the file and watch it.

While there are a plethora of video codecs in the market, there are several that are widely used and should probably be looked at as mainstays for your videos.

H.264

Perhaps the most commonly used video codec today is H.264, short for MPEG-4 Part 10, Advanced Video Coding. The original H.264 codec was finalized in 2003 and has become widely used because of its ability to compress video to much smaller file sizes with almost imperceptible changes to quality.

H.264 is the codec used for Blu-Ray DVD video, as well as the codec used to compress videos for playback on YouTube, Vimeo, and a host of other streaming services. It’s also one of the codecs used by Netflix for encoding video.

In recent years, the teams that developed H.264 have gone on to develop the assumed H.264 successor, which they cleverly called H.265. More and more pieces of software and hardware are incorporating H.265 compatibility, and we may soon see a shift from H.264 to H.265 as a standard.

Apple Pro-Res

Apple Pro-Res is another codec that became widely used with the rise of Final Cut Pro as an editing platform. The Apple Pro-Res codec will generally produce a final output closer to the original source than H.264, but with a larger file size as well.

A file compressed with the Apple Pro-Res codec can be 5 to 10 times larger than an H.264 compressed file, depending on the settings used.

Several hardware makers incorporate their own proprietary codecs for recording purposes that are compatible with various software setups for playback in editing.

For example, the Redcode Raw (.R3D) codec is a proprietary format used in RED cameras that stores each of the color channels separately for maximum control in post-production. Here the emphasis isn’t on file size, but instead on control.

Another codec developed by a camera maker is the CineForm codec by GoPro. In 2014, the Society of Motion Picture and Television Engineers (SMTPE) announced that the GoPro CineForm codec was the new open codec standard for video acquisition and post-production.

This means that the CineForm codec will be more widely supported for post-production workflows across the board.

Compression Settings

When outputting video from your post-production software of choice or video conversion software, there are a number of settings that you may see aside from the specific codec used. Altering these settings will affect the overall quality and filesize of your video as well.

Here are some of the more common settings that you may choose to adjust.

Quality: This is sometimes used as a basic control for the overall output quality. The lower you set the quality, the smaller the file size will be, but as you may guess, the overall quality will go down. This setting adjusts the bitrate per second.

Limit Bitrate: Another way to have more control of the size and quality is to set a specific bit rate limit using one of these controls.

Keyframes: You can often set the space in between keyframes in the codec. Essentially, a keyframe here is a frame where the codec takes a full picture with no compression. It will then calculate the differences between keyframes for compression.

Setting the number of frames between each keyframe lower will make a higher quality video with larger file size and vice versa.

Having a basic understanding of compression and codecs will save you tons of time later when you output, as you’ll have a better understanding of what your final video quality will be and the file size. It will also help you to know which codecs you are using based on where you will be delivering your files or what software you will be importing into. Take a little time to get familiar with these in your workflow so as to avoid having to re-render or re-export over and over again.