Cracking the Codec: What’s a Video Codec and How to Choose the Right One

Codecs are the silent heroes of video production. While you apply creativity and practical skill to get the right shot or edit a video together, the codecs working in the background make capturing, manipulating and sharing those videos possible. However, many filmmakers don’t actually know what codecs are or how they work. While you can get quite far without understanding codecs, once you do, you’ll be able to take full advantage of them to boost your production workflow.

What are codecs, and why do you need them?

In short, a codec is essentially software designed to encode and decode data in a particular manner. This process of encoding and decoding, from which the portmanteau “codec” is created (coder/decoder), transforms raw video into a more manageable digital format (encoding) before then reverting it back to a viewable form when needed (decoding).

Why codecs are so important to filmmakers comes down to 3 main reasons:

• Storage

Codecs can be used to significantly reduce the size of raw video files so that they can be stored on SD cards or other storage devices. Moreover, they can do so often without sacrificing noticeable quality. This is why codecs are so crucial to compression.

• Speed

By converting data in this way, codecs also enable your camera, PC/laptop and editing program to download, manipulate and upload files much faster.

• Compatibility

Depending on the production stage you are in, it may be necessary to use particular codecs to make your files compatible with their next destination (e.g., NLE or online video platform).

Codec vs. container

If you’ve read up on codecs, you’ll often see them referred to alongside containers, and it may not be instantly clear what the difference is. While codecs are the software that enables your video to be encoded and decoded in specific ways, the container is the “wrapper” that holds all the data (video files, audio files and metadata) to make this possible. It’s this container that also facilitates delivery and playback. While there are many options out there, the most common video formats use the MP4 and MOV containers.

Most common video codecs

In order to understand the most common video codecs, it’s helpful to first learn the terms “lossy” and “lossless.” As the names suggest, these refer to the amount of data lost in the encoding/decoding process.

• Lossy

Lossy codecs convert your raw video into a final product with technically less information/quality than before.

• Lossless

However, with lossless codecs, no information is lost between the raw video and its final format.

• Visually lossless

Visually lossless codecs claim to lose no quality while still technically being a lossy codec (e.g., ProRes and DNxHR).

While it may seem obvious that you should avoid lossy codecs, the benefits in storage, speed and compatibility often outweigh the drawbacks of losing information, especially as this often goes unnoticed.

Common lossy codecs

• H.264
• H.265
• MPEG-4
• XAVC (Sony’s customized version of H.264)
• VP9

Common lossless/visually lossless codecs

True lossless:

• H.264 Lossless
• H.265 Lossless
• JPEG 2000
• OpenEXR

Visually lossless:

• Apple ProRes (422, 422 HQ, 4444, 4444 XQ)
• Avid DNxHR/HD

Which codec should you choose?

Having looked at the various popular codec options, let’s start to address which ones are suitable for you. To do this, it’s helpful to understand that codecs are used for different purposes across the stages of production, each having different codec requirements and names.

1. Acquisition codec: When capturing media in-camera
2. Intermediate codec: When editing and color grading
3. Delivery codec: When delivering your final video.

It is quite possible that you have used the same codec throughout all 3 stages (e.g., H.264), and this may have worked well for your own needs. However, by making some changes to your codecs from acquisition to delivery, you could find yourself both saving time and improving your video quality.

Acquisition codecs

When it comes to media acquisition, you may be limited by your own camera’s options of codecs. Or, you may be restricted to certain codecs because of the speed limits of your SD cards or storage size. However, it’s important to realize that recording straight into a very lossy codec will limit the potential quality of your final video straight away. You will not be able to convert the file again later to regain information. Lossy codecs like H.264 are ideal for web delivery, but editing with H.264 (or other lossy codecs) can be very limiting. However, if you can shoot at a lossless (or “visually lossless” codec), you will have more flexibility in color grading and other editing advantages, which we’ll discuss shortly.

External recorders

Now is a good point to mention external recorders, such as the Atomos Ninja V.

Suppose your camera provides limited codec options, or your SD cards don’t offer the speed or storage you need to use less lossy codecs. In these cases, you might consider investing in an external recorder, which can often solve these problems. Many external recorders can use codecs that are otherwise unavailable to your camera, and they also tend to use SSDs as opposed to SD cards for storage, which can be both faster and cheaper. This may be an avenue to explore if you seek a higher quality acquisition codec in a relatively efficient way. Plus, most external recorders are also monitors, which are extremely useful.

To sum up, the best acquisition codec should be the most lossless/least lossy codec you can afford, given your camera’s limitations and your storage’s size and speed.

Recommended codecs for shooting:

• Apple ProRes
• Avid DNxHR

Intermediate codecs

Intermediate codecs are those used while editing and color grading. It’s natural to assume that a more compressed codec, producing a smaller file with less information, would also mean faster editing. However, due to the methods of compression used in many lossy codecs, the opposite can actually be true.

Is H.264 good for editing?

Codecs such as H.264 use a form of compression that groups multiple frames so that some frames contain the full image, while others only include the changes or differences in the image. This prevents similar or static components from needing to be processed separately in each frame (this is called long GOP recording). The plus side is that this massively reduces file size, but the downside is that NLEs will require much higher CPU and GPU usage for playback and basic editing to interpret every frame.

On the other hand, codecs such as ProRes and DNxHR use intraframe recording, which processes each frame separately, resulting in larger files but a more efficient editing experience.

Lossy codecs also limit your flexibility in color grading your footage, as there is simply less information there to manipulate. Unfortunately, you’ll find the image begins to degrade as you make more and more adjustments to color and exposure.

Special codecs

It’s also worth mentioning that some codecs have been specifically designed to work well with particular systems. For example, you have the Blackmagic Raw codec working best with DaVinci Resolve and the Apple ProRes for Final Cut Pro X or other NLEs running on Apple devices.

Depending on your NLE and computer, you may wish to factor this information into the codec you choose to shoot with.

If shooting in an intraframe codec is not possible, you can still benefit from the editing benefits of such codecs by using proxy files. Commonly, NLEs like Final Cut Pro X and Premiere Pro can produce temporary ProRes and DNxHR versions of your footage for the purpose of editing, but on export, apply your changes to the original footage. This is definitely recommended if shooting in such a codec isn’t possible at acquisition.

In summary, the best intermediate codecs are ones that are available for interframe recording (as opposed to long GOP recording) and are designed to edit efficiently.

Recommended codecs for editing:

• Apple ProRes
• Avid DNxHR

Delivery codecs

As the name suggests, delivery codecs are the ones you choose to deliver with (upload/send), either online or via hard copies. Since most video creators upload content online, we’ll focus on codec options for doing so.

You might assume your finished video should be as high in quality as possible. However, when uploading online, codecs that minimize file size are actually extremely helpful on either end of the upload. On the one end, lossy codecs can help you quicken upload time, and then once they are online, these codecs also help videos be streamed smoothly by users. If quality loss concerns you, this is much less of an issue at the delivery stage than during the editing stage. After all, codecs like H.264 and H.265 are optimized to minimize file size without sacrificing visible image quality. This shouldn’t be a problem as long as your bitrate is still proportionately high in relation to your resolution and frame rate.

To sum up, your delivery codec can afford to be lossy as this will likely work best on most online platforms and can also speed things up in the upload.

Recommended codecs for uploading online:

• H.264
• H.265
• VP9

What does this all mean for you?

In practical terms, the above can be summarized by considering each codec’s purpose as having certain principles to follow, if possible.

• In acquisition, shoot at the most lossless codec your camera and storage allow for.
• In editing/grading, prioritize a lossless codec and consider using proxies if your acquisition codec used long GOP recording.
• In delivery, feel free to compress your final product into a more lossy codec since the quality loss won’t be noticed online, but it will speed up your upload.
  Get unlimited high-quality music for your videos

  Start Free Now

  Pricing

Frequently asked questions about video codecs

What is the best codec for video?

This is a difficult question to provide a quick answer. But in essence, shoot and edit in the least lossy codec your camera, storage and computer can handle (ProRes, DNxHR, Blackmagic Raw, and export in H.264 for web delivery.

Is MP4 a video codec?

MP4 is a container, as opposed to a codec, but is used to hold the audio data, video data and metadata produced by the codec.

Is H.264 a codec?

It is, and a very popular codec at that. While it is possible to use H.264 from acquisition through to delivery, it is not an ideal codec to use in-camera since it compresses the image leading to limitations in color grading and the editing process in general.

Is MOV a codec?

Like MP4, MOV is not a codec but a container. MOV is a proprietary Apple format and is not universally the most compatible container available. MP4 would be most widely accepted (though MOV is not far behind).

What happens if a video codec is not supported?

In this instance, you will need to transcode your video into a different codec. You can do this within most NLEs or transcoders such as VLC.

Can H.264 be MOV?

The H.264 codec and the .MOV container are not mutually exclusive. Therefore, it is possible to use both together, though most NLEs will automatically assign the .MP4 container to the H.264 codec.

How do I change the codec of a video?

The codec of a video is most commonly changed when exporting from an NLE but can also be done using transcoders such as VLC or Handbrake. The process of doing so is program-specific, many of which are outlined in our post on video compression.