All About Video Codecs and Containers

These days, everyone seems to be an aspiring cinematographer. People use a variety of devices, including cell phones, small pocket camcorders, high-end camcorders, and digital cameras. Even the latest generation of professional-level digital SLRs include high-definition video recording capability.

Learning to shoot video can be a big learning curve by itself, but things only get more confusing when you want to show other people your masterpiece. Maybe you want to upload it to YouTube, create your own Blu-ray Discs or DVDs, or perhaps enable your video to be downloaded to mobile devices like smartphones.

It's a video-rich world, and most of us will be creating, editing, and showing off our work for our friends, family, and coworkers through a variety of media and formats. If you want to make the best impression with your masterpiece, you'll need to understand the basics of video formats and codecs.

First, I'll cover the basics of container formats, then move on to codecs. At the end, I'll present several usage examples, with a brief discussion of software. Many apps are out there, ranging from free, open-source ones to very pricey, professional-grade software, so I can only touch on a couple of examples. What you use depends on your budget, how you plan on delivering video, and how much control you want to have over your output.

What's the Difference Between a Codec and a Container?

Handbrake is an app that will let you convert video from one format to another, using whichever codec and compressor you wish.
First-time users often stumble when trying to figure out the difference between codecs and containers. A codec is a method for encoding or decoding data--specifically, compressed data. Though the word codec has become somewhat generic, the term was originally a shortened form of compressor-decompressor. That's what codecs do: they take digital media data and either compress them (for transport and storage) or decompress them (for viewing or transcoding.)

Raw video and audio data consume vast amounts of storage space. Uncompressed 1080i high-definition video recorded at 60 frames per second eats up 410 gigabytes per hour of video. CD audio, which is pretty passé by modern standards, puts about 74 minutes of audio on a 680MB CD. However, eight channels of audio encoded at 24-bit resolution churns through 16 megabits per second, or a little over 7 gigabytes per hour of audio. Even with high-speed broadband, that's too much data. That's why video and audio needs to be compressed for storage.

Once the media data is compressed into suitable formats and reasonable sizes, it needs to be packaged, transported, and presented. That's the purpose of container formats--to be discrete "black boxes" for holding a variety of media formats. Good container formats can handle files compressed with a variety of different codecs.

First, let's run down some of the different codecs.

Understanding Codecs

If you cruise the Web forums of people who work regularly with video, you'll stumble across raging debates about which codec is best. In reality, the right codec really depends on your intended use. So it's worth looking at the different codecs in terms of their usage model, instead of just a long list. We'll mostly focus on video codecs, but will touch on audio issues in the section on container formats.

Video Capture and Archiving

Most modern consumer electronic devices capture content in some type of compressed format; usually, only professional videographers work with uncompressed HD video. In an ideal world, with infinite storage, you'd maintain archives of video captured in the original format, if possible, because that's the highest-quality capture. Once you transcode the video from one compression type to another, it's possible to introduce subtle errors that may reduce the quality of the image. (Good transcoding software minimizes this.)

Dozens of codecs exist, with specific use models; most of them you may never encounter as a typical consumer. This section focuses on codecs as compression/decompression schemes, not as specific software that may encode or transcode video. That discussion comes later.

H.264 / MPEG-4 AVC: This is the most common codec used in modern camcorders and digital cameras that capture to file-based devices (hard drives, memory cards, and so on). Again, note that this is the codec, not the container format, which is typically AVCHD (see the container section on the next page.)

MJPEG (Motion JPEG): This is an older format used by some digital cameras and older devices to capture video. It was developed by the same group (Joint Picture Experts Group) that developed the JPEG photography compression codec, hence the name.

DV and HDV: DV was developed by a consortium of consumer electronics companies that manufacture and sell camcorders. DV is a tape-based standard and is common on camcorders that use mini-DV tape cartridges. (Some versions of DV are used in professional tape-based gear as well, like DVCPRO and DVCAM.) DV itself is limited to standard definition, so one version, called HDV, was created to allow capture of high-definition video to mini-DV tape cartridges.

Note that DV and HDV describe both the codec used and the container format.

Disc-Based Delivery Formats

Let's talk about the old-fashioned DVD or slightly more newfangled Blu-ray Discs for a bit. Despite the increasing popularity of streaming video, the capability to deliver disc-based media is still needed. I've created Blu-ray and DVD discs to hand out to parents of high school athletes, or to send to relatives, for example. Practically everyone has a DVD player, and you don't need an Internet connection to share a DVD.

MPEG-2: Again, we have to distinguish the codec (MPEG-2 Part 2, also known as H.262) from the MPEG-2 container format. MPEG-2 is used exclusively as the compression standard for DVD video. MPEG-2 was also used in the early days of Blu-ray Disc creation, though most of the newer Blu-ray movies no longer use MPEG-2. MPEG-2 is also used to compress video for over-the-air HDTV broadcast.

H.264 / MPEG-4 AVC: This is essentially the same codec used by modern camcorders for capturing video. It's also used for delivering Web video. H.264 offers robust image quality at relatively low bit rates and high compression ratios. It's very scalable, so you can also have high-bit-rate H.264 video that looks fabulous. This is what's typically used for Blu-ray playback.

Microsoft VC-1: VC-1 is one of three codecs used to encode Blu-ray content. It's also used in Microsoft's Silverlight alternative to Adobe Flash. Blu-ray movies using VC-1 mostly use VC-1 Advanced Profile, also known as Windows Media Video 9 Advanced Profile or WVC1.

Streaming From the Web

Delivering video over the Web necessarily means compromises, mostly trading off image quality for lower bit rates. Broadband bit rates vary depending on the ISP and transport technology. Most of what applies to Web content delivery also applies to video stored on mobile, handheld devices.

MPEG-1: MPEG-1 is the old warhorse for delivering video over the Web. While YouTube, Netflix, and other relatively sophisticated streaming video providers have moved away from MPEG-1, a ton of MPEG-1 standard-definition video is still available on other sites. I've included it mostly for completeness; if you're planning on using high definition at all, you'll want to avoid MPEG-1.

WMV (Windows Media Video): Once again, we're talking about the codec, not the Windows Media Video container format. While not as common as MPEG-1, there's still a lot of WMV content available. Again, it's probably best to avoid using this.

H.264 / MPEG-4 AVC: H.264, at lower bit rates, delivers fairly high-quality video. H.264 will likely become the most common codec used: Adobe supports it in Flash, HTML5 canvas can use H.264, YouTube is steadily moving to H.264, and Apple fully supports it. While creating a video compressed in H.264 might not play on older devices, it's a sure bet going forward.

Next: The Right Container

Subscribe to the Now Playing Newsletter