How to create web-optimized video with ffmpeg and the VP9 and H265 codecs

Every year, modern web development raises the bar on performance. Frameworks compete by optimizing network requests, images, and code to achieve blazing-fast loads. For images, almost any framework, headless CMS, or CDN offers built-in optimization options. Unfortunately, the same can't be said for video.

Still, there's a solution. Modern video codecs deliver excellent quality at a very reasonable size. A well-optimized video is always a better choice for animations than a GIF, and sometimes it even outperforms native implementations. In this article you'll see what the VP9 and H265 codecs are and how to optimize video with the ffmpeg tool.

Formats and codecs are two different things

This is a topic that lends itself to confusion, so it's worth getting the difference clear. Think of the video format as a container that defines the structure of the file carrying the video and audio tracks. Those tracks, inside the container, are media data encoded with a codec. Some popular video formats are mp4, WebM, or MKV. Among the best-known codecs are H264, H265, and VP8/9.

With that clarified, the next thing to know is that not every codec works with every format. WebM uses VP8 and VP9, while mp4 can carry almost anything.

For example, the video above, from Apple's new MacBook Air website, uses the mp4 format, but if you look closer you'll see it uses the H264 codec. H264 is very popular, but it's much less optimized than VP9 or H265.

WebM and VP9

WebM is an open, royalty-free media file format designed for the web. It can hold video tracks compressed with the VP8 or VP9 codecs and audio tracks compressed with the Vorbis or Opus codecs.

It lets you save between 20 and 50% in size compared to H264 while keeping the same quality level. It's supported by all modern desktop and mobile browsers, except Safari on iOS. Apple added WebM to desktop Safari starting with macOS Big Sur, but so far hasn't made any progress on the mobile version.

But don't worry: there's a solid alternative for iOS users.

MP4 and H265

H265, also known as HEVC (High-Efficiency Video Coding), is a codec supported by Apple devices that offers that same 20-50% efficiency improvement over its older sibling, H264. It isn't royalty-free, which may be why its adoption is quite a bit lower than WebM's.

So, by smartly combining webm + VP9 and mp4 + H265 for your videos, you can reach global coverage of 94%.

Tools for optimizing video

The two most popular tools for optimizing video are Handbrake and ffmpeg. Handbrake has a graphical interface, which makes it much easier to use; however, ffmpeg is more flexible if you want to render videos with an alpha channel (transparency) or use hardware acceleration, which speeds up compression enormously. Also, I personally find it much easier to share a single ffmpeg command with the team than to explain where to click in Handbrake.

Installing ffmpeg

If you're on macOS, install it with brew.

For other systems, follow the instructions on the official website.

ffmpeg basics

ffmpeg is a command-line tool with a huge number of options worth knowing. If you're just starting out, it's enough to run:

The result will be a WebM file encoded with VP9 and decent compression out of the box. If you want to master the tool, you'll need to learn how to use its flags.

To make this clearer, I'll show you how each ffmpeg option reduces file size and affects quality. All the tests were done with an animation created for a recent project.

The original video weighs about 26 MB and has a resolution of 5000x2700.

But with basic compression it already comes down to 1 MB. Not bad, but it can be improved! 🚀

There are a few ffmpeg options that apply whether you use H265 or VP9:

-an: removes the audio track. Even if the file has no sound, it may include information for an empty audio track that adds weight, so always use it on videos without sound.

-vf: the option for passing filters to ffmpeg. The most common use case is scaling.

As mentioned, the original resolution is 5000x2700, but it isn't displayed at that size, so you can reduce it to whatever the page actually needs. In this case, 3840 px wide. The scaling option keeps the original video's aspect ratio if you pass the width and an extra parameter with possible values -1, -2, or -n. The value -2 guarantees the resulting resolution is even. To learn more, take a look at this great demo here.

If you run the command above, the video drops to 542 KB.

Advanced settings

In addition to the arguments above, ffmpeg's VP9 and H265 codecs accept the -crf flag. It turns on constant quality mode, which guarantees a given perceptual quality level throughout the video. It takes a number from 0 to 63 (VP9) or from 0 to 51 (H265), where 0 is the best quality and the maximum value is the worst.

Any value between 18 and 40 can give quite decent quality. If you omit this option, ffmpeg defaults to -crf 32 for VP9.

Result: 314 KB.

We're getting closer to codec-specific options, but first let's see how to explicitly tell ffmpeg which codec to use, instead of leaving it on automatic. That's what the -c:v flag is for. Some interesting options:

libvpx-vp9: the default when converting to WebM with the VP9 codec

libx265: for the H265 codec and the mp4 container

hevc_videotoolbox: for H265 with hardware acceleration

Example:

VP9 specifics

libvpx-vp9 has the -deadline option, which accepts the following values:

• good is the default and recommended value for most cases.
• best is recommended if you have plenty of time and want maximum compression efficiency.
• realtime is recommended for live or fast encoding.

Using best can save you a few extra KB.

Result: 295 KB total.

Time to move on to H265 and libx265. It's a very efficient library, but unfortunately it doesn't use hardware acceleration, so encoding takes quite a while. For H265 you need to know a few more options.

-preset is similar to deadline in VP9, but accepts many more values:

• ultrafast
• superfast
• veryfast
• faster
• fast
• medium (default)
• slow
• slower
• veryslow
• placebo

placebo is the slowest but, according to official tests, barely improves encoding. That's why it makes more sense to use veryslow.

Another flag you can pass is -movflags faststart. It optimizes the video for the web and makes it ready to play sooner. We haven't used it with VP9 in WebM because its architecture doesn't need it and always guarantees progressive loading.

And finally, -tag:v hvc1, which makes the video work on iOS and macOS devices. The command ends up like this:

And that gets us down to 260 KB!

You've probably noticed that for H265 we used -crf 32, while for VP9 it was -crf 40. Both values give a very similar bitrate in the two videos; the difference is just syntax, because the accepted number range is different.

Pros and cons of the hevc_videotoolbox library for H265

hevc_videotoolbox is a macOS-exclusive library that offers native hardware acceleration for encoding H265. It's two orders of magnitude faster than libx265, but it has its downsides: it optimizes worse and doesn't have the constant quality mode (-crf) or the presets from the previous example. As an alternative to -crf you can use -b:v, which lets you manually set the video bitrate (more information).

Result: 654 KB.

The only advantage of hevc_videotoolbox over libx265 is that it lets you encode H265 videos with an alpha channel.

Sharpening the video

For some videos you can try lowering the resolution, but on retina screens the image may look blurry. Try applying the unsharp filter as shown below to improve perceived quality while keeping a low resolution:

For example, 5:5:1.0 gives a medium sharpening; for a soft one, try 3:3:1.0, and for a strong one, use 9:9:1.0. You can check this documentation for more details.

Perceived quality in moving images is very different from that of still images, so tricks like this help the video look good even if you shrink its size further.

With sharpening

Without sharpening

Alternatives to constant quality mode

ffmpeg offers more video optimization settings, such as Constrained Quality, two-pass encoding, constant bitrate, and Lossless. But they either drag out encoding time considerably, or require more trial and error to land on the right bitrate to get results similar to what you get with constant quality mode (crf).

Summary

Optimizing video for the web with ffmpeg is fairly straightforward in simple cases, and just two commands will do:

WebM (VP9)

Command flag breakdown:

• -c:v libvpx-vp9 sets which codec to use
• -crf 40 sets the constant quality mode settings
• -vf scale=3840:-2 sets the scaling filter settings
• -deadline best adjusts the compression speed/quality trade-off
• -an removes the audio

MP4 (H265)

Command flag breakdown:

• -c:v libx265 sets which codec to use
• -crf 32 sets the constant quality mode settings
• -vf scale=3840:-2 sets the scaling filter settings
• -preset veryslow adjusts the compression speed/quality trade-off
• -tag:v hvc1 makes the video work on iOS and macOS devices
• -movflags faststart optimizes the video for the web
• -an removes the audio

Tweaking the -crf values will help you find the best option at the smallest possible size.

And to embed it in the project for the best possible result, use this HTML:

Troubleshooting

• If the video comes out with a green screen when converting to H.265, try adding the -pix_fmt yuv420p flag
• If you want to check video quality with different settings without waiting for the whole thing to encode, use the -frames:v 1 option, which renders only the given number of frames from the start of the video
• If you're going to use the video for a frame-by-frame scroll animation, use the -g 1 flag. It will make the jump between frames smoother.