A standards group has completed its work on a compression technology called HEVC, or H.265, which promises to improve resolution, image quality and colour. Google's VP9 is waiting in the wings.
A key to efficient video compression is predicting future video frames on the basis of past ones. This diagram concerns "intra-angular prediction".
An array of companies have finished work on video-compression technology called HEVC or H.265 that promises better video to start with and paves the way for higher-end extensions next year, they announced today.
The High Efficiency Video Codec (HEVC) supports 4K "Ultra HD" video — and perhaps 8K as well, if the video industry can convince buyers that so many pixels are worthwhile. Perhaps more important, given how many people watch video online these days, it doubles video quality for a given network data capacity.
HEVC has the potential to spread very widely indeed. It's the successor to the Advanced Video Codec (AVC), H.264, used in video cameras, TV, Blu-ray discs, online video streaming and more. Codecs are technology used to encode and decode video or audio for more efficient transmission and storage.
To begin with, HEVC comes in two flavours: the "main profile" for conventional video, which supports 8-bit colour data, and the "main 10 profile" with 10-bit video for higher-end uses. On top of that, the standard supports a still-image format.
HEVC specification drafts have described its use up to a resolution of 7680x4320 pixels. That 33-megapixel image is hardly a near-term need, but the video and cinema industries are edging toward 4K video resolutions with images of more than 8 megapixels. HEVC's better use of bandwidth is designed to enable higher resolutions without crippling networks and overstuffing storage systems.
More HEVC work is afoot at the Moving Picture Experts Group (MPEG), which represents two standards groups, the International Organization for Standardization (ISO) and the International Telecommunications Union (ITU), which produced HEVC/H.265 and AVC/H.264.
New video uses
First are HEVC extensions that let the standard store more colour information by using in the full-colour 4:4:4 format and the horizontally sub-sampled 4:2:2 format, which preserves more colour information than the 4:2:0 format used in the first incarnation. These extensions will also support colour data "up to 12 bits and beyond", the ISO said.
Second, another range of HEVC extensions will enable 3D and multi-view video.
The colour, 3D and multi-view extensions will be done in January 2014, the ISO said.
One potentially unpleasant aspect of the technology, though, is that like H.264, it's encumbered by patents that potentially mean royalty payments for those using the technology. Such payments are common for devices such as cameras and computers, but web standards groups shun patent payment tollbooths on basic technology such as putting video on the web. That's why Google released its VP8 codec royalty free as part of the WebM project and why, presumably, it's working on VP9.
One aspect of of HEVC video compression, H.265, involves interpolation among various pixels to determine brightness. This diagram from the draft standard shows some of how that process takes place.
(Credit: ITU-T/ISO/IEC Joint Collaborative Team on Video Coding (JCT-VC))
In contrast, the ISO patent policy, which is shared with the ITU, doesn't object to patents. Instead, it requires organizations with patents that bear on standards to release them for royalty free use or license them "on a non-discriminatory basis on reasonable terms and conditions". That policy also is called RAND, short for reasonable and non-discriminatory, which means, for example, that one company can't refuse to license the technology to a rival.
According to the ISO's list of patent declarations, companies that have said they have patents that bear on HEVC (known formally as ISO/IEC 23008-2 at the ISO) include Microsoft, Intel, Qualcomm, Research In Motion (RIM), Huawei, Thomson Licensing and NTT DoCoMo.
It's not clear whether VP8 and VP9 will be able to completely sidestep patent constraints. The group that licenses pools of patents for MPEG standards, called MPEG LA, has said that 12 organizations reported that they own patents essential to VP8 use, but so far MPEG LA hasn't offered a licence to the VP8 patent pool despite looking into the matter for more than two and a half years.
MPEG LA plans to license a pool of HEVC-related patents. It issued a call for patents essential to HEVC (PDF) in June 2012. "MPEG LA is pleased to assist in facilitating a convenient, independently administered, one-stop patent-licensing alternative to assist users with implementation of their technology choices and invites all patent holders to participate," it said at the time.
More than two dozen patent owners have responded so far, and they're planning to hash out the details next month. "In February, we will be holding a third meeting with essential HEVC patent owners to discuss a patent pool license for HEVC. There are currently 25 essential patent owners participating," MPEG LA said in a statement yesterday.
MPEG LA declined to comment on any VP8 patent developments.
Google started looking into VP9 seriously in the third quarter of 2011, according to a Google presentation about "VP-Next" (PDF), or Next-Generation Open Video (PDF) (NGOV). Google is acutely aware of its competition. One VP9 goal is to "further reduce bitrates by at least 10 to 20 percent to move ahead of HEVC", Adrian Grange of the Chrome Media Group at Google, said in a presentation.
And as with HEVC, support for higher resolution is central. "HD is the new sweet spot," Grange said.
To get there, Google isn't starting from scratch. "It is an evolutionary development from VP8, not a completely new paradigm," Google said in a presentation.
According to another presentation about NGOV (PDF), a top priority is halving the network bandwidth requirement at a given image quality. Another priority is doing so with a video decoder that's only 40 percent more complex. Higher complexity for decoding translates heavier processing demands and therefore power usage, major constraints in the era of smartphones and tablets.
Although Google is leading the VP9 work, it's not alone. One element of the technology, called asymmetric discrete sine transform (PDF), is being developed as part of a collaboration with researchers at the University of California, Santa Barbara.