Mobile video codec policy

Provisional notes - flesh me out!

Background

Per existing file format policies (e.g. commons, WMF, en:wp; see also: file format policy), all video files that are uploaded to Wikimedia sites today use the Ogg Theora format, which is not encumbered by patents. This means that both encoding and decoding files does not require a patent license.

This is accomplished through separate means for upload and playback:

For upload, users are required to figure out how to properly encode videos based on documentation. Generally free software can be obtained for desktop platforms which can do this encoding.

For playback, we use a hybrid approach. Browsers natively supporting Ogg Theora playback using the html5 video element will use this, while others use various plugins (if available) or a fallback Java applet which is fast enough to decode moderately sized videos on desktop and laptop computers.

Mobile

On mobile (phone and tablet) operating systems that are rising in prominence today, the playback situation is different.

almost no native Theora playback
almost no native WebM playback
nearly ubiquitous H.264 playback in video element
no Java applets
JavaScript not fast enough to implement decoders on ARM-based devices
no system-pluggable codec support (eg QuickTime or DirectShow codecs for Theora and WebM can be installed on Mac and Windows)
no or non-standard plugin interfaces for browsers
custom apps could implement Theora or WebM in software, but are likely to be slow or use heavy battery life, and cannot be auto-installed as easily as a Java applet

H.264 background

H.264 is covered by many patents. These are licensed together by the MPEG Licensing Authority (MPEG-LA) as the H.264 Patent Pool. MPEG-LA collects royalties and distributes them to the patent holders.

Activities that could cause WMF to someday owe H.264 royalties, were we to use the codec:

Distribution of a "branded encoder product"
Distribution of H.264-encoded content

We can sidestep #1 by not distributing an H.264 encoder. Our video transcoding platform (Timed Media Handler) uses ffmpeg for its encoding, which in turn uses the x264 library. Under this scheme, x264 would be responsible for paying license fees.

Regarding #2, MPEG-LA does not charge royalties for content distributed via the Internet free-of-charge. This will be the case until at least 2015, after which they may choose to begin asking for royalty payments. Even though content can be distributed for free, we would still need to obtain a license.

Platforms affected

iOS
Android (4.0 has WebM support but may be slow, and has small share of existing market)
Blackberry etc
windows phone 7 (test me)
Windows 8 tablets
Nokia N-series (test this also)

Mac OS X / Safari and Windows 7 / IE9 users would also benefit from native H.264 playback, but can make use of other methods with additional software installations.

Brion's recommendations

This proposal recommends parallel distribution of video files in (Theora and/or WebM) and H.264.

Without H.264 transcodes of our videos, we essentially are locked out of showing video to casual users on smartphone and tablet devices. This is a category that's increasing in traffic and thus importance for Wikimedia readers.

The fallbacks we have available on desktop PCs (plugins, Java applet) are unavailable on these platforms.

Creating both WebM and H.264 transcodes would allow us to serve every major desktop and smartphone/tablet browser with native video.

considerations

conflates ingestion of H.264 for transcoding into a free format (which everyone is OK with) with serving H.264 (which is contentious)
- these are entirely separate matters; this proposal deals only with serving H.264
Violation of licence policies
- hence need to adjust license policies to match our desire to make content available to readers and viewers
Effects on free reusability of content
- no known effects; all files would still be available as ogg theora which is not patent-encumbered.
Violation of community expectations of free reusability of content we make available
- absolutely needs to be cleared properly with the communities, not just discussed on a tech list then passed by the board bypassing community discussion
  - Who to discuss, where, how, for how long, on what terms?
is doing the wrong thing just because Mozilla also did a good idea?
- What is the wrong thing?
Cost
- believed to be zero for the forseeable future?
Potential cost
- conceivably there could be encoder licensing costs for doing transcoding. Conceivably. Nobody seems too sure.
Effect on format wars and our partner organizations
- Mozilla has been our steadfastest compatriot in supporting Ogg Theora and WebM to the exclusion of H.264. This may change, in that Firefox may start supporting H.264 at least on some platforms, in order to play real-world existing content. Apple, Google, and other mobile platform vendors have been slow or explicitly hostile to supporting Theora or WebM at all (even Google, owner of WebM, can't make it available on existing Android devices and doesn't seem to have been pushing for hardware acceleration by device makers. We have to weigh the vague possibility that device vendors and operating system vendors will start actively pushing WebM (small) against the practical likelihood that they will mostly continue to ignore it, leaving only H.264 as a viable option for content distribution to those platforms. Generally, our problem is opposite of Mozilla's -- they want to play widespread existing content on a free platform, we want to make our content available on widespread existing platforms that don't support our preferred codec.
Would we be pushing a slippery slope to H.264-only?
- certainly one intends not.