iPad, Flash, HTML 5 and Standards

Lack of Flash Support by the iPad – Bad News or Good?

A post I wrote in November 2008 entitled “Why Did SMIL and SVG Fail?” has been referenced by the Stevie 5 is Alive blog. The post on the lack of Flash support for the iPad device says “Apple: Thank You for Leaving Flash Out“.

As the author, a ‘geek and entrepreneur’, correctly points out SMIL, the open XML-based multimedia standards developed by the W3C “was virtually assassinated from the landscape“. He goes on to point our that:

“Quicktime X no longer opens and runs SMIL files (Quicktime Player 7 does, and it’s still in the spec). Quicktime on the iPhone won’t handle SMIL. WYSIWYG SMIL editors now are nowhere to be found. Evolution of the SMIL specification slowed to a crawl. The once potentially vibrant ecosystem around open standards has withered to nearly nothing – with obscure projects like Ambulant remaining as last-chance efforts to keep an open format available to the world for interactive media.“

In its place we have seen Flash dominating the market place. The problem is that Flash “is a vendor proprietary format, with a closed ecosystem. Adobe makes the flash player. Adobe makes the flash development tools. Sure some other companies provide streamlined development tools based on Adobe’s APIs (like SWiSH Max) but Adobe controls what they can and can’t do with those APIs.”

Perhaps, then, the lack of Flash support in the iPad is to be welcomed, particularly in light of the recent announcement about YouTube’s HTML5 Video Player, which does not require Flash support, but instead supports native video streaming.

A desire to move away from Flash was expressed at a meeting I attended last week when I heard that Flash seems to be blocked by firewalls in certain public sector organisations. “HTML 5 will avoid the need for Flash” was a response made to this comment, although the lack of support for HTML 5 in current versions of Internet Explorer is likely to be a barrier to its deployment.

Complexities of Video and HTML 5

But rather than a lack of support for standards being a problem, once again, for just Microsoft, use of the open source FireFox browser to view HTML 5 pages used by services such as YouTube will not necessarily work. Although HTML5 defines a standard way to embed video in a Web page, using the element. FireFox currently supports the Ogg Theora, Ogg Vorbis and WAV formats – but not the widely used H.264 format (codec).

The H.264 family of standards were developed to “create a standard capable of providing good video quality at substantially lower bit rates than previous standards“. But despite its popularity as described in Wikipedia “vendors of products which make use of H.264/AVC are expected to pay patent licensing royalties for the patented technology that their products use“.  The costs of use of the format are difficult to determine: Christopher Blizzard, in a post looking at the history of patented technologies, points out that although “H.264 is currently liberally licensed [it] also has a license that changes from year to year, depending on market conditions. This means that something that’s free today might not be free tomorrow.” As for what those costs may be an article on “H.264 Royalties: what you need to know” states that:

“… a one-time payment of $2,500 “per AVC transmission encoder” or an annual fee starting at “$2,500 per calendar year per Broadcast Markets of at least 100,000 but no more than 499,999 television households, $5,000 per calendar year per Broadcast Market which includes at least 500,000 but no more than 999,999 television households, and $10,000 per calendar year per Broadcast Market which includes at 1,000,000 or more television households.“

A Dive Into HTML 5 post on Video on the Web also points out that “The fees are potentially somewhat steeper for internet broadcasts” and “starting in 2011, it’s going to cost a whole lot more“.

As well as the issues of the licensing costs (likely to be difficult to be paid for by an open source company such as FireFox which doesn’t have an income stream related to its core product), there is also a need to consider the principles involved: the success of the Web has been based on open standards for which use has not required payment of royalty feeds.

Is There an Open Alternative to H.264?

Are there open alternatives to H.264 which aren’t encumbered with licensing restrictions? The answer is yes:  Ogg is an open standard container format for video which is unencumbered by any known patents. Firefox 3.5, Chrome 4, and Opera 10 provide native support for the format without the need for any platform-specific plugins through use of the  Ogg container format, Ogg video (“Theora”) and Ogg audio (“Vorbis”) .

Surely the answer to the licensing complexities of H.264 is simple – make use of Ogg instead?  Robert Accettura has given his interpretations of the reasons why Apple and Google appear to be willing to support H.264:

Apple’s Argument: Hardware decoding for H.264 is available on various devices (including the iPhone). Hardware decoding means the devices CPU does not have to carry out this function, resulting in better performance and battery life. As there does not appear to be a hardware Theora decoder available use of the H.264 standard can be deployed using existing technologies.

Google’s Argument: In a message sent to the WhatWG mailing list last year Chris DiBona argued that “If [you] were to switch to theora and maintain even a semblance of the current youtube quality it would take up most available bandwidth across the internet“. Although others have queried this argument (and an Ars technica post on “Decoding the HTML 5 video codec debate” explored this issue in more detail) the bandwidth costs of accessing streaming video will be a factor in choosing an appropriate format, particularly for companies such as Google who are significant providers of streaming video due to their video streaming services such as YouTube.

What is To be Done?

In a recent post on “Reflections on CETIS’s “Future of Interoperability Standards” Meeting” I described how there was a view that policy makers tended to have a naive view of open standards, perhaps feeling that an open standard would be guaranteed to provide simple, elegant solutions whilst bringing down costs by avoiding reliance of vendors of proprietary formats.  In response Erik Duval pointed out that “I certainly strongly agree that policy makers sometimes have a somewhat naive view of the standards process – but then so did we when we started this?“.

Erik is certainly correct that developers and others working in IT will have a tendency to gloss over real world deployment issues – on reflection I was guilty of this in my article on “HTML Is Dead!” which argued that the future for HTML was based on XHTML. So here’s my brief summary related to the complexities of video and HTML 5.

The video element in the draft HTML 5 standard will allow Web pages to have embedded videos which do not require use of plugin technologies (such as the proprietary Flash format which is widely used twoay).  The format of such videos is not defined in the HTML 5 standard – it is being left to the market place, the browser vendors,  to provide such support. Google (with their Chrome browser) and Apple (with their Safari browser) currently support the H.264 video format, but since this format uses patented technologies use of this requires the browser vendors to pay a licence fee.   FireFox feel that the open Ogg / Theodora format  should be used, but Google and Apple argue that this format has limitations.

Since Google and Apple are both significant providers of video and multimedia content (with YouTube in the case of the former and iTunes for the latter)  the decisions they make regarding formats for the content they provide is likely to influence the user community’s preferences, since users will have no interest in the complexities of codeces, patents, etc.

There may be ways of circumventing these difficulties, by eventual agreements by the major software vendors or by the provision of alternative environment (e.g. the Google Chrome Frame plugin for Internet Explorer or, as described in Ryan Paul’s blog post as “The undesirable middle-ground” of “expos[ing] each platform’s underlying media playback engine through the HTML 5 video element” which appears to be technically possible but would “heighten the risk of fragmentation“).

What are your plans for streaming video?!