20 April 2011

The Virtues of Closed Versus Open

I keep hearing HTML 5 this and HTML 5 that. The funny part, to me, is that while people are making stuff in HTML 5, it is not a standard yet. The W3C (World Wide Web Consortium, the people who are in charge of defining HTML) is taking forever to settle the finer points of how the new web standard should work. I am not exaggerating too much when I say that. Consider that HTML 3 (January 1997) was released a little more than a year after HTML 2 (November 1995) had been published. HTML 4 was released less than a year after that (December 1997). In the year 2000, the W3C gave us HTML 4.01 and XHTML 1.1 to help make programmers lives easier when using new media content (notice that was when the internet started looking and feeling more like it does today). The newest version, HTML 5 is designed to make the programmers lives even easier and thus make the internet even more amazing of a place. Work on the new version started in 2008, now three years later we are still waiting. A "working draft" has been published meaning, "We think it will look something like this, so start programming but do not be shocked if we change it on you later on."

While the "draft" has been published and programmers have been slowly working with it, many companies such as Mozilla (makers of Fire Fox), Apple and Google are pretending that the draft is gold (a programming term that means, "this is the final release for production") and are running hard to make the browsers play well with the new almost standard. The problem with this is that they are almost standards, but not quite, and many of the particulars have not been settled which means that different browsers are handling things differently. For example, drop shadows in Fire Fox work differently enough from Safari and Chrome that web programmers (the people that these standards are supposed to help) have to right triple the code in order to use the cool new effect (once the HTML 5 way, once the Fire Fox way and once the Safari way). And triple is not even counting all the extra coding that has to be for Internet Explorer that does not play nice with anyone else.

Back to the main point of this post: the main reason for this splintering is because HTML 5 has taken longer to develop and get out the door than any prior version of the standard. You may ask why this is the case. Surely this is not because HTML 5 is so much technically superior that a lot of extra effort has to be taken to develop it, a lot of the technical coolness is in making the code flow freer. It certainly is not because the W3C is waiting to see how that world will continue to evolve, HTML 6 can take care of that. No, the reason that it is taking so long to get HTML 5 from draft to standard is because the big boys (that were not there, or at least were not as big, for previous versions) spend too much time fighting amongst themselves that they cannot decide anything. Google wants to support their WebM video format because it is free and open. Apple wants to use their H.264 MP4 format because they think it is the best (and they happen to control the H.264 standard), but no one else wants to use it because Apple often does crazy things that throws everyone for a loop. (For example, the whole Photoshop on the Mac issue, a summary follows.)

In case you missed it: When Mac OS X was released in March 2001, Apple caused a major stir amongst computer programmers. OS X was the first major operating system to severely break backward compatibility. Most other systems (namely Microsoft Windows) typically took great pains to remain reverse compatible while adding new features. But Apple, in a very "Steve Jobsish" way, suddenly said, "Um, world, we are going to change our core programming language in our next release (later that year) so start rewriting your programs because your old programs won't work as well." This threw everyone for a loop and the computer world was full of a lot of grumbling, mostly over the incredible expense of rewriting an application in a new language and on such short notice. Because of this last minute switch, there were few applications ready for OS X when it was released. Most of the programs that were available were from Apple itself. Adobe, creators of the extremely popular Photoshop software (and in many ways a competitor with Apple's popular Final Cut Pro and Garage Band software) rolled their eyes and released a compatible version of Photoshop a year later. Fast forward several years to October 2007 when, concurrent with the release of a new version of OS X, Apple announced that once again, they were changing the foundational programming of the operating system, this time they would end support for 32-bit programs. Microsoft made similar announcements concerning Windows Vista, Windows 7 and Windows 8. Adobe was ready with an updated version of Photoshop that moved in Microsoft's desired direction shortly after the launch of Windows Vista and 7 (Windows 8 has yet to be launched or even finalized, but is supposed to be the first version of Windows that only support 64-bit, this means that Microsoft gave the world more than 5 years advanced notice for what Apple gave a few month's notice). Does Adobe love Microsoft so much more than Apple? Maybe. Does Adobe love Windows users so much more than Mac users? Not really, especially considering that Adobe has a strong Mac customer base and the Windows customer base just recently caught up. Why then was Adobe ready for Microsoft's changes and not Apple's? The difference is in the subtleties of the timings. Apple announced their dramatic change months ahead of the release of the new OS (mid-2007), giving the company little time to do anything but sneeze. Microsoft announced their dramatic change in 2003 and 2005 for Windows Vista, 2005 and 2007 for Windows 7 and 2007 and 2009 for Windows 8. The difference is that Apple has a history of surprising everyone with big changes that are happening right away while Microsoft (and most other large companies like IBM, Oracle, Sun Microsystems and Google) tend to give other companies a lot of notice for big changes. By the way, Adobe didn't release an updated version of Photoshop until April 2010, two and a half years later. The lengthy delay and the "nasty-gram" sent just after Apple's announcement were meant to say, "knock it off or we will stop playing nice." Apple responded by not supporting Flash on the iPhone and then banned applications made using Adobe Flash tools (until the government started eying them for anti-trust law violations).

Adobe, who also sits on the W3C, wants to continue to push Flash to the forefront tauting that with Flash, users do not have to worry about the format, the people building the site do. While Microsoft is fighting for their favorite WMV format, which stands for Windows Media Video and thus no one wants to touch it even with a long stick. All this, and we have not even touched to the "groups" yet. Consider the Motion Picture Experts Group. Heard of them? Probably not directly, but they are the people that invented, and control, the MPEG standards including the incredibly popular and world famous MP3 format, MPEG 1 and 2 formats (used for DVDs and satellite TV respectively) and MP4 which is quickly replacing the previously mentioned formats. Each iteration allows for more compact files (read: faster download times) without sacrificing quality (but your computer has to do a lot of extra work). All of these companies and groups are arguing over which formats and standard must be support, and this is just for video. I am not even going to touch audio or graphics.

All of this lengthy process of deciding on which video codec to use as the new web standard, a standard that within the next couple of years will need to be revised and at that point, the most current and popular video codec can be added. But, what we see here is all of these companies realize the extreme value in being the "chosen" codec until the new standard can be published.

--

As a direct competitor to HTML 5 (at least in the mind of Apple Inc.,’s Steve Job) we have Adobe Flash. Originally released in 1995 as Future Splash by Smartsketch, Flash has been bought out twice: once by Macromedia, in 1996, and again by Adobe in 2005. According to Adobe’s computer census, Flash enjoys a nearly perfect market penetration, with recent (being the current and previous version) versions on 99% of computers on the internet. That is a huge market penetration, of which I can think of no other non-governmental product (i.e. electricity, water and sewer) that comes close. In more than 15 years, developers and consumers (mostly forced by developers) have consistently chosen Flash so strongly that even in an era of cutthroat business practices including the great Dot Com Bubble burst of 2000, Flash has expanded adoption in order to capture the entire market and has kept pace to maintain its market share. (In fairness, Flash was aided by being included as a default plugin with Microsoft Internet Explorer for a time.)

Now, let us consider what Flash has done in their 17 years of existence. Flash came from humble beginnings, as rightly it should. It was originally developed as a pen drawing tool that was built out to include vector animation for the then fledgling internet (very few people had internet access in those days). The few people who did have internet had very limited bandwidth. This bandwidth limitation kept most web pages confined to simple text and  basic backgrounds (in HTML, colors are set with a simple hex code such as #000000 for black or #FFFFFF for white, hex code takes up a lot less bandwidth than graphics). While some graphic formats were available for animation, such of Compuserve’s GIF, which included transparency too), these graphics were limited by color restrictions (GIFs can show 256 colors, 255 if one color is transparent), a lack of support (the Motion Picture Experts Group’s MPG format still is not well support in browser), a difficulty to produce (no easy-to-use MPG animation tools were around in 1995), and if developers had been able to overcome all of the aforementioned obstacles they were likely to have ended up with a “huge” file that, though small by today’s standards, would have taken web page viewers hours to download. All of this meant that the not so wonderful animation was not likely to be viewed by anyone and thus was billed as wasted effort.

The introduction of Flash changed all that. Flash used vector graphics. While not the best for every application—people pictures do not look too good in them—they are excellent for general shapes and lines. In computer speak, shapes and lines are simply geometric formulas that are easy to store, easy to transmit and easy to render.  Using HTML’s simple hex color codes made coloring them really easy too. Finally, because vector graphics are simple calculations, they are really easy to make bigger and smaller: for twice the size, simply have the computer multiply by 2 (or whatever other multiplier you want). As vectors get larger they maintain their crisp, sharp look, unlike the alternative graphic formats.

Macromedia produced two components to Flash: the authoring tool and the plugin which is used to view Flash. The authoring tool meant that designers could produce their own animations without having to wait for programmers or video producers to work with them, a huge benefit for the designer. While the authoring tool costs money, developers could show their work to the world without worrying about their consumers having the right plugin: if a consumer did not have Flash, or the correct version, they could quickly download it for free. This setup led to Flash’s strong adoption throughout the early internet among both producers and consumers.

After releasing the first version of Flash, Macromedia continued to add features that were of great benefit to developers, features that, in effect, severed the need pervasive trend (at least, to an ever increasing larger degree) of a programmer to work with the designer to make compelling content. While this may not seem significant, consider the two different worlds that programmers and designers come from and how difficult it can be to for one to communicate the vision of what they are trying to do in a way that the other can readily understand and accept.

In 1997, two years after Flash’s first release, Macromedia added a host of new features and laid the foundation that producers would be using for years to come. The next year, in 1998, Macromedia sensed that the relatively new MP3 format would become a powerful market force and added support for playing the new format from within Flash. This greatly expanded the capabilities of Flash. Prior to that point, Flash could only handle WAV and MIDI files. WAV files are actual recordings of real sound that generally include a full audio spectrum, but also includes a full file size (good quality wave audio is about 10mb a minute, far too large for the still restrictive bandwidth to allow, especially when compared to MP3’s 1mb a minute). MIDI is synthesized sounds and music; while the files are really small, they are also very limited in quality and sound more like a Queen song that the London Philharmonic Orchestra. The following year, streaming MP3 support was added allowing consumers to start listening to sounds and music without downloading the whole file.

The years 2000 and 2002 saw the introduction of Actionscript and video support respectively. The addition of Actionscript welcomed programmers back into Macromedia’s internet development arena by providing them a programmatic interface in which they could further enhance Flash animations by giving them custom logic sets (to do things that Macromedia had not imagined Flash doing) and allowed for dynamic content. No longer was the developer limited to text, words, video and information that was available at publication, using Actionscript, content could be added and updated on the fly without republishing the Flash project. The introduction of video not only extended the features of Flash, but also offered a solution to a prevalent problem across the maturing internet: producers, developers and consumer could not decide on a video production tool and plugin. Adding it to Flash made the choice simple, especially when their then more than 90% market penetration was considered. Almost everyone on the internet already had Flash, so why not use it. Additionally, Flash allow the developers to make custom interfaces so the video frame looked and felt like the rest of the site; the producers could protect their video against download and theft (though, later tools would emerge to work around this); customers often need do nothing, the magical plugin they downloaded in the past would magically play the video they wanted to watch.

The 2003 release of Flash completed the foundational process of turning Flash into a truly ubiquitous platform both on and off the internet. This release included Actionscript 2, extending and deepening Flash's programmatic abilities over the previous version that made advanced programming functions such as database interaction and dynamic content rendering more powerful and easier for the programmers; extended support for popular graphics formats, Adobe Illustrator files (a popular and powerful vector graphics program, think of it as Photoshop for vectors); inclusion in the popular (especially amongst business users) Adobe Acrobat; and charting features for business users. By this time, Flash had already achieved nearly universal distribution across internet connected computers and these additional features strengthened its position amongst both business users (who want to ensure the secure, as in difficult to steal, and reliable distribution of corporate information, and consumers who had begun to rely on the internet as not just a source of information, but a source of rich and engaging media.

After Adobe’s acquisition of Macromedia in 2005, the development of Flash continued including the release of Actionscript 3 (continuing the strengthening of the developers’ side of Flash while securing Flash by disallowing Flash to interact with the host without the users knowledge), and the inclusion of upcoming video encoding standards including H.264 and MP4, and tighter integration with Adobe’s other development environments such as Adobe Illustrator and Photoshop.

In 2009, Adobe delivered on their promised goals announced when Adobe bought Macromedia: Flash became an open standard. By publishing the specifications for Flash, Adobe knocked down one of the last barriers in adoption of Flash (not that very many people were actively opposed to it). Now everyone could see exactly what made Flash tick and could even build their own Flash plugins and development tools if they so desired. While some effort has been made to create a less expensive alternative to Adobe Flash (the development program) the comparative difficulty of programming a suitable development environment to the low cost of Adobe Flash has proved too difficult to overcome. Adobe has maintained a competitively priced product by separating the more advance Adobe Flash features into Adobe Flash Professional, leaving a lower featured and prices, though still adequate, Adobe Flash on the market and by including Adobe Flash in their Creative Suite bundles.

As to custom plugins, no one really trusts another company to develop a better, more stable plugin than what Adobe itself produces so Adobe’s Flash plugin is still the ubiquitous one plugin.

Publication of the Flash specifications signaled a clear direction for Adobe and their Flash product: they want to make money off the authoring tools, not the technology itself. This directional signal has been further reinforced by the diversification of Adobe Flash (less featured and lower cost versions) and Adobe’s strong efforts to extend Flash onto every available platform with variants including Linux, Macintosh, Windows, iPhone, Symbian, BlackBerry and Android with continued to attempts to get Flash onto ever more devices. In this context, the opening and broadening of Flash is the only course that makes sense. Any company can build Flash into their device, any programmer can build Flash extensions and any user can use and enjoy Flash.

With the completion of the Flash foundational platform in 2003, Macromedia added the last vital features needed to make the broad base of developers and consumers happy with their product. In fact, every addition and development made by Macromedia and later Adobe, have been delivered in direct response to the requests, suggestions and demands of their consumers. It is, as Adobe is well aware of, in their best interest to continue to develop the platform to meet their customer’s needs. Any slip, any mishap, will allow room for a competitor to enter. Once a competitor has a toe hold, they will have a niche within which to establish and eventually erode Adobe’s position.

--

While HTML and Flash may seem to be unrelated topics, the two stories, when compared, shed valuable insight on the processes and functionality of the two different styles of development: open (HTML) versus closed (Adobe), shared interest (or self-interest) versus vested interest (singular interest), responding to personal needs versus responding to customer needs.

In the HTML development process, the WC3 had great success early in the development process. Even though the process has continued to remain open, the self-interest of the individual companies involved has driven the development process from one of harmony and continuity into a contentious frenzy with each company seeking to position themselves in the most profitable position possible. On the other hand, the closed (though transparent) development path chosen by Adobe has led to continued dramatic improvement over time, specifically responding to customer needs over time and focusing on building a solid business framework. There is no one but Adobe to be responsible for a good product (which will make them money) or a bad product (which will loss them money). In the closed environment, the needs of the customers, their actual needs as opposed to the perceived needs that would be represented and skewed in favor of the company demanding them, are the point of focus instead of whatever will make the company a core component while leaving others to rely upon them.

(Thank you to Google and Wikipedia for providing the invaluable assistance in bringing all the details together for this essay.) 

11 April 2011

Masters of Camouflage

I was reading Alan Fletcher's The Art of Looking Sideways while pondering on my recent Disneyland experience. In his book he talk about a variety of things including the purpose of camouflage. Fletcher describes camouflage as "making the conspicuous inconspicuous or the inconspicuous conspicuous" and that "[t]he objective of camouflage is to mislead rather than conceal." I was struck at how well Disneyland is able to camouflage their operations, not by trying to make the disappear (a powerful feat that even the best magicians cannot sustain indefinitely) but rather to make them blend in or seem insignificant. This was particularly notable while waiting in line for Indiana Jones and the Temple of the Forbidden Eye, a ride that has an incredible long wait time and is entirely underground. The wait time does not seem so bad because we are presented with a vast array of fun (and detailed) things to look at. Even the fact that the entire ride is underground is misled from our mind by having us enter the "temple". Naturally, in the temple we do little to track our elevation and the gradual decline is further masked by even more intricate details to observe and look at. It is almost like an entire movie set put before us just to keep us distracted from the truth: we waited far too long, mostly underground, for a ride.

04 April 2011

The Difference Between Programmers and Designers

I was talking with codemaster in my HTML class about what all HTML 5 can do (a cool new version of web pages that can do stuff that programmers used to have to use Flash to do). He was talking about how cool HTML 5 was and I kept saying how cool Flash was. The next day I found Adobe Flash Catalyst and opened a whole new box of worms (by "box of worms" I mean "opened up a case of cans of worms and then opened up most of the cans"--that is a lot of worms everywhere). After thinking about our discussion, I realized some of the primary difference between programmers (codemaster) and designers (me):

Programmers Designers
loves to make things from scratch loves to make things look good
"cool" is making something simple that does amazing things "cool" is something that looks simple but is choked full of subtleties
wants everything to be open and free (HTML5) wants everything to "just work" (Adobe Flash)
wants to have a say in how things proceed (thus allowing everyone to have a say) wants better tools, sooner rather than later (thus allowing a few people to call the shots)
is willing to settle with fewer features, as long as they are universal is willing to sacrifice money to get features and ubiquity
can make cool stuff that works, but does not like waiting for a designer to make it look pretty can make good looking stuff, but does not like waiting for a programmer to make it work

In conclusion, I am grateful that Adobe makes tools that let me easier make Flash user interfaces (such as Flash Catalyst) and export Flash to the iPhone and Android with relative little programming effort. In fact, I feel a bit sorry that while people can make my life as a designer easier by getting me easy-to-use tools, it is a lot harder to make tools that can make visually appealing things more intuitive.