IBM Wins Pyrrhic Format Battle Over Microsoft

BNET TalkbackShare your ideas and expertise on this topic

Subscribe to this discussion via Email or RSS

•  
  1

  Gary Edwards

  05/11/09 | Report as spam

  Hoist on our own petard

  Very perceptive Michael. When ODF was rel="nofollow" href="http://lists.oasis-open.org/archives/tc-
  announce/200211/msg00001.html">submitted to OASIS in
  2002 by Sun, (then known as OASIS Open Office
  XML), the world was eager to believe that an open XML
  format was the answer to application specific vendor lock-
  in. All had felt the curse of the application specific binary
  format.
  
  We all know the drill. Information that belonged to end-
  users was locked into the specific versions of applications
  used to create those document/data compounds. Binary
  document formats were to blame; and open XML formats
  thought to be the answer.
  
  Today we have two open XML formats designed to service
  the complex domain of desktop office suite
  compound documents. And there is no solution in sight
  that meets all the basic expectations (see below).
  The XML formats turned out to be just as application
  specific and bound as the binaries they replaced. ODF has
  serious interop problems, even across applications that
  share the same source code core. And OOXML's interop is
  application-platform-vendor specific.
  
  So what went wrong? Was it Microsoft's refusal to
  participate in the ODF TC (technical committee)? Or was it
  Microsoft's refusal to implement ISO 26300 in compliance
  with the ODF 1.1 standards specification?
  
  No. It turns out that ODF 1.1 has some incredibly serious
  interoperability problems, and these problems are built
  into the specification (or left in shreds on the cutting room
  floor). Not much of a secret here. Lots of people have
  been screaming about the ODF 1.1 interop problems and
  the urgent need to address them.
  
  This problem has been brewing since December 16th,
  2002, when the OASIS ODF TC met for the first time and
  voted on the proposed charter.
  The seeds of the document wars were planted in that first
  vote, setting the stage for what we see today; the ODF
  vendors hoist on their own petard.
  
  There were two points of order made at that meeting to
  stick with the proposed charter language. The first issue
  to come up was the suggestion that to guarantee a higher
  quality of document interchange, the TC consider starting
  with a clean slate rather than the submitted
  OpenOffice/StarOffice specific XML Schema. The other
  point was to put into the founding charter the objective of
  backwards compatibility with legacy binary
  formats. Otherwise, how could the public convert
  their legacy documents and information systems to the
  new XML format?
  
  These efforts to enhance the ODF charter failed. Sun and
  OASIS insisted that the OpenOffice/StarOffice XML schema
  met all the requirements stated in the charter.
  And, amending the charter to recognize the need to be
  backward compatible with legacy binary documents was
  not practical since certain prominent vendors were not
  represented on the TC. It was also noted that some might
  interpret such a clause in the charter as suggesting that
  the work would be based on and biased towards a specific
  binary format corpus.
  
  This was an odd statement if ever there was one. The
  original ODF schema was an XML encoding of the
  OpenOffice/StarOffice binary dump. The magic of XML is
  not that it's inherently interoperable. It's that it's
  endlessly customizable! XML was developed to be a
  language for custom languages. If what you need is an
  application specific markup based encoding, XML is the
  perfect language. There is no question that every
  application feature and possible innovation going forward
  can be expressed in the highly customizable XML. If what
  you seek is interoperability though, you really need a
  structured markup schema that is clean slate - application
  neutral from the git go.
  
  The ODF vendors rightfully claim that Microsoft locks them
  out of the desktop office suite marketplace, while locking
  end-users into the MSOffice productivity environment.
  Unfortunately, the ODF vendors sold the public into
  believing that this was just an issue of file formats;
  arguing that if only Microsoft would support ISO 26300,
  documents could be exchanged between competing
  applications, and competition restored to the desktop.
  
  The problem is more complex than that. It's not just the
  documents that are bound to the MSOffice productivity
  environment. These are compound documents rich with
  content, data, workflow logic and multi media - which are
  in their own right very much bound to the MSOffice client-
  side environment. Unless every aspect of these
  compounds has been accounted for, conversion will break
  both the fidelity and the embedded business processing.
  
  The short story is that conversion breaks documents. The
  fidelity conversion could be handled by highly structured
  open XML format representation, but only if the aspects of
  content, presentation and logic can be mapped without
  loss. Since neither ODf or OOXML provide the semantics of
  their application specific layout models, fidelity is
  problematic. The embedded stuff is very difficult to
  convert because it's overwhelmingly application and
  productivity environment specific. Sure you can trap this
  stuff, but conversion to another application/environment is
  a challenge. But these things have to be done if the
  interoperability sold to the public by ODF vendors is to be
  achieved; and of course, that would mean fully
  accommodating the demands of a lossless conversion
  process in the ODF 1.1 specification!
  
  The XML format specification provided to ISO by the OASIS
  ODF TC in 2005 falls short of the interop challenge as it
  was framed for the public - and everyone on that TC knew
  it. But since that rev was a 1.0, everyone also knew there
  were years of work ahead. No big deal. Except for the
  unintended consequences.
  
  At the time of OASIS ODF 1.0 approval, there were only
  two "applciaiton vendors" represented on the TC: Sun with
  OpenOffice/StarOffice, and, KDE with KOffice. It was after
  the May 2005 approval that IBM lead a massive surge of
  vendors into the ODF TC. With so much work to be done,
  these giants were greeted with cheers. And for sure, the
  big vendors brought much needed resources. They brought
  massive loads of experts, applications, testing labs and
  marketing resources.
  
  Spurred by Massachusetts, government interest in ODF was
  surging. Many thought that ODF might be an important
  first step towards restoring desktop competition. Soon
  enough though, these expectations began to focus on ODF
  1.0 being the answer to all interoperability and market
  competition problems.
  
  Even with all the new resource, it still took six to nine
  months to get the ODF Open Formula and Metadata sub
  committee work off the ground. This work was essential to
  moving ODF towards the interoperability the world
  increasingly believed was in ODF 1.0. Unfortuantely,
  interoperability was not a primary objective within
  the ODF TC. Even the comparatively easy stuff like
  cleaning up the normative language, editing for
  interoperable compliance, and separating out informative
  semantics was shelved as the pursuit for accommodating
  innovative application features took precedence. For sure
  Oepn Formual and Metadata work will work wonders for
  ODF interop, but there is no excuse for letting compliance
  and implementation language cleanup slip.
  
  And so the stage was set. ISO approval of the
  interoperability challenged ODF 1.1 set the clock running.
  he world had expectations, and ODF vendors were pushing
  governments to push Microsoft into cooperation by
  adopting ODF.
  
  Sure, people do not want to admit this. Bu this was the
  essence of the ODF promise; by standardizing on ODF,
  governments would push Microsoft to support both ODF
  and the ODF standards process. With that support, ODF
  would then achieve the interoperability that would restore
  competition to the desktop, and provide a competitive
  basis for going forward with the Open Web as a central
  platform.
  
  Sounds great. But ISO approval of the interop challenged
  ODF 1.1 set in motion another outcome. Microsoft now
  had the opportunity to support ODF without having to
  provide the kind of interoperable outcome that might have
  finally leveled the playing field. To my surprise, this did
  not happen in 2005! I thought for sure they would jump
  the moment, and muck up the future of ODF simply by
  complying with a specification devoid of compliance
  guidelines required for interop.
  
  I wondered about this. We had a worse-case-scenario
  scoped out, that was often discussed in 2006 during the
  Massachusetts document wars. The scenario was that
  Microsoft would use the interop deficiencies of ISO 26300
  to get ISO approval of OOXML.
  
  How so? Simple really. Start with the hard fact that the
  interop deficiencies of ODF 1.1 make it a safe bet.
  Microsoft could implement and support ODF 1.1 without
  having to worry about their desktop franchise being
  threatened. Whether end-user selected OOXML, ODF, RTF
  or the native binary, the documents would still be bound
  and locked into the MSOffice productivity environment.
  The interop end-users and developers seek would be
  restricted to the Microsoft Office platform. Perhaps more
  importantly, interop between a re-purposed MSOffice
  rich client and the rich server represented
  by an emerging MS WebStack-Cloud-RiA platform would be
  feature full and greatly enhanced. Enough to result in a
  better Web than the Open Web.
  
  So, with ISO 26300 as safe back drop, Microsoft would be
  able to promise ISO members something they believed
  was vital. So vital that all the ODF vendors had staked
  the future on it. The promise is simple: in exchange for
  OOXML approval:
  
  ....... Microsoft would join the ODF TC and participate in
  the development of ODF.
  
  ....... Microsoft would implement a fully compliant version
  of ISO 26300 in MSOffice.
  
  ...... Microsoft would meet all ISO requirements and
  requests regarding OOXML.
  
  After the bare knuckle battles and all out war Microsoft had
  waged in Massachusetts, the subsequent cascade of
  concessions on OOXML came as a surprise. But you would
  have had to be a total fool not to see the opportunity the
  ODF TC had created for Microsoft. Keep in mind that the
  ODF vendors were waging an all out campaign assuring
  governments that if they adopted ODF, Microsoft would
  have to concede by joining the ODF TC, and providing
  support in MSOffice. Imagine the case where Microsoft
  promises to fulfill these quests in exchange for OOXML
  approval, and you can imagine the possible eagerness of
  governments to comply.
  
  The worse-case-scenario was just speculation. But i think
  given subsequent events, one could make the case that
  this is a bit more than just another single bullet theory.
  
  At the end of the day, governments want pretty much the
  same thing as all end users. They want a highly structured
  and open format that is application-platform-vendor
  independent, universally interoperable and backwards
  compatible, and, Web ready. They mistakenly
  thought that at least one of the two XML formats would
  meet these objectives.
  
  One last point. With over 60 billion public documents
  written in HTML (or increasingly advanced HTML), and an
  estimated 30 billion behind the firewall, HTML+ is arguably
  the most interoperable format the world has ever known.
  If in that time frame of 1998 - 2003, when the W3C had
  declared HTML-CSS dead, we
  had the advanced HTML supported by the ACiD3 browsers
  today, there would have been an uproar if either
  OpenOffice or MSOffice had proposed to side step Web
  ready interoperability in favor of an application specific
  XML. No matter how open that XML might be, or how
  powerful the vendor consortia behind it, interoperability
  and Open Web readiness matters.
  
  ..... an open technology based on highly structured markup
  ....... application-platform-vendor independent
  ......... universally interoperable with backwards
  compatibility
  ........... Web ready
  
  Hope this helps,
  
  ~ge~

•  
  2

  Gary Edwards

  05/12/09 | Report as spam

  ISO Governments

  Sorry Michael. I missed one important consideration in our
  worst-case-scenario theory; without which the motivations
  for governments coming to terms with Microsoft and the
  approval of OOXML are incomplete. Compatibility with
  legacy binary formats is big.
  
  The rule of thumb here is that the more sophisticated any
  governments information infrastructure is, the more
  dependent and bound they are to client side MSOffice
  productivity environments. Uncover a sprawl of legacy
  client/server systems, and you'll find with it an
  entanglement of complex documents, rich with the
  compounds of embedded data, media and workflow logic.
  
  The older the databases, the deeper the binding to
  MSOffice productivity environments. The bigger the IT
  budget, the stronger the grip of Redmond.
  
  In short, many of the ISO governments absolutely must
  have open format solutions compatible with legacy binary
  documents. And that means OOXML. ODF simply isn't a
  cost effective solution for many.
  
  So the the w-c-s theory considered the powerful need of
  governments to find a way to coerce, demand, bend, force
  or barter Microsoft into supporting ODF. Only Microsoft
  was in position to provide a measure of support sufficient
  to capture these business and workgroup document
  complexities. Just as they did with OOXML.
  
  Note here the assumption that these governments also
  believed Microsoft compliance with ISO 26300 would result
  in conversion without breakage or information loss. A
  MSOffice ODF format was thought to be as rich in fidelity
  and business systems processing as it's OOXML
  equivalent.
  
  It's okay to laugh here because unless and until a
  government goes through the depths of a pilot study, they
  are unlikely to have realistic view of the replacement cost
  and conversion breakage.
  
  The Massachusetts pilot study measured the disruption
  and cost of replacing MSOffice with ODF
  alternatives. It was too much. So they came up with the
  ODF plug-in idea, hoping to re-purpose MSOffice
  with an ODF plug-in in much the same was as the OOXML
  Compatibility Pack worked. Very clever, but difficult.
  
  The problem was not that of producing a compliant ODF
  that wasn't broken. We could do that. The problem was
  producing an unbroken ODF in MSOffice that was
  transparently interoperable with OpenOffice!
  
  The recently released SP2 is compliant with ODF 1.1, but
  breaks the documents and is not interoperable with
  OpenOffice. This would not work for Massachusetts any
  more than the Sun or Clever Age plug-ins did.
  
  No doubt many of the ISO governments had figured out
  the same thing Massachusetts learned. A Microsoft
  compliant ODF, as capable as OOXML, was the answer.
  
  What these governments perhaps didn't count on though
  was the fact that ISO 26300 had some serious
  interoperability problems. That, coupled with a compliance
  clause more fitting to the task of wiping an arse than
  breaking an unrepentant and determined monopolist,
  created the conditions of a perfect storm. A few promises
  here and theres, and the unthinkable happened: OOXML
  sailed on home at ISO.
  
  Note that immediately following ISO approval of OOXML,
  Microsoft did in fact join the ODF TC. Why didn't they do
  this prior to the ISO BRM? As a show of good faith? No,
  they went for approval of OOXML first, fulffilling promises
  immediately after. Or so it looks.
  
  By all accounts they have been an honorable but quiet
  participant on the ODF TC. There is no leaping up and
  down, shouting about the interoperability problems. But
  then, why wreck a good thing? All they had to do was
  stick to the only target with ISO approval, ODF 1.1, and
  their game plan remains intact and on track. All
  agreements and promises fulfilled. And every ISO
  government looking at Rob Weir and the ODF TC,
  wondering what went wrong.
  
  Many wonder what is the Microsoft plan? Having been
  close to four pilot studies, i have to say that ODF was
  never a threat to Microsoft. In spite of all the sturm
  und drang, this looks less and less about the desktop,
  and more like a battle for the future of the Web, and the
  information systems moving there.
  
  I wouldn't go as far to say that ODF was merely a
  diversion. OOXML is important in the grand scheme of
  things for Microsoft. Shifting the Web argument away from
  further developing highly interoperable HTML and browser
  technologies, and towards custom application-platform
  specific XML is close enough to be a keystone initiative
  everything else rests on. Especially when you consider the
  near total monopoly Microsoft has on business desktop
  environments.
  
  When it comes to enterprises, workgroups and
  client/server systems, efforts to rip out and
  replace the MSOffice productivity environment is too
  too costly and disruptive. Sure those business processes
  and systems are eventually going to transition to a Web
  center. The question is one of cost. Web centered
  collaboration, communication and connectivity can be
  either "value-added" to existing business systems. Or, the
  systems and processes can be re-engineered.
  
  "Value-added" is less costly, especially in terms of
  disruption to day-to-day work loads. Re-engineering frees
  one from the monopolist grip, but at great disruptive cost.
  
  It's been said that Google must replace MSOffice
  on the desktop before Microsoft can replace
  Google servers. The same holds for IBM. They must
  replace MSOffice on the desktop before Microsoft
  can replace all those Lotus Notes - WebSphere
  servers.
  
  Microsoft has the luxury of being in position to re-
  purpose the existing MSOffice monopoly base to
  integrate and interoperate with their proprietary WebStack-
  Cloud-RiA servers. This is not good news for Lotus Notes
  or Google Enterprise Services.
  
  All things considered, the real battle is the fight for the
  future, where open technologies and Open Web formats,
  protocols and interfaces challenge the entire proprietary
  .NET-WPF platform of converged desktop, device and
  server systems.
  
  One question to ask about our XML formats is which one is
  a better HTML than an Ajax or WebKit HTML+? (HTML5,
  CSS3-4, SVG/Canvas, JS, jQuery). Hint hint. Watch the
  transition of OOXML to XAML/Silverlight and back. If the
  embedded business processes and environment bindings
  are intact, this is going to be an uphill battle.
  
  Another question remains that backwards compatibility?
  Can the cowboys challenged by the thrill of reverse
  engineering Microsoft techniques re-purpose
  MSOffice in the same way Microsoft has? Only this time
  for the Open Web?
  
  Certainly the Open Web needs powerful, end-user facing
  office suite editors. Yet we're left wondering how it is that
  open source champions like OpenOffice can't speak native
  HTML+?
  
  Although there are an estimated 90 billion or so HTML
  documents out there, one has to ask how many HTML+
  documents would there be on the Web today, (especially
  behind the firewall and throughout the business
  enterprise), if either MSOffice or OpenOffice could natively
  speak the same Webkit HTML+ as those millions of
  devices out at the edge?
  
  Did OpenOffice miss it's calling, and leave the Open Web
  to the ravages of a monopolist?
  
  When we came out of the Massachusetts fiasco, we
  learned a few things:
  
  ..... Compatibility with legacy documents is a big deal.
  ...... Round tripping without breaking fidelity or the
  embedded business process logic and bindings is a beyond
  a big deal.
  ..... Replacement is costly and disruptive.
  ........Re-purposing may be the only way forward.
  ......... Interactive documents trump static - even for
  archiving. (You never know when a document is truly out-
  of-process)
  ...... All involved vendors are either protecting their server
  side systems, or plotting to replace someone else's. The
  desktop is but a pawn.
  ..... Don't get caught between waring vendors. There are
  no islands of safety, so watch your back.
  ....... Interoperability is so basic an expectation, that no
  one thinks to ask.
  ....... Same with the Open Web. Being Web ready is
  assumed.
  
  Are we having fun yet?
  ~ge~