Open Standards, Why They Are Important.

As we know, open standards are not a new thing. So why is it critical now as never before? In order to understand why open standards, especially within the context of documents and other data formats that need to traverse through the internet between heterogeneous systems is a major issue today and more so in the future, we need to understand the tremendous historical and social change that we are all participating in today.

Although the majority of the population of the world today have no access to the web and the numerous services that it delivers, this is anticipated to change with the increasing adoption of Open Source Software (OSS) and the lowering in the cost of computing. Add to this the firm commitment of many of the world's governments to bridge the digital divide, we can see that within the next decade a massive change is under way.

The potentials are enormous, estimates of some less developed countries show that on line access is available only to approximately 5% (or even less) of their population. The governments of these countries would like to bring the rest of the 95% on line as they too want to build the next Google, the next Ebay, the next Yahoo and the next big thing on the web in their countries. Projects like that of the  low cost One Laptop per Child (OLPC) should also help even out this divide between the digital haves and the have nots if it is successful. The ability to manipulate and adapt data formats for local requirements in an unrestricted manner without losing interoperability is very important in this new age to allow rapid and unfettered development.

It is estimated that 3 million people join the internet every week. Think about this. More people becoming dependant on on-line services, in good times and in bad. During disasters such as happened  in Thailand with the tsunami and hurricane Katrina in New Orleans, lives can be lost due to interoperability issues between government departments that need to collaborate in a fast and timely manner to save people, deploy valuable aid and help find loved ones.

This makes unprecedented demands on not just web infrastructure but also the demand for mission critical applications to serve new requirements often on-demand with an unfettered expansion of information and related information services, much of which is encompassed within the document. The document is essentially a tool for asynchronous communication, without the need for people to communicate face to face. A word processing file is a document, a web page is a document, a video file is a document, a sound file is a document and a document that is produced on the web may be the result of a highly dynamic and complex interwoven systems and solutions that need to interoperate seamlessly.

So what is the solution? The solution resides in utilizing data and document formats based on truly open and unencumbered standards that can be read across the board by heterogeneous systems irregardless of their applications and operating systems or the hardware used to host them. It should not matter that the host and client applications are proprietary or open. The information should be equally readable to some well heeled executive running the latest office suite on his Windows based system to the old Kelabit lady in East Malaysia using a low cost Linux based box in the village internet center. It  must also be accessible to her grandson using an OLPC at the center to access educational resources or submit an assignment on-line.

Does it make sense for the government to provide documents that the old Kelabit lady and her grandson does not have access to by virtue of being inaccessible to applications on their different operating system? As citizens of Malaysia, do they too not have the rights to equality of access to their government information and services? We must therefore be always be sensitive and vigilant as to the lowest common denominator within the accessibility equation in provisioning Government services.

As a final point, just focussing on interoperability alone is not enough as you can only guarantee that interoperability as long as the vendors cooperate. In the case of proprietary data and document formats, either you buy a license from these vendors or you have to reverse engineer the format which in turn may result in some show stopping legal issues. If the vendors decide to stop supporting whatever document version that have been standardized on, or go out of business, or decide to stop supporting the format needed to achieve interoperability, then we have a potentially terminal problem. So in effect, there is a lock in to the vendor which is highly undesirable especially for governments which must guarantee public access to certain classes of documents such as land titles, often in perpetuity.

With open standards, the situation is completely different. Interoperability is made possible forever. All the specifications necessary to ensure such interoperability are publicly available and that is one of the defining criteria of open standards, so even if the standardizing body doe not exist anymore, the documents specifying the standard have been published and are usable freely, forever.

The ideal proposition then, if we can realistically achieve this, is that any protocol including various data formats and their encoding that is designed to be carried over the Internet should be able to be freely reverse-engineered and it must not be restricted by patents. Think of the internet, could it grow as it did and manage to be as resilient to natural disasters such as the Kobe earthquake more than 10 years ago if it was not built on open and unencumbered standards? In fact the internet has proven itself to be highly resilient in virtually every disaster, it is consistently hard to kill because of its openness.

My humble 2.0 Malaysian sen (about 0.5 US Cents) worth.

Comments

[Nah Soo Hoe]

The recent decision by Adobe not to allow Microsoft to include as a native option in its forthcoming MS-Office software to enable a user to save or export the contents in PDF format is a very clear example of why open standards are needed and why de facto standards cannot take the place of open standards.

The PDF format is a de facto standard set and controlled by Adobe. It also contains several patents owned by Adobe but licensed royalty-free for use. Older versions and subsets of PDF (e.g. ver 1.4) have been adapted as ISO standards (e.g. PDF/X for printing and graphics, ISO 15930, and PDF/A for long term preservation of electronic documents, ISO 19005). However, the industry mainly makes use of the published PDF specifications from Adobe rather than the ISO standards in implementations of software to use PDF.

The PDF specification are actively being developed by Adobe with no means of open participation by interested parties and control of the specification always lies in the hands of Adobe. In the case of PDF while the specs are openly available there are specific constraints in the implementation of the features in the specs. Thus, Adobe can, when it sees fit, to impose specific constraints on another party attempting to make use of the specification. In this case Microsoft is the victim.

If the PDF format had been made available as a real open standard, like ODF, then all this will not have happened. This incident should open up the eyes of those who still insist that de facto standards that are not truely open are acceptable for use especially in the public sector.

[Required]

MS has a history of raping standards. So if Adobe wants to allow that, they might have to provide the source code, after which MS would probably "embrace and extend" it, which might lead to Microsoft Reader (TM) and the end of Adobe.

[yoonkit]

> This incident should open up the eyes of
> those who still insist that de facto standards
> that are not truely open are acceptable
> for use especially in the public sector.

Dr Nah,
That is an excellent point, worthy of an entry on its own.

The overused excuse of 'because it is the most common format' ... 'used by 98% of users out there' ... 'and by default, a "standard"' can now be seriously questioned.

Im just surprised that Adobe and MSoft are shooting themselves at such a crucial time as well.

yk.