Web Specs Logo HTML Plan

This is work in progress. It has only been looked at by Sam, Paul, PLH, and Robin. This document has no official status and was simply produced in response to a request from the AB. Some elements of this plan have been rejected previously and could undergo substantial changes before coming into effect.

Between the enactment of “Plan 2014” and the release of HTML 5.0 as a Recommendation, the HTML WG was for the most part fully dedicated to shipping, knowingly accepting the cost of such a strongly directed effort. Shipping HTML 5.0 was considered to be enough of an overwhelming objective that the group’s operation over that period can be considered mutatis mutandis as a wartime economy in which a singleminded objective primes over flourishing and innovation.

With this drive gone, the time has come to reorganise the HTML WG’s operation in the manner most conducive to improving the Web. The core ideas behind this reorganisation were published in the “After5” plan, much of which has already been deployed.

Resources available for the production of standards are scarce. Able editors with time to dedicate to new work are few and far apart, quality reviewers only have so much bandwidth, and test developers are always ridiculously hard to find. In order for us to contribute to the furthering of the Web, it therefore behooves us:

  1. To encourage greater participation — at the very least by not deterring it — and to ease the way for new participants to become contributors at all levels. Successful open projects have in common a relatively smooth path from minor fixes to expert contributions; something at which the Web standards community has been particularly inept.
  2. To deploy our limited resources in the most productive manner.

It should be noted that the ruthless efficiency applied to a single-minded time-driven release schedule entails an organisation of labour very different from the ruthless efficiency seeking the blossoming of innovation. The change is from a regimen of decrees and officialdom to one of experimentation and reciprocal obligations.

It is this philosophy and these constraints which together helm the whole of this plan. In fact, in more ways than one this is a plan against planification. It is grounded in the belief that the primary and perhaps sole function of “leadership” in the HTML project is to trust in the community’s intelligence and competence by providing support, infrastructure, and assistance for its work. Such a community can only be directed, planned, or scheduled with circumspection and Montesquieu’s trembling hand.

Before diving into further details, it should be noted that such an approach has direct implications that are worth highlighting.

Perhaps most notably, considering that duplication of work is a net loss to progressing the Web as are manual integration costs (e.g. from cherry-picking individual changes), this plan explicitly makes the WHATWG bugzilla instance at W3C the first location at which to report bugs about the HTML specification. While this may strike some as a radical change it is in truth nothing but the ratification of an existing reality. The primary benefit over denialism on this topic is the freeing of resources that it enables. The ability to escalate issues to the HTML WG is however provided for.

It must be noted however that this acquiescence to reality comes not without its pitfalls. Several attempts at finding a modus vivendi with the WHATWG have run aground on issues of IPR and escalation of decisions.

IPR problems are well-known and apply across the board to the documents produced by the WHATWG. There is little that can be done but keep the door open for a potential future agreement on IPR-friendly stable snapshots.

The question of escalation is more subtle. While the overwhelming majority of bugs processed in the development of the WHATWG HTML Living Standard are resolved in ways that meet the W3C definition of consensus, WHATWG HTML does however suffer from “too big to fail” externalities that decrease the editor’s responsibility for his actions and thereby negatively affect the specification’s evolution.

These issues must be accounted for in our plan. We must operate such that these additional desirable properties are externally bolted onto the WHATWG where needed, while keeping the cost of this forcible integration low and leaving the door open to future friendlier cooperation.

Some may be tempted to castigate this approach as “pro-WHATWG” (whatever that means) or that it carries an existential threat to W3C. Neither of these straw-people strikes us as carrying value. To the first one we shall simply point out that there are no sides but the Web. To the second we can only note that if the W3C’s advantage sits solely in its IPR and consensus policies but the WHATWG is superior in all other important aspects then the only existential threat to W3C would be W3C itself. It is the W3C’s sole responsibility to excel at standardisation, independently from what other organisations do.

To excel is not out of reach, or perhaps even so far as it may sometimes appear. Altogether too often W3C working groups have constrained their thinking about how to operate to a set of behaviours perceived to represent the requirements placed upon them by the W3C Process. Much of the time, these constraints are imaginary. While there is no denying that the production of royalty-free technology and a strong preference for broad consensus do place some demands on a group’s operation, there are many ways in which a group can carry out its day-to-day work without such requirements getting in the way.

Note that while this document specifically targets the HTML WG, it can also read as a broad set of suggestions for reform in the manner in which groups in general operate, most specifically large groups with broad remits.

Organisation & Work Mode

Over the past two years, the core HTML WG has been productive almost exclusively at shipping HTML5. The parts of the group that have produced novel, useful extension specifications are some of its task forces such as for example the Media TF or the Editing TF.

Certainly the adamantine focus on shipping that dominated the core group is part of this, but not alone. There are several issues with working as part of a broad group:

A single group is not without its advantages (notably built-in coordination) but in practice they are outweighed by the downsides. It is notable that groups that operate as one large entity have resorted to filtering mechanisms, and in effect many participants are known to tune out the parts they do not directly care about.

As a result, we recommend the following organisation for the group: focused topic areas, some of which are currently handled as separate Task Forces, are to be handled in their own Community Groups (replacing the existing Task Forces). The list of CGs need not be fixed, on the contrary it is expected to evolve over time. Examples of possible CGs are given below.

Naturally, this does not instate a mandate to produce any kind of work. Participation commitments in W3C working groups are made under the agreement that the group’s intellectual property scope be sufficiently clearly bounded. One does not simply add a deliverable to a working group if it is not in scope. We can however expect the upcoming HTML WG’s charter to define its IP boundaries in a manner similar to the current one. While not open-ended, these boundaries are broad enough to accommodate new work that cannot necessarily be fully predicted two years in advance. This flexibility is to our advantage since it affords us the ability to reach intelligent judgement calls through concrete discussions (on public-html-admin) as to whether a proposed new work item fits within the established boundaries rather than constraining us to operate within the limitation of prescriptive foresight that, with two years to look ahead to, none of us has.

CGs can organise as they wish to subject to the W3C Community Group process. They can involve broad participation. Their process can be radically lightweight, while still operating within the boundaries of the W3C Community Group IPR policy.

CGs can notably publish forkable specifications (see notes below on licensing) so as to encourage participation by avoiding contribution-hostile constraints [NOTE: the ability for CGs to use such licenses needs to be validated by W3C legal]. They need not rely on communication methods that are often disliked by today’s Web developers and can easily make use of more modern alternatives (more on this in the tooling section).

Under this regime the core HTML WG would have as one of its functions to serve as a coordination forum between CGs, helping to sort out problems that may need cooperation amongst them or with the core HTML specification (publication details for which are outlined below). It can also serve as a catch-all group for work that does not yet have a home of its own. At heart, this coordination is meant to enable the involvement of a broader community when it is needed. This may happen when a CG does not feel comfortable that it has the necessary knowledge or context to make progress on a topic, or when dissent amongst groups or inside of one would be helped by a discussion of the whole village. To extend the metaphor, it should be noted that if one is to call upon a gathering of the whole village for every minor issue and squabble, the villagers might not consider one favourably for long.

Such coordination is meant to be called upon solely as needed, and not enforced through a constant flow of artificial and seldom read activity reports as has been the case in the past in what used to be known as “Coordination Groups”.

Technical disputes are encouraged to be addressed through forking and merging (since CGs can avail themselves of forkable licenses), the goal being for dissent to operate on camera-ready documents. Only once a pull request has been thoroughly discussed with still no resolution in sight should a broader assembly be convened. Once it has, the usual machinery of group decisions apply. Disputes that are not grounded in a concrete proposal, implemented through a pull request or similar mechanism, expose themselves to summary dismissal. Perhaps counterintuitively given the naming, the primary goal of a forkable license is a merging process.

Over the past two years, the HTML WG has worked in tandem with CGs, notably the Responsive Images Community Group. One of the primary complaints has been the overhead imposed on such groups to report on their status and produce heartbeats. To address that, CGs that produce Final Specifications can bring them to the HTML WG as a way to get them to Recommendation relatively efficiently, provided they have undergone credible levels of review, notably horizontal review (I18N, A11Y, etc.) which naturally includes a reasonable amount of testing. As experience is gathered with this division of labour, the HTML WG could set up procedures allowing for the automatic publication of CG Final Specifications as Candidate Recommendations (which can be a First Public Working Draft at the same time).

When a Final Specification is accepted for publication by the HTML WG, if its content is intended to replace a given part of the HTML specification, the part in question is then removed.

Community Groups have a broad mandate to organise themselves in whatever creative manner they see fit, provided that:

Spreading the work in several locations does have the downside that it can make finding the proper place in which to participate harder. This will be addressed by making the HTML WG’s home page essentially be a participation map, very clear and above the fold, that points to every group with details about the topics covered there.

Additionally, in order to facilitate the work of people who wish to provide feedback, each group will document how it prefers to receive feedback and what its high-level policies (e.g. commit-then-review or the reverse). The goal here is notably to simplify the lives of horizontal groups that have to deal with many different organisations that can be hard to guess.

CGs will naturally require some guidance in order to produce good specifications. With this in mind it would be helpful to produce a resource pointing to the various existing (but scattered or unfinished) resources about writing good specifications, and to point out specific details about what make good extension specifications.

The expectation is that the HTML WG would operate under the 2014 Process (without which the useful mouthful that is the “First Public Working Draft Candidate Recommendation” is not possible).

The current (more recent) work mode of the HTML group is derived from that of the WebApps WG. Usage of regular telcons should be revisited by the group, while of course ad hoc ones can be scheduled (and CGs are allowed to make use of them is they so desire). If expressly requested by direct stakeholders, a coordination call between CGs may be held, for instance on a monthly basis. Occasional face to face meetings are to be expected, but may not involve the entire group.

Note: in the past some provisions of the Invited Expert agreement have forced individuals to choose between participating in a given W3C activity and a similar activity elsewhere, to the detriment of participation. By the time the charter for this new group is approved, the 2015/01 version of the Invited Expert and Collaborator Agreement is expected to be in effect, which will address this problem.

It is also worth noting that while CGs have been chosen as an organisational vehicle here in order to benefit from their flexibility notably with respect to the licensing and invited expert issues, it could be preferable to solve those issues at the WG level. Should that happen, using TFs instead of CGs should work just as well. Ideally, participants should be shielded from such discussions and from the operational differences they imply.

Open Tooling

Both the HTML WG and its related CGs can make use of whatever tools they see fit for a given specification, but we call out a number of options here as they are expected to take up a central place in these operations.

For drafts under the control of the HTML WG (since this is not an option we can offer CGs yet) the new “Echidna” automated publishing system that is to be deployed on labs.w3.org in early February 2015 is strongly encouraged. This tool essentially allows a group to use http://w3.org/TR/ for its Editor’s Drafts. Editors can set their draft up such that any commit triggers a new publication there. This is designed to avoid specifications in TR space from going stale. As many Working Drafts as are needed can be published, every day.

Groups that so desire are encouraged to make use of GitHub, not just for repositories but also for discussion if that is their preference. A tool that makes it possible to backup not just the git part but also the issues, comments, etc. is expected to be made available. This should address concerns over migration should the need arise.

Discourse is already available for broad discussions with the community. Groups are encouraged to request new categories that match their needs better than those already in existence (as was done by the asm.js folks). The current administrators are happy to mint moderators and support the mode of operation required by groups wishing to work there.

Finally, the WebSpecs project is open to every project that wishes to use it as a publishing platform at which to maintain its documents. The project is specifically designed to publish from GitHub repositories with very minimal overhead (once you have a specification ready, integrating the project is pretty much one pull request away) while avoiding the scattered nature and complexity of dozens of GitHub repositories that can be hard to find. It also operates on the core assumption that multiple forked version of the same specification can be maintained at the same time in order to facilitate discussion (and, eventually, merges).

Core HTML Draft Publication

While the CGs may publish as they please, the HTML WG at present retains the responsibility of publishing the HTML specification itself. The publication is expected to operate under the following requirements:

Note that “additive” describes features defined as part of the HTML WG or one of its related Community Groups. New features defined in the WHATWG, if acceptable and deemed to match reality, will simply be included.

Frequent publication is expected to be achieved through two-fold automation. First, the draft is maintained as a set of transformations to the WHATWG’s version using a tool called “Spork”. These transformations are specified, as code, by the editors in charge of the HTML draft at W3C (the list of whom still needs to be established). This avoids synchronisation issues that have made work slow and painful in the past and serves to maintain the Landscape Document as direct documentation of Spork’s operations. The ideal goal is to reach a point at which the W3C HTML specification is just a subset of WHATWG HTML with two types of excisions: parts that do not match reality, and parts that are being developed in separate documents. Second, the output from these transformations is to be fed into the new Echidna automated publishing system and published under /TR/html.

It should be noted that while a strict subset is the desirable option, there are cases in which it is not entirely possible. To take an example from HTML 5.0, while in an ideal situation the ruby section of the HTML 5.0 specification could have been dropped and replaced with a separate extension specification, the interactions of that feature with the HTML parsing algorithm did not make that a viable option. Keeping such cases to a minimum is, however, an express goal. When needed, they are to be handled in the manner described in the paragraph on technical disputes.

The astute reader may note at this point that this organisation takes steps towards establishing the WHATWG as one CG (which it already is) that publishes through its coordination with the HTML WG, with modifications made to account for the atypical nature of the document. There are several ways in which such a setup would be desirable, and the organisation is deliberate. However it should be noted that it is not at this point a political reality, that making it so would require a number of issues between the two groups to be addressed, and that no claim is made here about endorsement from the WHATWG for this plan.

During the drive to get HTML 5.0 to Recommendation, the process employed was one of manual cherry-picking of individual WHATWG changes. Editors would both apply their own changes and integrate upstream changes from WHATWG HTML. One purported benefit of this approach was that HTML WG members were given explicit batch notifications of changes and had the opportunity to disagree with the acceptance or rejection of upstream changes. In practice, this opportunity was seldom used and the cherry-picking process was essentially a very costly way of accepting WHATWG changes. This proposal removes the cost while still making it possible for participants to maintain the ability to review accepted or rejected changes simply by tracking the commit logs from the upstream WHATWG HTML and Spork repositories. We would be willing to entertain discussions of alternatives, but only if such proposals include details describing how the both financial and technical aspects will be addressed.

The DOM specification is expected to follow a similar treatment, modulo the fact that strict subsetting (corresponding exclusively to test-failing) is far more likely. Having said that, some interoperability issues with the DOM that cannot be subsetted away are currently pending implementer input, and without such input the specification may, according to the principles expressed herein, find itself on indefinite hold.

The expectation is that the HTML specification will go to Recommendation every year, based on the features it contains that have tests and pass them sufficiently well. Rules concerning which tests ought to pass or not tend to have nasty side-effects (e.g. removing tests to ensure that the document passes and other such weaselling). Instead, the recommended practice is to make use of judgement calls when assessing interoperability, the key components of which being rough usability by developers and improvements compared to the previous year’s situation.

Even with the improvements to the errata process that the AB is considering which includes a number of simplifications, one has to question whether it is useful to maintain errata against HTML. Errata are valuable in the case in which no new features are being added to a specification, and patent commitments that were made for an existing Recommendation may no longer be made for the revised one.

HTML is not in such a situation. HTML is seeing new features, and we are not seeing major shifts in participation that impact patent commitments. The expectation is that publishing a Recommendation every year is sufficient to obviate the need for errata on previous Recommendations. Where required, these could be exposed as a diff of the documents as is likely to become possible with Process 2015 should the AB enact its modifications to the handling of errata (note however that such a diff would likely be large and hard to review).

Bug Handling

Over the past few years HTML-related bugs have been filed in both the WHATWG and the HTML WG. This duplication is at best a waste of effort and at worst induces confusion on its own.

In the future, when finding a bug in the core HTML specification, reviewers are expected to file it first against the HTML component of the WHATWG product in W3C’s Bugzilla. This proposal may not be consensual, but experience with handling the duplication has proven unpleasant (and wasteful). Alternatives suggestions are welcome; but the goal is just to be simple and efficient since there’s enough work for everyone and then some.

If the resolution of a given bug is not satisfactory (either to the reporting individual or to a third-party), it should be duplicated to the W3C Bugzilla HTML product for further discussion in the HTML WG, or to the corresponding CG bug tracker if applicable. Specific groups or individuals who have had clearly unpleasant interactions with the WHATWG in the past and feel it would be demoralising or demotivating to interact directly with the WHATWG to report HTML bugs can, after discussing the issue with the chairs of the HTML WG, skip straight to reporting bugs to the HTML WG. But in no case should bugs be reported twice as a first step.

For new features, the default mode of operation is for them to be defined in a separate document. They should therefore be directed to a CG if one exists that matches in scope, or failing that to Discourse where the community can help orient the proposal to its proper home, setting it up if needed.

A large bug backlog has accumulated while the HTML WG focused on shipping HTML 5.0. These bugs need to be ventilated and triaged amongst those that have been overtaken by events, those that have been fixed, those that aren’t relevant to the HTML 5.0 or HTML 5.1 document and those that, shockingly enough, need to be fixed.

Triaging the latter will require knowing where they need to be sent. This depends largely on the setup described herein coming into effect. It should be noted that some CGs may prefer bug tracking systems other than Bugzilla. In such cases the bugs will be moved there and the original tracker appraised accordingly.

Licensing

It is the de facto expectation of would-be contributors today that a project calling itself “open”, be it source, science, data, or standards, would operate under a contributor-friendly license in which rights granted for usage and modification are reciprocal. When developers learn that the W3C does not publish under such an agreement, they are baffled and at a loss for words. It simply does not fit into the W3C’s image as a place defending openness, and it certainly deprives the W3C’s standards from being considered open, let alone for W3C to take a credible open stand for anything.

In view of our goals for increased participation, and considering that all standards should be open, this plan’s recommendation is for the production of all CGs related to the HTML WG to be licensed under the W3C Software License. This license not only can be applied to specifications, but it also affords the guarantees desired from CC-BY while remaining GPL-compatible. It is also the default copyright license in the W3C Member Agreement (last 2 sentences of 7b) which states that “copyrightable materials (…) will be made available to the general public pursuant to the then-current W3C Software Notice and License.”

Adopting the W3C Software License for the HTML specification itself is also highly desirable, but would require the agreement of the AC. CGs however can freely adopt the license they desire and we suggest that they should use the W3C Software License unless overwhelming reasons not to are put forth.

Note: using a license explicitly labelled as being for “Software” might strike some — notably lawyers — as strange, and justifiably so. PSIG is currently discussing an update that would change the license’s title and replace “software” with “work” to clarify this point. This approach is in line with the general thinking of the open licenses community on this topic.

Example CGs

The set of CGs related to the HTML WG is meant to be dynamic and evolve over time in a flexible manner, the sole constraint being the chartered boundaries of the HTML WG. The following are however a few examples that could provide a good starting point based on existing experience.

Editing
The existing joint HTML-WebApps TF is already well under way but it is hitting up a few Invited Expert issues and could live with being a CG rather than the always clunky joint Task Force.
Media APIs
New Media APIs are needed, and some coherence could usefully be brought amongst all the people extending HTMLMediaElement. Some recent ideas about media controls could also fit here. Note that there is an existing HTML WG Media TF. Whether it should be extended to include new deliverables or whether two groups with different media aspects ought to be considered is left as a decision of the interested parties.
Accessibility
Much useful work is still needed in order for the Web platform to be accessible. While accessibility-review of other groups’ work is best conducted directly in those groups, this CG should define the technologies that are directly accessibility-centric. There has been some degree of dissent concerning some of the outputs from this task force; it is currently working on defining its upcoming work items.
Canvas
The 2DContext is an area of continued interest and given its specific, focused work could constitute a useful CG on its own.
Markup for Application UIs
There may be a set of elements that could usefully be added to HTML for (semantic) application components. Some work has already kicked off (in a deliberately discreet corner) about this and there is interest from some browser vendors. Also: form controls are still a problem.

Schedule

While the new group is meant to be flexible and oriented towards action — and therefore less amenable to centralised planning — the following schedule is expected to be applied:

Discussion of this plan with the WG
ASAP.
Date of republication of HTML 5.1 as an Editors’ Draft
In February. The new Echidna system is expected to support the publication. The goal is for a Recommendation by year end 2015.
The mode of operation outlined in this document
Assuming the HTML WG can agree to it, it is expected to be deployed during the coming HTML WG face to face meeting in April 2015.
Rechartering of the group
Much of the HTML WG rechartering depends on agreement about this plan. Assuming the April milestone is hit for that, a new charter can be proposed to the W3C Membership shortly thereafter.
New CR for the DOM
As explained above, our requirements to focus on interoperability are blocking the DOM specification’s progression along Recommendation-track until at least bug 22960 and bug 19431 are addressed through implementer decisions. Given the current lack of interoperability any alternative blesses an option with insufficient real-world support.