User:Pi zero/Vision for the sisterhood

Here is an overview of how I hope to help the sisters grow, moving forward. The short-short version:

Enable ordinary wiki users to write interactive wiki pages, so that the wiki community can directly grow semi-automated assistants to aid expert wiki tasks.

The short version

Each of the sister projects, including Wikipedia, requires a lot of expert tasks. These can include content creation and review, and a wide variety of other overhead tasks such as (for a few scattered examples) opening and closing nominations, specialized archiving, and various forms of article curation. Semi-automated tools are needed, to facilitate these tasks without degrading the quality of human input into them. Human input is, after all, the whole point.

Developing these tools requires immersive knowledge of the tasks to be aided. Consequently, the WMF is inherently incapable of developing them. Successful development of such tools must be intensively managed by people with intimate knowledge of the tasks, and only the wiki community can have that intimate knowledge. This is a natural extension of the wiki model: the volunteer community grows wiki content directly instead of petitioning the Foundation for each edit they want made, and likewise the volunteer community must grow the assistants directly that help them to perform the community's expert wiki tasks.

The technical basis for this approach is a small set of straightforward templates that make ordinary wiki pages interactive. The clear objective here is to allow the building of assistants to be done entirely in wiki markup. Development of these templates, in their first production version, is nearing completion at en.wn. Once those basic elements are available, techniques need to be learned for using them effectively in growing assistants; some tentative efforts in that direction are already being made.

Design considerations

Interactivity templates

Some considerations in designing the basic interactivity templates:

Simplicity of function and ease of use, of course.
The template functionality has to be chosen to be extremely enabling of useful assistant functionality, but at the same time it has to be very robust against both (1) accidental breakage that has dire consequences, and (2) malicious abuse.
The implementation has to be robust against both (1) loss of data due to errors occurring during a dialog, and (2) breakage caused by changes to the wiki software.

The basic templates are driven under the hood by a mixture of wiki markup, html, JavaScript, and Lua, all kept as simple as could be arranged (to minimize the chance of, as noted, breakage caused by changes to the wiki software). It would clearly be possible to make it all faster by integrating it into the wiki software as an extension. However, that process would make the software less accessible for upgrades and fixes, would require a long and arduous process to get it approved technically, and imho would have no chance of ever being approved politically by the WMF, which has spent many years now committing itself to sequester structural control and flexibility away from the contributing community.

Assistants

Some considerations in designing the assistants, built on top of the basic templates:

Assistants should reduce incidental tasks, such as pure tedium and remembering to-do lists, without degrading human input into tasks.
Design should facilitate customization of assistants by the community. This may in fact be a major technological challenge, requiring an immense amount of accumulated experience to get right.
Design should help the user learn what is being done by the assistant, rather that merely do it for them. Experience with an assistant should translate into greater ability to do partly or entirely without the assistant if necessary — although assistants may be meant for experts too.
Design should degrade gracefully by naturally generating parallel how-to instructions that naturally present themselves if for some reason the assistant or its underlying infrastructure of basic templates fails (one hopes temporarily). This may be particularly challenging when assistants are provided to aid in the construction and maintenance of assistants.

Wiki markup

When handled properly, wiki markup is the best thing the wikis have going for them; for the long-term success of wikis, it needs to be streamlined and embraced — contrary to the peculiar myth, which has gained some purchase (even within the WMF, where they should know better), that wiki markup ought to be avoided. (If that myth were true, wikis would never have been successful in the first place.)

Treating wiki markup well requires understanding where its strengths come from. At heart wiki markup involves a handful of basic notations with minimal, distinctive, easily typed/read character patterns. That's only half the equation, though. The other half is the way it's learned: hands-on. When correcting a spelling mistake that happens to be near something in italics, the new contributor sees how the italics were made. When looking to add another item to a bulleted list, the contributor can see immediately how to add an item by looking at the markup of the existing list. And so on: new users are routinely exposed to how things are done, and because those things are also very easy to do once one sees how, the new users pick them up by osmosis. The learning user is generally aware of a bit more than what they've actually used themselves, because they noticed it nearby while doing something else.
A good recipe for long-term failure of wikis would be to impose a WYSIWYG interface on them, thus cutting users off from the primary vector by which they would otherwise be constantly exposed to how things are done. The whole point of a WYSIWYG interface is to prevent users from knowing how anything is done.
It is not necessary to choose between a WYSIWYG interface and a clumsy text box: I would recommend an editing interface where you click on the place where you want to edit and are taken into an editing mode with the cursor at the point where you clicked, in which you edit the wiki markup and, to whatever extent feasible, see a preview of its consequences in real time.
Requiring a segregated language for development is cutting off development from both the simplicity of wiki markup and the means for ordinary wiki users to aspire to that level of development. Building a closed high priesthood, so that expertise has to be something the user arrives already having rather than something they can incrementally accumulate. (En.wn has always pursued the incremental-accumulation model of expertise.)
Moreover, heavily procedural programming format needs to be avoided for the general wiki community; it has very high conceptual and syntactic overhead. (Notwithstanding that, of the procedural kind, Lua is a likeable language.) Initial experience applying the interactive wiki templates has demonstrated that a certain amount of "glue" code is needed to specify processing of text as it's passed from page to page; for this purpose, a template {{evalx}} has been provided (and continues to be grown, cautiously, as needed) that specifies powerful wiki-markup-oriented string manipulations using a highly customized dialect of Lisp. Lisp was considered ideal for the purpose because it has very low syntactic overhead and very low syntactic complexity. In practice, a small number of (usually quite small) dabs of glue code have been found highly effective in enabling impressive tasks, while relative to separate procedural code they have been far more tractable, smaller, and more closely integrated into the wiki markup. The Lisp dialect omits "important" traditional Lisp features that are unsuited to the purpose, such as trees (only proper lists are supported) and recursive functions (broadside techniques such as map are heavily exploited, and users can define functions but cannot deeply nest calls to them).

Implementation

interactivity templates	Help:Dialog
glue code	{{evalx}}
semi-automated assistants	WN:Assistant