Message substitution

Following is a discussion of possible enhancements to the MediaWiki namespace feature.

The mediawiki namespace is a great idea, but it tries to address three issues, and deals with two of them unsatisfactorily. The issues are:

User interface text and translation, which AFAIK seems to work fine
Meta text: housekeeping with messages about articles (vfd, stub, copyvio, npov), and things like compact TOC, etc.
- The natural way to do this in the current setup would be to use {{msg:}}, which would allow the text to be changed in all articles at once. However, it turns out that the messages are too sophisticated for simple text replacement, and the system also has the recently found problem with backlinks in changed messages, which messes up "what links here" for pages, linked from messages. It looks like {{subst:}} will have to be used, which kind of defeats the purpose of introducing {{msg:}} in the first place.
Insertion of repeating chunks of text, possibly for further editing (for example inserting templates of complicated tables)
- {{subst:}} is useful for this, but cumbersome: you have to save the page, then press edit again. It also hurts the servers somewhat.

The solution for the latter two issues is fairly simple:

Insert meta messages into text as links. Use [[msg:messagename]], to be replaced by the text of the meta message when displayed.

Store these links as backlinks to meta message pages and show articles with a certain message in "what links here" for the meta message's page. The pages that are suspected copyvio could then easily be seen on one page, without links from numerous other pages that point to wikipedia:wikipedia:copyvio.

Ignore links from messages in "what links here", because they are problematic and not of that much use anyway. Do we really want to know which articles link to wikipedia:wikipedia:Copyrights because they're listed as copyvio?

Possibly combined with a log of linking/delinking events, the "what links here" lists for meta messages could replace the current central housekeeping pages (vfd, copivio,...). Most discussion and voting on individual articles that is currently conducted in the housekeeping registers, could be moved to subpages of articles themselves (pagename/delete, pagename/copyvio), linked from the "what links here" lists. The system would be more reliable and simpler to use. Views and saves of some very long pages would happen far less often, which servers would probably enjoy. Central pages should be retained for dealing with issues involving more than one article.

Use {{msg:messagename}} to display a message on a page without showing the page in "what links here" for the message, to be used for displaying a message on a policy page or a how-to page, without showing the page in the list of, say, candidates for deletion.

Establish a general convention that {{{varname}}} is replaced with value of the variable when the page is saved, so {{sys:currenttime}} shows current time, while {{{sys:currenttime}}} shows time at page save.

Thus, {{{msg:messagename}}} would work exactly like {{subst:messagename}}works now.

Enable parameter expansion for meta messages. Use {{$1}},{{$2}} etc. to specify parameters in the message source. Use the [[msg:messagename|param 1|param 2|..]] syntax to specify parameter values in article source.

Render the text of messages in articles at page save and store statically. On page view for articles that contain one or more messages, check if messages are newer than stored rendering. If so, render again and show. It will allow expected behavior of dynamic messages without need for rendering at every page view and without updating all concerned messages at the same time. This means messages with parameter expansion should be even cheaper in processor usage terms than the current setup.

For semantic reasons, create a separate namespace, WikiTemplate, to store templates, i.e. useful chunks of text to be perused in article (and are not about the article). Treat them in much the same way as meta-messages.

Use [[template:templatename|param 1|param 2|..]] to insert things like compact TOC, standard tables, etc.

Use {{template:templatename|param 1|param 2|..}} to display a message on a page without showing the page in "what links here" for the template, so we can distinguish pages that use the template from those that talk about the template.

Use {{{template:templatename|param 1|param 2|..}}} to insert chunks of useful text into the article source, to be edited at will. Replace these at "Show preview", to avoid unnecessary page saves.

All of this should be useful and relatively non-problematic to implement and should actually decrease the load on servers, so IMO, it should be done.

</pompous mode> I've been thinking a lot about wikipedia's practical sides lately and have come up with some ideas that I tried to present on this page in brilliant prose (they somehow always turn into semi-formal proposals). But it looks like this page is hardly read - there are few reactions to any of the ideas on it.

Anyway, I may want to implement some of this myself. I can do a wonders in php and perl and to a somewehat lesser extent, in mysql, but I don't know how to go about it. Please help. I don't have access to a server where I could set up a test site and I'm no sysadm either. I'm not even that familiar with CVS, but I can learn.

I'm not paranoidly scared of mailing lists, but I prefer wikitalk, and I also think these ideas should be discussed here -- that's what the page is called. So please comment.

Zocky 10:28, 15 Dec 2003 (UTC)

So, is there any way this could be implemented with Field-value pairs instead? --Evan 00:29, 16 Dec 2003 (UTC)

The MediaWiki namespace was introduced for one purpose: internationalisation. To make for easy translation, it was necessary to allow message substitution in the Template:All messages pages. This is implemented using MSGNW. I realised that this could be useful for other things, but not wanting the article text to start looking like some sort of cryptic programming language, I added SUBST, which allows messages to be substituted into the article text.

Now I happily admit that I don't know the ideal way my features should be used, and I'm quite happy for people to use MSG for boilerplate text if they feel that is best. But problems with MSG do not defeat the purpose of implementing it -- the purpose of implementing it was for internationalisation.

If you don't want links in MSG substituted text to be included in the link table, you need to somehow disable the link cache while they are rendered.

Changing the syntax of MSG tags from Template:MSG to [[MSG]] will not automatically register the messages in the link table. In the code, the syntax used is distinct from linking issues. However, I would prefer to avoid square bracket syntax as much as possible, since it clashes with the article namespace.

I have considered the need for a syntax for variable substitution at save time. It should be possible to bring variables and messages into the same namespace -- I don't see any need for a "sys:" qualifier at the current time. However, I was thinking of a syntax which is more visually distinct than triple-brace versus double-brace. For example, Daniel Quinlan suggested using {{<message}} and {{>message}}.

If you have read our code, you would know that $1 and $2 alone are already used for parameter substitution in messages, and I see no need whatsoever to change it to something longer and uglier, like Template:$1 and Template:$2. Template:Message name syntax has been suggested (by mav?) and it seems fine to me. I guess {{>message_name|p1|p2|p3}} would be okay too.

As for rendering as the page is saved, and then checking if the message has been altered at view time, you might want to check the profiling data before you go implementing something like that. I doubt it will save much time, except perhaps for Template:All messages and similar pages with large numbers of uncached messages.

I'm not sure I understand why you want a template namespace.

The best way to get involved with programming is to do the following:

Download the code
Get it working on your home computer
Write something useful
Submit a patch to the mailing list
Repeat 3-4 until you get CVS access
Become addicted to developing for Wikipedia and write lots of good stuff

Note that the IRC channel #wikipedia at irc.freenode.net is a common hang-out for programmers. You can often get advice there.

-- Tim Starling 00:11, 16 Dec 2003 (UTC)

Macro, argument functionality edit

Moved from w:Wikipedia talk:MediaWiki namespace text

Copyvio and other messages that need to be filled in can be useful even without macro/argument functionality. They could be used with SUBST, so they are rendered, and edited to fill in. But this is cumbersome - you have to save the page, then open it for editing again. Maybe SUBSTED messages shuld be expanded on "show preview" or a "Expand substs" button is waranted? Zocky 20:37, 13 Dec 2003 (UTC)

"What links here" solution edit

Moved from w:Wikipedia talk:MediaWiki namespace text

I don't know if someone has already suggested this, but why not solve the "What links here" problem with msg links by making the msg link act like a regular link as far as "what links here" is concerned? i.e. under "What links here" for MediaWiki:stub would be all articles that contain msg:stub.

Also, MediaWiki namespace articles could act similar to redirects, i.e. Wikipedia:Find or fix a stub under "What links here" would show MediaWiki:stub, and all the pages that contain msg:stub nested under that. I hope this may solve the problem of links not being updated. Evil saltine 13:10, 19 Dec 2003 (UTC)

May be it's simpler to make software consider all msgs as links? Then we'll find all stub articles by looking at 'What links here' on 'MediaWiki:stub', all legal stuff on 'MediaWiki:legal' etc. It will be simple and reliable. As for substed old format I think it will be converted to msg at some point Ilya 05:45, 19 Dec 2003 (UTC)

That might make sense. It would also be nice if the links would be refreshed when the MediaWiki page is updated, which shouldn't happen too often. That's because a MediaWiki text might contain several links, especially when the [[Category:Xyz]] scheme is introduced.

If we all lobby User:Tim Starling maybe he will change it for us. He's a nice guy. :-)—Eloquence

Thanks for your support. :-) Evil saltine 13:22, 19 Dec 2003 (UTC)

Try searching for {{msg:stub}}. This will give you a lot more than "What links here" will for Wikipedia:Find or fix a stub. Actually, adding a URL to the latter which does the former would probably be a good idea ([1] seems to work :-) Phil 14:19, Dec 19, 2003 (UTC)

I forgot to say: I have now done this BTW. Phil 17:31, Dec 19, 2003 (UTC)

I like what Evil proposes :) but I would say that in addition to whatever link there already is, we add a link to the MediaWiki message itself (for example the full stop at the end). This way there will be at least two "What links here" ways of finding messages. Also, I don't think it is a good idea to point everything just to the MediaWiki pages. Dori | Talk 17:27, Dec 19, 2003 (UTC)

Yup, that's what I've been saying all along. msgs are a great way to do houskeeping. "What links here" for them could mostly replace vfd, cleanup, stubs, copyvio etc. pages. And the links that appear inside the message when it's shown should be ignored in "what links here" for relevant pages. Do we really want to know which texts point to wikipedia:copyrights because they have a copy vio notice?

Moved from w:Wikipedia talk:MediaWiki namespace text

We're trying to kill to many flies with one sledgehammer. This is now a translation/general text insertion tool, which we are trying to use to do housekeeping. I think we should concentrate on what needs to be done, not on what current tools allow. IMO, the following would be the rigt way to go. note: this is not about the particular syntax I use in examples below. It's about identifying functionality we need to make housekeeping easier and better.

What we now call messages should be status flags for articles. To avoid confusion and syntax creep, they should be inserted just like language links, but with a logical keyword: [[status:vfd]], anywhere in the article. The status message (if any) should always be displayed at the top of the article, and be visually different from the article text.

These links should show up in "what links here" for the status message pages. These "what links here" pages would almost entirely replace central housekeeping pages, like wikipeda:copyvio, wikipedia:vfd, wikipedia:cleanup, etc. The article-specific discussion should go to the articles's talk. All links produced by expanding messages should be ignored for "what links here" purposes (we don't want to know which articles link to wikipedia:Copyright because they're on copyvio).

Some parameter expansion in the messages ($1, $2) would be nice, but we can live without it as long as {{PAGE NAME}} macro is provided, so we can use it in the message defintion to provide links to article's talk page or subpages: [[talk:{{PAGENAME}}]] or [[{{PAGENAME}}/temp]]. The status messages should be uniform and all other variable text ("this text is same as URL" and signatures) should go to the article's talk page anyway.

Some additional, partly related ideas (optional reading)
- If a message needs to be displayed as an example (like in this talk), the same syntax that displays variables should be used, so {{status:copyvio}} should display the copyvio message, without making the page show on "what links here" for the copyvio message. Adopt this syntax for other things: {{21 December}} - format dates without linking!

- [[[page name]]] could be a hard link to the page, turning any automagic off. So [[[eo:Slovenio]]] should show up in text same as an intra-wiki link and lead to the Esperanto page.
- {{{state:vfd}}} could do what subst does now, that is be replaced with message text permanently. {{{VARIABLE_NAME}}} could be replaced with the variable value at save time.

That's it. Anybody following me? Zocky 01:05, 20 Dec 2003 (UTC)

Nope. :) Seems confusing. The <nowiki> is already used for doing some of what you suggest above. Angela. 01:19, 20 Dec 2003 (UTC)

Hm... maybe you should read it again. None of what I talk about has anything to do with <nowiki>. Zocky

The bit about turning automagic off and showing dates without linking read to me as wanting them to be <nowiki><nowiki></nowiki>'d (or use the colon option as Eloquence mentions below. Angela. 01:38, 20 Dec 2003 (UTC)

I don't want to <nowiki> dates. I want to have date formating automagic -> December 21, 21 December, without overlinking to the page for the date.

Anyway, this is not about the choice of syntax. It's about identifying what functionality we need for housekeeping to work with less manual work. Zocky 01:51, 20 Dec 2003 (UTC)

It's a bag of stuff. I sympathize with the notion that metadata and transclusion are separate things, and should be treated separately in terms of syntax. Magnus Manske has written a category scheme, so maybe it will be possible to associate a flag like [[Category:Stub]] with a MediaWiki text that is displayed at the top or bottom of all pages in that category.

We have a syntax for overriding "magic", it's a leading colon, e.g. [[:eo:Slovenio]]. A Message substitution macro would definitely be useful, but mostly I'm looking toward substitution macros for MediaWiki namespace text.—Eloquence 01:33, Dec 20, 2003 (UTC)

Re: Category:stub: As long as it's separate from real categories and show up in yellow. Maybe Wikipedia category:stub? But I wouldn't tie it to categories. This can be implemented simply and quickly, categories are a much tougher cookie then they seem (but that's a whole another discussion).

And as for the [[:eo:Slovenio]] thing - I know it exists, but I always found the syntax somewhat arbitraty. Oh, and I do wish we had normal character escaping with backslashes... Zocky

You're too perfectionist -- in my experience it's generally a bad idea to come up with a very complex scheme and then expect someone to implement it. Unless you're willing to do it yourself, it will not get done. People prefer to work on their own complex harebrained scheme. So it's easiest to convince others to make small incremental changes. As such, I would favor first getting the Category scheme in place and then refining it for things we are currently doing with [[mediawiki:{{{1}}}|message with id '{{{1}}}']] ([[mediawiki talk:{{{1}}}|talk]]). And the things we are currently doing with [[mediawiki:{{{1}}}|message with id '{{{1}}}']] ([[mediawiki talk:{{{1}}}|talk]]) we did manually previously. So we're getting better little by little.

That may come as a surprise to you, but it's not so difficult to name areas of improvement. Off the top of my head I could probably give you about 20K of important usability and functionality improvements -- discussions, image pages, missing navigational structures, grouping of related pages, scripting ... The tough part is either coding them yourself or convincing someone else to do it.

The exception are clearly defined group efforts, such as a rework of the syntax. But it takes a lot of social and organizational skills to keep people involved and get them emotionally attached to such an effort. Once they are the implementation is pretty much a given. It's almost impossible to pull this off if you're not already an active developer, because open source coders don't like to be told what to do by non-coders.—Eloquence 02:03, Dec 20, 2003 (UTC)

Hmm... First of all, I am willing to implement most of this myself. I've been looking at the code a bit and none of it is too hard to do (just a bit of regexp). I'm a wiz for PHP and perl, but don't really like tinkering with other people's databases.

But that's beside the point. I don't think the things it should work the way you describe them. This is an encyclopedia, not a test site for Media Wiki.

It is the way of things that the choice of software features and rules directly influences the way this encyclopedia is made. It is not for the code (or coders) to decide what kind of behaviour and functionality Wikipedia needs. What features are needed and how they should work should be discussed in public (preferably in meta, granted, but where on-topic, also in wikipedia), worked out and then implemented.

There are many things which can be made to work without major rewrites of the software and there is no reason to put up with waiting to see what will happen (when and by whom is another thing - you can't force people to do what you want, but you can establish the order in which things should be done).

So, no, I think the editor community should be the one to decide which way they want it to work, and then kindly ask the developers to implement it. Zocky 05:15, 20 Dec 2003 (UTC) has spoken.

Not really, you can only decide which order you think things should be done in. The order they actually happen in is very much up to the developers. It's their time, so they are likely to implement the things that interest them or that they want to work on rather than pandering to the desires of Wikipedians. That's not to say they won't take views into account - I'm sure they do - but the decision of what they code really has to be up to them. Angela. 05:27, 20 Dec 2003 (UTC)

Angela is quite right: we are volunteers, not paid staff, so we're under no obligation to do what you tell us to do, when you tell us to do it. Developers have to prioritise based not only on what is most desired but on what is easiest to implement. Clearly the current link table behaviour is broken, but it might not be obvious to most users that fixing it is technically non-trivial. I like nothing better than to make users happy, but there are many other factors which influence prioritisation.

The link table definitely needs fixing. So-called "parametric inclusion" and variables such as {{PAGENAME}}, {{USERNAME}}, etc. would certainly be nice. I'm sure everyone would be very grateful if Zocky were to implement them. I'd be quite happy if there were facilities for [[status:vfd]] style tags, as long as they work alongside the current scheme rather than replacing it. Initial colon syntax [[:eo:Slovenio]] style is widely used on the wikis, I don't think there's any need to change it. And backslash escaping might be normal for programming languages, but it isn't normal for HTML, which wikitext is based on. As such it would be a fairly large change, but if Zocky is willing to implement it, and assuming there are no major technical obstacles, I guess it would be fine too. As long as it doesn't break most backslash usage in ordinary text. -- Tim Starling 12:58, 21 Dec 2003 (UTC)

Of course, I'm not talking about anybody telling anyone what to do. It's just about whether development of Media Wiki should be centered on the needs of Wikipedia, and who should decide what those needs are.

IMO, the correct way to introduce changes is to discuss the ideas on Wikipedia, decide what functionality is needed, and then implement it. I may implement something, and developers may think, hey, this is neat, let's put it in, but what's the point in doing it if it's never going to be used, or is going to introduce confusion and cluttering, which is even worse.

So, I still think that the desired functionality at a certain stage is something for the editors to decide. When, how and in which minute order it is implemented, is of course up to the people who do it. Zocky 19:09, 23 Dec 2003 (UTC)

Oh, another thing... the whole thing with {{}} [[]]: I don't mind cryptic syntax myself, but I think that the great strength of wikipedia has all along been its simple and symmetrical syntax, which provides a nice learning curve. Everything is done by repeating characters, either at the start of line,

* 
**
***

#
##
###

 :
 ::
 :::

or symmetrically in the text:

'''asd''', ''asd'', [[asd]], {{ASD}}

I think that that's what makes it simple, and should be kept to. So, marking another way of linking with [[[]]] and another way of inserting text with {{{}}} makes sense to me. I even think //text// should produce ~~text~~.

Zocky 19:09, 23 Dec 2003 (UTC)

I prefer the syntax [$message$], which would go along with the markup being proposed at wikitax. [$message$] could be equivilent to

, [#message#] could be equivelent to <message>, and [@var@] could be the same as {{{1}}}. $message$, #message#, and @var@ could be links. Noldoaran 18:16, 21 Feb 2004 (UTC)

While trying to come up with a workable template for data about languages, I had a couple of ideas for template syntax "macro-expansion" features that would be very handy:

An "ifdef"-like conditional insertion feature for inserting wikitext only if a certain parameter is supplied. For example, inserting a table row for optional data.

Syntax: {{{ifdef:param | wikitest to be included}}}

Rationale: Sometimes it's useful to have optional fields. For example, some languages (such as French) have a central authority that controls usage, and it would be good to have a row in the table for that information (some language articles' databoxes have it) but for languages that don't the row is pointless. Under the current system, the row would always be present, and the parameter would have to be something like "agency=-" or "agency=N/A" or else some ugly template-code would show through.

Note: this would have to allow parameter expansions inside the wikitext to be included to be useful in a lot of cases.

A "for each"-like "looping" construct to support parameters that can occur more than once

Syntax: {{{foreach:param | wikitext to be included}}} Within the to-be-included text, {{{param#}}} would expand to the value of the current param. The multi-parameters could be specified either with named sub-parameters separated from the main parameter name by a number sign (param#family=, param#genus=) or just by the main parameter name more than once (with successive params being equivalent to param#1, param#2, etc.)

Rationale: Sometimes a particular property has several values that don't work as a simple list (e.g. when they must appear in separate table rows).

Note: It may also be useful to be able to use {{{otherparam#}}} inside a foreach for param, in the case of parameters that are supposed to be paired. It would expand to the value of the sub-parameter of otherparam with the same sub-parameter name as the current param#. Also, a way of getting the number of sub-parameters of a multi-parameter would be useful for e.g. specifying a rowspan in a table so that a single cell can sit next to an arbitrary number of foreach-generated rows.

That's probably not the best explanation. And the syntax can probably be improved so it doesn't look quite so much like modem noise. But I think the features themselves would be very useful. On the downside, it might make template editing a little more arcane. However, it could also make using templates in pages easier. - Gwalla 04:10, 23 Jun 2004 (UTC)

I've written up a more comprehensive proposal for this functionality at extended template syntax. The proposed syntax is slightly different (improved, I think), and hopefully that article is a little less cryptic. Gwalla 02:02, 30 Jun 2004 (UTC)