DokuWiki vs MediaWiki benchmarks

Methodology

Same system setup and benchmark method as that in MediaWiki 1.4 benchmarks. Tested with MediaWiki current REL1_4 branch, with more or less stock configuration. Using standard parser cache (objectcache table in db), not memcached which can be a smidge faster.

DokuWiki is dokuwiki-2004-11-10.tgz, freshly set up with no special configuration.

Results

Here summaries of two 'ab -n20' trials of each page are presented, with median/minimum/maximum times in milliseconds from the 'Connection Times' total summary.

Short start page

Page contained a short sentence and one link.

Turck on:

 Doku
     7 (  7 to  12)
     7 (  7 to  13)
 
 MW w/ file cache
    39 ( 39 to  92)
    39 ( 36 to  48)
 
 MW w/ file cache after additional code tweaking
    28 ( 27 to  38)
    28 ( 27 to  38)
 
 MW w/o file cache
    66 ( 63 to  72)
    66 ( 62 to  71)

Turck off:

 Doku
    30 ( 30 to  32)
    30 ( 29 to  32)
 
 MW w/ file cache
   180 (178 to 192)
   181 (179 to 325)
 
 MW w/ file cache after additional code tweaking
   169 (166 to 175)
   169 (166 to 176)
 
 MW w/o file cache
   211 (210 to 356)
   211 (209 to 255)

w:en:Zuiderzee Works

Copied the raw page text onto DokuWiki, so markup other than simple links is thus not rendered in DokuWiki.

Turck on:

 Doku
    16 ( 15 to 269)
    15 ( 15 to  22)
 
 MW w/ file cache
    44 ( 43 to  64)
    45 ( 43 to  56)
 
 MW w/ file cache after additional code tweaking
    33 ( 30 to 132)
    33 ( 32 to  37)
 
 MW w/o file cache
    78 ( 75 to  91)
    77 ( 74 to  80)

Turck off:

 Doku
    39 ( 37 to  52)
    38 ( 37 to  89)
 
 MW w/ file cache
   186 (183 to 307)
   186 (184 to 248)
 
 MW w/ file cache after additional code tweaking
   174 (172 to 192)
   174 (171 to 180)
 
 MW w/o file cache
   225 (319 to 309)
   223 (220 to 244)

Conclusions

DokuWiki has much lower startup overhead, particularly when a stock PHP configuration is used which requires parsing the code on every hit. Turck MMCache gives a much bigger boost to MediaWiki than it does to DokuWiki, indicating that Doku loads less code in the first place. As always, reducing the amount of code that is loaded on startup will help a great deal here; we have many too-big 'kitchen sink' modules which are prematurely loaded, and pruning these is ongoing work. Load-locality improvements will make a big difference for users with non-accelerated PHP environments, but should still provide some slight improvement on accelerated servers as well.

Final output and skin layout is still a large chunk of output time, making up the difference between the regular parser cache hit and the file cache hit. Further improvements here will help on all non-file, non-squid hits in all configurations.

Improvements to the file cache could certainly slash the file cache hit time to meet or beat DokuWiki by skipping most or all of the MediaWiki startup. (This requires some changes to the file cache to remove cache files on invalidation, just as we purge the squid cache in squid cache mode. If we hadn't switched to squids on Wikipedia this likely would have been done a long time ago. See Evan's work on an experimental '404 handler' variant for some sample code on this.)

Deferring message cache initialization knocks file cache hit time down by another 8-9ms, for a start. New figures added above.

Squid cache hits with MediaWiki in Squid mode ought to be faster than DokuWiki non-cached hits, but I have not tested this. DokuWiki's headers don't allow caching and it doesn't seem to give 304 responses, requiring a re-render and re-download on every hit. (At least in the stock configuration.)

I think a positive step forward would be to delay initialisation of most global variable objects. Instead of using $wgUser, $wgTitle, $wgArticle etc. directly, we should fetch the objects using a global or static member function. Then if an invocation doesn't need $wgUser (and we can work on making it so that it doesn't), it won't create that object. This will unfortunately break extensions... maybe a compatibility mode that initialises all the globals at the start?

The other thing is to make object initialisation quicker, by delaying initialisation of member variables. I've been doing that progressively, I'm sure other developers have been too. -- Tim Starling 11:51, 9 Dec 2004 (UTC)

In order to compare apples to apples, I think testing this with a large database would be appropriate. Of course software with a file-based backing store will be faster than a relational database, for small data sets. What about for large data sets? I think this is where MW would see an advantage over DokuWiki but I can't see that this scenario has been adjusted for. 72.15.90.142 14:17, 4 August 2005 (UTC)[reply]

To a large extent application performance comparison will be meaningless, as the "tricks" which could be applied to both MediaWiki and DokuWiki that will be installation specific, for example mounting DokuWiki pages into ramdisk;

mount --bind -ttmpfs /home/user/dokuwiki/data/pages /home/user/dokuwiki/data/pages

At the same time, measuring specific operations e.g. parse wiki page to HTML (measured independently from where the page came from) , fulltext search, etc. might have value. Comparing parsers, for example, would expect DokuWiki to be faster here , at least for larger documents containing a significant amount of wiki syntax, because it handles parsing of a wiki page in a single scan of the text, while MediaWiki's parser uses multiple scans for each element of wiki syntax to be parsed - that will means DokuWiki's parser scales better and is nearer to probably O(n) while MediaWiki's will likely be closer to O(n^2).

Meanwhile would expect MediaWiki to excel in operations where indexing is involved (e.g. searching, "What links here" etc.) as MySQL is doing the work. DokuWiki is currently either evaluating these on the fly (per request - slow) or by building file-based indexes managed by PHP scripts (which is significantly harder to develop and make scale well).

--[[1]] 10:11, 4 October 2005 (UTC)