Wikimedia Meta-Wiki

Wikimedia Blog/SVN-GitHub mirror of the WordPress theme

< Wikimedia Blog

This page contains some internal technical notes about the theme of https://blog.wikimedia.org/ as used around 2017, in particular on how to keep its public mirror at https://github.com/wikimedia/wikimediablog-wordpresscom in sync and deploy changes submitted there.

See Wikimedia Blog#WordPress Theme for some general information about the theme.

Merge changes from GitHub trunk into SVN

edit

This assumes that you have set up a "gateway" folder on your laptop (and the nice-merge bash alias has been created) as described below.


To sync a commit from GitHub:

Merge pull request on https://github.com/wikimedia/wikimediablog-wordpresscom to Master (using the web interface is fine), then enter the following commands from your local mirroring folder: 

$ git checkout master
$ git pull
$ git checkout svnsync
$ git svn rebase
$ nice-merge
$ git svn dcommit

Optionally, check the result before the final git svn dcommit as follows: 

$ git log --decorate --graph

Merge changes on SVN back into Github trunk

edit

This needs to be done every time after changes have been made directly to Automattic's SVN, in order to keep the GitHub mirror up to date.

Proceed exactly as in the original recipe: 

$ git checkout svnsync
$ git svn rebase
$ git checkout master
$ git merge svnsync
$ git push origin master

Initialize the mirroring "gateway"

edit

This describes how to set up (on a local laptop) the git repo through which the private SVN on Automattic's servers is mirrored on GitHub. It only needs to be done once.

Create the repository on GitHub if not already present (its content will be overwritten), in this case https://github.com/wikimedia/wikimediablog-wordpresscom

Open a terminal on your laptop and cd to the folder where the mirror gateway is going to be kept.

Check the git version (we used 1.9.1): 

$ git --version
git version 1.9.1

Initialize: 

$ git init
$ git svn init --trunk=/ --prefix=svn/ https://vip-svn.wordpress.com/wikimedia-v2/

Fetch the content of the (private) remote SVN repo. This may result in an error at first 

$ git svn fetch
W: Ignoring error from SVN, path probably does not exist: (160013): Filesystem has no item: '/!svn/rvr/100/wikimedia-v2' path not found
W: Do not be alarmed at the above message git-svn is just searching aggressively for old history.
This may take a while on large repositories
Checked Ahrough templates/featured-image.php
Atemplates/featured-posts-slider.php

[“Ahrough”: sic] [...] 

r250205 = 62ad30fffe65aaee362bfa5a594c870c4940784a (refs/remotes/svn/trunk)
Mfunctions.php
r254475 = 7330498a032cf6c089e7a8896e96c885fd6d3edf (refs/remotes/svn/trunk)
Checked out HEAD:
https://vip-svn.wordpress.com/wikimedia-v2 r254475
creating empty directory: lib/carbon-fields
error closing pipe: Bad file descriptor at /usr/lib/git-core/git-svn line 0.
error closing pipe: Bad file descriptor at /usr/lib/git-core/git-svn line 0.

Just try again: 

$ git svn fetch

Trying again appears to have read in the entire history correctly (see checks below). Altogether this fetch step took about 15 minutes. Apparently this is because git-svn runs a detection step for every possible revision number, and the maximum revision number in this particular repository is fairly high (>270k, not because the actual history has that many revisions, but because the repository is part of a larger one that apparently contains all plugins used on VIP).

Connect to the GitHub repo: 

$ git remote add origin https://github.com/wikimedia/wikimediablog-wordpresscom
$ git branch svnsync

(left out the --no-track option from the original recipe here) 

$ git fetch origin

Check the previous history from https://github.com/wikimedia/wikimediablog-wordpresscom, before overwriting it: 

$ git log origin/master



$ git reset --hard origin/master

Remove content from GitHub repo (to be overwritten with mirrored SVN repo): 

$ rm -r README.md lib/
$ git commit -a -m 'delete stuff to prepare for mirroring'

(It might be possible to leave the following command out:) 

$ git push --set-upstream origin master

$ git reset --hard svnsync
$ git push -f

(If this fails with "The remote end hung up unexpectedly", just try the same command again.)

Lastly, set up a bash alias as follows. (This keeps the history cleaner, by dropping the message corresponding to the merge of the pull request on GitHub, keeping the commit message(s) of the pull request itself, the author name and the change ID.) Should be put into .bashrc for convenience. 

alias nice-merge='( echo -e "Submitting changes from GitHub\n"; git log --format=format:"---------%n%an%n%B" --no-merges ..master) | ( git merge --no-edit --no-ff master; git commit --amend -F - )'


Optionally, after git svn fetch above, check its result before proceeding, as follows:

Check presence of files: 

$ ls .git/refs/remotes/svn/trunk

Get revision IDs: 

$ less .git/refs/remotes/svn/trunk

Check one of them: 

$ git show 7330498a032cf6c089e7a8896e96c885fd6d3edf

commit 7330498a032cf6c089e7a8896e96c885fd6d3edf
Author: [... etc.]
edit

The SVN <--> GitHub mirroring method above is based on http://ben.lobaugh.net/blog/147853/creating-a-two-way-sync-between-a-github-repository-and-subversion , with various adaptions and corrections by Gergő Tisza to make it work in our case. Other interesting links about this topic:

Retrieved from "https://meta.wikimedia.org/w/index.php?title=Wikimedia_Blog/SVN-GitHub_mirror_of_the_WordPress_theme&oldid=22460401"