This post has been published at https://blog.wikimedia.org/2014/04/10/wikimedias-response-to-the-heartbleed-security-vulnerability/. You are welcome to add translations here.
Wikimedia's response to the "Heartbleed" security vulnerability
On April 7th, a widespread issue in a central component of Internet security (OpenSSL) was disclosed. The vulnerability has now been fixed on all Wikimedia wikis. If you only read Wikipedia without creating an account, nothing is required from you. If you have a user account on any Wikimedia wiki, you will need to re-login the next time you use your account.
The issue, called Heartbleed, would allow attackers to gain access to privileged information on any site running a vulnerable version of that software. Wikis hosted by the Wikimedia Foundation were potentially affected by this vulnerability for several hours after it was disclosed. However, we have no evidence of any actual compromise to our systems or our users' information, and because of the particular way our servers are configured, it would have been very difficult for an attacker to exploit the vulnerability in order to harvest users' wiki passwords.
After we were made aware of the issue, we began upgrading all of our systems with patched versions of the software in question. We then began replacing critical user-facing SSL certificates and resetting all user session tokens. See the full timeline of our response below.
All logged-in users send a secret session token with each request to the site. If a nefarious person were able to intercept that token, they could impersonate other users. Resetting the tokens for all users has the benefit of making all users reconnect to our servers using the updated and fixed version of the OpenSSL software, thus removing this potential attack.
We recommend changing your password as a standard precautionary measure, but we do not currently intend to enforce a password change for all users. Again, there has been no evidence that Wikimedia Foundation users were targeted by this attack, but we want all of our users to be as safe as possible.
Thank you for your understanding and patience.
Greg Grossmeier, on behalf of the WMF Operations and Platform teams
Timeline of Wikimedia's response
(Times are in UTC)
- 04:03: We begin upgrading libssl on all of our servers, beginning with high-priority machines.
- 09:08: We begin replacing SSL certificates.
- 13:09: We forcibly upgrade libssl on WMF Tool Labs.
- 13:46: The upgrade of libssl on all public servers is complete.
- 16:45: All Wikimedia wiki user-facing SSL servers have new certificates in place.
- 23:08: We begin resetting user login tokens (forcing users to re-login using new libssl and certificates).
- 13:54: ticket.wikimedia.org's ssl certificate is replaced (the last one)
- 16:44: Email to all users of ticket.wikimedia.org (OTRS) and otrs-wiki.wikimedia.org to change their passwords.
- 22:33: Logged out all Bugzilla users
Frequently Asked Questions
(This section will be expanded as needed.)
- Why hasn't the "not valid before" date on your SSL certificate changed if you have already replaced it?
- Our SSL certificate provider keeps the original "not valid before" date (sometimes incorrectly referred to as an "issued on" date) in any replaced certificates. This is not an uncommon practice. Aside from looking at the change to the .pem files linked above in the Timeline, the other way of verifying that the replacement took place is to compare the fingerprint of our new certificate with our previous one.