Wikimedia Status - Incident History https://www.wikimediastatus.net Statuspage Sat, 26 Apr 2025 21:01:27 +0000 Wikis editing unavailability <p><small>Mar <var data-var='date'>31</var>, <var data-var='time'>04:26</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'>31</var>, <var data-var='time'>03:48</var> UTC</small><br><strong>Monitoring</strong> - The fix appears to be effective, and edit metrics are returning to normal, along with user reports of recovery. We will continue to monitor.</p><p><small>Mar <var data-var='date'>31</var>, <var data-var='time'>03:39</var> UTC</small><br><strong>Update</strong> - We have identified an issue that we believe is responsible for the impact to login sessions and edits, and are pursuing a fix.</p><p><small>Mar <var data-var='date'>31</var>, <var data-var='time'>03:27</var> UTC</small><br><strong>Investigating</strong> - We are investigating an issue impacting edits and / or logins, with impact beginning at approximately 2:55 UTC.</p> Mon, 31 Mar 2025 04:26:49 +0000 https://www.wikimediastatus.net/incidents/1w3rq4d2zljj https://www.wikimediastatus.net/incidents/1w3rq4d2zljj Partial media unavailability <p><small>Mar <var data-var='date'>29</var>, <var data-var='time'>16:46</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'>29</var>, <var data-var='time'>16:22</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Mar <var data-var='date'>29</var>, <var data-var='time'>15:37</var> UTC</small><br><strong>Investigating</strong> - We are experiencing issue affecting media upload and download. We are investigating.</p> Sat, 29 Mar 2025 16:46:26 +0000 https://www.wikimediastatus.net/incidents/tzb72slhy9cz https://www.wikimediastatus.net/incidents/tzb72slhy9cz Reduced availability for some users <p><small>Mar <var data-var='date'>21</var>, <var data-var='time'>03:01</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'>21</var>, <var data-var='time'>02:22</var> UTC</small><br><strong>Update</strong> - We are continuing to monitor for any further issues.</p><p><small>Mar <var data-var='date'>21</var>, <var data-var='time'>01:39</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Mar <var data-var='date'>21</var>, <var data-var='time'>01:07</var> UTC</small><br><strong>Investigating</strong> - Some users are experiencing connectivity issues to Wikimedia project pages. Staff are investigating the issue</p> Fri, 21 Mar 2025 03:01:34 +0000 https://www.wikimediastatus.net/incidents/8y10l7vpbygy https://www.wikimediastatus.net/incidents/8y10l7vpbygy Wikis read-only: datacentre switchover <p><small>Mar <var data-var='date'>19</var>, <var data-var='time'>14:22</var> UTC</small><br><strong>Completed</strong> - Maintenance completed.</p><p><small>Mar <var data-var='date'>19</var>, <var data-var='time'>14:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Mar <var data-var='date'>19</var>, <var data-var='time'>13:36</var> UTC</small><br><strong>Scheduled</strong> - The SRE team will run a planned data center switchover, moving all wikis from our data center in Texas to the data center in Virginia. This is an important periodic test of our tools and procedures, to ensure the wikis will continue to be available even in the event of major technical issues.<br /><br />The switchover process requires a brief read-only period for all Foundation-hosted wikis, which will start at 14:00 UTC on Wednesday March 19th, and will last for a few minutes while we execute the migration as efficiently as possible. All our public and private wikis will be continuously available for reading as usual, but no one will be able to save edits during the process.</p> Wed, 19 Mar 2025 14:22:35 +0000 Wed, 19 Mar 2025 15:00:00 +0000 https://www.wikimediastatus.net/incidents/q0y02m3k5z0v https://www.wikimediastatus.net/incidents/q0y02m3k5z0v Trouble with images <p><small>Mar <var data-var='date'>19</var>, <var data-var='time'>12:43</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'>19</var>, <var data-var='time'>11:29</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Mar <var data-var='date'>19</var>, <var data-var='time'>11:19</var> UTC</small><br><strong>Investigating</strong> - We are aware of issues with accessing images, and we are investigating.</p> Wed, 19 Mar 2025 12:43:19 +0000 https://www.wikimediastatus.net/incidents/xc1n0wbml3n4 https://www.wikimediastatus.net/incidents/xc1n0wbml3n4 Editing issues <p><small>Mar <var data-var='date'>13</var>, <var data-var='time'>14:29</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'>13</var>, <var data-var='time'>13:37</var> UTC</small><br><strong>Update</strong> - We are continuing to monitor for any further issues.</p><p><small>Mar <var data-var='date'>13</var>, <var data-var='time'>12:47</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Mar <var data-var='date'>13</var>, <var data-var='time'>11:36</var> UTC</small><br><strong>Investigating</strong> - We are aware that users are having trouble editing Wikipedia and other Wikimedia wikis, and we are investigating.</p> Thu, 13 Mar 2025 14:29:39 +0000 https://www.wikimediastatus.net/incidents/fy48j8lfsf6w https://www.wikimediastatus.net/incidents/fy48j8lfsf6w Editing issues <p><small>Mar <var data-var='date'>12</var>, <var data-var='time'>13:54</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'>12</var>, <var data-var='time'>13:50</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Mar <var data-var='date'>12</var>, <var data-var='time'>12:23</var> UTC</small><br><strong>Update</strong> - We are continuing to work on a fix for this issue.</p><p><small>Mar <var data-var='date'>12</var>, <var data-var='time'>11:35</var> UTC</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Mar <var data-var='date'>12</var>, <var data-var='time'>11:34</var> UTC</small><br><strong>Investigating</strong> - We are aware that users are having trouble editing Wikipedia and other Wikimedia wikis, and we are investigating.</p> Wed, 12 Mar 2025 13:54:24 +0000 https://www.wikimediastatus.net/incidents/t13w599t74sq https://www.wikimediastatus.net/incidents/t13w599t74sq Wikimedia Gerrit git repository is unavailable <p><small>Mar <var data-var='date'> 7</var>, <var data-var='time'>15:09</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'> 7</var>, <var data-var='time'>14:55</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Mar <var data-var='date'> 7</var>, <var data-var='time'>14:45</var> UTC</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Mar <var data-var='date'> 7</var>, <var data-var='time'>14:42</var> UTC</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Mar <var data-var='date'> 7</var>, <var data-var='time'>14:42</var> UTC</small><br><strong>Investigating</strong> - We are aware that developers are having trouble updating and checking out code from gerrit.wikimedia.org and our engineers are working on it.</p> Fri, 07 Mar 2025 15:09:35 +0000 https://www.wikimediastatus.net/incidents/5f9qm4x5xc11 https://www.wikimediastatus.net/incidents/5f9qm4x5xc11 www.wikipedia.org portal unavailable on some locations <p><small>Feb <var data-var='date'>28</var>, <var data-var='time'>10:57</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Feb <var data-var='date'>28</var>, <var data-var='time'>10:36</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Feb <var data-var='date'>28</var>, <var data-var='time'>09:57</var> UTC</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Feb <var data-var='date'>28</var>, <var data-var='time'>09:36</var> UTC</small><br><strong>Investigating</strong> - We are aware that many users are having trouble accessing the portal www.wikipedia.org for excessive redirects, and we are investigating.</p> Fri, 28 Feb 2025 10:57:30 +0000 https://www.wikimediastatus.net/incidents/nvh935bw6nbv https://www.wikimediastatus.net/incidents/nvh935bw6nbv Unscheduled maintenance to prevent database errors <p><small>Feb <var data-var='date'> 3</var>, <var data-var='time'>13:00</var> UTC</small><br><strong>Resolved</strong> - For some minutes between 13:07 to 13:14 UTC, wikis in the s3 section ( https://noc.wikimedia.org/conf/highlight.php?file=dblists/s3.dblist ) were in read only mode (reads could happen, but edits were not allowed) due to unscheduled maintenance in order to fix ongoing database errors. The maintenance is over and the issue should be now fixed. More info at: https://phabricator.wikimedia.org/T385457</p> Mon, 03 Feb 2025 13:00:00 +0000 https://www.wikimediastatus.net/incidents/sclkxsj73cmz https://www.wikimediastatus.net/incidents/sclkxsj73cmz Occasional errors with Wikimedia commons media access <p><small>Dec <var data-var='date'>23</var>, <var data-var='time'>21:38</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Dec <var data-var='date'>23</var>, <var data-var='time'>20:44</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Dec <var data-var='date'>23</var>, <var data-var='time'>20:08</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p> Mon, 23 Dec 2024 21:38:48 +0000 https://www.wikimediastatus.net/incidents/5v33zrdfq59s https://www.wikimediastatus.net/incidents/5v33zrdfq59s Errors for logged in and some logged-out users <p><small>Dec <var data-var='date'> 5</var>, <var data-var='time'>05:19</var> UTC</small><br><strong>Resolved</strong> - The remediation was successful.</p><p><small>Dec <var data-var='date'> 5</var>, <var data-var='time'>05:02</var> UTC</small><br><strong>Monitoring</strong> - A primary database server for cached data experienced an issue serving requests. We have corrected the issue and services are recovering.</p> Thu, 05 Dec 2024 05:19:46 +0000 https://www.wikimediastatus.net/incidents/3g2ckc7bp6l9 https://www.wikimediastatus.net/incidents/3g2ckc7bp6l9 Database issues in the Dallas TX datacenter <p><small>Oct <var data-var='date'>24</var>, <var data-var='time'>14:02</var> UTC</small><br><strong>Resolved</strong> - Traffic back to baseline, root cause investigation is still ongoing but the incident is resolved.</p><p><small>Oct <var data-var='date'>24</var>, <var data-var='time'>13:52</var> UTC</small><br><strong>Update</strong> - We are continuing to monitor for any further issues.</p><p><small>Oct <var data-var='date'>24</var>, <var data-var='time'>13:49</var> UTC</small><br><strong>Monitoring</strong> - We have repooled the primary datacenter for read traffic and are monitoring the situation while investigating the root cause.</p><p><small>Oct <var data-var='date'>24</var>, <var data-var='time'>11:33</var> UTC</small><br><strong>Update</strong> - We shifted read traffic to the secondary datacenter, which helped with the overloaded database. We are still investigating the root cause.</p><p><small>Oct <var data-var='date'>24</var>, <var data-var='time'>11:02</var> UTC</small><br><strong>Investigating</strong> - Reading and editing is affected. We are investigating the issue.</p> Thu, 24 Oct 2024 14:02:48 +0000 https://www.wikimediastatus.net/incidents/b406lmnx5s57 https://www.wikimediastatus.net/incidents/b406lmnx5s57 Trouble with some wikis <p><small>Oct <var data-var='date'> 9</var>, <var data-var='time'>05:00</var> UTC</small><br><strong>Resolved</strong> - For approximately one hour after 5am UTC some uncached requests, such as those coming from logged in users, edits and api calls observed high latencies and/or errors due to an overload on MediaWiki nodes at the primary datacenter.</p> Wed, 09 Oct 2024 05:00:00 +0000 https://www.wikimediastatus.net/incidents/89m28jzq6z1d https://www.wikimediastatus.net/incidents/89m28jzq6z1d Site slow for users in the Asia region (eqsin) <p><small>Sep <var data-var='date'>28</var>, <var data-var='time'>22:43</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Sep <var data-var='date'>28</var>, <var data-var='time'>22:23</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Sep <var data-var='date'>28</var>, <var data-var='time'>20:58</var> UTC</small><br><strong>Update</strong> - We are continuing to work on a fix for this issue.</p><p><small>Sep <var data-var='date'>28</var>, <var data-var='time'>20:55</var> UTC</small><br><strong>Identified</strong> - We are investigating issues with connectivity to our sites from the Asia region.</p> Sat, 28 Sep 2024 22:43:31 +0000 https://www.wikimediastatus.net/incidents/fvd9nkpdrlqv https://www.wikimediastatus.net/incidents/fvd9nkpdrlqv Wikis read-only: Datacenter switchover <p><small>Sep <var data-var='date'>25</var>, <var data-var='time'>15:30</var> UTC</small><br><strong>Completed</strong> - The read-only period completed at 15:01:28. All systems appear to be operating as expected following the switchover.</p><p><small>Sep <var data-var='date'>25</var>, <var data-var='time'>15:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Sep <var data-var='date'>25</var>, <var data-var='time'>14:18</var> UTC</small><br><strong>Scheduled</strong> - The SRE team will run a planned data center switchover, moving all wikis from our data center in Virginia to the data center in Texas. This is an important periodic test of our tools and procedures, to ensure the wikis will continue to be available even in the event of major technical issues.<br /><br />The switchover process requires a brief read-only period for all Foundation-hosted wikis, which will start at 15:00 UTC on Wednesday September 25, and will last for a few minutes while we execute the migration as efficiently as possible. All our public and private wikis will be continuously available for reading as usual, but no one will be able to save edits during the process.</p> Wed, 25 Sep 2024 15:30:39 +0000 Wed, 25 Sep 2024 16:00:00 +0000 https://www.wikimediastatus.net/incidents/8rtzwkd31yt6 https://www.wikimediastatus.net/incidents/8rtzwkd31yt6 Eventgate failure in codfw <p><small>Aug <var data-var='date'>22</var>, <var data-var='time'>12:47</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Aug <var data-var='date'>22</var>, <var data-var='time'>12:45</var> UTC</small><br><strong>Monitoring</strong> - Eventgate has returned to normal operation, we are continuing to monitor the situation.</p><p><small>Aug <var data-var='date'>22</var>, <var data-var='time'>12:36</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating this issue; in the mean time, events will not propagate properly, and some operations may fail.</p> Thu, 22 Aug 2024 12:47:26 +0000 https://www.wikimediastatus.net/incidents/58zltngkd6rm https://www.wikimediastatus.net/incidents/58zltngkd6rm Database issues <p><small>Aug <var data-var='date'>15</var>, <var data-var='time'>00:55</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved. We're continuing to address the root cause to reduce future recurrences.</p><p><small>Aug <var data-var='date'>14</var>, <var data-var='time'>21:17</var> UTC</small><br><strong>Monitoring</strong> - We are monitoring the situation and continuing to investigate the root cause of brief recurring lag spikes.</p><p><small>Aug <var data-var='date'>14</var>, <var data-var='time'>21:00</var> UTC</small><br><strong>Investigating</strong> - We're currently experiencing a recurrence of lag on our MariaDB S4 primary instance.</p> Thu, 15 Aug 2024 00:55:51 +0000 https://www.wikimediastatus.net/incidents/rnf81fh7x423 https://www.wikimediastatus.net/incidents/rnf81fh7x423 Database issues <p><small>Aug <var data-var='date'>14</var>, <var data-var='time'>20:39</var> UTC</small><br><strong>Resolved</strong> - We've made changes to reduce primary database load, and things are stable for now.</p><p><small>Aug <var data-var='date'>14</var>, <var data-var='time'>14:40</var> UTC</small><br><strong>Monitoring</strong> - We have mitigated the issue, but we are still searching for the root cause. We are monitoring the situation. Short recurrences are likely.</p><p><small>Aug <var data-var='date'>14</var>, <var data-var='time'>14:22</var> UTC</small><br><strong>Investigating</strong> - We're currently experiencing a recurrence of lag on our MariaDB S4 primary instance. We are investigating to determine the root cause.</p> Wed, 14 Aug 2024 20:39:46 +0000 https://www.wikimediastatus.net/incidents/l9zg5ngs32w6 https://www.wikimediastatus.net/incidents/l9zg5ngs32w6 Database issues <p><small>Aug <var data-var='date'>14</var>, <var data-var='time'>10:01</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Aug <var data-var='date'>14</var>, <var data-var='time'>09:51</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Aug <var data-var='date'>14</var>, <var data-var='time'>09:49</var> UTC</small><br><strong>Investigating</strong> - We're currently experiencing a recurrence of lag on our MariaDB S4 primary instance.</p> Wed, 14 Aug 2024 10:01:17 +0000 https://www.wikimediastatus.net/incidents/3x86p48v5tx6 https://www.wikimediastatus.net/incidents/3x86p48v5tx6 Database issues <p><small>Aug <var data-var='date'>13</var>, <var data-var='time'>22:02</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Aug <var data-var='date'>13</var>, <var data-var='time'>21:37</var> UTC</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Aug <var data-var='date'>13</var>, <var data-var='time'>21:25</var> UTC</small><br><strong>Investigating</strong> - We're currently experiencing lag on our MariaDB S4 primary instance</p> Tue, 13 Aug 2024 22:02:36 +0000 https://www.wikimediastatus.net/incidents/fqj5hmtrrmkd https://www.wikimediastatus.net/incidents/fqj5hmtrrmkd Database issues <p><small>Aug <var data-var='date'>13</var>, <var data-var='time'>18:38</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Aug <var data-var='date'>13</var>, <var data-var='time'>18:15</var> UTC</small><br><strong>Investigating</strong> - We're currently experiencing lag on our MariaDB S4 primary instance</p> Tue, 13 Aug 2024 18:38:55 +0000 https://www.wikimediastatus.net/incidents/jhq8qcw6bwz7 https://www.wikimediastatus.net/incidents/jhq8qcw6bwz7 Problems with our secondary datacenter <p><small>Jul <var data-var='date'>18</var>, <var data-var='time'>12:50</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jul <var data-var='date'>18</var>, <var data-var='time'>12:49</var> UTC</small><br><strong>Update</strong> - Everything is back to normal.</p><p><small>Jul <var data-var='date'>18</var>, <var data-var='time'>12:02</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Jul <var data-var='date'>18</var>, <var data-var='time'>11:56</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p> Thu, 18 Jul 2024 12:50:13 +0000 https://www.wikimediastatus.net/incidents/rvxz4x7gtzmh https://www.wikimediastatus.net/incidents/rvxz4x7gtzmh Trouble with some wikis <p><small>Jul <var data-var='date'>10</var>, <var data-var='time'>15:26</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jul <var data-var='date'>10</var>, <var data-var='time'>15:18</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Jul <var data-var='date'>10</var>, <var data-var='time'>15:05</var> UTC</small><br><strong>Investigating</strong> - We are aware of issues with accessing some wikis, and we are investigating.</p> Wed, 10 Jul 2024 15:26:16 +0000 https://www.wikimediastatus.net/incidents/rlfwcrx6l4z7 https://www.wikimediastatus.net/incidents/rlfwcrx6l4z7 Issues with multiple wikis <p><small>Jun <var data-var='date'>24</var>, <var data-var='time'>20:48</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jun <var data-var='date'>24</var>, <var data-var='time'>20:28</var> UTC</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Jun <var data-var='date'>24</var>, <var data-var='time'>20:12</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p> Mon, 24 Jun 2024 20:48:18 +0000 https://www.wikimediastatus.net/incidents/p1g649x023sk https://www.wikimediastatus.net/incidents/p1g649x023sk