rdc@fri Status - Incident history https://status.rdc.si rdc@fri Sat, 19 Sep 2026 18:21:36 +0000 Login is back up Type: Incident Affected Components: Login Sep 19, 18:21:36 GMT+0 - Investigating - Login is down at the moment. This incident was automatically created by Instatus monitoring. Sep 19, 18:25:45 GMT+0 - Resolved - Login is back up. This incident was automatically resolved by Instatus monitoring. Type: Incident

Affected Components:

<p><small>Sep <var data-var='date'> 19</var>, <var data-var='time'>18:21:36</var> GMT+0</small><br><strong>Investigating</strong> - Login is down at the moment. This incident was automatically created by Instatus monitoring..</p> <p><small>Sep <var data-var='date'> 19</var>, <var data-var='time'>18:25:45</var> GMT+0</small><br><strong>Resolved</strong> - Login is back up. This incident was automatically resolved by Instatus monitoring..</p> ]]> Sat, 19 Sep 2026 18:21:36 +0000 https://status.rdc.si/incident/cmu8prsk401g007scxqmxc5cs https://status.rdc.si/incident/cmu8prsk401g007scxqmxc5cs Login is back up Type: Incident Affected Components: Login Sep 19, 16:53:07 GMT+0 - Investigating - Login is down at the moment. This incident was automatically created by Instatus monitoring. Sep 19, 16:57:14 GMT+0 - Resolved - Login is back up. This incident was automatically resolved by Instatus monitoring. Type: Incident

Affected Components:

<p><small>Sep <var data-var='date'> 19</var>, <var data-var='time'>16:53:07</var> GMT+0</small><br><strong>Investigating</strong> - Login is down at the moment. This incident was automatically created by Instatus monitoring..</p> <p><small>Sep <var data-var='date'> 19</var>, <var data-var='time'>16:57:14</var> GMT+0</small><br><strong>Resolved</strong> - Login is back up. This incident was automatically resolved by Instatus monitoring..</p> ]]> Sat, 19 Sep 2026 16:53:07 +0000 https://status.rdc.si/incident/cmu8mm1jk020p07oybye3h1n0 https://status.rdc.si/incident/cmu8mm1jk020p07oybye3h1n0 Regular annual inspection of transformer stations Type: Maintenance Duration: 6 hours Affected Components: Compute Jul 31, 12:21:54 GMT+0 - Identified - A regular annual inspection of the UL FRI & FKKT transformer stations is scheduled for 22\. 8\. 2026, 6:30-11:00; during that time electricity service will be interrupted. To avoid potential loss of work or data we are putting the cluster into maintenance mode for that time. Most of the compute cluster will be shut down and jobs will be queued. We appreciate your patience. Aug 22, 04:00:01 GMT+0 - Identified - Maintenance is now in progress Aug 22, 10:00:00 GMT+0 - Completed - Maintenance has completed successfully Type: Maintenance

Duration: 6 hours

Affected Components:

<p><small>Jul <var data-var='date'> 31</var>, <var data-var='time'>12:21:54</var> GMT+0</small><br><strong>Identified</strong> - A regular annual inspection of the UL FRI & FKKT transformer stations is scheduled for 22\. 8\. 2026, 6:30-11:00; during that time electricity service will be interrupted. To avoid potential loss of work or data we are putting the cluster into maintenance mode for that time. Most of the compute cluster will be shut down and jobs will be queued. We appreciate your patience..</p> <p><small>Aug <var data-var='date'> 22</var>, <var data-var='time'>04:00:01</var> GMT+0</small><br><strong>Identified</strong> - Maintenance is now in progress.</p> <p><small>Aug <var data-var='date'> 22</var>, <var data-var='time'>10:00:00</var> GMT+0</small><br><strong>Completed</strong> - Maintenance has completed successfully.</p> ]]> Sat, 22 Aug 2026 04:00:00 +0000 https://status.rdc.si/maintenance/cms8wwm8s0p1v13pfqex8bfcx https://status.rdc.si/maintenance/cms8wwm8s0p1v13pfqex8bfcx Intervention on UL FRI transformer station Type: Maintenance Duration: 12 hours Affected Components: Storage, Login, Compute Aug 4, 14:02:52 GMT+0 - Identified - A intervention on the UL FRI transformer station is scheduled for 13\. 8\. 2026, 16:00-21:00; during that time electricity service will be interrupted. To avoid potential loss of work or data we are putting the cluster into maintenance mode for that time. Running jobs will be terminated, and the entire cluster will be shut down, including the login node and storage. During bringup we will perform regular maintenance of the cluster to avoid additional disruptions. We appreciate your patience. Aug 13, 13:00:01 GMT+0 - Identified - Maintenance is now in progress Aug 14, 01:00:00 GMT+0 - Completed - Maintenance has completed successfully Type: Maintenance

Duration: 12 hours

Affected Components: , ,

<p><small>Aug <var data-var='date'> 4</var>, <var data-var='time'>14:02:52</var> GMT+0</small><br><strong>Identified</strong> - A intervention on the UL FRI transformer station is scheduled for 13\. 8\. 2026, 16:00-21:00; during that time electricity service will be interrupted. To avoid potential loss of work or data we are putting the cluster into maintenance mode for that time. Running jobs will be terminated, and the entire cluster will be shut down, including the login node and storage. During bringup we will perform regular maintenance of the cluster to avoid additional disruptions. We appreciate your patience..</p> <p><small>Aug <var data-var='date'> 13</var>, <var data-var='time'>13:00:01</var> GMT+0</small><br><strong>Identified</strong> - Maintenance is now in progress.</p> <p><small>Aug <var data-var='date'> 14</var>, <var data-var='time'>01:00:00</var> GMT+0</small><br><strong>Completed</strong> - Maintenance has completed successfully.</p> ]]> Thu, 13 Aug 2026 13:00:00 +0000 https://status.rdc.si/maintenance/cmseq9vgu0fi80zppvcvvpyy3 https://status.rdc.si/maintenance/cmseq9vgu0fi80zppvcvvpyy3 Login is back up Type: Incident Affected Components: Login Jul 13, 00:00:48 GMT+0 - Investigating - Login is down at the moment. This incident was automatically created by Instatus monitoring. Jul 13, 00:02:45 GMT+0 - Resolved - Login is back up. This incident was automatically resolved by Instatus monitoring. Type: Incident

Affected Components:

<p><small>Jul <var data-var='date'> 13</var>, <var data-var='time'>00:00:48</var> GMT+0</small><br><strong>Investigating</strong> - Login is down at the moment. This incident was automatically created by Instatus monitoring..</p> <p><small>Jul <var data-var='date'> 13</var>, <var data-var='time'>00:02:45</var> GMT+0</small><br><strong>Resolved</strong> - Login is back up. This incident was automatically resolved by Instatus monitoring..</p> ]]> Mon, 13 Jul 2026 00:00:48 +0000 https://status.rdc.si/incident/cmrigi80y006h0ks8n30fg5nu https://status.rdc.si/incident/cmrigi80y006h0ks8n30fg5nu Login is back up Type: Incident Affected Components: Login Jul 12, 02:59:31 GMT+0 - Investigating - Login is down at the moment. This incident was automatically created by Instatus monitoring. Jul 12, 03:01:31 GMT+0 - Resolved - Login is back up. This incident was automatically resolved by Instatus monitoring. Type: Incident

Affected Components:

<p><small>Jul <var data-var='date'> 12</var>, <var data-var='time'>02:59:31</var> GMT+0</small><br><strong>Investigating</strong> - Login is down at the moment. This incident was automatically created by Instatus monitoring..</p> <p><small>Jul <var data-var='date'> 12</var>, <var data-var='time'>03:01:31</var> GMT+0</small><br><strong>Resolved</strong> - Login is back up. This incident was automatically resolved by Instatus monitoring..</p> ]]> Sun, 12 Jul 2026 02:59:30 +0000 https://status.rdc.si/incident/cmrh7g6nq0ftkqvo2jf5ntqno https://status.rdc.si/incident/cmrh7g6nq0ftkqvo2jf5ntqno Login is back up Type: Incident Affected Components: Login Jul 2, 11:06:47 GMT+0 - Resolved - Login is back up. This incident was automatically resolved by Instatus monitoring. Jul 2, 11:04:38 GMT+0 - Investigating - Login is down at the moment. This incident was automatically created by Instatus monitoring. Type: Incident

Affected Components:

<p><small>Jul <var data-var='date'> 2</var>, <var data-var='time'>11:06:47</var> GMT+0</small><br><strong>Resolved</strong> - Login is back up. This incident was automatically resolved by Instatus monitoring..</p> <p><small>Jul <var data-var='date'> 2</var>, <var data-var='time'>11:04:38</var> GMT+0</small><br><strong>Investigating</strong> - Login is down at the moment. This incident was automatically created by Instatus monitoring..</p> ]]> Thu, 2 Jul 2026 11:04:38 +0000 https://status.rdc.si/incident/cmr3edjo2009gdapaw8re5mou https://status.rdc.si/incident/cmr3edjo2009gdapaw8re5mou Login is back up Type: Incident Affected Components: Login Jul 1, 18:10:43 GMT+0 - Investigating - Login is down at the moment. This incident was automatically created by Instatus monitoring. Jul 1, 18:31:19 GMT+0 - Resolved - Login is back up. This incident was automatically resolved by Instatus monitoring. Type: Incident

Affected Components:

<p><small>Jul <var data-var='date'> 1</var>, <var data-var='time'>18:10:43</var> GMT+0</small><br><strong>Investigating</strong> - Login is down at the moment. This incident was automatically created by Instatus monitoring..</p> <p><small>Jul <var data-var='date'> 1</var>, <var data-var='time'>18:31:19</var> GMT+0</small><br><strong>Resolved</strong> - Login is back up. This incident was automatically resolved by Instatus monitoring..</p> ]]> Wed, 1 Jul 2026 18:10:43 +0000 https://status.rdc.si/incident/cmr2e5mxb005u0krmfsr353xz https://status.rdc.si/incident/cmr2e5mxb005u0krmfsr353xz Login is back up Type: Incident Affected Components: Login Jul 1, 18:04:27 GMT+0 - Investigating - Login is down at the moment. This incident was automatically created by Instatus monitoring. Jul 1, 18:06:37 GMT+0 - Resolved - Login is back up. This incident was automatically resolved by Instatus monitoring. Type: Incident

Affected Components:

<p><small>Jul <var data-var='date'> 1</var>, <var data-var='time'>18:04:27</var> GMT+0</small><br><strong>Investigating</strong> - Login is down at the moment. This incident was automatically created by Instatus monitoring..</p> <p><small>Jul <var data-var='date'> 1</var>, <var data-var='time'>18:06:37</var> GMT+0</small><br><strong>Resolved</strong> - Login is back up. This incident was automatically resolved by Instatus monitoring..</p> ]]> Wed, 1 Jul 2026 18:04:26 +0000 https://status.rdc.si/incident/cmr2dxkrq007p0kp84ycg8ix3 https://status.rdc.si/incident/cmr2dxkrq007p0kp84ycg8ix3 Login is back up Type: Incident Affected Components: Login Jul 1, 06:10:38 GMT+0 - Resolved - Login is back up. This incident was automatically resolved by Instatus monitoring. Jun 30, 20:04:58 GMT+0 - Investigating - Login is down at the moment. This incident was automatically created by Instatus monitoring. Type: Incident

Affected Components:

<p><small>Jul <var data-var='date'> 1</var>, <var data-var='time'>06:10:38</var> GMT+0</small><br><strong>Resolved</strong> - Login is back up. This incident was automatically resolved by Instatus monitoring..</p> <p><small>Jun <var data-var='date'> 30</var>, <var data-var='time'>20:04:58</var> GMT+0</small><br><strong>Investigating</strong> - Login is down at the moment. This incident was automatically created by Instatus monitoring..</p> ]]> Tue, 30 Jun 2026 20:04:58 +0000 https://status.rdc.si/incident/cmr12spwj000r0ks4lj776srv https://status.rdc.si/incident/cmr12spwj000r0ks4lj776srv Login is back up Type: Incident Affected Components: Login Jun 23, 17:03:18 GMT+0 - Investigating - Login is down at the moment. This incident was automatically created by Instatus monitoring. Jun 23, 17:05:29 GMT+0 - Resolved - Login is back up. This incident was automatically resolved by Instatus monitoring. Type: Incident

Affected Components:

<p><small>Jun <var data-var='date'> 23</var>, <var data-var='time'>17:03:18</var> GMT+0</small><br><strong>Investigating</strong> - Login is down at the moment. This incident was automatically created by Instatus monitoring..</p> <p><small>Jun <var data-var='date'> 23</var>, <var data-var='time'>17:05:29</var> GMT+0</small><br><strong>Resolved</strong> - Login is back up. This incident was automatically resolved by Instatus monitoring..</p> ]]> Tue, 23 Jun 2026 17:03:17 +0000 https://status.rdc.si/incident/cmqqw841a01z4qvo2hqyyxb8x https://status.rdc.si/incident/cmqqw841a01z4qvo2hqyyxb8x Login is back up Type: Incident Affected Components: Login May 21, 13:46:32 GMT+0 - Investigating - Login is down at the moment. This incident was automatically created by Instatus monitoring. May 21, 13:49:33 GMT+0 - Resolved - Login is back up. This incident was automatically resolved by Instatus monitoring. Type: Incident

Affected Components:

<p><small>May <var data-var='date'> 21</var>, <var data-var='time'>13:46:32</var> GMT+0</small><br><strong>Investigating</strong> - Login is down at the moment. This incident was automatically created by Instatus monitoring..</p> <p><small>May <var data-var='date'> 21</var>, <var data-var='time'>13:49:33</var> GMT+0</small><br><strong>Resolved</strong> - Login is back up. This incident was automatically resolved by Instatus monitoring..</p> ]]> Thu, 21 May 2026 13:46:31 +0000 https://status.rdc.si/incident/cmpfjny1403b8ry15xvh8fn0g https://status.rdc.si/incident/cmpfjny1403b8ry15xvh8fn0g Login is back up Type: Incident Affected Components: Login May 21, 13:10:28 GMT+0 - Investigating - Login is down at the moment. This incident was automatically created by Instatus monitoring. May 21, 13:22:29 GMT+0 - Resolved - Login is back up. This incident was automatically resolved by Instatus monitoring. Type: Incident

Affected Components:

<p><small>May <var data-var='date'> 21</var>, <var data-var='time'>13:10:28</var> GMT+0</small><br><strong>Investigating</strong> - Login is down at the moment. This incident was automatically created by Instatus monitoring..</p> <p><small>May <var data-var='date'> 21</var>, <var data-var='time'>13:22:29</var> GMT+0</small><br><strong>Resolved</strong> - Login is back up. This incident was automatically resolved by Instatus monitoring..</p> ]]> Thu, 21 May 2026 13:10:27 +0000 https://status.rdc.si/incident/cmpfidkqx002pjz15wlta2z3y https://status.rdc.si/incident/cmpfidkqx002pjz15wlta2z3y Problems with Ceph S3 storage cluster Type: Incident Duration: 1 hour and 20 minutes Affected Components: Storage May 20, 13:29:43 GMT+0 - Identified - We were notified that FRI's Ceph cluster is currently unavailable and that FRI IT support is investigating incident. As FRIDA uses Ceph to offload older files from our Weka storage (tiering), this may affect access to your files if they were already offloaded to Ceph. May 20, 14:49:46 GMT+0 - Resolved - This incident has been resolved. FRI's Ceph S3 storage is accessible again. We will notify you In case of any new performance or availability issues. Type: Incident

Duration: 1 hour and 20 minutes

Affected Components:

<p><small>May <var data-var='date'> 20</var>, <var data-var='time'>13:29:43</var> GMT+0</small><br><strong>Identified</strong> - We were notified that FRI's Ceph cluster is currently unavailable and that FRI IT support is investigating incident. As FRIDA uses Ceph to offload older files from our Weka storage (tiering), this may affect access to your files if they were already offloaded to Ceph..</p> <p><small>May <var data-var='date'> 20</var>, <var data-var='time'>14:49:46</var> GMT+0</small><br><strong>Resolved</strong> - This incident has been resolved. FRI's Ceph S3 storage is accessible again. We will notify you In case of any new performance or availability issues..</p> ]]> Wed, 20 May 2026 13:29:43 +0000 https://status.rdc.si/incident/cmpe3mirt08odnrmoez3n335u https://status.rdc.si/incident/cmpe3mirt08odnrmoez3n335u FRIDA network upgrade Type: Maintenance Duration: 27 days, 21 hours and 52 minutes Affected Components: Storage, Login, Compute Apr 7, 06:00:01 GMT+0 - Identified - Maintenance is now in progress May 5, 03:52:02 GMT+0 - Completed - Maintenance has completed successfully. The network has been upgraded with new top tier switches, all optics has been re-cabled. Plus, new additional, top tier nodes have been installed. Slurm has been upgraded to the latest version with important steps set in place to secure as short as possible future maintenance windows. The new nodes are currently held under reservation for final benchmarking and fine-tuning before their release to general availability. In the upcoming weeks we will gradually do storage cleaning, increase storage space, and introduce storage quotas, but any maintenance if not unforeseen will be on a limited section of the cluster. We appreciate your patience. Apr 7, 06:00:00 GMT+0 - Identified - We are planning a major and longer maintenance on FRIDA. During this time the cluster will be unavailable, as we will be upgrading the cluster's network and storage. A maintenance reservation is already in place. The cluster will continue processing jobs that are scheduled to finish before the maintenance start time. Once the maintenance starts all active and pending jobs will be cancelled and all queues cleared. We appreciate your patience. Type: Maintenance

Duration: 27 days, 21 hours and 52 minutes

Affected Components: , ,

<p><small>Apr <var data-var='date'> 7</var>, <var data-var='time'>06:00:01</var> GMT+0</small><br><strong>Identified</strong> - Maintenance is now in progress.</p> <p><small>May <var data-var='date'> 5</var>, <var data-var='time'>03:52:02</var> GMT+0</small><br><strong>Completed</strong> - Maintenance has completed successfully. The network has been upgraded with new top tier switches, all optics has been re-cabled. Plus, new additional, top tier nodes have been installed. Slurm has been upgraded to the latest version with important steps set in place to secure as short as possible future maintenance windows. The new nodes are currently held under reservation for final benchmarking and fine-tuning before their release to general availability. In the upcoming weeks we will gradually do storage cleaning, increase storage space, and introduce storage quotas, but any maintenance if not unforeseen will be on a limited section of the cluster. We appreciate your patience..</p> <p><small>Apr <var data-var='date'> 7</var>, <var data-var='time'>06:00:00</var> GMT+0</small><br><strong>Identified</strong> - We are planning a major and longer maintenance on FRIDA. During this time the cluster will be unavailable, as we will be upgrading the cluster's network and storage. A maintenance reservation is already in place. The cluster will continue processing jobs that are scheduled to finish before the maintenance start time. Once the maintenance starts all active and pending jobs will be cancelled and all queues cleared. We appreciate your patience..</p> ]]> Tue, 7 Apr 2026 06:00:00 +0000 https://status.rdc.si/maintenance/cmnd1g16q02x67y43tubnmw5g https://status.rdc.si/maintenance/cmnd1g16q02x67y43tubnmw5g Cluster outage Type: Incident Duration: 3 hours and 24 minutes Affected Components: Compute Feb 14, 15:04:51 GMT+0 - Investigating - Cluster cannot be accessed at the moment. This incident was created by an automated monitoring service. Feb 14, 18:28:23 GMT+0 - Resolved - This incident has been resolved. Type: Incident

Duration: 3 hours and 24 minutes

Affected Components:

<p><small>Feb <var data-var='date'> 14</var>, <var data-var='time'>15:04:51</var> GMT+0</small><br><strong>Investigating</strong> - Cluster cannot be accessed at the moment. This incident was created by an automated monitoring service..</p> <p><small>Feb <var data-var='date'> 14</var>, <var data-var='time'>18:28:23</var> GMT+0</small><br><strong>Resolved</strong> - This incident has been resolved..</p> ]]> Sat, 14 Feb 2026 15:04:51 +0000 https://status.rdc.si/incident/cmlmg6wjh09si1vw3capzx98i https://status.rdc.si/incident/cmlmg6wjh09si1vw3capzx98i Reduced performance Type: Maintenance Duration: 4 hours and 30 minutes Affected Components: Compute Feb 3, 02:00:00 GMT+0 - Identified - We're putting the cluster in maintenance mode, due to an intervention on the main power lines scheduled for 3\. 2\. 2026, 3:00 - 7:00\. Since during this the cluster will rely entirely on UPS, we are doing so to prevent undesired outages. Plan jour jobs accordingly. We appreciate your patience. Feb 3, 02:00:01 GMT+0 - Identified - Maintenance is now in progress Feb 3, 06:30:00 GMT+0 - Completed - Maintenance has completed successfully Type: Maintenance

Duration: 4 hours and 30 minutes

Affected Components:

<p><small>Feb <var data-var='date'> 3</var>, <var data-var='time'>02:00:00</var> GMT+0</small><br><strong>Identified</strong> - We're putting the cluster in maintenance mode, due to an intervention on the main power lines scheduled for 3\. 2\. 2026, 3:00 - 7:00\. Since during this the cluster will rely entirely on UPS, we are doing so to prevent undesired outages. Plan jour jobs accordingly. We appreciate your patience..</p> <p><small>Feb <var data-var='date'> 3</var>, <var data-var='time'>02:00:01</var> GMT+0</small><br><strong>Identified</strong> - Maintenance is now in progress.</p> <p><small>Feb <var data-var='date'> 3</var>, <var data-var='time'>06:30:00</var> GMT+0</small><br><strong>Completed</strong> - Maintenance has completed successfully.</p> ]]> Tue, 3 Feb 2026 02:00:00 +0000 https://status.rdc.si/maintenance/cml4y74hr0flow7f511t5sjt2 https://status.rdc.si/maintenance/cml4y74hr0flow7f511t5sjt2 Login is down Type: Incident Affected Components: Login Dec 3, 14:07:46 GMT+0 - Investigating - Login is down at the moment. This incident was automatically created by Instatus monitoring. Dec 3, 14:09:53 GMT+0 - Resolved - Login is back up. This incident was automatically resolved by Instatus monitoring. Type: Incident

Affected Components:

<p><small>Dec <var data-var='date'> 3</var>, <var data-var='time'>14:07:46</var> GMT+0</small><br><strong>Investigating</strong> - Login is down at the moment. This incident was automatically created by Instatus monitoring..</p> <p><small>Dec <var data-var='date'> 3</var>, <var data-var='time'>14:09:53</var> GMT+0</small><br><strong>Resolved</strong> - Login is back up. This incident was automatically resolved by Instatus monitoring..</p> ]]> Wed, 3 Dec 2025 14:07:45 +0000 https://status.rdc.si/incident/cmiq30b20003jddpi99tk7me4 https://status.rdc.si/incident/cmiq30b20003jddpi99tk7me4 FRIDA stress test Type: Maintenance Duration: 10 hours Affected Components: Compute Dec 3, 07:00:00 GMT+0 - Identified - We are planning for a scheduled stress test and operation fine-tuning. During that time all jobs will be queued. We appreciate your patience. Dec 3, 07:00:01 GMT+0 - Identified - Maintenance is now in progress Dec 3, 17:00:00 GMT+0 - Completed - Maintenance has completed successfully Type: Maintenance

Duration: 10 hours

Affected Components:

<p><small>Dec <var data-var='date'> 3</var>, <var data-var='time'>07:00:00</var> GMT+0</small><br><strong>Identified</strong> - We are planning for a scheduled stress test and operation fine-tuning. During that time all jobs will be queued. We appreciate your patience..</p> <p><small>Dec <var data-var='date'> 3</var>, <var data-var='time'>07:00:01</var> GMT+0</small><br><strong>Identified</strong> - Maintenance is now in progress.</p> <p><small>Dec <var data-var='date'> 3</var>, <var data-var='time'>17:00:00</var> GMT+0</small><br><strong>Completed</strong> - Maintenance has completed successfully.</p> ]]> Wed, 3 Dec 2025 07:00:00 +0000 https://status.rdc.si/maintenance/cmiel2kon0091qmjctztxacwd https://status.rdc.si/maintenance/cmiel2kon0091qmjctztxacwd Storage, relocation of tiered data Type: Incident Duration: 10 days, 23 hours and 5 minutes Affected Components: Storage Nov 26, 17:30:00 GMT+0 - Identified - In the last couple of days we've been doing relocation of tiered data. Although transparent for most users, it may be having impact on work job performance. Be mindful if you are doing any benchmarking. The relocation should be finished by December 5th. We appreciate your patience. Dec 7, 16:34:37 GMT+0 - Resolved - Data relocation completed successfully. We've increased the amount of storage available. Note that both `/shared/home` and `/shared/workspace` are tiered; common datasets should be shared within groups using the groups' appropriate `WORK` path. You can check data location on an individual file basis with the command `weka fs tier location <file_path>` (write-cache, read-cache, or object storage), fetch data with `weka fs tier fetch <file_path>` (copy from object storage to read-cache), and release data with `weka fs tier release <file_path>` (remove from read-cache). Note that these commands work only on files, not directories, see `weka fs tier <command> --help` for more advanced usage. Type: Incident

Duration: 10 days, 23 hours and 5 minutes

Affected Components:

<p><small>Nov <var data-var='date'> 26</var>, <var data-var='time'>17:30:00</var> GMT+0</small><br><strong>Identified</strong> - In the last couple of days we've been doing relocation of tiered data. Although transparent for most users, it may be having impact on work job performance. Be mindful if you are doing any benchmarking. The relocation should be finished by December 5th. We appreciate your patience..</p> <p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>16:34:37</var> GMT+0</small><br><strong>Resolved</strong> - Data relocation completed successfully. We've increased the amount of storage available. Note that both `/shared/home` and `/shared/workspace` are tiered; common datasets should be shared within groups using the groups' appropriate `WORK` path. You can check data location on an individual file basis with the command `weka fs tier location <file_path>` (write-cache, read-cache, or object storage), fetch data with `weka fs tier fetch <file_path>` (copy from object storage to read-cache), and release data with `weka fs tier release <file_path>` (remove from read-cache). Note that these commands work only on files, not directories, see `weka fs tier <command> --help` for more advanced usage. .</p> ]]> Wed, 26 Nov 2025 17:30:00 +0000 https://status.rdc.si/incident/cmik3qr7x07j43hmhraal00gl https://status.rdc.si/incident/cmik3qr7x07j43hmhraal00gl FRIDA relocation Type: Maintenance Duration: 20 days Affected Components: Storage, Login, Compute Nov 16, 12:02:30 GMT+0 - Identified - Relocation is taking longer than expected, we are forced to extend the maintenance window. We appreciate your patience. Nov 4, 23:00:00 GMT+0 - Identified - We are planning a major maintenance on FRIDA. During this time the cluster will be unavailable, as we will be relocating the cluster to a new data centre. A maintenance reservation is already in place. The cluster will continue processing jobs that are scheduled to finish before the maintenance start time. Once maintenance starts all active and pending jobs will be cancelled and all queues cleared. We appreciate your patience. Nov 4, 23:00:01 GMT+0 - Identified - Maintenance is now in progress Nov 24, 23:00:00 GMT+0 - Completed - Cluster relocation has completed successfully. We're bringing FRIDA back on-line. We appreciate your patience. Type: Maintenance

Duration: 20 days

Affected Components: , ,

<p><small>Nov <var data-var='date'> 16</var>, <var data-var='time'>12:02:30</var> GMT+0</small><br><strong>Identified</strong> - Relocation is taking longer than expected, we are forced to extend the maintenance window. We appreciate your patience..</p> <p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>23:00:00</var> GMT+0</small><br><strong>Identified</strong> - We are planning a major maintenance on FRIDA. During this time the cluster will be unavailable, as we will be relocating the cluster to a new data centre. A maintenance reservation is already in place. The cluster will continue processing jobs that are scheduled to finish before the maintenance start time. Once maintenance starts all active and pending jobs will be cancelled and all queues cleared. We appreciate your patience..</p> <p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>23:00:01</var> GMT+0</small><br><strong>Identified</strong> - Maintenance is now in progress.</p> <p><small>Nov <var data-var='date'> 24</var>, <var data-var='time'>23:00:00</var> GMT+0</small><br><strong>Completed</strong> - Cluster relocation has completed successfully. We're bringing FRIDA back on-line. We appreciate your patience..</p> ]]> Tue, 4 Nov 2025 23:00:00 +0000 https://status.rdc.si/maintenance/cmh1vp5d3000exl13g0ftlmtf https://status.rdc.si/maintenance/cmh1vp5d3000exl13g0ftlmtf Major malfunction of one of the nodes Type: Incident Duration: 5 days, 21 hours and 5 minutes Affected Components: Compute Sep 24, 15:49:56 GMT+0 - Resolved - We have successfully replaced GPU2 and node ixh has been put back into production. Thank you for your patience. Sep 18, 18:45:00 GMT+0 - Identified - Node ixh is in maintenance due to overheating of GPU2\. We are working on a resolution with support. Thank you for your patience. Type: Incident

Duration: 5 days, 21 hours and 5 minutes

Affected Components:

<p><small>Sep <var data-var='date'> 24</var>, <var data-var='time'>15:49:56</var> GMT+0</small><br><strong>Resolved</strong> - We have successfully replaced GPU2 and node ixh has been put back into production. Thank you for your patience..</p> <p><small>Sep <var data-var='date'> 18</var>, <var data-var='time'>18:45:00</var> GMT+0</small><br><strong>Identified</strong> - Node ixh is in maintenance due to overheating of GPU2\. We are working on a resolution with support. Thank you for your patience..</p> ]]> Thu, 18 Sep 2025 18:45:00 +0000 https://status.rdc.si/incident/cmfpwt8ob011h35cmyrfk4rs9 https://status.rdc.si/incident/cmfpwt8ob011h35cmyrfk4rs9 We are experiencing some issues with the cluster Type: Incident Duration: 59 minutes Affected Components: Storage, , Login, Compute, Frida → Sep 8, 13:12:03 GMT+0 - Resolved - This incident has been resolved. Sep 8, 12:13:22 GMT+0 - Investigating - We are currently investigating this incident. Type: Incident

Duration: 59 minutes

Affected Components: , , ,

<p><small>Sep <var data-var='date'> 8</var>, <var data-var='time'>13:12:03</var> GMT+0</small><br><strong>Resolved</strong> - This incident has been resolved..</p> <p><small>Sep <var data-var='date'> 8</var>, <var data-var='time'>12:13:22</var> GMT+0</small><br><strong>Investigating</strong> - We are currently investigating this incident..</p> ]]> Mon, 8 Sep 2025 12:13:22 +0000 https://status.rdc.si/incident/cmfb31yg1002d4wxygb484cqj https://status.rdc.si/incident/cmfb31yg1002d4wxygb484cqj Regular maintenance Type: Maintenance Duration: 1 day Affected Components: Storage, , Login, Compute, Frida → Aug 8, 22:00:00 GMT+0 - Identified - On August 9th a regular maintenance on the DC power grid is scheduled. We'll take this opportunity to perform also regular maintenance on all FRIDA nodes. Running jobs may get canceled. Aug 9, 22:00:00 GMT+0 - Completed - Maintenance has completed successfully Aug 8, 22:00:01 GMT+0 - Identified - Maintenance is now in progress Type: Maintenance

Duration: 1 day

Affected Components: , , ,

<p><small>Aug <var data-var='date'> 8</var>, <var data-var='time'>22:00:00</var> GMT+0</small><br><strong>Identified</strong> - On August 9th a regular maintenance on the DC power grid is scheduled. We'll take this opportunity to perform also regular maintenance on all FRIDA nodes. Running jobs may get canceled..</p> <p><small>Aug <var data-var='date'> 9</var>, <var data-var='time'>22:00:00</var> GMT+0</small><br><strong>Completed</strong> - Maintenance has completed successfully.</p> <p><small>Aug <var data-var='date'> 8</var>, <var data-var='time'>22:00:01</var> GMT+0</small><br><strong>Identified</strong> - Maintenance is now in progress.</p> ]]> Fri, 8 Aug 2025 22:00:00 +0000 https://status.rdc.si/maintenance/cmdym3wck001c4wtumust8b1i https://status.rdc.si/maintenance/cmdym3wck001c4wtumust8b1i Major malfunction of one of the nodes Type: Incident Duration: 12 days, 22 hours and 19 minutes Affected Components: Compute Jul 24, 16:33:20 GMT+0 - Identified - Node ixh is down as during the replacement of GPU6 issues with GPU0 have been detected. We're coordinating a resolution with support. Thank you for your patience. Jul 17, 12:42:58 GMT+0 - Identified - Node ixh is down due to overheating of GPU6\. We are working on a resolution with support. Thank you for your patience. Jul 17, 12:21:17 GMT+0 - Investigating - We are currently investigating this incident. Jul 30, 10:40:43 GMT+0 - Resolved - The GPUs on node ixh has been successfully replaced and the node is back in production. Please, benchmark your runs against earlier ones and report any discrepancies. Thank you for your patience. Type: Incident

Duration: 12 days, 22 hours and 19 minutes

Affected Components:

<p><small>Jul <var data-var='date'> 24</var>, <var data-var='time'>16:33:20</var> GMT+0</small><br><strong>Identified</strong> - Node ixh is down as during the replacement of GPU6 issues with GPU0 have been detected. We're coordinating a resolution with support. Thank you for your patience. .</p> <p><small>Jul <var data-var='date'> 17</var>, <var data-var='time'>12:42:58</var> GMT+0</small><br><strong>Identified</strong> - Node ixh is down due to overheating of GPU6\. We are working on a resolution with support. Thank you for your patience..</p> <p><small>Jul <var data-var='date'> 17</var>, <var data-var='time'>12:21:17</var> GMT+0</small><br><strong>Investigating</strong> - We are currently investigating this incident..</p> <p><small>Jul <var data-var='date'> 30</var>, <var data-var='time'>10:40:43</var> GMT+0</small><br><strong>Resolved</strong> - The GPUs on node ixh has been successfully replaced and the node is back in production. Please, benchmark your runs against earlier ones and report any discrepancies. Thank you for your patience..</p> ]]> Thu, 17 Jul 2025 12:21:17 +0000 https://status.rdc.si/incident/cmd7czzg8004johyiojdt9nzj https://status.rdc.si/incident/cmd7czzg8004johyiojdt9nzj Ceph congestion Type: Incident Duration: 2 hours and 21 minutes Affected Components: Storage Jul 3, 08:10:00 GMT+0 - Identified - We are currently experiencing an outage of the external Ceph tired storage, your /shared/home folders may be affected. We are investigating the issue and trying to resolve it as swiftly as possible. We apologise for the inconvenience. Jul 3, 10:31:10 GMT+0 - Resolved - This incident has been resolved. Access to all affected files should be restored. Type: Incident

Duration: 2 hours and 21 minutes

Affected Components:

<p><small>Jul <var data-var='date'> 3</var>, <var data-var='time'>08:10:00</var> GMT+0</small><br><strong>Identified</strong> - We are currently experiencing an outage of the external Ceph tired storage, your /shared/home folders may be affected. We are investigating the issue and trying to resolve it as swiftly as possible. We apologise for the inconvenience..</p> <p><small>Jul <var data-var='date'> 3</var>, <var data-var='time'>10:31:10</var> GMT+0</small><br><strong>Resolved</strong> - This incident has been resolved. Access to all affected files should be restored..</p> ]]> Thu, 3 Jul 2025 08:10:00 +0000 https://status.rdc.si/incident/cmcn550ic018lmxxh1e6eliui https://status.rdc.si/incident/cmcn550ic018lmxxh1e6eliui Storage maintenance Type: Maintenance Duration: 6 hours Affected Components: Storage Jun 18, 07:00:01 GMT+0 - Identified - Maintenance is now in progress Jun 18, 07:00:00 GMT+0 - Identified - We are planning an upgrade of the storage software; the process should go largely undetected, but in case of issues we may be forced to cancel active jobs. We appreciate your patience. Jun 18, 13:00:00 GMT+0 - Completed - Maintenance has completed successfully Type: Maintenance

Duration: 6 hours

Affected Components:

<p><small>Jun <var data-var='date'> 18</var>, <var data-var='time'>07:00:01</var> GMT+0</small><br><strong>Identified</strong> - Maintenance is now in progress.</p> <p><small>Jun <var data-var='date'> 18</var>, <var data-var='time'>07:00:00</var> GMT+0</small><br><strong>Identified</strong> - We are planning an upgrade of the storage software; the process should go largely undetected, but in case of issues we may be forced to cancel active jobs. We appreciate your patience..</p> <p><small>Jun <var data-var='date'> 18</var>, <var data-var='time'>13:00:00</var> GMT+0</small><br><strong>Completed</strong> - Maintenance has completed successfully.</p> ]]> Wed, 18 Jun 2025 07:00:00 +0000 https://status.rdc.si/maintenance/cmc07g10y000hn5l42t84ehrl https://status.rdc.si/maintenance/cmc07g10y000hn5l42t84ehrl RDC cooling malfunction Type: Incident Duration: 11 days, 7 hours and 23 minutes Affected Components: Storage, Login, Compute Jun 15, 15:13:43 GMT+0 - Identified - We are continuing to work on a fix for this incident. Jun 19, 14:37:56 GMT+0 - Monitoring - Due to continual cooling malfunction we're forced to cancel all jobs and shut down the cluster. A larger maintenance of the RDC cooling that will hopefully resolve the issues is planned for tomorrow morning. We appreciate your patience and we'll keep you posted of the progress. Jun 20, 16:52:20 GMT+0 - Monitoring - The maintenance of the RDC cooling has finished (partially). Some procedures are more extensive and will need to be performed in the upcoming week. We'll keep the operation of FRIDA under close monitoring and try to keep disruptive actions to the minimum possible. We appreciate your patience. Jun 15, 13:43:16 GMT+0 - Investigating - The RDC cooling is experiencing malfunction. As a preventative measure we're forced to shutdown the cluster; all running jobs were canceled. The cluster has been temporarily put into maintenance, with a planned release after 22:00 or once the datacenter's cooling resumes normal operation and temperatures reach acceptable levels. Due to high external temperatures and regular cooling malfunctions job cancelations of the most demanding jobs may become more frequent. We therefore advise to run demanding jobs overnight and perform regular and frequent checkpoints. Before and when running jobs please consult the dashboard “RDC heat load”. We appreciate your patience. Jun 16, 14:30:53 GMT+0 - Monitoring - The RDC cooling has returned to normal values, and the cluster has been put back into operation. We will be monitoring the cooling. We are working on resolving the issue. Note that in case of high external temperatures we may be forced to temporarily pause the cluster or cancel jobs to prevent additional fallouts. We appreciate your patience. Aug 11, 07:23:04 GMT+0 - Identified - Due to cooling malfunction we need to stop all operations. We're actively working on a fix. We appreciate your patience. Aug 11, 12:29:57 GMT+0 - Monitoring - The evaporator of one of the cooling units has been successfully repaired; some repairs will need to be performed at a later time. We're bringing the cluster back online, but will continue monitoring the cooling behaviour. We appreciate your patience. Nov 24, 21:05:53 GMT+0 - Resolved - FRIDA was successfully relocated to a new facility. Type: Incident

Duration: 11 days, 7 hours and 23 minutes

Affected Components: , ,

<p><small>Jun <var data-var='date'> 15</var>, <var data-var='time'>15:13:43</var> GMT+0</small><br><strong>Identified</strong> - We are continuing to work on a fix for this incident..</p> <p><small>Jun <var data-var='date'> 19</var>, <var data-var='time'>14:37:56</var> GMT+0</small><br><strong>Monitoring</strong> - Due to continual cooling malfunction we're forced to cancel all jobs and shut down the cluster. A larger maintenance of the RDC cooling that will hopefully resolve the issues is planned for tomorrow morning. We appreciate your patience and we'll keep you posted of the progress..</p> <p><small>Jun <var data-var='date'> 20</var>, <var data-var='time'>16:52:20</var> GMT+0</small><br><strong>Monitoring</strong> - The maintenance of the RDC cooling has finished (partially). Some procedures are more extensive and will need to be performed in the upcoming week. We'll keep the operation of FRIDA under close monitoring and try to keep disruptive actions to the minimum possible. We appreciate your patience. .</p> <p><small>Jun <var data-var='date'> 15</var>, <var data-var='time'>13:43:16</var> GMT+0</small><br><strong>Investigating</strong> - The RDC cooling is experiencing malfunction. As a preventative measure we're forced to shutdown the cluster; all running jobs were canceled. The cluster has been temporarily put into maintenance, with a planned release after 22:00 or once the datacenter's cooling resumes normal operation and temperatures reach acceptable levels. Due to high external temperatures and regular cooling malfunctions job cancelations of the most demanding jobs may become more frequent. We therefore advise to run demanding jobs overnight and perform regular and frequent checkpoints. Before and when running jobs please consult the dashboard “RDC heat load”. We appreciate your patience..</p> <p><small>Jun <var data-var='date'> 16</var>, <var data-var='time'>14:30:53</var> GMT+0</small><br><strong>Monitoring</strong> - The RDC cooling has returned to normal values, and the cluster has been put back into operation. We will be monitoring the cooling. We are working on resolving the issue. Note that in case of high external temperatures we may be forced to temporarily pause the cluster or cancel jobs to prevent additional fallouts. We appreciate your patience..</p> <p><small>Aug <var data-var='date'> 11</var>, <var data-var='time'>07:23:04</var> GMT+0</small><br><strong>Identified</strong> - Due to cooling malfunction we need to stop all operations. We're actively working on a fix. We appreciate your patience..</p> <p><small>Aug <var data-var='date'> 11</var>, <var data-var='time'>12:29:57</var> GMT+0</small><br><strong>Monitoring</strong> - The evaporator of one of the cooling units has been successfully repaired; some repairs will need to be performed at a later time. We're bringing the cluster back online, but will continue monitoring the cooling behaviour. We appreciate your patience..</p> <p><small>Nov <var data-var='date'> 24</var>, <var data-var='time'>21:05:53</var> GMT+0</small><br><strong>Resolved</strong> - FRIDA was successfully relocated to a new facility..</p> ]]> Sun, 15 Jun 2025 13:43:16 +0000 https://status.rdc.si/incident/cmbxpu5h7004tw28x5fvhjzr1 https://status.rdc.si/incident/cmbxpu5h7004tw28x5fvhjzr1 RDC cooling malfunction Type: Incident Duration: 1 day, 8 hours and 36 minutes Affected Components: , Compute, Frida → Jun 10, 13:00:00 GMT+0 - Identified - The RDC cooling is experiencing malfunction. As a preventative measure we're forced to shutdown the cluster; all jobs will be canceled. While working to fix the issue we'll also perform the scheduled FRIDA maintenance to keep the downtime at minimum. We appreciate your patience. Jun 11, 21:35:47 GMT+0 - Resolved - We implemented a fix, we're bringing the cluster back into operation. We'll be monitoring the cooling. Please make sure to perform regular checkpoints to avoid data loss in case of additional shutdowns. We appreciate your patience. Type: Incident

Duration: 1 day, 8 hours and 36 minutes

Affected Components: ,

<p><small>Jun <var data-var='date'> 10</var>, <var data-var='time'>13:00:00</var> GMT+0</small><br><strong>Identified</strong> - The RDC cooling is experiencing malfunction. As a preventative measure we're forced to shutdown the cluster; all jobs will be canceled. While working to fix the issue we'll also perform the scheduled FRIDA maintenance to keep the downtime at minimum. We appreciate your patience..</p> <p><small>Jun <var data-var='date'> 11</var>, <var data-var='time'>21:35:47</var> GMT+0</small><br><strong>Resolved</strong> - We implemented a fix, we're bringing the cluster back into operation. We'll be monitoring the cooling. Please make sure to perform regular checkpoints to avoid data loss in case of additional shutdowns. We appreciate your patience..</p> ]]> Tue, 10 Jun 2025 13:00:00 +0000 https://status.rdc.si/incident/cmbs22z7m00048p6kx1gwscao https://status.rdc.si/incident/cmbs22z7m00048p6kx1gwscao Major malfunction of one of the nodes Type: Incident Duration: 4 days and 46 minutes Affected Components: Compute May 6, 09:30:00 GMT+0 - Investigating - We are currently investigating this incident. May 7, 10:56:40 GMT+0 - Monitoring - We are reinstating the node and we'll monitor the status. May 7, 12:46:06 GMT+0 - Identified - Node ixh is down due to overheating of GPU0\. We are working on a resolution with support. Thank you for your patience. May 10, 10:15:46 GMT+0 - Resolved - GPU0 on node ixh has been successfully replaced and the node is back in production. Please, benchmark your runs against earlier ones and report any discrepancies. Thank you for your patience. Type: Incident

Duration: 4 days and 46 minutes

Affected Components:

<p><small>May <var data-var='date'> 6</var>, <var data-var='time'>09:30:00</var> GMT+0</small><br><strong>Investigating</strong> - We are currently investigating this incident..</p> <p><small>May <var data-var='date'> 7</var>, <var data-var='time'>10:56:40</var> GMT+0</small><br><strong>Monitoring</strong> - We are reinstating the node and we'll monitor the status..</p> <p><small>May <var data-var='date'> 7</var>, <var data-var='time'>12:46:06</var> GMT+0</small><br><strong>Identified</strong> - Node ixh is down due to overheating of GPU0\. We are working on a resolution with support. Thank you for your patience..</p> <p><small>May <var data-var='date'> 10</var>, <var data-var='time'>10:15:46</var> GMT+0</small><br><strong>Resolved</strong> - GPU0 on node ixh has been successfully replaced and the node is back in production. Please, benchmark your runs against earlier ones and report any discrepancies. Thank you for your patience..</p> ]]> Tue, 6 May 2025 09:30:00 +0000 https://status.rdc.si/incident/cmadryr01005325ugbf1h0xgm https://status.rdc.si/incident/cmadryr01005325ugbf1h0xgm Regular maintenance Type: Maintenance Duration: 6 days Affected Components: Storage, Compute, , Login, Frida → Apr 30, 20:00:00 GMT+0 - Identified - We are planning for a scheduled maintenance during that time. May 6, 20:00:00 GMT+0 - Completed - Maintenance has completed successfully Apr 30, 20:00:01 GMT+0 - Identified - Maintenance is now in progress May 6, 20:59:12 GMT+0 - Completed - Maintenance has completed successfully. We performed the full set of updates & upgrades on all cluster nodes. The storage cluster has also undergone the full set of updates & upgrades. Most of the cluster is operational and ready to accept jobs. One node is currently kept in maintenance mode due to HW issues that require physical maintenance, it should resume operation in the next few days. Type: Maintenance

Duration: 6 days

Affected Components: , , ,

<p><small>Apr <var data-var='date'> 30</var>, <var data-var='time'>20:00:00</var> GMT+0</small><br><strong>Identified</strong> - We are planning for a scheduled maintenance during that time..</p> <p><small>May <var data-var='date'> 6</var>, <var data-var='time'>20:00:00</var> GMT+0</small><br><strong>Completed</strong> - Maintenance has completed successfully.</p> <p><small>Apr <var data-var='date'> 30</var>, <var data-var='time'>20:00:01</var> GMT+0</small><br><strong>Identified</strong> - Maintenance is now in progress.</p> <p><small>May <var data-var='date'> 6</var>, <var data-var='time'>20:59:12</var> GMT+0</small><br><strong>Completed</strong> - Maintenance has completed successfully. We performed the full set of updates & upgrades on all cluster nodes. The storage cluster has also undergone the full set of updates & upgrades. Most of the cluster is operational and ready to accept jobs. One node is currently kept in maintenance mode due to HW issues that require physical maintenance, it should resume operation in the next few days..</p> ]]> Wed, 30 Apr 2025 20:00:00 +0000 https://status.rdc.si/maintenance/cm9jlkmgb006caoc7ewtcdxek https://status.rdc.si/maintenance/cm9jlkmgb006caoc7ewtcdxek Login is down Type: Incident Affected Components: Login Apr 29, 20:49:16 GMT+0 - Investigating - Login is down at the moment. This incident was automatically created by Instatus monitoring. Apr 29, 20:49:43 GMT+0 - Resolved - Login is back up. This incident was automatically resolved by Instatus monitoring. Type: Incident

Affected Components:

<p><small>Apr <var data-var='date'> 29</var>, <var data-var='time'>20:49:16</var> GMT+0</small><br><strong>Investigating</strong> - Login is down at the moment. This incident was automatically created by Instatus monitoring..</p> <p><small>Apr <var data-var='date'> 29</var>, <var data-var='time'>20:49:43</var> GMT+0</small><br><strong>Resolved</strong> - Login is back up. This incident was automatically resolved by Instatus monitoring..</p> ]]> Tue, 29 Apr 2025 20:49:13 +0000 https://status.rdc.si/incident/cma2zcvs0000xrzrq6t06h5jb https://status.rdc.si/incident/cma2zcvs0000xrzrq6t06h5jb Cluster outage Type: Incident Duration: 7 days, 4 hours and 39 minutes Affected Components: Compute May 6, 20:50:20 GMT+0 - Resolved - This incident has been resolved. Apr 29, 16:10:58 GMT+0 - Investigating - Cluster cannot be accessed at the moment. This incident was created by an automated monitoring service. Type: Incident

Duration: 7 days, 4 hours and 39 minutes

Affected Components:

<p><small>May <var data-var='date'> 6</var>, <var data-var='time'>20:50:20</var> GMT+0</small><br><strong>Resolved</strong> - This incident has been resolved..</p> <p><small>Apr <var data-var='date'> 29</var>, <var data-var='time'>16:10:58</var> GMT+0</small><br><strong>Investigating</strong> - Cluster cannot be accessed at the moment. This incident was created by an automated monitoring service..</p> ]]> Tue, 29 Apr 2025 16:10:58 +0000 https://status.rdc.si/incident/cma2pf1zh003tmeazo11lou02 https://status.rdc.si/incident/cma2pf1zh003tmeazo11lou02 RDC cooling malfunction Type: Incident Duration: 12 days, 18 hours and 21 minutes Affected Components: Storage, Compute, , Login, Frida → May 10, 10:51:05 GMT+0 - Identified - The RDC cooling is experiencing malfunction. As a preventative measure we're forced to shutdown the cluster; all jobs will be canceled. We appreciate your patience. Apr 29, 15:37:24 GMT+0 - Identified - The RDC cooling is experiencing malfunction. As a preventative measure we're forced to shutdown the cluster; all jobs will be canceled. While working to fix the issue we'll also perform the scheduled FRIDA maintenance to keep the downtime at minimum. We appreciate your patience. May 1, 09:47:34 GMT+0 - Monitoring - The RDC cooling was fixed, we're currently bringing the cluster back to life, performing the scheduled FRIDA maintenance, and monitoring the status. May 12, 09:58:11 GMT+0 - Resolved - The RDC cooling was partially fixed, we're bringing the cluster back to production. We'll be monitoring the status. During the week the remaining RDC cooling issues will be resolved. We appreciate your patience Type: Incident

Duration: 12 days, 18 hours and 21 minutes

Affected Components: , , ,

<p><small>May <var data-var='date'> 10</var>, <var data-var='time'>10:51:05</var> GMT+0</small><br><strong>Identified</strong> - The RDC cooling is experiencing malfunction. As a preventative measure we're forced to shutdown the cluster; all jobs will be canceled. We appreciate your patience..</p> <p><small>Apr <var data-var='date'> 29</var>, <var data-var='time'>15:37:24</var> GMT+0</small><br><strong>Identified</strong> - The RDC cooling is experiencing malfunction. As a preventative measure we're forced to shutdown the cluster; all jobs will be canceled. While working to fix the issue we'll also perform the scheduled FRIDA maintenance to keep the downtime at minimum. We appreciate your patience..</p> <p><small>May <var data-var='date'> 1</var>, <var data-var='time'>09:47:34</var> GMT+0</small><br><strong>Monitoring</strong> - The RDC cooling was fixed, we're currently bringing the cluster back to life, performing the scheduled FRIDA maintenance, and monitoring the status..</p> <p><small>May <var data-var='date'> 12</var>, <var data-var='time'>09:58:11</var> GMT+0</small><br><strong>Resolved</strong> - The RDC cooling was partially fixed, we're bringing the cluster back to production. We'll be monitoring the status. During the week the remaining RDC cooling issues will be resolved. We appreciate your patience.</p> ]]> Tue, 29 Apr 2025 15:37:24 +0000 https://status.rdc.si/incident/cma2o7wd1002jvquvg8bhcr5c https://status.rdc.si/incident/cma2o7wd1002jvquvg8bhcr5c Storage upgrade Type: Maintenance Duration: 14 days, 8 hours and 57 minutes Affected Components: Storage, Compute, , Login, Frida → Feb 28, 14:00:01 GMT+0 - Identified - Maintenance is now in progress Feb 28, 14:00:00 GMT+0 - Identified - We are planning a major maintenance on FRIDA. During this time the cluster will be unavailable. A maintenance reservation is already in place. The cluster will continue processing jobs that are scheduled to finish before the maintenance start time, other jobs will wait in queue. As this is a major maintenance waiting jobs may be cancelled. You can list the active reservations by issuing the following command `scontrol show reservations` Mar 9, 22:04:52 GMT+0 - Identified - Due to the complexity of the storage deploy procedure associated with the heterogeneous nature of our environment we are extending the maintenance window. We appreciate your patience. Mar 14, 22:57:00 GMT+0 - Completed - Maintenance on FRIDA has completed successfully. The key aspects of the cluster that have been updated are a new shared storage system based on the Weka Data Platform, and upgrade of Teleport to version v17\. Install the latest v17.X version of the Teleport client (<https://goteleport.com/download>) before connecting. In the upcoming weeks we will progressively increase storage space (mostly lab/group/project space) even further. For higher performance we advise you to move your lab/group/project related data to the corresponding shared folder. We appreciate your patience. Type: Maintenance

Duration: 14 days, 8 hours and 57 minutes

Affected Components: , , ,

<p><small>Feb <var data-var='date'> 28</var>, <var data-var='time'>14:00:01</var> GMT+0</small><br><strong>Identified</strong> - Maintenance is now in progress.</p> <p><small>Feb <var data-var='date'> 28</var>, <var data-var='time'>14:00:00</var> GMT+0</small><br><strong>Identified</strong> - We are planning a major maintenance on FRIDA. During this time the cluster will be unavailable. A maintenance reservation is already in place. The cluster will continue processing jobs that are scheduled to finish before the maintenance start time, other jobs will wait in queue. As this is a major maintenance waiting jobs may be cancelled. You can list the active reservations by issuing the following command `scontrol show reservations`.</p> <p><small>Mar <var data-var='date'> 9</var>, <var data-var='time'>22:04:52</var> GMT+0</small><br><strong>Identified</strong> - Due to the complexity of the storage deploy procedure associated with the heterogeneous nature of our environment we are extending the maintenance window. We appreciate your patience..</p> <p><small>Mar <var data-var='date'> 14</var>, <var data-var='time'>22:57:00</var> GMT+0</small><br><strong>Completed</strong> - Maintenance on FRIDA has completed successfully. The key aspects of the cluster that have been updated are a new shared storage system based on the Weka Data Platform, and upgrade of Teleport to version v17\. Install the latest v17.X version of the Teleport client (<https://goteleport.com/download>) before connecting. In the upcoming weeks we will progressively increase storage space (mostly lab/group/project space) even further. For higher performance we advise you to move your lab/group/project related data to the corresponding shared folder. We appreciate your patience. .</p> ]]> Fri, 28 Feb 2025 14:00:00 +0000 https://status.rdc.si/maintenance/cm7hsk2mv009hd0ohik7fqy0u https://status.rdc.si/maintenance/cm7hsk2mv009hd0ohik7fqy0u Storage upgrade Type: Maintenance Duration: 8 hours and 44 minutes Affected Components: Compute, Storage Feb 12, 14:00:01 GMT+0 - Identified - Maintenance is now in progress Feb 12, 14:00:00 GMT+0 - Identified - We are planning a major maintenance on FRIDA. During this time the cluster will be unavailable. A maintenance reservation is already in place. The cluster will continue processing jobs that are scheduled to finish before the maintenance start time, other jobs will wait in queue. As this is a major maintenance waiting jobs may be cancelled. You can list the active reservations by issuing the following command `scontrol show reservations ` Feb 12, 22:44:06 GMT+0 - Completed - The scheduled maintenance has been postponed due to a delay in the delivery of essential network infrastructure. We will provide an updated schedule once the required components are received and ready for installation. Type: Maintenance

Duration: 8 hours and 44 minutes

Affected Components: ,

<p><small>Feb <var data-var='date'> 12</var>, <var data-var='time'>14:00:01</var> GMT+0</small><br><strong>Identified</strong> - Maintenance is now in progress.</p> <p><small>Feb <var data-var='date'> 12</var>, <var data-var='time'>14:00:00</var> GMT+0</small><br><strong>Identified</strong> - We are planning a major maintenance on FRIDA. During this time the cluster will be unavailable. A maintenance reservation is already in place. The cluster will continue processing jobs that are scheduled to finish before the maintenance start time, other jobs will wait in queue. As this is a major maintenance waiting jobs may be cancelled. You can list the active reservations by issuing the following command `scontrol show reservations ` .</p> <p><small>Feb <var data-var='date'> 12</var>, <var data-var='time'>22:44:06</var> GMT+0</small><br><strong>Completed</strong> - The scheduled maintenance has been postponed due to a delay in the delivery of essential network infrastructure. We will provide an updated schedule once the required components are received and ready for installation..</p> ]]> Wed, 12 Feb 2025 14:00:00 +0000 https://status.rdc.si/maintenance/cm6dlr2uo003x13ue8t9iylwp https://status.rdc.si/maintenance/cm6dlr2uo003x13ue8t9iylwp