Ops One AG platform status and maintenance announcements. Times are indicated in local Swiss time. https://opsone.ch/.
All systems are operational
Scheduled Maintenance
GitLab Fleet Scaling: Migrate to Debian 13 as Base Image

The underlying VM on which GitLab CI/CD jobs are executed will be migrated to Debian 13.
This has no effect on the Docker image in which the respective job is executed, which is normally already debian:stable.

Reference: #67

Start maintenance window: 22/09/2025 23:00
Stop maintenance window: 23/09/2025 02:00

Mailcow important changes for update from legacy to master

We will update Mailcow from legacy to the newest master version, which contains the following breaking changes:

  1. Login With the update, the login screens for users, domain administrators, and administrators are now separate:

Users log in as usual at the following URL: https://mail.example.com/

Domain administrators now use https://mail.example.com/domainadmin

Administrators use https://mail.example.com/admin

The SOGo web interface is now only accessible via the user login at https://mail.example.com/; after a standard direct redirect to SOGo, the Mailcow settings are still accessible via the link (wrench icon, top right). The Login URL for Roundcube User does not change.

  1. Two-factor authentication (2FA) Users/mail accounts that have 2FA enabled must now use app passwords to log in via mail protocols such as SMTP/IMAP, etc., for example when using a local email client or the Roundcube web interface. Each user can create these after logging into their Mailcow account under "App Passwords".

Take a close look at the changes, update your documentation accordingly, and feel free to contact us at any time if you have any questions or concerns.

Start maintenance window: 29/09/2025 23:00
Stop maintenance window: 30/09/2025 02:00

Past Incidents

Saturday 24th June 2023

Cluster Maintenance, scheduled 2 years ago

Two of our SSD disks do report problematic wearout indicator values. For the moment, they still do run fine, but we will replace them out of caution nevertheless. During the rebuild, a higher disk i/o latency is to be expected.

Update 25/06/2023 10:00: Both disk have been replaced successfully.

Start maintenance window: 24/06/2023 18:00
Stop maintenance window: 26/06/2023 06:00

Monday 19th June 2023

Confluence 7.19.10, scheduled 2 years ago

We will update Confluence to version 7.19.10.

Start maintenance window: 19/06/2023 23:00
Stop maintenance window: 20/06/2023 02:00

Traefik 2.10.2, scheduled 2 years ago

We will update Traefik to version 2.10.2.

Start maintenance window: 19/06/2023 23:00
Stop maintenance window: 20/06/2023 02:00

Sentry 23.6.1, scheduled 2 years ago

We will update Sentry to version 23.6.1.

Start maintenance window: 19/06/2023 23:00
Stop maintenance window: 20/06/2023 02:00

OnlyOffice 7.4.0, scheduled 2 years ago

We will update OnlyOffice to version 7.4.0.

Start maintenance window: 19/06/2023 23:00
Stop maintenance window: 20/06/2023 02:00

Discourse 3.4.0, scheduled 2 years ago

We will update Discourse to version 3.4.0.

Start maintenance window: 19/06/2023 23:00
Stop maintenance window: 20/06/2023 02:00

Shibboleth: Security Update XMLTooling, scheduled 2 years ago

An updated version of the XMLTooling library that is part of the OpenSAML and Shibboleth Service Provider software is now available which corrects a server-side request forgery (SSRF) vulnerability.

For details, see Shibboleth Service Provider Security Advisory [12 June 2023].

Start maintenance window: 19/06/2023 23:00
Stop maintenance window: 20/06/2023 02:00

Sunday 18th June 2023

No incidents reported

Saturday 17th June 2023

No incidents reported

Friday 16th June 2023

No incidents reported

Thursday 15th June 2023

No incidents reported

Wednesday 14th June 2023

Network Partial Network Outage

At 14/06/2023 15:44 we detected increased high latencies in the network during maintenance work. This delayed the internal data replication, which in turn led to further latencies in the system. As a result, certain services were no longer available. The situation slowly returned to normal after a structured restart of the affected services.

Our analysis showed that this was not an attack. We have identified a human error as the cause. The confidentiality and integrity of the data was guaranteed at all times.

As an immediate measure, first organisational measures have already been taken. In addition, further infrastructure optimisation is planned for the next few weeks to prevent similar problems in the future.

We apologise for the inconvenience experienced by some of our customers. Thank you for your patience and understanding.

Incident timeline

  • 17:48 [Update] - Services are back in operation. We will closely monitor the situation over the next few hours and then analyse the root cause.
  • 16:53 [Update] - We are still experiencing limited availability of certain services. The affected systems are gradually becoming available again
  • 16:23 [Update] - Affected services are partially available again. But still suffering from poor performance
  • 15:53 - We are experiencing a partial outage of our network and are currently investigating the cause

Tuesday 13th June 2023

Managed Server Version 7: Debian Updates, scheduled 2 years ago

We will update the following packages to the latest version.

vim-common/oldoldstable 2:8.1.0875-5+deb10u5 all [upgradable from: 2:8.1.0875-5+deb10u4]
vim/oldoldstable 2:8.1.0875-5+deb10u5 amd64 [upgradable from: 2:8.1.0875-5+deb10u4]
vim-runtime/oldoldstable 2:8.1.0875-5+deb10u5 all [upgradable from: 2:8.1.0875-5+deb10u4]
vim-tiny/oldoldstable 2:8.1.0875-5+deb10u5 amd64 [upgradable from: 2:8.1.0875-5+deb10u4]
xxd/oldoldstable 2:8.1.0875-5+deb10u5 amd64 [upgradable from: 2:8.1.0875-5+deb10u4]

Start maintenance window: 13/06/2023 15:00
Stop maintenance window: 13/06/2023 16:00

Managed Server Version 7: TYPO3 v12.4.2

The versions 12.4.2 of the TYPO3 Enterprise Content Management System have just been released.

https://typo3.org/article/typo3-1242-and-11528-maintenance-releases-published

Managed Server Version 7: TYPO3 v11.5.28

The versions 11.5.28 of the TYPO3 Enterprise Content Management System have just been released.

https://typo3.org/article/typo3-1242-and-11528-maintenance-releases-published

Managed Server Version 8: TYPO3 v11.5.28

The versions 11.5.28 of the TYPO3 Enterprise Content Management System have just been released.

https://typo3.org/article/typo3-1242-and-11528-maintenance-releases-published

Managed Server Version 8: TYPO3 v12.4.2

The versions 12.4.2 of the TYPO3 Enterprise Content Management System have just been released.

https://typo3.org/article/typo3-1242-and-11528-maintenance-releases-published

Managed Server Version 8: Composer 2.5.8

Composer was updated to version 2.5.8.

Reference: #1333