Ops One AG platform status and maintenance announcements. Times are indicated in local Swiss time. https://opsone.ch/.

Wednesday 14th June 2023

Network Partial Network Outage

At 14/06/2023 15:44 we detected increased high latencies in the network during maintenance work. This delayed the internal data replication, which in turn led to further latencies in the system. As a result, certain services were no longer available. The situation slowly returned to normal after a structured restart of the affected services.

Our analysis showed that this was not an attack. We have identified a human error as the cause. The confidentiality and integrity of the data was guaranteed at all times.

As an immediate measure, first organisational measures have already been taken. In addition, further infrastructure optimisation is planned for the next few weeks to prevent similar problems in the future.

We apologise for the inconvenience experienced by some of our customers. Thank you for your patience and understanding.

Incident timeline

  • 17:48 [Update] - Services are back in operation. We will closely monitor the situation over the next few hours and then analyse the root cause.
  • 16:53 [Update] - We are still experiencing limited availability of certain services. The affected systems are gradually becoming available again
  • 16:23 [Update] - Affected services are partially available again. But still suffering from poor performance
  • 15:53 - We are experiencing a partial outage of our network and are currently investigating the cause