NORDUnet status page

Incident feeds: RSS JSON Maintenance RSS

All Systems Operational

Network

Operational

Zoom

Operational

Kaltura

Operational

MediaSite

Operational

Panopto

Operational

Metadata

Operational

Info

Kaltura transcoding flavours

It is now again possible to choose which transcoding flavours to enable. But before you go ahead and enable all flavours please read this post about transcoding settings , as the number of flavours will impact your storage consumption, and thereby the cost for the service.

KMS go for iOS

Due to a recent update to the KMS Go for iOS, it is not possible to login to the NORDUnet hosted MediaSpaces. Kaltura is currently investigating possible solutions to this problem.


2020-07-24 06:41 UTC

Zoom Info

Zoom concurrent meeting participants
More Zoom statistics

Zoom has located all information about the latest client versions and Release notes on a single page

NORDUnet Zoom: GDPR and Privacy Facts

A helpful article from zoom about error codes

Zoom for education FAQ

2020-07-22 07:04 UTC


Zoom users running licensed are experiencing problems (Degraded)

Resolved -

The Zoom AWS cluster has been stable for 24 hours, and users can migrate back to the AWS cluster by changing setting to licensed.


2020-07-23 13:12 UTC
Monitoring -

During the incident more than 50% of accounts have been moved to run on-prem. We will be monitoring the situation for the next 24 hours before taking any further action. We are waiting for a more detailed RFO from Zoom.


2020-07-22 14:12 UTC
Resolved -

The issue has been resolved and service for the customers running licensed has been restored.


2020-07-22 14:09 UTC
Investigating -

We are working with Zoom to find the cause of the problem with the AWS cluster.


2020-07-22 08:24 UTC
Work-a-round -

As a temporare meassure, to get things to work, it is required to change the setting from cloud to on-prem.


2020-07-22 07:13 UTC
Investigating -

We are investigating and looking at mitigation options.


2020-07-22 07:06 UTC

2020-06-12 11:19 UTC


Kaltura - KMS Go for iOS is not able to login to the NORDUnet hosted MediaSpaces (Degraded)

Investigating -

Kaltura is currently investigating possible solutions to this problem.


2020-06-12 11:21 UTC

2020-07-23 20:00 UTC


Planned maintenance affecting Zoom on-prem users (Degraded)

2020-07-16 19:00 UTC


The Zoom service is affected by maintenance on a core router (Degraded)

Monitoring -

Service restored investigating, failover issue.


2020-07-16 19:24 UTC

2020-06-24 15:40 UTC


Panopto backend storage migration (Maintenance)

Closed -

Migration was completed as planned without any issues.


2020-06-30 06:55 UTC

2020-06-16 09:59 UTC


Kaltura - there is currently a network problem cauing longer video load and upload times (Degraded)

Resloved -

The Netwok problem has been resolved.


2020-06-17 10:12 UTC
Investigating -

The cause of the problem is being invetigated.


2020-06-16 12:15 UTC

2020-06-09 06:35 UTC


Panopto index search not working (Degraded)

Update -

We are working with Panopto to find a solution.


2020-09-09 13:37 UTC
Resolved -

The problem has been resolved.


2020-06-11 13:27 UTC
Update -

Work is still ongoing to establish a solution.


2020-06-10 13:38 UTC

2020-06-08 15:00 UTC


Planned maintenance - Panopto service upgrade - June 8, 15-22 UTC (Maintenance)

2020-05-27 10:58 UTC


Zoom Datacentre labeling (Operational)

2020-06-04 14:15 UTC


Issues with infrastructure affecting multiple services (Degraded)

Resolved -

The Kaltura database re-sync has completed and everything is back to normal operation.


2020-06-05 14:29 UTC
Update -

The Kaltura database re-sync will not be completed until aproximately 14 UTC.


2020-06-05 09:56 UTC
Monitoring -

The problem has been resolved. The Kaltura database is currently re-syncing and Kaltura can therefore not accept new video material. When the re-sync is completed transcoding will resume. The Kaltura service is expected to be fully operational at 11 UTC.


2020-06-05 09:02 UTC
Investigating -

The problem is Intermediate packet loss. This is specially affecting Kaltura due to recovery times.


2020-06-05 07:10 UTC

2020-05-17 07:24 UTC


NORDUnet Zoom platform - Licensed meetings hosted in AM datacentre (Operational)

Update -

We have received information from Zoom that even if the Datacentre is labled as AM the meetings are actually hosted in the AWS cluster in Stockholm. The Datacentre will be relabled to show SK.


2020-05-19 08:38 UTC
Update -

Still being worked on


2020-05-18 07:28 UTC
Uodate -

We are working with Zoom to migrate licensed accounts back to the Stockholm datacentre.


2020-05-17 18:00 UTC

2020-05-20 16:00 UTC


Planned maintenance - Kaltura service - Wednesday 20/5, 16-19 UTC. (Maintenance)

2020-05-18 06:11 UTC


Issues with Kaltura (Degraded)

Resolved -

No further issues have been seen or reported.


2020-05-20 09:04 UTC
Monitor -

The problem was caused by a backend storage job moving large amount of data and causing poor performance for the Kaltura API servers communicating with the storage backend. The backend job has completed, but we continue to monitor the situation.


2020-05-19 21:44 UTC
Update -

The issue is caused by communication problems between the Kaltura servers and the storage system.


2020-05-18 10:39 UTC

2020-04-22 14:46 UTC


Zoom skype for business connector traffic routed via US (Operational)

Update -

We have received a confirmation that Zoom will not be deploying a EU based connector and not be providing the local virtual connector for Skype for business. The reasoning is that SFB is coming to the end of life for the product and Zoom will not be providing any more updates to the virtual connector if something were to break and Zoom want to prevent new customers from jumping on the software that is not going to be supported. Zoom is working with the documentation team to remove the option from the admin profile.


2020-05-18 16:41 UTC
Recommended workaround -

The recommended workaround is to install the Zoom client and use the plugin https://support.zoom.us/hc/en-us/articles/204481169-Zoom-Skype-for-Business-Lync-Plugin.

If it is not possible to use the Zoom client, please refer to the other workaround options.


2020-05-12 15:10 UTC
Workaround -

As a temporare meassure it is possible to use either SIP when connecting from Skype for business to a Zoom meeting, or use the clientless direct browser access.


2020-04-22 14:52 UTC

2020-05-15 09:03 UTC


Issues with Kaltura transcoding (Degraded)

Root cause -

The root case was that due to the network problems causing firwall flaps, the Kaltura master and slave datatbase got out of sync. The system was unable to accept new data until the databases were re-sync'd and database consistancy was restored.


2020-05-16 09:24 UTC
Resolved -

The issue has been resolved, but there is a large backlog of videos that need to be transcoded.


2020-05-15 14:46 UTC
Update -

The issue is still being worked on and is also affecting the performance of other parts of the Kaltura service.


2020-05-15 11:25 UTC

2020-05-15 01:03 UTC


Problems on one NORDUnet core router in Denmark is causing packet loss affecting all services (Degraded)

Resolved -

The issue was resolved by reloading the software on a core router component.


2020-05-15 10:08 UTC

2020-05-14 05:42 UTC


Zoom instability (Degraded)

2020-05-13 19:39 UTC


There is currently an issue with NORDUnet Zoom on-prem (Degraded)

2020-05-02 05:00 UTC


Zoom on-premise in Copenhagen down due to failed software update (Outage)

Fixed -

The problem has been resolved and all users previously running with Copenhagen as the primary location have been migrated back.


2020-05-02 18:46 UTC
Workaround in place -
2020-05-02 16:25 UTC
Workaround -

As a workaround users are being migrated to Stockholm.


2020-05-02 16:00 UTC

2020-04-21 12:01 UTC


Firewall software version upgrade (Maintenance)

completed -

upgrade completed.


2020-04-22 04:54 UTC

2020-04-18 10:00 UTC


Firewall capacity upgrade (Maintenance)

2020-04-20 05:00 UTC


Kaltura Outage (Outage)

Resolved -

The problem has been resolved. A Reason For Outage (RF) report will be made available.


2020-04-20 06:32 UTC
Update -

The problem has been located to a routing problem for the Storage network.


2020-04-20 05:58 UTC

2020-04-14 07:46 UTC


Kaltura upload (Degraded)

Resolved -

The backlog has been processed and upload times are back to normal.


2020-04-14 17:05 UTC

2020-03-31 09:37 UTC


Kaltura Storage system (Maintenance)

Maintenance successfully completed. -
2020-04-11 21:44 UTC

2020-04-03 08:00 UTC


(Degraded)

Resolved -

Problem was fixed on April 5, at 04:00.


2020-04-06 06:55 UTC
Latest Update -

Waiting for information from Zoom. Next update Monday April 6, at 08:00 UTC


2020-04-03 10:00 UTC
Investigating -

We are investigating the issue to identify the cause. Next update at 10:00 UTC


2020-04-03 08:14 UTC

2020-04-05 11:00 UTC


Datacentre re-cabling (Maintenance)

2020-03-31 14:16 UTC


Datacenter cabling (Maintenance)

2020-03-26 06:10 UTC


NORDUnet Kaltura Service (Degraded)

Closed -
2020-03-30 08:41 UTC
Resolved -

The issue has been fixed.


2020-03-30 05:49 UTC
Update -

Waiting for Kaltura support. Next update, March 30, at 09 UTC.


2020-03-27 14:58 UTC
Update -

There is currently a problem with livestreaming on the Kaltura platform. The issue is causing some live streams not being able to start. The issue has been escalated to Kaltura support.


2020-03-26 07:55 UTC
Investigating -

There is currently a problem with livestreaming on the Kaltura platform. The issue is being investigated. Next update at 08:00 UTC.


2020-03-26 07:12 UTC

2020-03-23 09:15 UTC


NORDUnet Kaltura Service. (Degraded)

Resolved -

The problem has been resolved.


2020-03-20 11:45 UTC
Investigating -

The issue is being investigated with Kaltura on-premise engineering. Next update at 12:00 CET.


2020-03-20 09:25 UTC

2020-03-20 09:25 UTC


NORDUnet Kaltura Service. (Degraded)

Resolved -

Additional API server added.


2020-03-20 15:00 UTC
Investigating -

The issue is being investigated with Kaltura on-premise engineering.


2020-03-20 09:25 UTC

2020-03-19 08:25 UTC


NORDUnet Kaltura Service. (Degraded)

Update -

Additional API servers were added at 03:00 UTC


2020-03-20 03:25 UTC
Investigating -

The issue is being investigated with Kaltura on-premise engineering. Work is ongoing to add more system resources


2020-03-19 08:25 UTC

2020-03-18 14:00 UTC


NORDUnet Kaltura Service. (Degraded)

Resolved -

The issue has been resolved.


2020-03-18 16:00 UTC
Investigating -

The issue is being investigated with Kaltura on-premise engineering, to establish what can be done to improve system performance.


2020-03-18 14:00 UTC

2020-03-17 20:20 UTC


The NORDUnet Panopto on-premise service is currently experiencing problems that prevent videos from being uploaded. (Critical)

Resolved -

The issue has been resolved but users will experience a degraded performance as the upload backlog is being processed.


2020-03-18 09:02 UTC

2020-03-16 19:20 UTC


Most services on our virtualization platform (Outage)

Resolved -

All systems are comming back online, and returning to normal.


2020-03-16 20:08 UTC
Investigating -

During maintenance we experienced a large outage on ndn services running on our virtualisation setup. The cause was human error, but we are looking into how/why it ended up affecting most of our systems.


2020-03-16 19:20 UTC

2020-03-16 08:30 UTC


Creating zoom meetings (Degraded)

Update -

We have found the cause and are working with Zoom to see if we can find a work around.
For now please change users to Licensed instead of On-Premise as the system hits low logical limits
and then block new meeting creation.


2020-03-16 09:50 UTC

2020-03-13 09:30 UTC


Creating zoom meetings (Degraded)

Monitoring -

The issue has now been mitigated, we continue to monitor the situation. Official zoom status


2020-03-11 15:10 UTC
Creating zoom meetings -

We are currently having errors creating new meetings on the Zoom service for both Cloud and On-Premise. We are working with Zoom to resolve this issue.


2020-03-11 9:30 UTC

2020-03-12 10:00 UTC


Creating zoom meetings (Degraded)

Monitoring -

The issue has now been mitigated, we continue to monitor the situation.


2020-03-12 14:30 UTC
Investigating -

We are currently having errors creating new meetings on the Zoom service. We are working with Zoom to resolve this issue.


2020-03-12 10:00 UTC