NORDUnet status page

Incident feeds: RSS JSON Maintenance RSS

All Systems Operational














Pingdom statistics

Additional statistics for external monitoring of services web reachability and response times can be seen on Pingdom

On its way; Kaltura - KMS go for iOS

Due to a recent update to the KMS Go for iOS, it is not possible to login to the NORDUnet hosted MediaSpaces using the generic Kaltura app's. The problem will be resolved by NORDUnet lauching it's own app's in the relevant app-stores.

Zoom Info

Zoom statistics

Important info regarding: LTI Pro Data Storage and Processing

Zoom has located all information about the latest client versions and Release notes on a single page

NORDUnet Zoom: GDPR and Privacy Facts

A helpful article from zoom about error codes

2021-04-30 09:46 UTC

Panopto - Transcoding problems (Degraded)

Resolved -

All back to normal, delay caused by an organisation doing a bulk upload af a lot of material.

2021-05-01 11:02 UTC
Update -

We are working on resolving the problem.

2021-04-30 09:48 UTC

2021-04-28 06:58 UTC

Kaltura - Problems with analytics (Operational)

Resolved. -

Problem has been resolved.

2021-04-28 10:45 UTC
Update -

Problem has been reported to Kaltura on-prem support.

2021-04-28 07:12 UTC

2021-04-28 10:46 UTC

Kaltura - Problem with transcoding (Operational)

Update -

The transcoding que has been processed and normal operations has been restored.

2021-04-29 06:07 UTC
Update -

Problem has been resolved, and transcoding has resumed. There is a backlog in que, that will be transcoded over the next couple of hours.

2021-04-28 13:13 UTC

2021-04-09 09:04 UTC

Panopto - Problems uploading content with slidedecks (Degraded)

Resolved -

A permanent fix will be implemented late June.

2021-04-28 10:47 UTC
Update -

A work a round has been implemented. We are working with Panopto to establish a fix for the root cause.

2021-04-12 13:35 UTC
Update -

Problem is being investigated with Panopto support.

2021-04-09 11:06 UTC

2021-04-20 07:35 UTC

Zoom - problems with web and admin pages (Operational)

Resolved -
2021-04-20 13:03 UTC
Update -

Functionality is restored. Waiting for further information and updates from Zoom.

2021-04-20 10:04 UTC
Update -

Progress updates available on

2021-04-20 08:35 UTC
Update -

Problem has been reported to Zoom and is being handled.

2021-04-20 08:14 UTC

2021-04-14 18:00 UTC

Panopto - Planned operating system updates and disk expansions (Operational)

2021-04-13 08:40 UTC

Panopto - DNS problem (Operational)

Update -

Problem has been located and resolved.

2021-04-13 09:15 UTC

2021-04-08 07:23 UTC

Network Maintenance - April 8'th 2021, 18:00 - 20:00 UTC (Operational)

2021-04-07 17:00 UTC

Kaltura, MediaSite and Panopto - Emergency update to Firewall (Operational)

2021-02-01 10:50 UTC

CANCELLED Kaltura software version upgrade - March 26'th at 18:00 UTC till March 28'th at 16:00 UTC (Operational)

2021-03-25 16:21 UTC

MediaSite - Planned work (Operational)

2021-03-07 19:11 UTC

Kaltura service issue (Operational)

Resolved -
2021-03-19 08:07 UTC
Monitoring -

The re-sync of the indexing servers has been completed. We continue to montor the situation.

2021-03-14 20:21 UTC
Monitoring -

At 10 UTC an optimisation of the indexing servers were implemented. Since then, the service has been stable and no issues have been seen. We continue to monitor the situation closely.

2021-03-12 21:11 UTC
Update -

The work a round has worked and the service is performing as should be expected. The re-sync is running again and is being monitored closely. While the re-sync process is running there may be sporadic erros encountered regarding activities that causes changes to the service content i.e. uploads, trimming, renaming, publishing and deletion of videos.

2021-03-11 09:43 UTC
Update -

The initail re-sync has failed. There continues to be problems. We will be implementing a work around that should of load the process and improve the performance by tomorrow (Thursday) morning.

2021-03-10 20:13 UTC
Update -

The Kaltura indexing servers will have to be resynced. During the process, the Kaltura service will be degraded regarding all activities that causes changes to the service i.e. uploads, trimming, renaming, publishing and deletion of videos, will all take longer time to process and will occasionally fail, especially during peak hours. The resync process is expected to take at least another 48 hours to complete.

2021-03-09 07:03 UTC
Update -

There are still some issues seen with upload and deletion of videos.

2021-03-08 12:13 UTC
Update -

A work-around has been implemented that appears to have improved and stabilised the situation for now.

2021-03-08 10:15 UTC
Update -

There continues to be instability regarding the API performance affecting the service.

2021-03-08 07:04 UTC
Update -

API servers have been restarted and normal operation has been restored.

2021-03-08 06:09 UTC
Update -

Stopping the scripts triggered a problem with the API's that is currently being investigated.

2021-03-08 05:49 UTC
Update -

Degraded performance is caused by some additional synchronisation scripts. These will be stopped before 06:00 UTC, if not completed by then, to restore performance

2021-03-08 00:19 UTC

2021-03-15 10:28 UTC

Kaltura service issue (Degraded)

Monitoring -

Service has been restored. The cause is being investigated.

2021-03-15 11:17 UTC

2021-03-09 09:16 UTC

Zoom - Live streaming on licensed AWS not working (Degraded)

Resolved -
2021-03-12 16:37 UTC
Update -

Issue has been reported to Zoom and is being investigated.

2021-03-09 12:48 UTC

2021-02-15 09:11 UTC

Storage network upgrade (Operational)

Completed -
2021-03-13 12:53 UTC
Update -

Work on site #1 completed

2021-02-21 13:09 UTC
Update -

Work on site #1 has commenced.

2021-02-20 09:00 UTC

2021-02-22 17:06 UTC

CANCELLED _Major fiber work affecting media services - Friday February 26'th at 23:00 UTC till Saturday February 27'th at 11 UTC (Operational)

2021-02-19 07:35 UTC

Issues for NORDUnet Zoom on-prem customers in Sweden (Degraded)

Resolved -
2021-02-20 09:14 UTC
Resloved -

Issue was caused by routing problems between the Zones.

2021-02-19 08:36 UTC

2021-02-18 21:00 UTC

Firewall issues affecting multple services (Degraded)

Reolved -
2021-02-20 09:11 UTC
Update -

To finally resolve the issue and restore redundancy there will be an EMERGENCY maintenance window tonight Friday 19'th between 20:00 and 21:00 UTC. During the window there will be a short interruption of all services.

2021-02-19 13:01 UTC
Monitoring -

Services restored. Currently without firewall redundancy.

2021-02-18 21:57 UTC
Update -

Services are in the process of fully restoring.

2021-02-18 21:46 UTC
Update -

Manually forcing backup firewall online.

2021-02-18 21:40 UTC

2021-02-13 12:42 UTC

Zoom Network changes - February 18'th from 20:00 - to 21:00 UTC (Operational)

2021-02-13 12:41 UTC

Zoom Network changes - February 16'th from 20:00 - to 21:00 UTC (Operational)

Completed -
2021-02-17 07:59 UTC

2021-02-13 12:40 UTC

Zoom Network changes - February 15'th from 19:00 - to 20:00 UTC (Operational)

Completed -
2021-02-16 07:21 UTC

2021-02-15 06:49 UTC

Panopto on-prem service inaccessible (Critical)

Resolved -

The problem was resolved at 07:44 UTC.

2021-02-15 07:52 UTC

2021-02-10 14:24 UTC

Zoom Network changes - February 11'th from 19:00 - to 20:00 UTC (Operational)

2021-02-10 13:44 UTC

Problems conneting to Zoom via ZIP and H.323 using (Degraded)

Resolved -

Problem was relating to DNS, and has been resolved.

2021-02-10 15:52 UTC
Update -

As a work-a-round you can connect using the ZIP and H.323 gateway load balancer IP's directly; IP's:

2021-02-10 13:47 UTC

2021-01-29 10:43 UTC

Network node upgrades - February 5'th between 18:00 and 23:00 UTC (Operational)

2021-02-01 08:56 UTC

Issues for NORDUnet Zoom on-prem customers in Denmark (Operational)

Update -

The interconnect with TDC has been upgraded to 40Gig.

2021-02-03 10:02 UTC
Update -

As a workaround there have been implemented some traffic engineering solutions that will i.e. route traffic to TDC via our IP Transit provider.

2021-02-02 08:26 UTC
Update -

Issue also experienced between 12:00 and 12:45 UTC.

2021-02-01 13:10 UTC
Update -

We have been working with TDC to establish a 100G interconnect since November 2020, but this is still not in place. Currently we are working with TDC to put some emergency meassures in place, to resolve the issue.

2021-02-01 13:02 UTC

2021-01-25 09:02 UTC

Issues with Zoom direct webclient (Operational)

resolved -
2021-01-28 07:32 UTC
Monitoring -

The problem should be fixed.

2021-01-27 07:57 UTC
Update -

The issue has be escalated.

2021-01-26 10:02 UTC
Update -

The issue has been reported to Zoom.

2021-01-25 11:04 UTC

2021-01-19 09:00 UTC

Zoom capacity and quality issues (Operational)

Montoring -

The changes implemented looks to have resolved the problem.

2021-01-20 10:57 UTC
Update -

Various meassures are being deployed to resolve the problem and improve user experience.

2021-01-19 09:01 UTC

2021-01-19 09:49 UTC

Zoom client login problems (Operational)

Resolved -

Zoom has resolved the problem.

2021-01-20 10:55 UTC
Update -

The issue has been reported to Zoom. The problem is a generic issue not related to on-prem nor to SSO.

2021-01-19 12:51 UTC

2021-01-22 10:53 UTC

Zoom on-prem issue (Degraded)

2020-10-30 16:03 UTC

Kaltura - Live streaming issue (Degraded)

Resolved -
2021-01-07 08:34 UTC
Update -

A work-a-round for the Epiphan Pearl devices is to downgrade the firmware version to 4.11.0h.

2020-11-10 08:55 UTC
Update -

The issue has been escalated with Kaltura support.

2020-11-09 13:16 UTC
Update -

The problem has been reported to Kaltura support and is being investigated.

2020-10-30 16:05 UTC

2020-11-17 09:08 UTC

Kaltura - problems with uploading videos with capture (Degraded)

Closed -

Awaiting upgrade of KMS.

2021-01-04 13:34 UTC
Update -

As a work-a-round, affected users can reinstall the old version 4.2.83.


2020-11-17 09:11 UTC

2020-12-15 07:15 UTC

MediaSite - Planned sofware update (Operational)

Completed -
2020-12-17 07:53 UTC

2020-12-03 14:30 UTC

Zoom service update - December 5, at 23:00 - 23:59 UTC (Operational)

Completed -
2020-12-06 21:10 UTC

2020-12-03 14:26 UTC

MediaSite Emergency database update - December 4, at 17:00 UTC (Operational)

Completed -
2020-12-06 21:09 UTC

2020-11-23 07:55 UTC

Panopto - Planned software update (Operational)

Completed -

Maintenance sucessfully completed. Total down time 20 minutes.

2020-11-28 15:43 UTC

2020-11-23 11:22 UTC

Zoom - LTI-PRO errors (Operational)

Monitoring -
2020-11-23 12:18 UTC
Update -

Problem should be fixed.

2020-11-23 12:17 UTC
Update -

This is a global Zoom problem

2020-11-23 12:10 UTC
Investigating -

We are investigating and have raised the issue with Zoom.

2020-11-23 11:28 UTC

2020-11-18 07:51 UTC

Changes to Datacenter infrastuture - November 19'th from 19-21 UTC (Operational)

2020-11-20 07:53 UTC

Panopto - transcoding delays (Degraded)

Resolved -

Que is back to normal.

2020-11-22 11:36 UTC
Update -

The +700 files are now processed and the regular backlog from this morning is being processed.

2020-11-20 12:27 UTC

2020-11-18 08:31 UTC

Panopto issue with uploads (Operational)

Resolved -

The problem has been resolved we are investigating the cause of the problem.

2020-11-18 08:52 UTC

2020-11-09 13:44 UTC

MediaSite, Kaltura and Panopto - Maintenance on backend storage system (Operational)

completed -
2020-11-18 07:01 UTC

2020-11-11 07:20 UTC

Changes to Datacenter infrastuture (Operational)

Completed -
2020-11-17 10:31 UTC

2020-11-09 13:10 UTC

Kaltura - Database extraction (Operational)

completed -
2020-11-13 05:57 UTC

2020-11-02 09:00 UTC

Panopto issue (Operational)

Monitoring -

Following some changes to the setup, things have become stable.

2020-11-09 13:04 UTC
Investigating -

We are investigating with Panopto support.

2020-11-02 12:47 UTC

2020-11-03 07:49 UTC

Zoom on-prem service infastructure updates (Operational)

completed -
2020-11-09 13:07 UTC

2020-11-06 00:09 UTC

Kaltura service issue (Operational)

Resolved -

full backend capacity restored.

2020-11-06 07:28 UTC
Update -

No user impact as system load is low,

2020-11-06 02:10 UTC

2020-11-04 11:51 UTC

MediaSite and Panopto increased logging (Degraded)

2020-11-02 14:48 UTC

Panopto - Planned operating system updates (Operational)

Resloved -

Updates completed.

2020-11-05 08:53 UTC

2020-10-29 13:46 UTC

Data Center Infrastructure maintenance (Operational)

2020-10-29 08:13 UTC

Media service issue (Operational)

Resolved -

Power restored and traffic back on normal route.

2020-10-29 11:45 UTC
Update -

Issue was caused by a power outage at a main communication site. Service was affected as trafic between our two datacenters were rerouted. The site is still without power.

2020-10-29 08:51 UTC
Update -

Service is restored.

2020-10-29 08:25 UTC

2020-10-21 11:07 UTC

MediaSite version upgrade (Operational)

Update -

Service window to be extended, till 02:00 UTC.

2020-10-28 21:15 UTC

2020-10-28 10:43 UTC

Kaltura service issue (Operational)

Closed -
2020-10-28 11:29 UTC
Resolved -

Fix applied.

2020-10-28 10:52 UTC

2020-10-26 11:17 UTC

Panopto issue (Operational)

Resolved -
2020-10-27 11:12 UTC
Monitoring -

No cause was indentified.

2020-10-26 13:34 UTC

2020-10-25 13:06 UTC

Kaltura service issue (Outage)

Closing -
2020-10-26 09:02 UTC
Resolved -

Problem has been resolved.

2020-10-25 16:22 UTC
Monitoring -
2020-10-25 16:22 UTC
Update -

The issue is caused by a problem with the backend storage.

2020-10-25 14:51 UTC

2020-10-23 07:56 UTC

Kaltura play list problems (Operational)

Resolved -

Problem has been resolved.

2020-10-23 09:47 UTC

2020-10-23 05:29 UTC

Kaltura service issue (Degraded)

Reolved -
2020-10-23 07:58 UTC
Update -

Service is functioning again. We are monitoring the situation.

2020-10-23 07:24 UTC

2020-10-22 14:50 UTC

Problem with Kaltura live streams (Degraded)

Resolved -

The problem has been resolved.

2020-10-22 19:24 UTC

2020-10-21 18:40 UTC

Kaltura service issue (Outage)

Update -

The database completed restoring at 12:55 UTC. Service is operational and functioning again.

2020-10-22 12:57 UTC
Update -

The job is now 92% completed, but has slowed down, new extimate for completion is 13 UTC.

2020-10-22 11:00 UTC
Update -

New expected restore time is aproximately 11 UTC.

2020-10-22 10:20 UTC
Update -

Service should be back up approximately 10:30 UTC.

2020-10-22 08:47 UTC
Update -

The Database is still recovering, it is expected to take a few more hours for the recovery to complete.

2020-10-22 06:10 UTC
Update -

Database recovery is still ongoing....

2020-10-22 04:40 UTC
Update -

Kaltura is down again. Database issue.

2020-10-22 00:35 UTC
Resolved -

All services fully restored.

2020-10-21 20:09 UTC
Update -

Problem has been resolved and services are restoring.

2020-10-21 19:57 UTC
Update -

Issue is caused by database problems.

2020-10-21 18:58 UTC

2020-10-19 08:25 UTC

Panopto issue (Outage)

Resolved -
2020-10-20 13:39 UTC
Monitoring -

As load is building up the system and response times remain low and stable.

2020-10-20 07:10 UTC
Update -

Corrections have been implemented, but will have to be verified during real load.

2020-10-19 20:04 UTC
Update -

Issue is stille being investigated.

2020-10-19 13:27 UTC
update -

The Moodle integration issue has caused the platform to be unresponsive, and considered to be down. We are working on the isuues.

2020-10-19 11:11 UTC
Update -

We are working with Panopto to resolve the problem.

2020-10-19 09:13 UTC

2020-10-20 06:43 UTC

Kaltura ingest problem (Operational)

Resloved -

Problem has bee resolved

2020-10-20 10:38 UTC

2020-09-14 11:12 UTC

Slow response on the Panopto and MediaSite services (Operational)

Monitoring -

The migration of the Panopto front-end serveres has been sucessfully completed.

2020-10-14 11:18 UTC
Update -

The changes in the Datacenter has improved the the situation. We will continue our action plan and will as a next step migrate all Panopto Front-End servers from the virtualised environment to physical servers.

2020-10-07 13:09 UTC
update -

There continues to be load related issues affecting Panopto. Over this week we will be optimising things in the datacentre infrastructure. In addition later in the week we will deploy new load balancers and add cashing capacity.

2020-10-06 06:26 UTC
Monitoring -

After the upgrades done last night, we will monitor the situation.

2020-10-01 10:06 UTC
resolved -

the load related issues have been resolved.

2020-09-30 10:19 UTC
update -

To night September 30, there will be additional work done that should further improve the situation for both MediaSite and Panopto.

2020-09-30 10:19 UTC
Update -

There have been several updates and performance optimisations during the weekend. MediaSite on-demand playback should be much improved even though loading the player itself, could take some seconds during peak hours.

2020-09-28 06:43 UTC
Update -

There are still load related issues during peak hours. We are working to resolve the issue.

2020-09-21 09:19 UTC
Monitoring -

We are still monitoring the situation but the capacity upgrade and rebalance of all systems seems to have resolved the issue.

2020-09-18 15:00 UTC
Monitoring -

Resources allocated to Mediasite and Panopto has been rebalanced. We continue monitoring and will decided on further actions if the issues persist.

2020-09-17 05:14 UTC
Monitoring -

More system capacity was added again last night (15/9-2020). We continue to monitor the situation.

2020-09-16 06:16 UTC
Monitoring -

More capacity has been added. We continue to monitor the situation, and will be adding more system capacity if the problem persist.

2020-09-15 07:04 UTC

2020-10-13 08:20 UTC

MeiaSite problem (Operational)

Monitoring -

A fix has been implemented and so far the metrics are looking good.

2020-10-14 18:58 UTC

2020-10-18 18:00 UTC

The NORDUnet Kaltura service is unavailable (Operational)

Update -

service is back up. We are investigating the cause.

2020-10-18 18:38 UTC

2020-10-15 06:39 UTC

Zoom on-prem issue (Operational)

Closed -
2020-10-18 18:25 UTC
Resolved -

We are working with Zoom to find the root cause.

2020-10-15 07:28 UTC

2020-10-14 13:34 UTC

Maintenance on backend storage system (Degraded)

Closed -

Maintenance sucessfully completed

2020-10-14 18:57 UTC

2020-10-12 20:46 UTC

Kaltura service issues (Outage)

Closing -

Things remain stable.

2020-10-13 07:43 UTC
Monitoring -
2020-10-12 22:07 UTC
Resloved -

The issue was caused by a problem in the Datacenter and the service has been restored.

2020-10-12 22:05 UTC

2020-10-05 11:25 UTC

Changes to Datacenter infrastuture (Operational)

Closed -

Maintenance completed.

2020-10-07 06:26 UTC

2020-10-05 06:45 UTC

Kaltura - Live streaming capacity upgrade (Operational)

Closed -

Capacity was added as planed.

2020-10-06 10:39 UTC

2020-09-29 17:03 UTC

MedisSite planned work (Operational)

2020-09-28 06:49 UTC

Zoom AWS Stockholm down (Outage)

Resloved -

Zoom AWS is back to normal operations.

2020-09-28 17:15 UTC
Monitoring -

We are monitoring, and waiting for feedback from Zoom on the cause of the outage.

2020-09-28 08:18 UTC
update -

Zoom AWS is back up.

2020-09-28 08:17 UTC

2020-09-26 09:08 UTC

Problem with Zoom AWS (Operational)

Monitoring -

Organisations that migrated to on-prem, can migrate back to licensed.

2020-09-28 06:41 UTC
Monitoring -

The service is fully restored.

2020-09-27 18:50 UTC
Update -

The Zoom AWS cluster is back up, and working, but some users will still get various error messages.

2020-09-27 06:14 UTC
Update -

Users with an urgent need to host meetings can migrate to on-prem by having an administrator change setting from from licensed to on-prem.

2020-09-26 09:11 UTC

2020-09-24 10:00 UTC

Site and Datacenter maintenance (Degraded)

completed -

Maintenance activities sucessfullt completed

2020-09-25 05:03 UTC

2020-09-16 21:00 UTC

Downtime of Mediasite and Panopto (Operational)

Maintenance completed -

The resource rebalancing was completed and the systems are back in operation.

2020-09-16 21:46 UTC

2020-08-11 09:28 UTC

Kaltura KMC Analytics (Degraded)

Investigating -

Analytics has stopped working again.

2020-09-14 12:21 UTC
Monitoring -

We will continue to monitor the situation over the coming week.

2020-09-11 12:06 UTC
Resolved -

A fix has been found and implemented.

2020-09-11 12:05 UTC

2020-09-08 10:04 UTC

Slow response in the Panoto service (Degraded)

Closed -
2020-09-09 10:01 UTC
Monitoring -

We will closely monitor the situation during the day.

2020-09-09 06:17 UTC
Update -

No cause for the slow response experinced has been identifyed.

2020-09-09 06:16 UTC

2020-08-28 05:15 UTC

Zoom on-prem dashboard issues (Operational)

Closed -
2020-09-09 10:02 UTC
Resolved -

Problem has been resolved for all accounts.

2020-09-04 07:33 UTC
Update -

The data is starting to become available again.

2020-09-01 13:39 UTC

2020-08-28 10:44 UTC

Performance problems with the Kaltura and Panopto service (Degraded)

Resolved -

The problem relating to the storage infrastructure, has been identified and fixed. Normal operations will be fully restored within 1 hour.

2020-08-28 11:25 UTC
Investigating -

We are currently investigating.

2020-08-28 10:46 UTC

2020-08-17 08:02 UTC

Problem with Kaltura live streams (Degraded)

Resolved -

Problem has been resolved. The issue was caused by a packaged deployed with inconsistant module versions.

2020-08-27 10:45 UTC
Investigating -

Problem is being investigated.

2020-08-27 08:04 UTC

2020-08-26 07:30 UTC

Kaltura - Issues with transcoding (Degraded)

Update -

Backlog is still being processed. Normal operations are expected at 14:30 UTC.

2020-09-26 13:40 UTC
Resolved -

Back to normal opertions.

2020-08-26 14:34 UTC
Update -

Problem has been identified and rectified. Backlog is still being processed.

2020-08-26 09:38 UTC
Investigating -

Issue is being investigated.

2020-08-26 07:31 UTC

2020-08-24 12:55 UTC

Problems logging in to the website and joing meetings from the web browser (Operational)

Monitoring -

We will keep monitoring the situation.

2020-08-24 18:42 UTC
Resolved -

The issue has been resolved. Waiting for more information regarding the cause of the outage.

2020-08-24 18:20 UTC
Update -

Zoom have identified the issue causing users to be unable to authenticate to the Zoom website ( and unable to start and join Zoom Meetings and Webinars, and are working on a fix for this issue.

2020-08-24 14:13 UTC
Investigating -

The issues is being investigated with the highest priority.

2020-08-24 13:07 UTC

2020-08-20 21:00 UTC

Planned maintenance affecting Zoom on-prem users (Outage)

Closed -

Maintenance sucessfully completed.

2020-08-21 06:44 UTC

2020-08-11 10:15 UTC

Embed codes in Kaltura (Operational)

Resolved -

This issue has been resolved

2020-08-12 09:01 UTC
Investigating -

The problem is currently being investigated by Kaltura.

2020-08-11 10:15 UTC

2020-08-11 09:27 UTC

Thumbnails in Kaltura (Operational)

Resolved -

The issus have been resolved

2020-08-11 15:59 UTC
Investigating -

The problem with thumbnails affects all Kaltura deployments, not only the NORDUnet instance.

2020-08-11 11:01 UTC

2020-07-22 07:04 UTC

Zoom users running licensed are experiencing problems (Degraded)

Resolved -

The Zoom AWS cluster has been stable for 24 hours, and users can migrate back to the AWS cluster by changing setting to licensed.

2020-07-23 13:12 UTC
Monitoring -

During the incident more than 50% of accounts have been moved to run on-prem. We will be monitoring the situation for the next 24 hours before taking any further action. We are waiting for a more detailed RFO from Zoom.

2020-07-22 14:12 UTC
Resolved -

The issue has been resolved and service for the customers running licensed has been restored.

2020-07-22 14:09 UTC
Investigating -

We are working with Zoom to find the cause of the problem with the AWS cluster.

2020-07-22 08:24 UTC
Work-a-round -

As a temporare meassure, to get things to work, it is required to change the setting from cloud to on-prem.

2020-07-22 07:13 UTC
Investigating -

We are investigating and looking at mitigation options.

2020-07-22 07:06 UTC

2020-06-12 11:19 UTC

Kaltura - KMS Go for iOS is not able to login to the NORDUnet hosted MediaSpaces (Degraded)

Investigating -

Kaltura is currently investigating possible solutions to this problem.

2020-06-12 11:21 UTC

2020-07-23 20:00 UTC

Planned maintenance affecting Zoom on-prem users (Degraded)

2020-07-16 19:00 UTC

The Zoom service is affected by maintenance on a core router (Degraded)

Monitoring -

Service restored investigating, failover issue.

2020-07-16 19:24 UTC

2020-06-24 15:40 UTC

Panopto backend storage migration (Maintenance)

Closed -

Migration was completed as planned without any issues.

2020-06-30 06:55 UTC

2020-06-16 09:59 UTC

Kaltura - there is currently a network problem cauing longer video load and upload times (Degraded)

Resloved -

The Netwok problem has been resolved.

2020-06-17 10:12 UTC
Investigating -

The cause of the problem is being invetigated.

2020-06-16 12:15 UTC

2020-06-09 06:35 UTC

Panopto index search not working (Degraded)

Update -

We are working with Panopto to find a solution.

2020-09-09 13:37 UTC
Resolved -

The problem has been resolved.

2020-06-11 13:27 UTC
Update -

Work is still ongoing to establish a solution.

2020-06-10 13:38 UTC

2020-06-08 15:00 UTC

Planned maintenance - Panopto service upgrade - June 8, 15-22 UTC (Maintenance)

2020-05-27 10:58 UTC

Zoom Datacentre labeling (Operational)

2020-06-04 14:15 UTC

Issues with infrastructure affecting multiple services (Degraded)

Resolved -

The Kaltura database re-sync has completed and everything is back to normal operation.

2020-06-05 14:29 UTC
Update -

The Kaltura database re-sync will not be completed until aproximately 14 UTC.

2020-06-05 09:56 UTC
Monitoring -

The problem has been resolved. The Kaltura database is currently re-syncing and Kaltura can therefore not accept new video material. When the re-sync is completed transcoding will resume. The Kaltura service is expected to be fully operational at 11 UTC.

2020-06-05 09:02 UTC
Investigating -

The problem is Intermediate packet loss. This is specially affecting Kaltura due to recovery times.

2020-06-05 07:10 UTC

2020-05-17 07:24 UTC

NORDUnet Zoom platform - Licensed meetings hosted in AM datacentre (Operational)

Update -

We have received information from Zoom that even if the Datacentre is labled as AM the meetings are actually hosted in the AWS cluster in Stockholm. The Datacentre will be relabled to show SK.

2020-05-19 08:38 UTC
Update -

Still being worked on

2020-05-18 07:28 UTC
Uodate -

We are working with Zoom to migrate licensed accounts back to the Stockholm datacentre.

2020-05-17 18:00 UTC

2020-05-20 16:00 UTC

Planned maintenance - Kaltura service - Wednesday 20/5, 16-19 UTC. (Maintenance)

2020-05-18 06:11 UTC

Issues with Kaltura (Degraded)

Resolved -

No further issues have been seen or reported.

2020-05-20 09:04 UTC
Monitor -

The problem was caused by a backend storage job moving large amount of data and causing poor performance for the Kaltura API servers communicating with the storage backend. The backend job has completed, but we continue to monitor the situation.

2020-05-19 21:44 UTC
Update -

The issue is caused by communication problems between the Kaltura servers and the storage system.

2020-05-18 10:39 UTC

2020-04-22 14:46 UTC

Zoom skype for business connector traffic routed via US (Operational)

Update -

We have received a confirmation that Zoom will not be deploying a EU based connector and not be providing the local virtual connector for Skype for business. The reasoning is that SFB is coming to the end of life for the product and Zoom will not be providing any more updates to the virtual connector if something were to break and Zoom want to prevent new customers from jumping on the software that is not going to be supported. Zoom is working with the documentation team to remove the option from the admin profile.

2020-05-18 16:41 UTC
Recommended workaround -

The recommended workaround is to install the Zoom client and use the plugin

If it is not possible to use the Zoom client, please refer to the other workaround options.

2020-05-12 15:10 UTC
Workaround -

As a temporare meassure it is possible to use either SIP when connecting from Skype for business to a Zoom meeting, or use the clientless direct browser access.

2020-04-22 14:52 UTC

2020-05-15 09:03 UTC

Issues with Kaltura transcoding (Degraded)

Root cause -

The root case was that due to the network problems causing firwall flaps, the Kaltura master and slave datatbase got out of sync. The system was unable to accept new data until the databases were re-sync'd and database consistancy was restored.

2020-05-16 09:24 UTC
Resolved -

The issue has been resolved, but there is a large backlog of videos that need to be transcoded.

2020-05-15 14:46 UTC
Update -

The issue is still being worked on and is also affecting the performance of other parts of the Kaltura service.

2020-05-15 11:25 UTC

2020-05-15 01:03 UTC

Problems on one NORDUnet core router in Denmark is causing packet loss affecting all services (Degraded)

Resolved -

The issue was resolved by reloading the software on a core router component.

2020-05-15 10:08 UTC

2020-05-14 05:42 UTC

Zoom instability (Degraded)

2020-05-13 19:39 UTC

There is currently an issue with NORDUnet Zoom on-prem (Degraded)

2020-05-02 05:00 UTC

Zoom on-premise in Copenhagen down due to failed software update (Outage)

Fixed -

The problem has been resolved and all users previously running with Copenhagen as the primary location have been migrated back.

2020-05-02 18:46 UTC
Workaround in place -
2020-05-02 16:25 UTC
Workaround -

As a workaround users are being migrated to Stockholm.

2020-05-02 16:00 UTC

2020-04-21 12:01 UTC

Firewall software version upgrade (Maintenance)

completed -

upgrade completed.

2020-04-22 04:54 UTC

2020-04-18 10:00 UTC

Firewall capacity upgrade (Maintenance)

2020-04-20 05:00 UTC

Kaltura Outage (Outage)

Resolved -

The problem has been resolved. A Reason For Outage (RF) report will be made available.

2020-04-20 06:32 UTC
Update -

The problem has been located to a routing problem for the Storage network.

2020-04-20 05:58 UTC

2020-04-14 07:46 UTC

Kaltura upload (Degraded)

Resolved -

The backlog has been processed and upload times are back to normal.

2020-04-14 17:05 UTC

2020-03-31 09:37 UTC

Kaltura Storage system (Maintenance)

Maintenance successfully completed. -
2020-04-11 21:44 UTC

2020-04-03 08:00 UTC


Resolved -

Problem was fixed on April 5, at 04:00.

2020-04-06 06:55 UTC
Latest Update -

Waiting for information from Zoom. Next update Monday April 6, at 08:00 UTC

2020-04-03 10:00 UTC
Investigating -

We are investigating the issue to identify the cause. Next update at 10:00 UTC

2020-04-03 08:14 UTC

2020-04-05 11:00 UTC

Datacentre re-cabling (Maintenance)

2020-03-31 14:16 UTC

Datacenter cabling (Maintenance)

2020-03-26 06:10 UTC

NORDUnet Kaltura Service (Degraded)

Closed -
2020-03-30 08:41 UTC
Resolved -

The issue has been fixed.

2020-03-30 05:49 UTC
Update -

Waiting for Kaltura support. Next update, March 30, at 09 UTC.

2020-03-27 14:58 UTC
Update -

There is currently a problem with livestreaming on the Kaltura platform. The issue is causing some live streams not being able to start. The issue has been escalated to Kaltura support.

2020-03-26 07:55 UTC
Investigating -

There is currently a problem with livestreaming on the Kaltura platform. The issue is being investigated. Next update at 08:00 UTC.

2020-03-26 07:12 UTC

2020-03-23 09:15 UTC

NORDUnet Kaltura Service. (Degraded)

Resolved -

The problem has been resolved.

2020-03-20 11:45 UTC
Investigating -

The issue is being investigated with Kaltura on-premise engineering. Next update at 12:00 CET.

2020-03-20 09:25 UTC

2020-03-20 09:25 UTC

NORDUnet Kaltura Service. (Degraded)

Resolved -

Additional API server added.

2020-03-20 15:00 UTC
Investigating -

The issue is being investigated with Kaltura on-premise engineering.

2020-03-20 09:25 UTC

2020-03-19 08:25 UTC

NORDUnet Kaltura Service. (Degraded)

Update -

Additional API servers were added at 03:00 UTC

2020-03-20 03:25 UTC
Investigating -

The issue is being investigated with Kaltura on-premise engineering. Work is ongoing to add more system resources

2020-03-19 08:25 UTC

2020-03-18 14:00 UTC

NORDUnet Kaltura Service. (Degraded)

Resolved -

The issue has been resolved.

2020-03-18 16:00 UTC
Investigating -

The issue is being investigated with Kaltura on-premise engineering, to establish what can be done to improve system performance.

2020-03-18 14:00 UTC

2020-03-17 20:20 UTC

The NORDUnet Panopto on-premise service is currently experiencing problems that prevent videos from being uploaded. (Critical)

Resolved -

The issue has been resolved but users will experience a degraded performance as the upload backlog is being processed.

2020-03-18 09:02 UTC

2020-03-16 19:20 UTC

Most services on our virtualization platform (Outage)

Resolved -

All systems are comming back online, and returning to normal.

2020-03-16 20:08 UTC
Investigating -

During maintenance we experienced a large outage on ndn services running on our virtualisation setup. The cause was human error, but we are looking into how/why it ended up affecting most of our systems.

2020-03-16 19:20 UTC

2020-03-16 08:30 UTC

Creating zoom meetings (Degraded)

Update -

We have found the cause and are working with Zoom to see if we can find a work around.
For now please change users to Licensed instead of On-Premise as the system hits low logical limits
and then block new meeting creation.

2020-03-16 09:50 UTC

2020-03-13 09:30 UTC

Creating zoom meetings (Degraded)

Monitoring -

The issue has now been mitigated, we continue to monitor the situation. Official zoom status

2020-03-11 15:10 UTC
Creating zoom meetings -

We are currently having errors creating new meetings on the Zoom service for both Cloud and On-Premise. We are working with Zoom to resolve this issue.

2020-03-11 9:30 UTC

2020-03-12 10:00 UTC

Creating zoom meetings (Degraded)

Monitoring -

The issue has now been mitigated, we continue to monitor the situation.

2020-03-12 14:30 UTC
Investigating -

We are currently having errors creating new meetings on the Zoom service. We are working with Zoom to resolve this issue.

2020-03-12 10:00 UTC