NORDUnet status page

Incident feeds: RSS JSON

NORDUnet Kaltura Service

Incident start: 2020-03-26 06:10 UTC

System status: Operational
User impact: Degraded

Update

Still waiting for Kaltura support. Next update, March 30, at 09 UTC.


2020-03-27 14:58 UTC
Update

Waiting for Kaltura support. Next update, March 27, at 15 UTC.


2020-03-27 08:46 UTC
Update

Next update, March 27, at 09 UTC.


2020-03-26 15:11 UTC
Update

The issue is still being worked on by NORDUnet and Kaltura support. Next update at 15 UTC.


2020-03-26 11:52 UTC
Update

There is currently a problem with livestreaming on the Kaltura platform. The issue is causing some live streams not being able to start. The issue has been escalated to Kaltura support. Next update at 12 UTC.


2020-03-26 07:55 UTC
Investigating

There is currently a problem with livestreaming on the Kaltura platform. The issue is being investigated. Next update at 08:00 UTC.


2020-03-26 07:12 UTC

Network

Operational

Zoom

Operational

Kaltura

Degraded

MediaSite

Operational

Panopto

Operational

Metadata

Operational

Info

About "poor" conectivity from home

Please note that “poor” connectivity from home when accessing media services can cause slower response and longer upload times.

Please read the blog post from Robert Vietzke from internet2 regarding this, it provides some good information and considerations. https://www.internet2.edu/blogs/detail/17583/

Kaltura News

The load on the Kaltura service and system has increased by +400% during last week and additional growth is expected. The NORDUnet technical teams have been working continuously over the weekend to implement a number of additions, proactive measures and capacity to the NORDUnet Kaltura service, including:

  • Adding more transcoding capacity - √
  • Reduce the number of transcoded flavors - √
  • Put a commercial CDN in front of the system - √
  • Adding more API servers - √

PLEASE NOTE: In the current situation with new and different workflows being applied it would be beneficial if the uploads were spread more evenly over the day to prevent concentrated usage peaks between 9-11 and 13-15 CET..

Zoom News

The Zoom meeting capacity has over the weekend been upgraded to support a capacity of +250.000 concurrent meeting participants.

PLEASE NOTE: In the current situation with new and different workflows being applied it would be beneficial if meetings and sessions were spread more evenly over the day to prevent concentrated usage peaks between 9-11 and 13-15 CET

Zoom has created a page that will help (new) Zoom users navigate and get started with using Zoom in teaching. https://zoom.us/docs/en-us/covid19.html

A helpful article from zoom about error codes.

2020-03-23 09:15 UTC

NORDUnet Kaltura Service. (Degraded)

Resolved -

The problem has been resolved.


2020-03-20 11:45 UTC
Investigating -

The issue is being investigated with Kaltura on-premise engineering. Next update at 12:00 CET.


2020-03-20 09:25 UTC

2020-03-20 09:25 UTC

NORDUnet Kaltura Service. (Degraded)

Resolved -

Additional API server added.


2020-03-20 15:00 UTC
Investigating -

The issue is being investigated with Kaltura on-premise engineering.


2020-03-20 09:25 UTC

2020-03-19 08:25 UTC

NORDUnet Kaltura Service. (Degraded)

Update -

Additional API servers were added at 03:00 UTC


2020-03-20 03:25 UTC
Investigating -

The issue is being investigated with Kaltura on-premise engineering. Work is ongoing to add more system resources


2020-03-19 08:25 UTC

2020-03-18 14:00 UTC

NORDUnet Kaltura Service. (Degraded)

Resolved -

The issue has been resolved.


2020-03-18 16:00 UTC
Investigating -

The issue is being investigated with Kaltura on-premise engineering, to establish what can be done to improve system performance.


2020-03-18 14:00 UTC

2020-03-17 20:20 UTC

The NORDUnet Panopto on-premise service is currently experiencing problems that prevent videos from being uploaded. (Critical)

Resolved -

The issue has been resolved but users will experience a degraded performance as the upload backlog is being processed.


2020-03-18 09:02 UTC

2020-03-16 19:20 UTC

Most services on our virtualization platform (Outage)

Resolved -

All systems are comming back online, and returning to normal.


2020-03-16 20:08 UTC
Investigating -

During maintenance we experienced a large outage on ndn services running on our virtualisation setup. The cause was human error, but we are looking into how/why it ended up affecting most of our systems.


2020-03-16 19:20 UTC

2020-03-16 08:30 UTC

Creating zoom meetings (Degraded)

Update -

We have found the cause and are working with Zoom to see if we can find a work around.
For now please change users to Licensed instead of On-Premise as the system hits low logical limits
and then block new meeting creation.


2020-03-16 09:50 UTC

2020-03-13 09:30 UTC

Creating zoom meetings (Degraded)

Monitoring -

The issue has now been mitigated, we continue to monitor the situation. Official zoom status


2020-03-11 15:10 UTC
Creating zoom meetings -

We are currently having errors creating new meetings on the Zoom service for both Cloud and On-Premise. We are working with Zoom to resolve this issue.


2020-03-11 9:30 UTC

2020-03-12 10:00 UTC

Creating zoom meetings (Degraded)

Monitoring -

The issue has now been mitigated, we continue to monitor the situation.


2020-03-12 14:30 UTC
Investigating -

We are currently having errors creating new meetings on the Zoom service. We are working with Zoom to resolve this issue.


2020-03-12 10:00 UTC