Affected services:

  • Storage

Issue with /scratch

Opened on Monday 25th September 2023, last updated

Resolved — We haven't seen the issues return after a thorough check of the file system.

Posted by Fokke

Monitoring — /scratch has been repaired. We are still investigating what usage pattern triggers these storage failures.

Posted by Fokke

Investigating — One of the storage targets had become unresponsive. We've fixed this yesterday afternoon, but the problem has returned early this morning. We suspect that the system can't handle certain usage patterns and are trying to identify what might trigger this.

Posted by Fokke

Investigating — One of the storage devices in /scratch is unresponsive. We are investigating the issue.

Posted by Fokke