Elevated API error rates

Incident Report for AgileMD

Resolved

Everything appears to be operating normally across all search nodes.
Posted Jun 10, 2016 - 16:20 PDT

Monitoring

We have applied several configuration changes and rebooted two search nodes. Resource consumption is returning to normal and API request delays and errors have returned to normal levels. We will continue to actively monitor performance of the file search service over the next hour.
Posted Jun 10, 2016 - 14:08 PDT

Identified

We have identified a backlog in file indexing and related sharding activities which, in turn, is delaying some inbound search requests. We are now working to expand capacity for the resource-constrained search cluster.
Posted Jun 10, 2016 - 13:54 PDT

Investigating

We are investigating elevated API error rates caused by unresponsive file search requests.
Posted Jun 10, 2016 - 13:24 PDT