Relate Functionality Outage

Incident Report for Hearsay Systems

Postmortem

Timeline:

Start Time: 04/04/23 2:04 PM PT

End Time: 04/04/23 2:56 PM PT

Customer Impact:

A version update to an internal database caused a conflict with Hearsay Relate, resulting in the inability to send messages, make phone calls, and create or update contacts.

Resolution:

Hearsay’s engineers immediately responded to this incident by reverting back to the previous version of the database, which effectively solved the issue for the affected users.

Future Improvements:

Hearsay already has proactive alerting mechanisms in place, which enabled us to identify and respond to this incident promptly. To help prevent similar occurrences in the future, we plan to update our existing end-to-end testing practices to ensure more edge cases are considered before future production deployments.

Posted Apr 11, 2023 - 10:12 PDT

Resolved

Hearsay's engineers have resolved the ongoing issue with Relate. Users should no longer be experiencing issues sending messages, making calls, and creating/editing contacts. We would like to express our sincere apologies for any inconvenience caused by the recent disruption. We greatly appreciate your patience and understanding as we worked to resolve this issue.

Posted Apr 04, 2023 - 15:09 PDT

Investigating

We're currently experiencing an issue with Hearsay Relate in which users may encounter problems sending messages and making calls. Users may also run into intermittent issues when creating or updating contacts. Hearsay's engineers are actively looking into this matter and are treating this as our top priority. We truly apologize for any disruption and will provide an update shortly.

Posted Apr 04, 2023 - 14:40 PDT

This incident affected: Relate.