Intermittent Filters and Reporting Incident

Incident Report for Hearsay Systems

Postmortem

Timeline:

Start Time: 02/26/2025 7:49 AM PT

End Time: 02/26/2025 8:56 AM PT

Customer Impact:

A recent production deployment caused filtering on the reports to be temporarily unavailable. Reports may have shown blank results and the download button did not appear. This incident only affected those listed as administrators in their instance of Hearsay. There was no data loss during this incident.

Resolution:

Hearsay’s engineers rolled back the change that caused the problem and redeployed production services to use the earlier stable version.

Future Improvements:

Moving forward, additional automated testing will be employed to ensure a similar issue does not occur. In the interim, we are executing manual regression tests after each change in this area of our platform.

Posted Mar 10, 2025 - 11:38 PDT

Resolved

Hearsay successfully reverted to a previous version of the platform, and users should no longer experience any reporting issues. After a thorough investigation has been completed, we will provide a post-mortem on this incident.

Posted Feb 26, 2025 - 09:01 PST

Investigating

Hearsays engineers are investigating an incident with our reporting where some admins may notice the filter options are temporarily unavailable. Some admins may experience an empty report, but clicking the refresh icon should load the data. At this moment, downloading reports may be temporarily unavailable. Hearsay is treating this as our top priority and will provide an update once we have more information.

Posted Feb 26, 2025 - 08:14 PST

This incident affected: Analytics.