Logs cluster failure
Incident Report for Fasterize
Resolved
Due to logs cluster issues, we have lost some logs from friday 23 to tuesday 27 th of july 2021.
Our logs cluster had some performance issues since sunday 16:30 UTC. It seems that some logs indices experimented sharding issue that have corrupted 23 to 24 th of july 2021 indexes. Indexes from sunday 25 to tuesday 27 th 9h00 UTC of july 2021 may be incomplete.
We deleted corrupted indexes. This allowed us to delete unhealthy nodes and re-balance the cluster.
Logs are now fully operational. We are sorry for logs lost during this incident.
Posted Jul 27, 2021 - 11:00 CEST