Postmortem -
Read details
Jan 20, 16:42 EST
Resolved -
We are no longer observing any issues across our APIs. This incident is now marked as resolved. A root cause analysis will be published by tomorrow..
Jan 20, 05:26 EST
Update -
Errors are very low and limited to brief blips during restarts; we will keep monitoring closely before marking this incident as resolved.
Jan 20, 05:20 EST
Update -
After We have restarted the affected backend services then pending account/journal work has cleared. Error rates and timeouts are now very low, and requests are succeeding again. We are continuing to monitor closely for any other issue.
Jan 20, 05:11 EST
Update -
overall performance has improved for all affected endpoints. We will keep monitoring closely before marking this incident as resolved.
Jan 20, 05:01 EST
Monitoring -
Error rates and latency are improving after remedial actions, we are continuously monitoring for any issue.
Jan 20, 04:51 EST
Investigating -
Our team is identified issues with one of the service being stale to refresh data and we are working on it.
Jan 20, 04:46 EST
Identified -
Core databases and replication appear healthy; impact is limited to specific journal /account operations and is intermittent.
Engineering is actively working to stabilize the affected service and reduce latency; we will provide another update as we learn more.
Jan 20, 04:43 EST
Update -
Accounts are taking longer than expected to fully load, which is occasionally causing some users to see an error (a timeout) when they try to check their journals or orders. We are actively working to quickly finish loading these accounts and stabilize the system.
Jan 20, 04:32 EST
Investigating -
API response for orders and JNLC were delayed intermittently
Jan 20, 04:20 EST