Live order partial outage

Incident Report for Alpaca

Resolved

One of the order requests was in an unexpected state and our order routing started to fail every retry. While in this situation, the order processing was slow. We have identified such situation and put a workaround to recover the entire system. We are working on the permanent fix in the multiple layers to prevent such issues from happening again.
Posted Jun 05, 2020 - 11:01 EDT

Monitoring

A fix has been implemented and we are monitoring the results.
Posted Jun 05, 2020 - 10:23 EDT

Investigating

Some orders are not being sent to the market venues
Posted Jun 05, 2020 - 10:05 EDT
This incident affected: Live Trading API (Orders API).