Fractional order fills
Incident Report for Alpaca
Postmortem

prop engine did not fill market orders due to fault on halts

A day before, at 03/27 on this PR we changed the source of market data in prop for otc assets to be marketcache-otc instead marketcache-otc-expert

Then marketcache-otc had some leftover symbols for non-OTC assets with halts.

prop engine is aggregating all the notifications for all marketdata sources so it did not know if this is coming from OTC or non-OTC market data source.

So even though eg MO was not an OTC symbol, OMS consumed the erroneous halt information from marketcache-otc and this resulted to halting market orders for erroneous symbols.

Posted Mar 28, 2024 - 13:34 EDT

Resolved
This incident has been resolved.
Posted Mar 28, 2024 - 13:31 EDT
Update
n/a
Posted Mar 28, 2024 - 13:31 EDT
Update
We currently still have impacted orders on DOYU
Posted Mar 28, 2024 - 13:05 EDT
Monitoring
Impacted orders are now filled
Posted Mar 28, 2024 - 11:02 EDT
Update
We are continuing to work on a fix for this issue.
Posted Mar 28, 2024 - 10:20 EDT
Identified
Alpaca is currently experiencing an issue filling some fractional orders on 15 symbols

ACB
ATNF
ETAO
IFBD
KNTE
LABP
MO
MODG
NYCB
SAVE
TKC
TNP
TSVT
UPST
VRM
Posted Mar 28, 2024 - 10:16 EDT
This incident affected: Live Trading API (Orders API) and Paper Trading API (Paper Order API).