Market Data: Incorrect pricing data for NASDAQ listed symbols
Incident Report for Alpaca

What Happened?

On the morning of Saturday, April 10th, as early as 9:30 AM Eastern time, some users started to see their account equity value flickering and eventually displaying inaccurate values. Around the same time, our engineering team started to look into the issue and realized that we were receiving real time market data updates via SIP feed with test data prices for NASDAQ listed stocks despite the market being closed at the time.

What Was The Issue?

NASDAQ schedules test trade sessions on a pre-scheduled Saturday of each month, and the market updates are sent through the SIP as “unmarked data”.  As we rolled out our new market data integration last month, this was our first time going through this process. The real-time account equity calculation is based on the last sale prices from this market data feed, and a large number of tickers, including AAPL and QQQ, had prices such as $1.1 as the last sale. Ideally, this data would be excluded from our data consumption and processing service but it was treated as if the test data was real trade data (as it was unmarked). As a result, a number of customer accounts temporarily displayed wrong market values on positions for these tickers, and the equity value, which is based on the position market value, temporarily displayed an inaccurate amount.

What Action Was Taken?

Our engineers got together as soon as we noticed the issue and went into the war room. The first few hours were spent to isolate and identify the problems. Then, we remediated data stored in several components in our backend system consuming the real time market data for various purposes. This took us another few hours. By around 7 PM Eastern time, we successfully purged any test data that was stored in our system and also confirmed the new data updates for the test session had stopped a few hours earlier.


We will implement a solution so that this event doesn’t occur again. As our new market data integration just went live last month, we realize we didn’t have enough cases to cover such events. We apologize for the surprises on Saturday morning and appreciate your patience. As always, your feedback helps us make our platform more robust as we continue to improve our API platform.

Posted Apr 11, 2021 - 23:17 EDT

This incident has been resolved.
Posted Apr 10, 2021 - 20:48 EDT
The issue has been identified and a fix is being implemented.
Posted Apr 10, 2021 - 15:00 EDT
We are currently remediating an issue where we received incorrect pricing data from NASDAQ feed for all NASDAQ listed symbols. We are purging the faulty data and expect correct pricing to resume shortly.
Posted Apr 10, 2021 - 15:00 EDT