We were able to deploy an additional scheduler, and apply a filter to the abnormal messages that were obstructing the queue. It took awhile to process the backlog but latency has now returned to normal. Tomorrow morning from 7am-8am ET we will be conducting emergency database maintenance to address what we believe is a faulty database node at the root of the problem. During this maintenance there will be approximately 1 minute of service interruption.
Posted 2 months ago. Mar 24, 2018 - 21:26 EDT
We are again experiencing delays with our scheduler subsystem, resulting in scheduled orders firing late to the POS.
Posted 2 months ago. Mar 24, 2018 - 17:38 EDT
This incident affected: Scheduler and Developer Portal.