On Thursday 23 March 2023, our buying and selling engine was suspended from 12:55 UTC to 13:20 UTC, with orders executed in cancel solely mode throughout this time.
This suspension was triggered by our audit course of detecting a potential inconsistency within the change state and halting the change as a precautionary measure.
At BitMEX our positions and margins are checked a number of instances a minute. With balances cross-checked in opposition to on-chain information, each 10 minutes. Bugs, flaws or intrusions, inflicting positions to not match, instantly halt our change.
We program our change to halt, as defending the property of our customers is central to our operations. We don’t compromise safety for comfort, which is why we’ve misplaced zero cryptocurrency. And as communicated through the downtime by way of our Status web page and social media channels, all of our customers’ funds remained secure always.
Following a complete evaluation of the downtime, we’re offering you, our customers, with a extra detailed abstract of the incident as set out beneath. Ought to you might have any extra questions, as all the time you’ll be able to attain us by contacting Support.
What occurred on 23 March?
As shared above this suspension was triggered by our audit course of detecting a potential inconsistency within the change state and halting the change as a precautionary measure.
A race situation surrounding inside transfers induced a momentary state inconsistency. The audit course of detected this and took motion – as supposed.
Our engineering workforce have been capable of assess the state of affairs, determine the basis trigger and deem it secure for buying and selling to proceed.
What was the consumer impression?
The change operated in cancel-only mode whereas buying and selling and matching was suspended for 25 minutes. There was some market disruption when the change resumed buying and selling.
Presently, our help workforce is responding to these clients who have been impacted. No clients have been liquidated because of the market disruption.
What’s being accomplished to make sure this doesn’t occur once more?
As a 24/7 change, we try to supply good uptime and uninterrupted providers, nonetheless this can’t be assured. We don’t make compromises on the subject of the protection of our customers’ funds. You possibly can learn extra about how we maintain funds secure here.
A working group has been fashioned to discover how we, at BitMEX, can higher resume buying and selling shifting ahead. Moreover, we’re reviewing the behaviour of the audit course of, to see if full suspension might be averted below sure strict circumstances.
How can I shield myself in opposition to the dangers related to downtime occasions?
We offer a variety of refined instruments and settings via which it’s potential for merchants to regulate the chance posed to them by downtime occasions.
For merchants who use our API, we provide a “Lifeless Man’s Change” characteristic, which allows merchants to set a timeout for cancellation of orders in case they’re unable to succeed in the change. For implementation particulars and an instance of setting a timeout, please see the documentation for this characteristic here.
For enhanced safety merchants can set the set off worth of a cease order to reference Final Value, Mark Value or Index Value. When the change is reopened after a downtime occasion, dramatic swings within the Final Value can happen. To keep away from cease orders being triggered by †any dramatic swings in Final Value, merchants can set their stops to set off primarily based on the Mark Price or Index Value, relatively than the Final Value.
These are simply two instance options of the BitMEX platform that permit our merchants to regulate danger and scale back the impression of downtime. Our Support team can present additional details about these (and some other) danger administration instruments on the platform.
Conclusion
We hope this weblog put up supplies all customers with a transparent description of this downtime incident and the sturdy ongoing steps we’re taking to reinforce platform stability and our general buying and selling expertise.
As all the time, we encourage any involved customers and people impacted by this downtime to contact Support.
Leave a Reply