Authentication issues - unable to connect

[Kevin B]

We are currently experiencing a major issue with the eSignal servers. We appreciate your patience while we investigate the issue.

We sincerely apologise for the inconvenience caused.

[ScottJ]

The issue is related to authentication so users that are currently connected are working OK but some customers attempting to connect are unable to successfully access the service. All engineers are working diligently to isolate the issue and resolve ASAP.

Thank you for your patience.

[ScottJ]

We continue to have issues with some users connecting to our servers. As a result, we are experiencing long wait times on chat and in our phone queues. Engineering is making every effort to fully restore service for all users ASAP.

Thank you for your continued patience.

[ScottJ]

We continue to restore the service to as many customers as possible. We believe we're close to completion.

Thank you.

[eostrom]

If you are still having issues connecting please call 510-264-1700 and remain on the line as it may take some time to go through. We continue to process connections and we appreciate your patience.

The best thing for users to do at this point is shutdown the program for about 10 to 15 minutes and then attempt to reconnect. We hope to have this fully resolved soon.

Update 1: This looks to have been resolved at this point. If you are still having connection issues. Please shut down the program along with the computer and restart. If you continue to have an issue please contact our tech support team. Thank you

Update 2: Although the initial issue has been resolved, it is taking more time than anticipated to process users and get everyone connected. We are still working on this. Thank you for your continued patience.

Update 3: We are continuing to process accounts...

Update 4: We continue to process users and believe that things are nearing completion. More updates to come as I have them. Thank you

Update 5: Issues have been mostly resolved, our tech support team is clearing up those that are still unable to connect. Please call If you have further issues. Thank you

[System Status Update]

Earlier today we experienced a major outage that affected a significant amount of users. While this issue has now been resolved, we would like to apologize for the inconvenience this may have caused. We will post a full explanation tomorrow morning on what occurred and the steps we're taking to prevent this from happening again.

Thank you for your continued patience.

Mike Ramos
eSignal Support

[ScottJ]

In regards to yesterday's outage that impacted some customers, we've learned the following:

As a part of our routine entitlement system maintenance, a legacy entitlement bit was removed from the table. This removal triggered a bug in the updating of the user record database from our Customer Management System and resulted in the deletion of thousands of valid usernames. The file was updated on 2/5 but not applied to downstream authentication systems until 1:00 am PT on 2/6.

After several hours of troubleshooting by all engaged teams, and once the issue was identified, usernames needed to be refreshed and reinserted into authentication database to allow user access. Service was fully restored once all accounts were updated.

We are taking numerous measures to prevent any chance of a recurrence of this issue and we're confident that our improvements will significantly reduce the chance of this event repeating.

We apologize for any inconvenience this event had on your business or your trading and we want to assure you that every effort will be made to ensure greater reliability of the service going forward.

Thank you.