Announcement

Collapse
No announcement yet.

IB Plugin API causing excessive CPU utilization

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • IB Plugin API causing excessive CPU utilization

    I am testing the new QuoteTrader IB Plugin APIs in eSignal 10.
    The eSignal version is version 858, and QuoteTrader is build 959.

    There are a few serious problems I have identified. First, whenever I send out an new order through API by clicking a button, right after the order is sent out, the CPU utilization shoots up to almost 100%. The CPU time consumed by winsig may shoot from 10 to 98 or 99 and lasts for quite a long time (from a few seconds to much more than 10 seconds). In the IntegratedTrading window, it can be seen there are alway more than a few seconds delay between "Place order; id = xxx ..." and "Order xxx New" messages. The timestamp of "Order xxx New", "Order xxx Submitted" and "Order xxx Filled" may be of less delay, however there may be quite a long time delay before the messages are really shown in the "Messages" window. Basically during that period the computer just hangs up. If during that freezing time period, more orders are sent out by APIs by clicking the buttons, those orders (actions) may just be queued somewhere in eSignal and got executed later.

    For example, in the screenshot I made, at 13:28:44 I mistakenly sent out a buy market order with order size without enough buying power, the order was rejected. The problem is that this event causing eSignal hanging up for about 20 seconds, during that time, I clicked sell buttons for three times, these three orders were queued in eSignal untill at around 13:29:18 they were sent to IB TWS in a burst, the orders were executed in IB TWS about three seconds later. however those messages from #482 to #491 were shown in "Messages" window after more than 90 seconds. In IB execution window, there last three sell orders were executed at 13:29:19, 13:29:20, 13:29:21 respectively, and in "Messages" window of eSignal IntegratedTrading window, they were reported being filled at 13:29:20. 13:29:23 and 13:29:26 respectively, and the actual display time was after 13:31:00. During these three minutes the eSignal just hangs up for most of the time and the CPU is close to 100% busy.

    So in general, I think there are some implementation issues in IB Plugin design regarding to the communication between IB TWS and IB Plugin, which may be something like busy polling causing the excessive CPU utilization. Before such unpredicatable excessive delay issue being solved, it is not practical to use these generic trading APIs in real trading.

    - Clearpicks
    Attached Files

  • #2
    the screenshot of the "Messages" window.
    Attached Files

    Comment


    • #3
      Hi Clearpicks,

      Many thanks for the detailed findings. Will forward to engineering for review.

      Brent

      Comment


      • #4
        Hi Brent,

        Could you confirm that the developpers of QuoteTrader have reproduced the issue and added it to the bug list? Thanks.

        - Clearpicks

        Originally posted by eSignal_Brent
        Hi Clearpicks,

        Many thanks for the detailed findings. Will forward to engineering for review.

        Brent

        Comment


        • #5
          Hi Clearpicks,

          Yes, I can confirm that

          Any additional suggestions would be much appreciated! They'll be immediately forwarded if sent to [email protected].
          We'll be looking to significantly enhance eSignal's trading integration services beginning in the near future.

          Trading integration requests are reviewed with the objective of optimizing their implementation vs constraints such as prioritization, scheduling, degree of difficulty, resources, etc..

          If you'd like to discuss in real-time, please send your contact information and a convenient time to call to [email protected].

          Thanks in advance,

          Brent
          Last edited by eSignal_Brent; 09-13-2007, 06:54 AM.

          Comment


          • #6
            Hello Brent,

            Has this problem been fixed or not? I installed the newest QuoteTrader (922) and found out the route field in configuration window could not be modified and thus could not to use it to trade.

            - Clearpicks

            Comment


            • #7
              Should be fixed in Octobers build (approximately October, 15)

              Comment


              • #8
                Hello Amol,


                Thank you for letting me know that this excessive CPU usage issue will be fixed by OCt. 15.

                There is another issue which may be caused by an eSignal IB Plug-in bug or you just designed it in this way. If I buy a contract then set a stop sell order below it. Than I short the same contract and set a stop buy order above it. Since then, there is no position and there are two active orders (one buy stop and one sell stop). In this case, I can not use the cancelOrders API in EFS to cancel these orders. Could you check whether this happens on your computer?

                - Clearpicks

                Comment


                • #9
                  Hi, Clearpicks

                  It works good for us. Please check if it's possible to cancel them from TWS.

                  Comment


                  • #10
                    It seems both cancelAllOrders and cancalOrder APIs do not work. I was using the gbroker_tester.efs to test them.

                    Also when using closeAllPositions(), is it possible to exclude FX positions in portfolio which is an option in IB TWS Account window?

                    - Clearpicks


                    Originally posted by amol
                    Hi, Clearpicks

                    It works good for us. Please check if it's possible to cancel them from TWS.

                    Comment


                    • #11
                      Originally posted by clearpicks
                      It seems both cancelAllOrders and cancalOrder APIs do not work. I was using the gbroker_tester.efs to test them.

                      Also when using closeAllPositions(), is it possible to exclude FX positions in portfolio which is an option in IB TWS Account window?

                      - Clearpicks
                      Not sure if this is what you are seeing:

                      Whether or not I checked the Include FX Position, I had a problem that was never resolved in my simulated account. I had only 15k GBP position, but a 850K value appeared in the Portfilio section of the Account screen, under the column "position".

                      But in my TWS I only see the the position for 15k that was correct as I had sent an order to buy 15k, got filled and now see the 15k long +15000 GBP position in TWS that wasn't there previously.

                      Now I get a signal to reverse, in my EFS I cssue an closePosition(getsymbol()) function and and order for 865K, yes 865K, the 850k amount listed under Account-Portfolio column heading "Position" amount plus the 15k long trade I executed.

                      I spoke to IB and they said the the 850K wasn't a real "position" but rather a cumlative running total for GBP and closePosition() would only close an actual position not the 850K listed in portfolio. I reported the problem several times to customer support, technical support, opened several trouble tickets and never could resolved it.

                      I began trading in a real IB account today for the first time, and had a 15000 GBP position in got a reversal, closePosition() and a ofsetting order for 14999 goes out, which is too small for IDEALPRO the trading exchange and gets routed to IDEAL not for trading and I get filled 19 pips away from the real market.

                      Nowhere in my EFS is there a string 14999, my position was 15000 and the order was generated by a closePosition(). THe trade audit trail shows 14999 getting sent and the insist I made a typo. Which I could if I was typing, but have no idea what's going on yet?

                      Glen
                      Glen Demarco
                      [email protected]

                      Comment


                      • #12
                        It's not possible to exclude FX position when using closeAllPositions().

                        As to the issue with position closing i believe it could be related with specific TWS version problem.

                        Comment


                        • #13
                          this bug has not been solved in the newest IBBridge

                          Amol,

                          I just tested the newest IBBridge (1019). It seems the causing excessive CPU usage problem is still not solved yet. For the first a few actions (sending/cancelling limit orders and stop orders), it seems working fine (low latency between the my clicking buttons on charts and the corresponding actions in IB TWS). However after I enter then cancel orders a few times, especially after some orders are executed, the latency becomes morenoticable and the CPU utilization gets higher and higher. In some cases it eventually freezes the computer. Even if the CPU utilization drops back to very low level after some time, it seems some thing in IBBridge has been messed up and it would shoot up when I enter orders again unless I restart eSignal to reset everything so that to get relatively low CPU utilization. I have not been able to identify what is the critical factor causing the problem. I don't know whether it was because I entered and cancelled the orders too frequently (I don't think so because I just manually clicking buttons to call broker apis to enter/cancel orders) or I called apis when the status of existing orders in IB is in transient state.

                          - Clearpicks

                          Comment


                          • #14
                            One more thing might be helpful to identify the problem. After the IBBridge becomes slow, trying to send more orders or call cancelOrders() when CPU utilization is high would definitely change the situation from bad to worse.

                            Comment


                            • #15
                              You right, we were able to reproduce. Will be fixed in November build.

                              Comment

                              Working...
                              X