Forums Development Check the Quality of your Broker's MT Price Feed

Viewing 15 posts - 1 through 15 (of 15 total)
  • Author
    Posts
  • #10129
    simplex
    Moderator

      Hi Guys!

      This is a follow-up of this post I made yesterday. I decided to start a thread on this topic because data quality is essential for many indicators to work properly, especially sophisticated ones which rely on calculating artificial pairs or short term currency strength.

      To check the quality of timestamps in my MT4 history I wrote the script which I attached. For 28 pairs and all native MT4 timeframes, it starts at bar 1 and compares timestamps of all pairs at that bar index. When it finds a mismatch, this is logged into a csv file, and the next timeframe is processed. This means: it only logs the FIRST mismatch of each timeframe. Runtime on my system is less than 1 second.

      On my development machine, connected to Global Prime Demo Server, the results were horrendous.

      Examples:

      • On M1, I hardly find more than 20 bars (starting at bar 1) that have the same timestamp for all 28 pairs on the same bar index. Depending on data reloaded, the result is different from run to run.
      • On M5, it’s only a little better.
      • On M15, I managed after some manual reload to get a stable number of about 1200 bars that can be used.
      • On H4, there are obviously faulty data (several H4 bars marked 10:00 instead of 12:00)
      • … and many more similar cases.

      Pic shows results for M1 with bar 1 out of sync for GBPCHF.

      I ask everyone around to download the script, run it on your system(s), and report back here, naming your broker and posting the logfiles. This might be interesting for everyone around in order to find brokers offering a high quality data feed.

      Feel free to modify the script and post your enhancements.

      Every additional advice regarding high quality price feeds for MT4 or methods to enhance data quality is highly appreciated!

      s.

      EDIT: Saver0 started a thread about data feed over here about one year ago – useful links in there!

      Attachments:
      You must be logged in to view attached files.

      A good trader is a realist who wants to grab a chunk from the body of a trend, leaving top- and bottom-fishing to people on an ego trip. (Dr. Alexander Elder)

      #10141
      Anti
      Participant

        Tried to execute CheckBarTimes1 by double clicking on it in navigator panel. But nothing happened. Neither custom indicator window opened nor any file has been saved. Recompiling gave no errors.

        • This reply was modified 8 years, 6 months ago by Anti.
        • This reply was modified 8 years, 6 months ago by Anti.
        #10144
        simplex
        Moderator

          Neither custom indicator window opened nor any file has been saved.

          Any message in in the ‘Experts’ log?

          Do your symbols require a prefix or suffix character? If yes, please modify the symbols array in the code, because I forgot them.

          A good trader is a realist who wants to grab a chunk from the body of a trend, leaving top- and bottom-fishing to people on an ego trip. (Dr. Alexander Elder)

          #10145
          Anti
          Participant

            @simplex: Sry, my mistake. Add it to the wrong folder (indicators instead of script).

            Here are my results – Broker GKFX

            Attachments:
            You must be logged in to view attached files.
            #10150
            BlackStack
            Participant

              I could not find the log file.  I looked in both log directories but they did not have any output file like shown above.

              #10152
              mak
              Participant

                I get wird results:

                 

                 

                Attachments:
                You must be logged in to view attached files.
                #10155
                gg53
                Participant

                  simplex,

                  I don’t understand your analysis.

                  Ticks are not coming in equal rates, but only when price move. Therefor they are not synchronized across all pairs.

                  Or, I don’t quite understand something….

                   

                  G.

                  #10156
                  simplex
                  Moderator

                    I could not find the log file. I looked in both log directories but they did not have any output file like shown above.

                    First: this is a script, not an indicator. So it has to be installed in the scripts folder. Please check!

                    Then look in the files directory for results. Filename is the same as script name, filetype .csv

                    A good trader is a realist who wants to grab a chunk from the body of a trend, leaving top- and bottom-fishing to people on an ego trip. (Dr. Alexander Elder)

                    #10157
                    Anti
                    Participant

                      Ticks are not coming in equal rates, but only when price move. Therefor they are not synchronized across all pairs. Or, I don’t quite understand something…. G.

                      Problems will occur if we use indicators that use data from multiple pairs. When some candle’s data are missed, the indices between those pairs asynchronize. Therefor it would be great if MT4 history would have prizes for every pair on every candle. Although prizes don’t change, we can give a prize for every single time point.

                      Another problem is that some histories contain big gaps (also of hours for my broker) that most probably can’t be explained by missing tick activity.

                      #10159
                      simplex
                      Moderator

                        @Anti, @makmak:

                        I tried to force similar results on my system by running the analysis including a pair I’ve never used, like SGDJPY. See pic for results: in this case no output for TF smaller than H1, and H1 pointing to timestamp 2015.03.26,02:00 at bar index = 1.

                        So not completely the same result as yours, but anyway I would interpret your results in a way that you simply do not have any local history for those pairs indicating a timestamp like 1970.01.01.

                        I don’t understand your analysis. Ticks are not coming in equal rates, but only when price move. Therefor they are not synchronized across all pairs.

                        Good point! Yet I don’t analyse tick data here, I simply check whether bar index and timestamp of a bar are matching for different pairs in local history.

                        Why did I start this?

                        When working with simple indicators like Stoch, an MA or whatever, it does not matter that much when a bunch of bars are missing in history. When coding more sophisticated indicators, which have to analyse many pairs to calculate an output (like working with synthetic symbols, or CS algorithms) we usually are linking the price data of all underlying pairs using the bar index. Most indicators have a simple loop over all bars for that purpose. To get precise results, it’s essential that at any bar index the timestamps of all underlying pairs are identical. And this is what most coders of such algorithms (including me) are assuming, at least implicitly.

                        This is the condition my script is checking, showing results I don’t like.

                        Now when using those indicators in an EA, we can easily check in realtime whether all required bars within a certain lookback period are having correct timestamps, thus suppressing trade signals if faulty bars occur. When trying to test our signals statistically using price history, this won’t work. Any missing bar will add a certain time gap (according to timeframe chosen) to all bars left (the past) of the missing bar, and the underlying pairs are out of sync. It’s the statistical tests that don’t work precisely under these conditions.

                        Any thoughts anybody?

                        s.

                        Attachments:
                        You must be logged in to view attached files.

                        A good trader is a realist who wants to grab a chunk from the body of a trend, leaving top- and bottom-fishing to people on an ego trip. (Dr. Alexander Elder)

                        #10161
                        mak
                        Participant

                          This should be beter..

                          Attachments:
                          You must be logged in to view attached files.
                          #10165
                          simplex
                          Moderator

                            @makmak:

                            This should be beter..

                            It says on FXCM demo you’re having 285 M1 bars in sync, on Oanda Live the first async fault occurs on bar 1 (see column IDX). The script will list timestamp data of the first bar (starting at IDX = 1) that is out of sync (faulty).

                            Now you’ll have to decide for yourself whether this is a problem regarding the strategies / indicators you’re using or testing on those accounts.

                            If you want to run the script only on a part of those symbols, the script code would have to be modified. It is kind of a quick and dirty hack of last night.

                            s.

                            A good trader is a realist who wants to grab a chunk from the body of a trend, leaving top- and bottom-fishing to people on an ego trip. (Dr. Alexander Elder)

                            #10169
                            mak
                            Participant

                              @simplex

                               

                              Thanks for explanation. Actualy it is not a problem for the strategy i m using.

                              #10197
                              BlackStack
                              Participant

                                OK found the file. I was looking in the log directories.

                                FXPro broker

                                 

                                 

                                • This reply was modified 8 years, 6 months ago by BlackStack.
                                Attachments:
                                You must be logged in to view attached files.
                                #11308
                                simplex
                                Moderator

                                  Quite a good explanation of CrapT4 crappy price feed can be found over here:

                                  http://privatelink.de/?http://www.forexfactory.com%2Fshowthread.php%3Fp%3D5638648%23post5638648

                                  A good trader is a realist who wants to grab a chunk from the body of a trend, leaving top- and bottom-fishing to people on an ego trip. (Dr. Alexander Elder)

                                Viewing 15 posts - 1 through 15 (of 15 total)
                                • You must be logged in to reply to this topic.
                                Scroll to Top