Backfill Exploration for Real-Time data

Backfilling data can be done in many ways, and different methods may be needed for different data providers. The method shown below was developed for eSignal data. If you use another data provider, you may need to modify the code and the procedure.

When backfilling data, you should confirm that data are backfilled properly and have some indication regarding the presence of data holes. As pointed out in Data Holes in Real-Time Trading, to be able to detect holes, you need a perfect data array against which to compare the data. Since AmiBroker doesn’t have such a data array, the method presented here uses the QQQQ as a reference ticker. Turn ON and set Backtester Settings -> General -> Pad and align all data to reference symbol -> QQQQ. To have all tickers backfilled, you should also turn ON Wait for Backfill(RT only). To speed up real-time backfills you may want to display only a simple price chart, instead of complex code. Refreshing complex Indicators or Systems will slow down backfill. The following Exploration will Backfill all tickers in your Watchlist and report on it’s success:

1 Star2 Stars3 Stars4 Stars5 Stars (2 votes, average: 3.50 out of 5)

One Response to “Backfill Exploration for Real-Time data”

  1. brian_z
    October 17th, 2007 | 5:05 am

    Nice work Herman and Al.
    In a later post in the DataMangement series I propose a method very similar to your ‘hole counting’ as a pseudo ‘data quality’ metric for EOD data (looks like we are on the same wavelength there!)