Collecting and Plotting Ticks v2

The program below shows how you can collect tick-based data and display it in its own pane.

Note: There was an error in the earlier version that is fixed, v2 only collect ticks if the Total Volume for the day changes. Feel free to report any further problems (on the main list or by private email please).

The example shows Last, Bid and Ask price. You can repeat the four lines that plot each price as often as you like to add other types of tick-based data. A typical chart looks like this:

function TickArray2TickerTickPriceNameNumTicksReset )
{
Tickprice GetRTDataForeignTickPriceNameTicker );
PrevVolume NzStaticVarGet"~TotalVolume_" TickPriceName ) );
TotVol GetRTDataForeign"TotalVolume"Ticker );
StaticVarSet"~TotalVolume_" TickPriceNameTotVol );

if ( TotVol &gtPrevVolume )
{
InIndicator Status"Action" ) == 1;
StaticVarSet"LastQuotePrice"TickPrice );
TA Null                                            // Clear Output array
NumTicks MinBarCount 2NumTicks ); // Stay in array size
for ( NumTicks>= 0n-- )
{
StaticVarGetTickPriceName + ( ) );
StaticVarSetTickPriceName n);
TABarCount n] = T    // Fill return array
}
StaticVarSetTickPriceName 0TickPrice );
TA[BarCount-1] = TickPrice;
}
else
{
TA Null                                               // Clear Output array
NumTicks MinBarCount 2NumTicks );     // Stay in array size
for ( NumTicks>= 0n-- )
{
StaticVarGetTickPriceName );
TABarCount n] = T    // Fill return array
}
}
return TA;
}

function TickArraysResetTickPriceName )
{
global NumTicks;
StaticVarSet"Init"+TickPricename);
for ( NumTicks>= 0n-- ) StaticVarRemoveTickPriceName  );
}

GraphXSpace 20;
TickerName Name();
Clear        ParamTrigger"Clear Tick Charts""CLEAR" );
NumTicks    Param"Tick-Array Length"2031000);

TickPriceName "Last";
if ( IsNullStaticVarGet"Init"+TickPriceName ) ) OR Clear TickArraysResetTickPriceName );
TA TickArray2TickerNameTickPriceNameNumTicksClear);
PlotTATickPriceName11);

TickPriceName "Bid";
if ( IsNullStaticVarGet"Init"+TickPriceName ) ) OR Clear TickArraysResetTickPriceName );
TA TickArray2TickerNameTickPriceNameNumTicksClear);
PlotTATickPriceName41);

TickPriceName "Ask";
if ( IsNullStaticVarGet"Init"+TickPriceName ) ) OR Clear TickArraysResetTickPriceName );
TA TickArray2TickerNameTickPriceNameNumTicksClear);
PlotTATickPriceName51);
1 Star2 Stars3 Stars4 Stars5 Stars (11 votes, average: 4.18 out of 5)
Loading...Loading...

Reading/Backing-up the TWS Exported Execution Report

Please be sure you set up the TWS according to the instructions provided in Setting Up your TWS before you test this code. The code presented here reads the execution report, converts it to a .csv format, date-stamps it, backs it up for later use, and optionally displays it in the chart Title. The code doesn’t do anything important besides displaying the information in the Title. The idea is to show you how to read the file so that you can extract real execution prices, use them in your calculations, and plot them on your chart. The Param options are self-explanatory:

clip_image002

The name used for the execution report generated by the TWS is not date-stamped. For example, if you set up the TWS to export executions under the name Simulated.Trades, this same name will be used on successive days. If the TWS finds a tradelist from the previous day, it will simply overwrite it. To prevent losing this AFL readable file it is important to back up the tradelist at the end of the day. The format of the execution report exported by the TWS looks like this:

ATVI;SLD;75;26.19;21:29:33;20080125;ARCA;DU1195;;;DEMO;
ALTR;BOT;100;18.54;21:53:12;20080125;ARCA;DU1195;;;DEMO;
ALTR;BOT;100;18.58;21:55:59;20080125;ISLAND;DU1195;;;DEMO;
ALTR;BOT;100;18.55;21:56:00;20080125;ARCA;DU1195;;;DEMO;
ALTR;BOT;100;18.58;21:58:47;20080125;ISLAND;DU1195;;;DEMO;

The .csv format of the backup file produced by the code below can be directly imported into Excel and looks like this (note the semicolons have been replaced by commas):

ATVI,SLD,75,26.19,21:29:33,20080125,ARCA,DU1195,,,DEMO,
ALTR,BOT,100,18.54,21:53:12,20080125,ARCA,DU1195,,,DEMO,
ALTR,BOT,100,18.58,21:55:59,20080125,ISLAND,DU1195,,,DEMO,
ALTR,BOT,100,18.55,21:56:00,20080125,ARCA,DU1195,,,DEMO,
ALTR,BOT,100,18.58,21:58:47,20080125,ISLAND,DU1195,,,DEMO,

In Excel, the file will look like this after activating Text to columns:

clip_image004

Please be aware that the minimum update interval that the TWS exports the execution report is approximately one-minute. This means it will take some time for your trades to show up in the list.

Before tackling the main backup function, there are a few helper functions you will need. While these are available elsewhere on this site, they are repeated below for your convenience. To prevent conflicts between static variables used in different programs, you should key their names with those charted; see Keying Static Variables for more information on this. The DateNumToStr() converts DateNumbers to a standard date string.

The TWSBackupTradeList( TWSInputPath ) listed below reads the TWS tradelist, extracts the date, converts it to the .csv format, saves it in a different location, and optionally displays both tradelists in the chart Title. To test this function, Apply it to a new Indicator, open the Param window, set up the parameters, and click BACKUP. The backup file is saved in the path defined by the TradebackupFolder variable. If the function finds the execution report and its display is turned on in the Param window, this should look like this in the Title (only a few lines shown):

clip_image006

And, when displayed, the backup file should look like that below:

clip_image008


InIndicator     Status"Action" ) == 1;
StaticVarKey     GetChartID();

procedure xStaticVarSetSNameSValue )
{
    global StaticVarKey;

    if ( InIndicator )
        StaticVarSetSname StaticVarKeySvalue );
}

function xStaticVarGetSName )
{
    global StaticVarKey;
    return StaticVarGetSname StaticVarKey );
}

procedure xStaticVarSetTextSNameSValue )
{
    global StaticVarKey;

    if ( InIndicator )
        StaticVarSetTextSname StaticVarKeySvalue );
}

function xStaticVarGetTextSName )
{
    global StaticVarKey;
    return StaticVarGetTextSname StaticVarKey );
}

function DateNumToStrDtNum )
{
    DayNm roundfracDtNum 100 ) * 100 );
    MthNm roundfracDtNum 10000 ) * 100 );
    YrNm intDtNum 10000 ) + 1900;
    return NumToStrMthNm1.0 ) + "/" NumToStrDayNm1.0 ) + "/" NumToStrYrNm1.0False );
}

procedure TWSBackupTradeListTWSInputPath )
{
    global TradebackupFolderAccountType;
    fh1 fopenTWSInputPath"r" );

    if ( fh1 )
    {
        Line                 StrReplacefgetsfh1 ), ";""," );
        DateStr             StrExtractLine);
        YearNum            StrToNumStrLeftDateStr) );
        MonthNum            StrToNumStrMidDateStr4) );
        DayNum            StrToNumStrRightDateStr) );
        DateNumber         = ( YearNum 1900 ) * 10000 100 MonthNum DayNum;
        DateNumStr         NumToStrDateNumber1.0False );
        BackupFilename AccountType DateNumStr ".csv";
        BackupPath         TradebackupFolder BackupFilename;
        fclosefh1 );
    }

    fh1 fopenTWSInputPath"r" );

    fdeleteBackupPath );
    fh2 fopenBackupPath"a" );
    LineNum 0;
    TWSTradeList CSVTradelist "";

    if ( fh1 )
    {
        if ( fh2 )
        {
            while ( ! feoffh1 ) )
            {
                Line fgetsfh1 );
                TWSTradeList TWSTradeList Line;
                Line StrReplaceLine";""," );
                CSVTradelist CSVTradelist Line;
                LineNum++;

                if ( Line != "" )
                {
                    fputsLinefh2 );
                }
            }
        }

        xStaticVarSetText"TWSTradelist"TWSTradelist );

        xStaticVarSetText"CSVTradelist"CSVTradelist );
    }
    else
    {
        if ( fh1 == )
        {
            PopupWindow"Could NOT Open InputPath: " TWSInputPath,
                         "TWS EXPORTED TRADELIST"timeout 5left = -1top = -);
        }

        if ( fh2 == )
        {
            PopupWindow"Could not open OutputPath: " OutputPath,
                         "TWS EXPORTED TRADELIST"timeout 5left = -1top = -);
        }
    }

    if ( fh1 )
        fclosefh1 );

    if ( fh2 )
        fclosefh2 );

    Caption "TWS EXPORTED TRADELIST";

    Message "The TWS Tradelist: \n   " TWSInputPath " [" NumToStrLineNum1.0False ) +
              " Trades/" DateNumToStrDateNumber ) + "]" +
              " \nHas been saved in csv format as:\n   " BackupPath;

    PopupWindowMessageCaptiontimeout 20left = -1top = -);
}

_SECTION_BEGIN"BACKUP TWS TRADELIST" );
TWSInputPath                ParamStr"TWS Tradelist (Folder)""C:\\Jts\\" );
AccountType                    ParamList"TWS Account Type (Filename)""Real|Simulated|Demo");
TWSInputFilename            AccountType ".Trades";
TWSInputPath                 TWSInputPath TWSInPutFilename;
TradebackupFolder            ParamStr"Backup Destination Folder""C:\\Jts\\TWSTrades\\" );
BackupTWSTradeList        ParamTrigger"Create Backup Tradelist""BACKUP" );
ShowTWSTradeList            ParamToggle"TWS Tradelist""HIDE|SHOW");
ShowCSVTradeList            ParamToggle"CSV Tradelist""HIDE|SHOW");
_SECTION_END();

if ( BackupTWSTradeList )
    TWSBackupTradeListTWSInputPath );

TWSStr WriteIfShowTWSTradeList"\nTWS Exported Tradelist: \n" xStaticVarGetText"TWSTradelist" ) + "\n""" );

CSVStr WriteIfShowCSVTradelist"\nCSV Exported Tradelist: \n" xStaticVarGetText"CSVTradelist" ), "" );

Title TWSStr CSVStr;

Edited by Al Venosa.

1 Star2 Stars3 Stars4 Stars5 Stars (2 votes, average: 4.00 out of 5)
Loading...Loading...

Real-Time Bar Period Timing

When trading in real-time, one often needs to know when a new period starts and how much time there is left before the period ends. The code below will give you the time remaining to the next bar, the time elapsed since the start of the bar, and the second count since date-change. The timing values will automatically adjust to the selected chart interval. You can use the variables in your system’s code to time various events.

This code is shown in a demo configuration, and you will have to adapt it to your personal requirements. To test, simply Apply the code to an Indicator window. For a quick test, select the one-minute time interval.

function GetSecondNum()
{
    Time         Now);
    Seconds     intTime 100 );
    Minutes     intTime 100 100 );
    Hours     intTime 10000 100 );
    SecondNum intHours 60 60 Minutes 60 Seconds );
    return SecondNum;
}

RequestTimedRefresh);
TimeFrame Interval();
SecNumber GetSecondNum();
Newperiod SecNumber TimeFrame == 0;
SecsLeft     SecNumber intSecNumber TimeFrame ) * TimeFrame;
SecsToGo     TimeFrame SecsLeft;

if ( NewPeriod )
{
    Say"New period" );
    Plot1""colorYellowstyleArea styleOwnScale0);
}

Title "\n" +

        "  Current Time: " Now) + "\n" +
        "Chart Interval: " NumToStrTimeFrame1.0 ) + " Seconds\n" +
        " Second Number: " NumToStrSecNumber1.0False ) + "\n" +
        "  Seconds Left: " NumToStrSecsLeft1.0False ) + "\n" +
        " Seconds To Go: " NumToStrSecsToGo1.0False );

PlotC""1128 );

For verification, timing is displayed in the chart title:

clip_image002

Edited by Al Venosa.

1 Star2 Stars3 Stars4 Stars5 Stars (5 votes, average: 4.60 out of 5)
Loading...Loading...

Real-Time Bar-Period Timing

In real-time trading one often needs to know when a new period starts and how much time there is left before the period ends. The code below provides this information. Be sure to synchronize your system’s clock.

<p>function GetSecondNum() 
<p>{ 
<p>Time Now(4); 
<p>Seconds int(Time%100); 
<p>Minutes int(Time/100%100); 
<p>Hours int(Time/10000%100); 
<p>SecondNumint(Hours*60*60+Minutes*60+Seconds); 
<p>return SecondNum; 
<p>} 
<p>
<p>RequestTimedRefresh(1); 
<p>TimeFrame Interval(); <p>SecNumber GetSecondNum(); 
<p>Newperiod SecNumber%TimeFrame == 0; 
<p>SecsLeft SecNumber-int(SecNumber/TimeFrame)*TimeFrame; 
<p>SecsToGo TimeFrame SecsLeft; 
<p>if( NewPeriod )  
<p>{ 
<p>Say("New period"); 
<p>Plot(1,"",colorYellow,styleArea|styleOwnScale,0,1); 
<p>} 
<p>Title "\n"+ 
<p>"time: "+Now(2)+"\n"+ 
<p>"Interval: "+NumToStr(TimeFrame,1.0)+"\n"+ 
<p>"Second Number: "+NumToStr(SecNumber,1.0,False)+"\n"+ 
<p>"Seconds Left: "+NumToStr(SecsLeft,1.0,False)+"\n"+ 
<p>"Seconds To Go: "+NumToStr(SecsToGo,1.0,False); 
<p>

For testing and code verification timing is displayed in the chart title:

clip_image002

1 Star2 Stars3 Stars4 Stars5 Stars (6 votes, average: 4.00 out of 5)
Loading...Loading...

High-Precision Delay and Interval Timing

Before continuing with this post you should carefully read the AmiBroker Help topic for the getPerformanceCounter().

Measuring time is an important aspect of all real-time intraday trading systems. Typical tasks requiring high-resolution timing include:

  1. Limiting the message rate to Interactive Brokers (IB) to 50/sec (API Error 100).
  2. Inserting small delays before polling IB Status, to allow for Internet delays.
  3. Staggering (interlacing) portfolio trades to spread out the action.
  4. Measuring and optimizing AFL execution time.
  5. Modifying orders after a small delay (to ensure fills).
  6. Periodic execution of tasks, for example, Watchlist scanning, Display and Status refresh, calculations based on slow changing variables, etc.
  7. Time-Stamping events, for example order placement.
  8. Collecting/preprocessing quotes.
  9. Overlay live tick-charts on faster and easier to manage 1-minute charts.

Most of these tasks can be accomplished using just three custom timing functions:

  1. GetElapsedTime(): A function that returns elapsed time since reset.
  2. SetDelay(): A function that returns time left to reach a future time.
  3. GetDelayTrigger(): A function that returns a trigger when a delay times out.

This post provides example functions in a demo application. To allow the use of many timers each function requires you to provide a TimerName, which will be used to retrieve timer information. Static variables are Global and can be read from anywhere; this means you have to be careful not to cross-reference the timers by using the same TimerName from different panes or windows. When running multiple copies of the same code you will need to key the TimerNames. For more on how to do this, see Keying Static Variables in the Real-Time AFL Programming category.

The timers below are implemented using the getPerformanceCounter(). This function returns the amount of time elapsed since the computer was last started. Tomasz recently explained this as follows: “The underlying high frequency counter runs all the time since computer start. What ‘reset’ flag really does is to store last value so next time you read it, it gets subtracted from last value giving you the difference. If reset is false, the last value is set to zero, and you get the original number of ‘clock ticks’ since computer start“.

The timers in this post do their own sampling of the underlying high frequency counter, and the getPerformanceCounter() Reset argument is always left set to False.

The getPerformanceCounter() returns values with microsecond resolution; however, the practical accuracy is severely limited by interruptions from the computer’s operating system. Do not expect much better than about 50-millisecond absolute accuracy. Aside from designing your own dedicated trading hardware (to replace the PC) there isn’t much that can be done about this. If you are brave, you can experiment with increasing program priority in your Task Manager window.

Chart refreshes are most often initiated by an arriving quote, but they can also be initiated by mouse clicks, tooltip, and various chart operations. This means that, when market activity is low and things are not happening as fast as you would like, you can force extra AFL executions by clicking on your chart. You can verify this by running the code below and, while clicking rapidly on the chart, observe that the timer counts displayed will update more rapidly.

You can ensure a one-second chart refresh by adding a RequestTimedRefresh(1) to your code. If the frequency of your arriving data is slow, your AFL code may execute only sporadically. Since your code must execute to read your timers, the resolution of your timers will be limited by the chart refresh rate. If your chart refreshes once a second your timing resolution will be one second!

Normally most of the AFL code in an Indicator window executes when your chart refreshes; however, to obtain speed advantages, you may execute non-critical sections (like account information and System Status) of your code less frequently using a timer. You can also execute small sections of code more frequently by placing them inside a well-controlled loop. If you do this, be sure to limit the maximum time your code can spend inside the loop to one second or less.

For fast trading systems, the frequency of AFL executions (chart refreshes) may be slow and this may make it difficult for you to get LMT fills. There is no way to have a program that requires 50 milliseconds per pass to execute 20 times per second.

Considering the interval between AFL executions, it is important to plan the layout of your code so that all events are handled in the most efficient order. If you don’t, transmittance of your order could well be delayed by up to a full second. There are situations where you want to invoke an immediate re-execution of your code. In some cases you might want to do this after placing an order to check order status before the next quote or refresh. Although it should be used sparingly this is possible by calling the RefreshAll():


<p>This function can only be called once a secondcalling it faster will not result in more frequent chart refreshesThis means you should only call it when really needed. <p>The code presented below is for demonstration onlyThe getElapsedTime() lets you measure elapsed time from the moment of ResetThe first argument passes the name you assign to the static timerthis allows you to use the same function to time different eventsThe second argument is a Reset flagWhen this Reset is Truethe function samples the underlying high frequency counter and uses it for later referenceWhen you call the function with the Reset argument set to Falseit calculates the elapsed time by subtracting the earlier sampled value from the current value of the Performance Counter. <p>The setDelay() function lets you StartRead, and Cancel a time delayThe TimerName argument functions as in the getElapsedTime(). Calling the setDelay() with the mSecDelay argument set to a non-zero value will start the Delay timerCalling it with the mSecDelay argument set to zero will make it return the current count-down time in millsecondsCalling the function with the cancel argument set to True will terminate the delay. <p>The getDelayTrigger() function returns a triggerThis is a signal that is true for only one pass through the codeTriggers are frequently used in real-time trading systemsThey are needed to prevent multiple actions when a signal becomes True. <p>To run the codecopy the formula to an Indicator and click InsertYou'll see a chart window like Figure 1 below: 

<p align="center">Figure 1. Result from running the example code. 
<a href='http://www.amibroker.org/userkb/2007/11/10/high-precision-delay-and-interval-timing/timerdisplayjpg/' rel='attachment wp-att-1422' title='timerdisplay.jpg'><img src='http://www.amibroker.org/userkb/wp-content/uploads/2007/11/timerdisplay.jpg' alt='timerdisplay.jpg' /></a>

<p>The example code maintains three timersT1T2 and T3All timing values are expressed in millisecondsIn Figure-1 the Elapsed Time shown is measured from timer ResetThe Delay shown is the time remaining after Startuntil the delay times outThe line for Timer T2 shows that its Delay just timed-out and produced a triggerTimer T3 still has a Delay in progressRight-click on the chart to open the Param window: 

<p align="center">Figure 2. Param window. 
<a href='http://www.amibroker.org/userkb/2007/11/10/high-precision-delay-and-interval-timing/timerparampng/' rel='attachment wp-att-1423' title='timerparam.png'><img src='http://www.amibroker.org/userkb/wp-content/uploads/2007/11/timerparam.png' alt='timerparam.png' /></a>

<p>If you click one of the timer Resets in the Param window you'll see the ElapsedTime in the corresponding row go to zero, and then start to increment sporadically when your chart refreshes. Without live data this would be at approximately 1-second intervals, as determined by the RequestTimedRefresh(1); <p>If you click Start for one of the timers this will start a delay. You can see how it counts down in the Delay column. Click the timer's Cancel to terminate the DelayNote that whenever a Delay times outthe word "Trigger" briefly appears in the third column. 


function RefreshAll()
    {
    oAB CreateObject("Broker.Application");
    oAB.RefreshAll();
    }

function getElapsedTimeTimerNameReset )
    {
    if( Reset ) 
        {
        TimeRefGetPerformanceCounter(False);
        StaticVarSet(TimerName,TimeRef);
        }
    TimeRef Nz(StaticVarGet(TimerName));
    ElapsedTime GetPerformanceCounter(False) - TimeRef;
    return ElapsedTime;
    }

function setDelayTimerNameMsecDelayCancel )
    {
    HRCounterGetPerformanceCounter(False);
    if( Cancel ) 
        {
        StaticVarSet("TO"+TimerName,-1);
        StaticVarSet("DS"+TimerName0);
        }
    else if( MsecDelay ) 
        {
        StaticVarSet("TO"+TimerName,HRCounter+MsecDelay );
        }
    TimeOutTime Nz(StaticVarGet("TO"+TimerName));
    DelayCount Max(0TimeOutTime HRCounter );
    StaticVarSet("DC"+TimernameDelayCount);
    return DelayCount;
    }

function getDelayTriggerTimerName )
    {
    DelayCount Nz(StaticVarGet("DC"+TimerName));
    DelayState DelayCount 0;
    PrevDelayState Nz(StaticVarGet("DS"+TimerName));
    StaticVarSet("DS"+TimerNameDelayState);
    return DelayState PrevDelayState;
    }
 
RequestTimedRefresh1);

_SECTION_BEGIN("TIMER 1");
Reset1 ParamTrigger("1 - Reset","RESET");
MSecDelay1 Param("1 - Delay (mS)",1000,0,10000,10);
if( ParamTrigger("1 - Start""START") ) setDelay"Timer1"MSecDelay1);
if( ParamTrigger("1 - Cancel""CANCEL") ) setDelay"Timer1"MSecDelay1);
_SECTION_END();

_SECTION_BEGIN("TIMER 2");
Reset2 ParamTrigger("2 - Reset","RESET");
MSecDelay2 Param("2 - Delay (mS)",4000,0,10000,10);
if( ParamTrigger("2 - Start""START") ) setDelay"Timer2"MSecDelay2);
if( ParamTrigger("2 - Cancel""CANCEL") ) setDelay"Timer2"MSecDelay2);
_SECTION_END();

_SECTION_BEGIN("TIMER 3");
Reset3 ParamTrigger("3 - Reset","RESET");
MSecDelay3 Param("3 - Delay (mS)",8000,0,10000,10);
if( ParamTrigger("3 - Start""START") ) setDelay"Timer3"MSecDelay3);
if( ParamTrigger("3 - Cancel""CANCEL") ) setDelay"Timer3"MSecDelay3);
_SECTION_END();

ET1     getElapsedTime"Timer1"Reset1 );
ETA1     setDelay"Timer1"0);
TT1     getDelayTrigger"Timer1");

ET2     getElapsedTime"Timer2"Reset2 );
ETA2     setDelay"Timer2"0);
TT2     getDelayTrigger"Timer2");

ET3     getElapsedTime"Timer3"Reset3 );
ETA3     setDelay"Timer3"0);
TT3     getDelayTrigger"Timer3");

Title "\nFilename: "+Filename+"\n\n"+
"MilliSeconds Since Computer Startup: "+NumToStr(GetPerformanceCounter(False),1.3)+"\n\n"+
"Timer     ElapsedTime     Delay     Trigger\n\n"+
"T1      "+NumToStr(ET1,10.0)+"     "+NumToStr(ETA1,7.0)+"      "+WriteIf(TT1,"TRIGGER1","")+"\n\n"+
"T2      "+NumToStr(ET2,10.0)+"     "+NumToStr(ETA2,7.0)+"      "+WriteIf(TT2,"TRIGGER2","")+"\n\n"+

Edited by Al Venosa.

1 Star2 Stars3 Stars4 Stars5 Stars (2 votes, average: 3.50 out of 5)
Loading...Loading...

Backfill Exploration for Real-Time data

Backfilling data can be done in many ways, and different methods may be needed for different data providers. The method shown below was developed for eSignal data. If you use another data provider, you may need to modify the code and the procedure.

When backfilling data, you should confirm that data are backfilled properly and have some indication regarding the presence of data holes. As pointed out in Data Holes in Real-Time Trading, to be able to detect holes, you need a perfect data array against which to compare the data. Since AmiBroker doesn’t have such a data array, the method presented here uses the QQQQ as a reference ticker. Turn ON and set Backtester Settings -> General -> Pad and align all data to reference symbol -> QQQQ. To have all tickers backfilled, you should also turn ON Wait for Backfill(RT only). To speed up real-time backfills you may want to display only a simple price chart, instead of complex code. Refreshing complex Indicators or Systems will slow down backfill. The following Exploration will Backfill all tickers in your Watchlist and report on it’s success:

1 Star2 Stars3 Stars4 Stars5 Stars (2 votes, average: 3.50 out of 5)
Loading...Loading...

Keeping Your Chart Right-Justified

In real-time trading it is vital to keep your chart right justified, i.e., ensuring the right-most price bar is visible at all times. Failing to do so and looking at the wrong bar when trading real-money can invoke a panic response by the trader that results in erroneously closing a position or worse, placing a wrong order. This can easily happen if you are running several computers and are not always monitoring the chart being traded.

The following code shows you how you can right-justify your chart at system startup or from the Param window. A visible alert that changes the chart back ground color when the last bar is not visible is included.

The Alarm uses DateTime() to automatically correct for blank bars at the right edge of your chart. The code is kept explicit and you may be able to optimize it for speed.

<p>

Edited by Al Venosa.

1 Star2 Stars3 Stars4 Stars5 Stars (3 votes, average: 4.00 out of 5)
Loading...Loading...

Measuring AFL Execution Times (v3)

When your code increases in complexity, it will start to execute more slowly, and you will eventually reach a point where you will want to optimize your code for speed of execution. Even if you are willing to buy a faster computer, it will rarely give you a greater speed advantage than optimizing your code.

Important note: The AmiBroker 5.01.1 beta released on 5/10/2007 offers a new profiling function that enables you to get accurate performance timing for all AmiBroker and custom functions called. It is available from the AFL Editor: Tools -> Code Check and Profile menu. To use this new tool effectively you should convert your code modules to function calls as much as possible. Try it!

The first requirement for analyzing your AFL execution speed is the ability to measure the execution times. This is best done using the GetPerformanceCounter(), which gives microsecond resolution. You can display the execution time in the chart Title, or log them to DebugView. To get an appreciation of how fast AFL executes you may want to time a few of your most frequently used AFL statements. As Tomasz pointed out in his comment to an earlier version of this post, this can be done using a simple loop:


<p>On my 2GHz laptopthe Param() takes about 4 microseconds to executeThe "Statement under test" can be any AFL statement or section of code you want to analyzeThis simple code will give you a fairly accurate idea of the true execution timePlease do not forget that all&nbsp;programs executing under Microsoft Operating Systems&nbsp;are subject to interruptionsThis means that your readings may fluctuate and will generally be greater than their absolute true valuesYour normal program executions are&nbsp;also subject to these interruptions and, if you perform&nbsp;precision timing tasksyour should consider their effect.</p> 

<p>If, in view of processing overheadyou prefer not to use DebugViewyou can log execution times either to the Interpretation screenwhich invokes an additional AFL execution, or to the chart Title if you output just a few measurements. Use of DebugView is outlined in several posts in the <a href="http://www.amibroker.org/userkb/category/real-time-auto-trading/debugging-afl/">Debugging AFL</acategoryWhen you are logging to DebugViewexecution times will be inflated by the amount of time needed to call and execute the _Trace() functionsYou can measure the timing overhead for a single&nbsp;_Trace()&nbsp;function using the same method we used above:</p> 


<p>On my 2GHz laptopthe _Trace() takes about 35 microseconds to executeCompared to the Param(), this would appear like a major errorbut in practice it is not. For examplewhen calling the&nbsp;_Trace() function thirty times in your codethis would add only about 1 millisecond to the overall execution time of your program. </p> 

<p>The example code below logs to DebugView and to the chart TitleIt uses a small Trace() function that is called each time you want to log a timing measurementYou can customize this function to log additional informationAlthough you can call this function on each lineyou do not have toCalling Trace() introduces a small measurement errorThis means that when you remove the measurement codeperformance should be slightly better than your measurements predict. If you do not use DebugViewyou can delete all DebugView related statements. If you do thisI suggest you comment them out until your modified code has been verified.</p> 

<p>Be careful when calling the Trace() from inside a loop. If your loop steps through 50,000 bars of datainserting one or more Trace() calls will slow down your code substantially and result in a significant measurement errorThe key to reasonable accuracy is to use the minimum number of _Trace() statements and keep them out of loops and frequently called functions.</p> 

<p>Inserting/removing TRACE() statements in a test program is easily done using the EditReplace function in the formula editorTo add _Trace() statementsreplace the ";" at the end of the line with "; TRACE("");":</p> <p align="center"><img height="178" src="http://www.amibroker.org/userkb/wp-content/uploads/2007/09/clip-image0023.jpg" width="351" border="0"></p> 

<p>To remove the _Trace() statements substitute "; TRACE("");" with ";":</p> <p align="center"><img height="178" src="http://www.amibroker.org/userkb/wp-content/uploads/2007/09/clip-image0043.jpg" width="351" border="0"></p> 

<p>Place your cursor just before the last ";" that precedes the first line of code you want to timeStop substitutions where you no longer want to time statementsYou should initialize the LineNumber to the line number that contains your first Trace() statementLine numbers for your cursor position are displayed at the right bottom of your editor screen.</p> 

<p>Here is a do-nothing program where _Trace() statements were added using the above methodYou can modify the function Trace() to output any information you like: 

<p>function TraceComment )</p> <p>{</p> <p>global LineNumberTitleStr;</p> <p>PC GetPerformanceCounterTrue );</p> <p>Str "# LN: "+NumToStr(LineNumber++,1.0,False)+", PC: "+NumToStr(PC,9.3)+" mSec., "+Comment;</p> <p>TitleStr TitleStr Str+"\n";</p> <p>_TRACE(Str);</p> <p>}</p> 
<p>

The output produced in DebugView looks like this:

dbvmeasurement.png

The output in the Title looks like this:

titlemeasurement.png

The first reading in the Title should be ignored as it initializes the PerformanceCounter. This reading is suppressed by the DBViewClear command in the DebugView log. The above method can be used to determine execution times of all AFL functions. Note that in the above example SetBarsRequired() is used to process all available data. This would be the worst-case measurement. Commenting out SetBarsRequired() will make your code execute under Quick-AFL. This may be significantly faster than running over all data.


Measuring Execution Order and Timing for Window Panes

When you are executing AFL formulas in multiple panes or windows and experience a timing problem, the first step to take, before using the above method, is to determine which pane uses up too much time. You can turn on Tools -> Preferences ->Miscelaneous -> Display Chart Timing to read the execution time at the bottom of the charts. Alternatively, the simple code below lets you log the execution order and time for the formulas in each open pane to DebugView. Since the AmiBroker Performance Counter readout is unique for each pane, the code below does not use it. In this case you will have to turn on Options -> Show Milliseconds in DebugView. This will give you Millisecond resolution, not micro-second as with the performance Counter and is a very approximate measurement. It should only be used to identify the problem formula. The execution order of the panes is revealed by the order in which the lines appear in the DebugView.

Filename StrLeft(_DEFAULT_NAME(),StrLen(_DEFAULT_NAME())-2);
function TraceChartID() 
{ 
Msg "# ChartID, "+NumToStr(GetChartID(),1.0,False)+", "+Filename;
_TRACEMsg ); 
return Msg;
} 

TraceChartID();

//
for(n=0n<BarCount;n++) ; // your code would be here
//

Title TraceChartID(); 

The Filename assignment and the TraceChartID() should be placed at the very top of your code, or you can add it to your default Include file. The single line used to call this function should be placed at both the beginning and the end of the code in each window/pane. This is what it looks like in DebugView:

dbvtrace.png

Edited by Al Venosa.

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...Loading...

Real-Time Delays

In real-time trading many situations arise when you want to delay action until a specific criterion is met. In AmiBroker you can base your delays in many different ways, the only requirement being that the delay variable increments or decrements. If the selected variable doesn’t revert towards your timeout value (target), your delay function would never time out. In this case you would have to add code to handle that condition. A few variables you might use are:

– RT TimeNumber (Now(4)).
– Elapsed Seconds (redrawaction).
– Real-time Data time-stamp (TimeNum()).
– Tick-count (New Data).
– Volume (Change in Volume).
– Price change (Change in Price).
– Chart Refresh (any AFL Execution).
– Indicator values.

Which of the above variables you would use for your delay depends on the requirements of your trading system. There may be times when you may need to combine several methods to get the required results. For example, if a delay were based on the data-timestamp, it might not time out during a data dropout or a no-trading period. In this case you need to back up your data-timestamp delay with a real-time (seconds) delay.

Delays play a critical role in real-time system design. For example, in real-time systems, signals may have a short lifetime. The signal is strongest when it triggers and than quickly decays until, perhaps after a few bars, it has lost all significance. Letting the order fill at the time when it has lost significance is pure gambling. To prevent this, you can cancel the order after a delay, or decrease the position size proportional to the perceived decay in signal strength (perhaps based on elapsed bars?)

Since in a real-time system the time-lapse between AFL executions can be significant, you should place your LMT price calculations ahead of the ordering code. Calculating the LMT price after the order has been placed postpones order placement until the next AFL execution occurs, i.e., when the next quote arrives; by then, the price has probably changed. Especially during periods of low volume, this could be significant. When these delays would be insignificant in EOD systems, they could make-or-break your system in fast-trading systems.

To ensure frequent AFL execution in the absence of quotes, you can place a RequestTimedRefresh(1) statement at the top of your code, where the variable ‘1’ refers to a 1-second refresh. This guarantees an AFL execution at least once per second.

If your code is lengthy and takes a significant amount of time to execute, you may have to check order status at several places in your code. If changed status demands immediate action, you can force an immediate AFL execution by calling the following function:

1 Star2 Stars3 Stars4 Stars5 Stars (2 votes, average: 3.50 out of 5)
Loading...Loading...

Preventing Repeat Orders and Whipsaws

1 Star2 Stars3 Stars4 Stars5 Stars (10 votes, average: 4.60 out of 5)
Loading...Loading...
Next Page »