Users' Knowledge Base

Please be sure you set up the TWS according to the instructions provided in Setting Up your TWS before you test this code. The code presented here reads the execution report, converts it to a .csv format, date-stamps it, backs it up for later use, and optionally displays it in the chart Title. The code doesn’t do anything important besides displaying the information in the Title. The idea is to show you how to read the file so that you can extract real execution prices, use them in your calculations, and plot them on your chart. The Param options are self-explanatory:

The name used for the execution report generated by the TWS is not date-stamped. For example, if you set up the TWS to export executions under the name Simulated.Trades, this same name will be used on successive days. If the TWS finds a tradelist from the previous day, it will simply overwrite it. To prevent losing this AFL readable file it is important to back up the tradelist at the end of the day. The format of the execution report exported by the TWS looks like this:

ATVI;SLD;75;26.19;21:29:33;20080125;ARCA;DU1195;;;DEMO;
ALTR;BOT;100;18.54;21:53:12;20080125;ARCA;DU1195;;;DEMO;
ALTR;BOT;100;18.58;21:55:59;20080125;ISLAND;DU1195;;;DEMO;
ALTR;BOT;100;18.55;21:56:00;20080125;ARCA;DU1195;;;DEMO;
ALTR;BOT;100;18.58;21:58:47;20080125;ISLAND;DU1195;;;DEMO;

The .csv format of the backup file produced by the code below can be directly imported into Excel and looks like this (note the semicolons have been replaced by commas):

ATVI,SLD,75,26.19,21:29:33,20080125,ARCA,DU1195,,,DEMO,
ALTR,BOT,100,18.54,21:53:12,20080125,ARCA,DU1195,,,DEMO,
ALTR,BOT,100,18.58,21:55:59,20080125,ISLAND,DU1195,,,DEMO,
ALTR,BOT,100,18.55,21:56:00,20080125,ARCA,DU1195,,,DEMO,
ALTR,BOT,100,18.58,21:58:47,20080125,ISLAND,DU1195,,,DEMO,

In Excel, the file will look like this after activating Text to columns:

Please be aware that the minimum update interval that the TWS exports the execution report is approximately one-minute. This means it will take some time for your trades to show up in the list.

Before tackling the main backup function, there are a few helper functions you will need. While these are available elsewhere on this site, they are repeated below for your convenience. To prevent conflicts between static variables used in different programs, you should key their names with those charted; see Keying Static Variables for more information on this. The DateNumToStr() converts DateNumbers to a standard date string.

The TWSBackupTradeList( TWSInputPath ) listed below reads the TWS tradelist, extracts the date, converts it to the .csv format, saves it in a different location, and optionally displays both tradelists in the chart Title. To test this function, Apply it to a new Indicator, open the Param window, set up the parameters, and click BACKUP. The backup file is saved in the path defined by the TradebackupFolder variable. If the function finds the execution report and its display is turned on in the Param window, this should look like this in the Title (only a few lines shown):

And, when displayed, the backup file should look like that below:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138

 
InIndicator 	= Status( "Action" ) == 1;
StaticVarKey 	= GetChartID();
 
procedure xStaticVarSet( SName, SValue )
{
    global StaticVarKey;
 
    if ( InIndicator )
        StaticVarSet( Sname + StaticVarKey, Svalue );
}
 
function xStaticVarGet( SName )
{
    global StaticVarKey;
    return StaticVarGet( Sname + StaticVarKey );
}
 
procedure xStaticVarSetText( SName, SValue )
{
    global StaticVarKey;
 
    if ( InIndicator )
        StaticVarSetText( Sname + StaticVarKey, Svalue );
}
 
function xStaticVarGetText( SName )
{
    global StaticVarKey;
    return StaticVarGetText( Sname + StaticVarKey );
}
 
function DateNumToStr( DtNum )
{
    DayNm = round( frac( DtNum / 100 ) * 100 );
    MthNm = round( frac( DtNum / 10000 ) * 100 );
    YrNm = int( DtNum / 10000 ) + 1900;
    return NumToStr( MthNm, 1.0 ) + "/" + NumToStr( DayNm, 1.0 ) + "/" + NumToStr( YrNm, 1.0, False );
}
 
procedure TWSBackupTradeList( TWSInputPath )
{
    global TradebackupFolder, AccountType;
    fh1 = fopen( TWSInputPath, "r" );
 
    if ( fh1 )
    {
        Line 				= StrReplace( fgets( fh1 ), ";", "," );
        DateStr 			= StrExtract( Line, 5 );
        YearNum			= StrToNum( StrLeft( DateStr, 4 ) );
        MonthNum			= StrToNum( StrMid( DateStr, 4, 2 ) );
        DayNum			= StrToNum( StrRight( DateStr, 2 ) );
        DateNumber 		= ( YearNum - 1900 ) * 10000 + 100 * MonthNum + DayNum;
        DateNumStr 		= NumToStr( DateNumber, 1.0, False );
        BackupFilename = AccountType + DateNumStr + ".csv";
        BackupPath 		= TradebackupFolder + BackupFilename;
        fclose( fh1 );
    }
 
    fh1 = fopen( TWSInputPath, "r" );
 
    fdelete( BackupPath );
    fh2 = fopen( BackupPath, "a" );
    LineNum = 0;
    TWSTradeList = CSVTradelist = "";
 
    if ( fh1 )
    {
        if ( fh2 )
        {
            while ( ! feof( fh1 ) )
            {
                Line = fgets( fh1 );
                TWSTradeList = TWSTradeList + Line;
                Line = StrReplace( Line, ";", "," );
                CSVTradelist = CSVTradelist + Line;
                LineNum++;
 
                if ( Line != "" )
                {
                    fputs( Line, fh2 );
                }
            }
        }
 
        xStaticVarSetText( "TWSTradelist", TWSTradelist );
 
        xStaticVarSetText( "CSVTradelist", CSVTradelist );
    }
    else
    {
        if ( fh1 == 0 )
        {
            PopupWindow( "Could NOT Open InputPath: " + TWSInputPath,
                         "TWS EXPORTED TRADELIST", timeout = 5, left = -1, top = -1 );
        }
 
        if ( fh2 == 0 )
        {
            PopupWindow( "Could not open OutputPath: " + OutputPath,
                         "TWS EXPORTED TRADELIST", timeout = 5, left = -1, top = -1 );
        }
    }
 
    if ( fh1 )
        fclose( fh1 );
 
    if ( fh2 )
        fclose( fh2 );
 
    Caption = "TWS EXPORTED TRADELIST";
 
    Message = "The TWS Tradelist: \n   " + TWSInputPath + " [" + NumToStr( LineNum, 1.0, False ) +
              " Trades/" + DateNumToStr( DateNumber ) + "]" +
              " \nHas been saved in csv format as:\n   " + BackupPath;
 
    PopupWindow( Message, Caption, timeout = 20, left = -1, top = -1 );
}
 
_SECTION_BEGIN( "BACKUP TWS TRADELIST" );
TWSInputPath				= ParamStr( "TWS Tradelist (Folder)", "C:\\Jts\\" );
AccountType					= ParamList( "TWS Account Type (Filename)", "Real|Simulated|Demo", 1 );
TWSInputFilename			= AccountType + ".Trades";
TWSInputPath 				= TWSInputPath + TWSInPutFilename;
TradebackupFolder			= ParamStr( "Backup Destination Folder", "C:\\Jts\\TWSTrades\\" );
BackupTWSTradeList		= ParamTrigger( "Create Backup Tradelist", "BACKUP" );
ShowTWSTradeList			= ParamToggle( "TWS Tradelist", "HIDE|SHOW", 1 );
ShowCSVTradeList			= ParamToggle( "CSV Tradelist", "HIDE|SHOW", 1 );
_SECTION_END();
 
if ( BackupTWSTradeList )
    TWSBackupTradeList( TWSInputPath );
 
TWSStr = WriteIf( ShowTWSTradeList, "\nTWS Exported Tradelist: \n" + xStaticVarGetText( "TWSTradelist" ) + "\n", "" );
 
CSVStr = WriteIf( ShowCSVTradelist, "\nCSV Exported Tradelist: \n" + xStaticVarGetText( "CSVTradelist" ), "" );
 
Title = TWSStr + CSVStr;

Edited by Al Venosa.

Filed by Herman at 2:48 pm under Real-Time AFL Programming
No Comments

(1 votes, average: 3 out of 5)

 Loading ...

When trading in real-time, one often needs to know when a new period starts and how much time there is left before the period ends. The code below will give you the time remaining to the next bar, the time elapsed since the start of the bar, and the second count since date-change. The timing values will automatically adjust to the selected chart interval. You can use the variables in your system’s code to time various events.

This code is shown in a demo configuration, and you will have to adapt it to your personal requirements. To test, simply Apply the code to an Indicator window. For a quick test, select the one-minute time interval.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

function GetSecondNum()
{
    Time 		= Now( 4 );
    Seconds 	= int( Time % 100 );
    Minutes 	= int( Time / 100 % 100 );
    Hours 	= int( Time / 10000 % 100 );
    SecondNum = int( Hours * 60 * 60 + Minutes * 60 + Seconds );
    return SecondNum;
}
 
RequestTimedRefresh( 1 );
TimeFrame = Interval();
SecNumber = GetSecondNum();
Newperiod = SecNumber % TimeFrame == 0;
SecsLeft	 = SecNumber - int( SecNumber / TimeFrame ) * TimeFrame;
SecsToGo	 = TimeFrame - SecsLeft;
 
if ( NewPeriod )
{
    Say( "New period" );
    Plot( 1, "", colorYellow, styleArea | styleOwnScale, 0, 1 );
}
 
Title = "\n" +
 
        "  Current Time: " + Now( 2 ) + "\n" +
        "Chart Interval: " + NumToStr( TimeFrame, 1.0 ) + " Seconds\n" +
        " Second Number: " + NumToStr( SecNumber, 1.0, False ) + "\n" +
        "  Seconds Left: " + NumToStr( SecsLeft, 1.0, False ) + "\n" +
        " Seconds To Go: " + NumToStr( SecsToGo, 1.0, False );
 
Plot( C, "", 1, 128 );

For verification, timing is displayed in the chart title:

Edited by Al Venosa.

Filed by Herman at 2:31 pm under Real-Time AFL Programming
2 Comments

(2 votes, average: 4 out of 5)

 Loading ...

In real-time trading one often needs to know when a new period starts and how much time there is left before the period ends. The code below provides this information. Be sure to synchronize your system’s clock.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

function GetSecondNum() 
{ 
Time = Now(4); 
Seconds = int(Time%100); 
Minutes = int(Time/100%100); 
Hours = int(Time/10000%100); 
SecondNum= int(Hours*60*60+Minutes*60+Seconds); 
return SecondNum; 
} 
 
RequestTimedRefresh(1); 
TimeFrame = Interval(); 
SecNumber = GetSecondNum(); 
Newperiod = SecNumber%TimeFrame == 0; 
SecsLeft = SecNumber-int(SecNumber/TimeFrame)*TimeFrame; 
SecsToGo = TimeFrame - SecsLeft; 
if( NewPeriod )  
{ 
Say("New period"); 
Plot(1,"",colorYellow,styleArea|styleOwnScale,0,1); 
} 
Title = "\n"+ 
"time: "+Now(2)+"\n"+ 
"Interval: "+NumToStr(TimeFrame,1.0)+"\n"+ 
"Second Number: "+NumToStr(SecNumber,1.0,False)+"\n"+ 
"Seconds Left: "+NumToStr(SecsLeft,1.0,False)+"\n"+ 
"Seconds To Go: "+NumToStr(SecsToGo,1.0,False);

For testing and code verification timing is displayed in the chart title:

Filed by Herman at 12:42 pm under Real-Time AFL Programming
1 Comment

(5 votes, average: 4.4 out of 5)

 Loading ...

Before continuing with this post you should carefully read the AmiBroker Help topic for the getPerformanceCounter().

Measuring time is an important aspect of all real-time intraday trading systems. Typical tasks requiring high-resolution timing include:

1. Limiting the message rate to Interactive Brokers (IB) to 50/sec (API Error 100).
2. Inserting small delays before polling IB Status, to allow for Internet delays.
3. Staggering (interlacing) portfolio trades to spread out the action.
4. Measuring and optimizing AFL execution time.
5. Modifying orders after a small delay (to ensure fills).
6. Periodic execution of tasks, for example, Watchlist scanning, Display and Status refresh, calculations based on slow changing variables, etc.
7. Time-Stamping events, for example order placement.
8. Collecting/preprocessing quotes.
9. Overlay live tick-charts on faster and easier to manage 1-minute charts.

Most of these tasks can be accomplished using just three custom timing functions:

1. GetElapsedTime(): A function that returns elapsed time since reset.
2. SetDelay(): A function that returns time left to reach a future time.
3. GetDelayTrigger(): A function that returns a trigger when a delay times out.

This post provides example functions in a demo application. To allow the use of many timers each function requires you to provide a TimerName, which will be used to retrieve timer information. Static variables are Global and can be read from anywhere; this means you have to be careful not to cross-reference the timers by using the same TimerName from different panes or windows. When running multiple copies of the same code you will need to key the TimerNames. For more on how to do this, see Keying Static Variables in the Real-Time AFL Programming category.

The timers below are implemented using the getPerformanceCounter(). This function returns the amount of time elapsed since the computer was last started. Tomasz recently explained this as follows: “The underlying high frequency counter runs all the time since computer start. What ‘reset’ flag really does is to store last value so next time you read it, it gets subtracted from last value giving you the difference. If reset is false, the last value is set to zero, and you get the original number of ‘clock ticks’ since computer start“.

The timers in this post do their own sampling of the underlying high frequency counter, and the getPerformanceCounter() Reset argument is always left set to False.

The getPerformanceCounter() returns values with microsecond resolution; however, the practical accuracy is severely limited by interruptions from the computer’s operating system. Do not expect much better than about 50-millisecond absolute accuracy. Aside from designing your own dedicated trading hardware (to replace the PC) there isn’t much that can be done about this. If you are brave, you can experiment with increasing program priority in your Task Manager window.

Chart refreshes are most often initiated by an arriving quote, but they can also be initiated by mouse clicks, tooltip, and various chart operations. This means that, when market activity is low and things are not happening as fast as you would like, you can force extra AFL executions by clicking on your chart. You can verify this by running the code below and, while clicking rapidly on the chart, observe that the timer counts displayed will update more rapidly.

You can ensure a one-second chart refresh by adding a RequestTimedRefresh(1) to your code. If the frequency of your arriving data is slow, your AFL code may execute only sporadically. Since your code must execute to read your timers, the resolution of your timers will be limited by the chart refresh rate. If your chart refreshes once a second your timing resolution will be one second!

Normally most of the AFL code in an Indicator window executes when your chart refreshes; however, to obtain speed advantages, you may execute non-critical sections (like account information and System Status) of your code less frequently using a timer. You can also execute small sections of code more frequently by placing them inside a well-controlled loop. If you do this, be sure to limit the maximum time your code can spend inside the loop to one second or less.

For fast trading systems, the frequency of AFL executions (chart refreshes) may be slow and this may make it difficult for you to get LMT fills. There is no way to have a program that requires 50 milliseconds per pass to execute 20 times per second.

Considering the interval between AFL executions, it is important to plan the layout of your code so that all events are handled in the most efficient order. If you don’t, transmittance of your order could well be delayed by up to a full second. There are situations where you want to invoke an immediate re-execution of your code. In some cases you might want to do this after placing an order to check order status before the next quote or refresh. Although it should be used sparingly this is possible by calling the RefreshAll():

1
2
3
4
5

function RefreshAll() 
{ 
oAB = CreateObject("Broker.Application"); 
oAB.RefreshAll(); 
}

This function can only be called once a second; calling it faster will not result in more frequent chart refreshes. This means you should only call it when really needed.

The code presented below is for demonstration only. The getElapsedTime() lets you measure elapsed time from the moment of Reset. The first argument passes the name you assign to the static timer; this allows you to use the same function to time different events. The second argument is a Reset flag. When this Reset is True, the function samples the underlying high frequency counter and uses it for later reference. When you call the function with the Reset argument set to False, it calculates the elapsed time by subtracting the earlier sampled value from the current value of the Performance Counter.

The setDelay() function lets you Start, Read, and Cancel a time delay. The TimerName argument functions as in the getElapsedTime(). Calling the setDelay() with the mSecDelay argument set to a non-zero value will start the Delay timer. Calling it with the mSecDelay argument set to zero will make it return the current count-down time in millseconds. Calling the function with the cancel argument set to True will terminate the delay.

The getDelayTrigger() function returns a trigger. This is a signal that is true for only one pass through the code. Triggers are frequently used in real-time trading systems. They are needed to prevent multiple actions when a signal becomes True.

To run the code, copy the formula to an Indicator and click Insert. You’ll see a chart window like Figure 1 below:

Figure 1. Result from running the example code.

The example code maintains three timers, T1, T2 and T3. All timing values are expressed in milliseconds. In Figure-1 the Elapsed Time shown is measured from timer Reset. The Delay shown is the time remaining after Start, until the delay times out. The line for Timer T2 shows that its Delay just timed-out and produced a trigger. Timer T3 still has a Delay in progress. Right-click on the chart to open the Param window:

Figure 2. Param window.

If you click one of the timer Resets in the Param window you’ll see the ElapsedTime in the corresponding row go to zero, and then start to increment sporadically when your chart refreshes. Without live data this would be at approximately 1-second intervals, as determined by the RequestTimedRefresh(1);

If you click Start for one of the timers this will start a delay. You can see how it counts down in the Delay column. Click the timer’s Cancel to terminate the Delay. Note that whenever a Delay times out, the word “Trigger” briefly appears in the third column.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88

Filename = StrLeft(_DEFAULT_NAME(),StrLen(_DEFAULT_NAME())-2);
 
function RefreshAll()
	{
	oAB = CreateObject("Broker.Application");
	oAB.RefreshAll();
	}
 
function getElapsedTime( TimerName, Reset )
	{
	if( Reset ) 
		{
		TimeRef= GetPerformanceCounter(False);
		StaticVarSet(TimerName,TimeRef);
		}
	TimeRef = Nz(StaticVarGet(TimerName));
	ElapsedTime = GetPerformanceCounter(False) - TimeRef;
	return ElapsedTime;
	}
 
function setDelay( TimerName, MsecDelay, Cancel )
	{
	HRCounter= GetPerformanceCounter(False);
	if( Cancel ) 
		{
		StaticVarSet("TO"+TimerName,-1);
		StaticVarSet("DS"+TimerName, 0);
		}
	else if( MsecDelay ) 
		{
		StaticVarSet("TO"+TimerName,HRCounter+MsecDelay );
		}
	TimeOutTime = Nz(StaticVarGet("TO"+TimerName));
	DelayCount = Max(0, TimeOutTime - HRCounter );
	StaticVarSet("DC"+Timername, DelayCount);
	return DelayCount;
	}
 
function getDelayTrigger( TimerName )
	{
	DelayCount = Nz(StaticVarGet("DC"+TimerName));
	DelayState = DelayCount > 0;
	PrevDelayState = Nz(StaticVarGet("DS"+TimerName));
	StaticVarSet("DS"+TimerName, DelayState);
	return DelayState < PrevDelayState;
	}
 
RequestTimedRefresh( 1);
 
_SECTION_BEGIN("TIMER 1");
Reset1 = ParamTrigger("1 - Reset","RESET");
MSecDelay1 = Param("1 - Delay (mS)",1000,0,10000,10);
if( ParamTrigger("1 - Start", "START") ) setDelay( "Timer1", MSecDelay1, 0 );
if( ParamTrigger("1 - Cancel", "CANCEL") ) setDelay( "Timer1", MSecDelay1, 1 );
_SECTION_END();
 
_SECTION_BEGIN("TIMER 2");
Reset2 = ParamTrigger("2 - Reset","RESET");
MSecDelay2 = Param("2 - Delay (mS)",4000,0,10000,10);
if( ParamTrigger("2 - Start", "START") ) setDelay( "Timer2", MSecDelay2, 0 );
if( ParamTrigger("2 - Cancel", "CANCEL") ) setDelay( "Timer2", MSecDelay2, 1 );
_SECTION_END();
 
_SECTION_BEGIN("TIMER 3");
Reset3 = ParamTrigger("3 - Reset","RESET");
MSecDelay3 = Param("3 - Delay (mS)",8000,0,10000,10);
if( ParamTrigger("3 - Start", "START") ) setDelay( "Timer3", MSecDelay3, 0 );
if( ParamTrigger("3 - Cancel", "CANCEL") ) setDelay( "Timer3", MSecDelay3, 1 );
_SECTION_END();
 
ET1 	= getElapsedTime( "Timer1", Reset1 );
ETA1 	= setDelay( "Timer1", 0, 0 );
TT1 	= getDelayTrigger( "Timer1");
 
ET2 	= getElapsedTime( "Timer2", Reset2 );
ETA2 	= setDelay( "Timer2", 0, 0 );
TT2 	= getDelayTrigger( "Timer2");
 
ET3 	= getElapsedTime( "Timer3", Reset3 );
ETA3 	= setDelay( "Timer3", 0, 0 );
TT3 	= getDelayTrigger( "Timer3");
 
Title = "\nFilename: "+Filename+"\n\n"+
"MilliSeconds Since Computer Startup: "+NumToStr(GetPerformanceCounter(False),1.3)+"\n\n"+
"Timer     ElapsedTime     Delay     Trigger\n\n"+
"T1      "+NumToStr(ET1,10.0)+"     "+NumToStr(ETA1,7.0)+"      "+WriteIf(TT1,"TRIGGER1","")+"\n\n"+
"T2      "+NumToStr(ET2,10.0)+"     "+NumToStr(ETA2,7.0)+"      "+WriteIf(TT2,"TRIGGER2","")+"\n\n"+
"T3      "+NumToStr(ET3,10.0)+"     "+NumToStr(ETA3,7.0)+"      "+WriteIf(TT3,"TRIGGER3","");

Edited by Al Venosa.

Filed by Herman at 9:09 pm under Real-Time AFL Programming
No Comments

(1 votes, average: 3 out of 5)

 Loading ...

Backfilling data can be done in many ways, and different methods may be needed for different data providers. The method shown below was developed for eSignal data. If you use another data provider, you may need to modify the code and the procedure.

When backfilling data, you should confirm that data are backfilled properly and have some indication regarding the presence of data holes. As pointed out in Data Holes in Real-Time Trading, to be able to detect holes, you need a perfect data array against which to compare the data. Since AmiBroker doesn’t have such a data array, the method presented here uses the QQQQ as a reference ticker. Turn ON and set Backtester Settings -> General -> Pad and align all data to reference symbol -> QQQQ. To have all tickers backfilled, you should also turn ON Wait for Backfill(RT only). To speed up real-time backfills you may want to display only a simple price chart, instead of complex code. Refreshing complex Indicators or Systems will slow down backfill. The following Exploration will Backfill all tickers in your Watchlist and report on it’s success:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

SN = Status("StockNum"); 
DT = DateTime(); 
DN = DateNum(); 
NumBars = Cum(1); 
NewDay = DateNum() != Ref(DateNum(),-1); 
DayCount = Cum(Newday); 
HoleCount = Cum(V==0); 
Plot(C,"A filled Bar is Red",IIf(V==0,colorRed,colorBlack),styleBar); 
Title = "\nBACKFILL EXPLORATION"+"\n"+ 
" Stock: "+Name()+"\n"+ 
"Stock Number: "+NumToStr(SN,1.0,False)+"\n"+ 
" Time frame: "+Interval(2)+"\n"+ 
" First Date: "+NumToStr(DT[0],formatDateTime)+"\n"+ 
" Last Date: "+NumToStr(DT[BarCount-1],formatDateTime)+"\n"+ 
" Day Count: "+NumToStr(DayCount,1.0,False)+"\n"+ 
" Bar Count: "+NumToStr(BarCount,1.0,False)+"\n"+ 
" Hole Count: "+NumToStr(HoleCount,1.0,False); 
Filter=Status("LastBarInTest"); 
SetOption("NoDefaultColumns",False); 
AddColumn(HoleCount,"#Holes",1.0); 
AddColumn(SN,"Stock#",1.0); 
AddColumn(NumBars,"#Bars",1.0); 
AddColumn(DayCount,"#Days",1.0); 
AddColumn(DT[0],"From",formatDateTime); 
AddColumn(DT[BarCount-1],"To",formatDateTime);

When executed on a Watchlist, the above Exploration will backfill all stocks and produce the report shown below:

Edited by Al Venosa.

Filed by Herman at 11:56 pm under Real-Time AFL Programming
1 Comment

(1 votes, average: 3 out of 5)

 Loading ...

In real-time trading it is vital to keep your chart right justified, i.e., ensuring the right-most price bar is visible at all times. Failing to do so and looking at the wrong bar when trading real-money can invoke a panic response by the trader that results in erroneously closing a position or worse, placing a wrong order. This can easily happen if you are running several computers and are not always monitoring the chart being traded.

The following code shows you how you can right-justify your chart at system startup or from the Param window. A visible alert that changes the chart back ground color when the last bar is not visible is included.

The Alarm uses DateTime() to automatically correct for blank bars at the right edge of your chart. The code is kept explicit and you may be able to optimize it for speed.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

procedure RightJustifyChart( ChartWidth )
{
DT = DateTime();
BI = BarIndex();
FirstDateTime = LastValue( Nz(Ref( DT,-Max(2, ChartWidth) ) ) );
FirstDateTimestr = DateTimeToStr( FirstDateTime );
LastDateTimestr = DateTimeToStr( LastValue(DT) );
AB = CreateObject("Broker.Application");
AW = AB.ActiveWindow;
AW.ZoomToRange( FirstDateTimestr, LastDateTimestr );
}  
 
RightJustify  = ParamTrigger("Right Justify Chart","RIGHT JUSTIFY");
ChartWidth = Param("Chart Width",20,1,200,1);
NotRightJustifiedColor = ParamColor("Chart Right Justified",colorPink);
if( IsNull(StaticVarGet("CodeInitialized")) OR RightJustify )
    {
    StaticVarSet("CodeInitialized", 1);
    RightJustifyChart( ChartWidth );
    }  
Plot(C,"",1,128);  
DT=DateTime();
IsRightJustified = LastValue(DT) == LastValue(ValueWhen(Status("BarVisible"), DT));
if( NOT IsRightJustified ) SetChartBkColor(NotRightJustifiedColor);

Edited by Al Venosa.

Filed by Herman at 12:41 pm under Real-Time AFL Programming
No Comments

(2 votes, average: 4.5 out of 5)

 Loading ...

When your code increases in complexity, it will start to execute more slowly, and you will eventually reach a point where you will want to optimize your code for speed of execution. Even if you are willing to buy a faster computer, it will rarely give you a greater speed advantage than optimizing your code.

Important note: The AmiBroker 5.01.1 beta released on 5/10/2007 offers a new profiling function that enables you to get accurate performance timing for all AmiBroker and custom functions called. It is available from the AFL Editor: Tools -> Code Check and Profile menu. To use this new tool effectively you should convert your code modules to function calls as much as possible. Try it!

The first requirement for analyzing your AFL execution speed is the ability to measure the execution times. This is best done using the GetPerformanceCounter(), which gives microsecond resolution. You can display the execution time in the chart Title, or log them to DebugView. To get an appreciation of how fast AFL executes you may want to time a few of your most frequently used AFL statements. As Tomasz pointed out in his comment to an earlier version of this post, this can be done using a simple loop:

1
2
3
4
5
6

GetPerformanceCounter( True );
for( i = 0; i < 1000; i++ )
{
x = Param("test", 1, 0, 100, 1 ); // Statement under test
}
Title=""+GetPerformanceCounter(True)/i+" milliseconds";

On my 2GHz laptop, the Param() takes about 4 microseconds to execute. The “Statement under test” can be any AFL statement or section of code you want to analyze. This simple code will give you a fairly accurate idea of the true execution time. Please do not forget that all programs executing under Microsoft Operating Systems are subject to interruptions. This means that your readings may fluctuate and will generally be greater than their absolute true values. Your normal program executions are also subject to these interruptions and, if you perform precision timing tasks, your should consider their effect.

If, in view of processing overhead, you prefer not to use DebugView, you can log execution times either to the Interpretation screen, which invokes an additional AFL execution, or to the chart Title if you output just a few measurements. Use of DebugView is outlined in several posts in the Debugging AFL category. When you are logging to DebugView, execution times will be inflated by the amount of time needed to call and execute the _Trace() functions. You can measure the timing overhead for a single _Trace() function using the same method we used above:

1
2
3
4
5
6

GetPerformanceCounter( 1 );
for( i = 0; i < 1000; i++ )
{
_TRACE(" Test");
}
Title="\nApproximate _Trace() Overhead: "+GetPerformanceCounter(1)/i+" milliseconds per call";

On my 2GHz laptop, the _Trace() takes about 35 microseconds to execute. Compared to the Param(), this would appear like a major error, but in practice it is not. For example, when calling the _Trace() function thirty times in your code, this would add only about 1 millisecond to the overall execution time of your program.

The example code below logs to DebugView and to the chart Title. It uses a small Trace() function that is called each time you want to log a timing measurement. You can customize this function to log additional information. Although you can call this function on each line, you do not have to. Calling Trace() introduces a small measurement error. This means that when you remove the measurement code, performance should be slightly better than your measurements predict. If you do not use DebugView, you can delete all DebugView related statements. If you do this, I suggest you comment them out until your modified code has been verified.

Be careful when calling the Trace() from inside a loop. If your loop steps through 50,000 bars of data, inserting one or more Trace() calls will slow down your code substantially and result in a significant measurement error. The key to reasonable accuracy is to use the minimum number of _Trace() statements and keep them out of loops and frequently called functions.

Inserting/removing TRACE() statements in a test program is easily done using the Edit- Replace function in the formula editor. To add _Trace() statements, replace the “;” at the end of the line with “; TRACE(”");”:

To remove the _Trace() statements substitute “; TRACE(”");” with “;”:

Place your cursor just before the last “;” that precedes the first line of code you want to time. Stop substitutions where you no longer want to time statements. You should initialize the LineNumber to the line number that contains your first Trace() statement. Line numbers for your cursor position are displayed at the right bottom of your editor screen.

Here is a do-nothing program where _Trace() statements were added using the above method. You can modify the function Trace() to output any information you like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

function Trace( Comment ) 
{ 
global LineNumber, TitleStr; 
PC = GetPerformanceCounter( True ); 
Str = "# LN: "+NumToStr(LineNumber++,1.0,False)+", PC: "+NumToStr(PC,9.3)+" mSec., "+Comment; 
TitleStr = TitleStr + Str+"\n"; 
_TRACE(Str); 
} 
 
function TestLoop() 
{ 
SumC=0; 
for(b=1;b<BarCount;b++) 
{ 
SumC[b] = SumC[b] + C[b]; 
} 
return b; 
} 
global LineNumber; 
TitleStr="\n"; 
LineNumber = 20; Trace("Ignore first reading"); 
LB=Param("LookBack",10,1,100,1); Trace("Param()"); 
_TRACE("# DBGVIEWCLEAR"); Trace("Clear DebugView"); 
SetBarsRequired(1000000,1000000); Trace("Set bars Required"); 
n = TestLoop(); Trace("TestLoop "+NumToStr(n,1.0)+" Loops"); 
Plot(C,"",1,128); Trace("Plot Bars"); 
Plot(C,"",1,1); Trace("Plot C Line"); 
H1=HHV(H,LB); Trace("HHV"); 
L1=LLV(L,LB); Trace("LLV"); 
PlotOHLC(H1,H1,L1,L1,"",2,styleCloud); Trace("Plot Cloud"); 
Title = TitleStr;

The output produced in DebugView looks like this:

The output in the Title looks like this:

The first reading in the Title should be ignored as it initializes the PerformanceCounter. This reading is suppressed by the DBViewClear command in the DebugView log. The above method can be used to determine execution times of all AFL functions. Note that in the above example SetBarsRequired() is used to process all available data. This would be the worst-case measurement. Commenting out SetBarsRequired() will make your code execute under Quick-AFL. This may be significantly faster than running over all data.

Measuring Execution Order and Timing for Window Panes

When you are executing AFL formulas in multiple panes or windows and experience a timing problem, the first step to take, before using the above method, is to determine which pane uses up too much time. You can turn on Tools -> Preferences ->Miscelaneous -> Display Chart Timing to read the execution time at the bottom of the charts. Alternatively, the simple code below lets you log the execution order and time for the formulas in each open pane to DebugView. Since the AmiBroker Performance Counter readout is unique for each pane, the code below does not use it. In this case you will have to turn on Options -> Show Milliseconds in DebugView. This will give you Millisecond resolution, not micro-second as with the performance Counter and is a very approximate measurement. It should only be used to identify the problem formula. The execution order of the panes is revealed by the order in which the lines appear in the DebugView.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

Filename = StrLeft(_DEFAULT_NAME(),StrLen(_DEFAULT_NAME())-2);
function TraceChartID() 
{ 
Msg = "# ChartID, "+NumToStr(GetChartID(),1.0,False)+", "+Filename;
_TRACE( Msg ); 
return Msg;
} 
 
TraceChartID();
 
//
for(n=0; n<BarCount;n++) ; // your code would be here
//
 
Title = TraceChartID();

The Filename assignment and the TraceChartID() should be placed at the very top of your code, or you can add it to your default Include file. The single line used to call this function should be placed at both the beginning and the end of the code in each window/pane. This is what it looks like in DebugView:

Edited by Al Venosa.

Filed by Herman at 3:44 pm under Real-Time AFL Programming
5 Comments

(No Ratings Yet)

 Loading ...

In real-time trading many situations arise when you want to delay action until a specific criterion is met. In AmiBroker you can base your delays in many different ways, the only requirement being that the delay variable increments or decrements. If the selected variable doesn’t revert towards your timeout value (target), your delay function would never time out. In this case you would have to add code to handle that condition. A few variables you might use are:

- RT TimeNumber (Now(4)).
- Elapsed Seconds (redrawaction).
- Real-time Data time-stamp (TimeNum()).
- Tick-count (New Data).
- Volume (Change in Volume).
- Price change (Change in Price).
- Chart Refresh (any AFL Execution).
- Indicator values.

Which of the above variables you would use for your delay depends on the requirements of your trading system. There may be times when you may need to combine several methods to get the required results. For example, if a delay were based on the data-timestamp, it might not time out during a data dropout or a no-trading period. In this case you need to back up your data-timestamp delay with a real-time (seconds) delay.

Delays play a critical role in real-time system design. For example, in real-time systems, signals may have a short lifetime. The signal is strongest when it triggers and than quickly decays until, perhaps after a few bars, it has lost all significance. Letting the order fill at the time when it has lost significance is pure gambling. To prevent this, you can cancel the order after a delay, or decrease the position size proportional to the perceived decay in signal strength (perhaps based on elapsed bars?)

Since in a real-time system the time-lapse between AFL executions can be significant, you should place your LMT price calculations ahead of the ordering code. Calculating the LMT price after the order has been placed postpones order placement until the next AFL execution occurs, i.e., when the next quote arrives; by then, the price has probably changed. Especially during periods of low volume, this could be significant. When these delays would be insignificant in EOD systems, they could make-or-break your system in fast-trading systems.

To ensure frequent AFL execution in the absence of quotes, you can place a RequestTimedRefresh(1) statement at the top of your code, where the variable ‘1′ refers to a 1-second refresh. This guarantees an AFL execution at least once per second.

If your code is lengthy and takes a significant amount of time to execute, you may have to check order status at several places in your code. If changed status demands immediate action, you can force an immediate AFL execution by calling the following function:

1
2
3
4
5

function RefreshAll() 
{ 
oAB = CreateObject("Broker.Application"); 
oAB.RefreshAll(); 
}

The highest rate that you can call this function is once per second.

Due to the lack of chart resolution and the static character of historical data in longer time frames (minutes and higher), you may only be able to optimize delays during live simulation or real trading. This is because only the last bar will show live changes.

If delay functions are used in different programs, they must use unique static variable names in each program. See the post on Keying Static Variables for more information on this.

The code below introduces a DelayManager() that can be used to implement most delay functions. The delay function uses the following variables:

1) DelayName is needed to allow its use for different delays.
2) DelayValue is used to set the DelayTarget, if set to zero the function returns the DelayRemaining.
3) DelayReference is the variable that is sampled when the delay is initialized.
4) DelayTarget is calculated as (DelayReference at initialization) + DelayValue.
5) DelayRemaining is the distance to the DelayTarget.

In your final system formula, you may want to use an in-line equivalent for better performance. The code below lists multiple choices for the DelayReference; you should comment out the ones not used or place the one to use in the last position.

When testing this code you can observe the DelayValue being initialized, and counting down to timeout, in the Title.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

function getTickCount() 
{ 
LastVol = LastValue(Volume); 
PrevVol = Nz(StaticVarGet("PrevVol")); 
NewTick = PrevVol != LastVol; 
if( NewTick ) 
{ 
TickCount = Nz(StaticVarGet("TickCounter")); 
StaticVarSet("TickCounter",++TickCount); 
StaticVarSet("PrevVol",LastVol); 
} 
TickCount = StaticVarGet("TickCounter"); 
return TickCount; 
}

1
2
3
4
5
6

function getRefreshCount() 
{ 
RefreshCount = Nz(StaticVarGet("RefreshCounter")); 
StaticVarSet("RefreshCounter",++RefreshCount); 
return RefreshCount; 
}

1
2
3
4
5
6
7
8
9

function getElapsedSeconds() 
{ 
if( Status("redrawaction") ) 
{  
ElapsedSeconds = Nz(StaticVarGet("ElapsedSeconds")); 
StaticVarSet("ElapsedSeconds",++ElapsedSeconds