US-Stocks Database (v2)

Jim, who is a long term AmiBroker user, regularly provides an updated template that automatically sets up a database of all US stocks available from Yahoo!Finance.

The stocks are initially categorized by Market and then as Stocks, Indexes or Funds at the Group level. They are also organized into Sectors, using the Hemscott classification system.

A link to the latest message from Jim is available at AmiBrokerYahooGroup message # 119057 – "AmiBroker/Yahoo Database File":  http://finance.groups.yahoo.com/group/amibroker/message/119057

Or link directly to the site to obtain the download:

http://www.icc-az.com

Go to Links >> AmiBroker at that site (instructions are available at the download page).

After the database is setup, use AmiQuote to download data from the Yahoo Historical server (allow approximately 1/2 – 1 hour download time, using a mid speed broadband connection, per year of history).

The length of back history available varies with each symbol (the author has downloaded up to 10 years history using this template).

Typically the ticker list, as setup, will contain a couple of hundred redundant symbols (symbols that return an error message when attempting to download historical data for them). Most of the redundant symbols are .OB (over the counter stock) and are not relevant to the majority of traders. Users can elect to delete them from the database or go to the Yahoo site, or other sites, to find the cause of the ‘download error’ e.g. the stock may have been de-listed or undergone a name change .

Instructions on how to find and delete the redundant (NoQuotes) symbols can be found in the UKB post >> Setup A Custom Database – Nasdaq

Information, or discussion, on the pros and cons of using Yahoo data is included in articles in the Yahoo sub-category at the Users’ Knowledge Base:

http://www.amibroker.org/userkb/?cat=99

Further information, about using the US-Stocks database, can be obtained by searching the AmiBrokerYahooGroup message board (use an Advanced Search and enter Subject contains "US-Stocks" OR "AmiBroker/Yahoo Database" as the search criteria .

 

Version 2 – Feb 02/2008 – by Brian_z – written using Windows Live Writer – updated links to current message and new download site

ASCII Import – Standard and Poor’s Global 1200

Objective 

The objective, for this post, is to outline a basic method for setting up a custom database, using ASCII importation to install the categorized structure.

A Global 1200 database, suitable for use with Yahoo data, is used as the example.

Note: it is an outline only and the procedure is not taken to completion i.e. it is not set-up as a working database (interested readers can complete that task for themselves).

 Pre-Requisites

It is assumed that readers are familiar with earlier posts in the UKB >> Database Management series (the basic skills introduced there are not recapped in this post) and also the ASCII Importer Reference (from the Users Guide).

http://www.amibroker.com/guide/d_ascii.html

 Introduction

"American Standard Code for Information Interchange (ASCII) allows digital devices to communicate with each other and to process, store, and communicate character-oriented information" 1.

It is the format used to import data into AmiBroker, via AmiQuote, or manually, via the ASCII Import function.

In this example, a list of the Global 1200 constituents, including Market and Sector categorization, is downloaded from the Standard and Poors (US) site and pre-conditioned into an ASCII compliant format, before importing it into AmiBroker.

Note: the example is a basic one, but, users can easily extend the method to more complex databases.

Preparation

Step1

A constituent list, in Comma Separated Value (CSV) format, can be downloaded from the Standard and Poors Homesite (US):

http://www2.standardandpoors.com/portal/site/sp/en/us/page.home/home/0,0,0,0,0,0,0,0,0,0,3,0,0,0,0,0.html

Go to Indices >>  + Equity Indices >> S and P Global Indices >> S and P Global 1200

OR follow the link below to go directly to the page:

http://www2.standardandpoors.com/portal/site/sp/en/us/page.topic/indices_gbl1200/2,3,2,1,0,0,0,0,0,0,3,0,0,0,0,0.html

 

Step 2

Click on Download Table, at the top of the page, to save a local copy of the Constituent List (save it in a spreadsheet compliant format)

ASC001

Note: not all of the Countries, with stocks in the Global 1200, are supported by Yahoo!Finance (refer to the following link for a list of Yahoo’s international exchanges  – http://finance.yahoo.com/exchanges)

 

Step 3

Prepare the file for ASCII Import, using the spreadsheet functions:

a) Trim excess ‘whitespace’ from the symbols, manually enter Yahoo exchange suffixes and append them to the symbols, sort by Market (Country) and manually add a column for the Market ID (0-23).

b) Delete symbols that are from unsupported exchanges, sort by Sector and manually add a column for the Sector ID’s (0-9).

c) Copy the Fullname, Ticker, Market ID and Sector ID columns to a separate Global1200.CSV file, ready for importing.

Note: Refer to the attached files 24-March-2008_GBL1200.xls and Global1200.xls (the Global1200 file is attached in .xls format as the UKB does not permit uploading of .csv files).

 

Step 4

Prepare a broker.industries and broker.sector file, using the template in the 24-March-2008_GBL1200.xls file, and save them in the root of the Programs/AmiBroker directory (overwrite the existing broker.industries and broker.sector files).

 

Step 5

Prepare an ASCII formats file and save it in the Programs/AmiBroker/Formats folder.

The file, for this example, needs to be in the following format (the file can be written in a plain text editor, like NotePad, and saved as Global1200.format):

ASC002

Note: The file name will be automatically added to the import.types list, which is also in the Programs/AmiBroker/Formats folder.

 

Step 6

Create a new EOD database, called Data_Global1200

Go to File >> Import ASCII and elect to Open the Global1200.csv file as file type Global1200(*.*)

Note: the import.types (formats file) list will be the default Files of type list in the file browser window that opens.

ASC003

The symbol list and database structure, as defined in the Global1200.csv file, will now be set-up in the current database.

Note: the Markets and Groups categories will remain as the default settings (they can only be renamed manually via Symbol >> Categories).

ASC005

Step 7

Data can now be downloaded, for the database, from the Yahoo historical server.

On downloading some symbols will return a 404 error.

In some cases the errors can be repaired by ‘researching’ via the Yahoo sites e.g. Hong Kong stocks use four digits and require leading zeros, which are not included in the symbols used by Standard and Poors.

In other cases there are multiple exchanges, supported by Yahoo, for a given country, so the ticker may need an alternative suffix (for this example the author assumed all tickers included in the Global 1200 were listed on the constituent countries major exchange).

 

ATTACHED FILES:

24-mar-2008_gbl1200.xls
global1200.xls

Browse the spreadsheet files on line, or, right click and select Save As to download them as an editable version.

REFERENCES:

1 – http://en.wikipedia.org/wiki/ASCII

AmiBroker Supported Sites

 

This post contains a list of AmiBroker ‘supported’ sites (most of the sites are officially supported by AmiBroker but some are not).

Search the main sites by using Google’s "Advanced Search" feature at http://www.google.com (search within a site, or domain, by using the URL address).

Alternatively visit any of the ‘supported’ sites for categorized information on AmiBroker features, purchase, installation, testimonials, news, FAQ’s, downloads, documents, knowledge bases, tips, code and mailing lists.

For personal support please email – support [at] AmiBroker.com

1) AmiBroker’s Homesite

This site is searchable using the "Search" box in the top left corner of the "Welcome" page.

 

Links to specific pages at the AmiBroker Homesite

a) A downloadable PDF version of the Users’ Guide (Help Manual).

Note: the PDF version has superior search features compared to the other available versions.

http://www.amibroker.com/bin/UsersGuide.pdf

b) An online version of the Users’ Guide (Help Manual).

http://www.amibroker.com/guide/

c) FAQ’s.

http://www.amibroker.com/faq.html

d) AmiBroker Knowledge Base (KB).

The ‘official’ Knowledge Base site with tips & usage hints from AmiBroker.com

http://www.amibroker.com/kb/

e) AmiBroker Tips Newsletter.

http://www.amibroker.com/newsletter/

NOTE: The Newsletter has been now replaced by the AmiBroker Knowledge Base at: http://www.amibroker.com/kb/

f) AmiBroker support page.

The support page features additional documentation and videos.

http://www.amibroker.com/support.html

g) AFL Library.

The AFL on-line library provides a very convenient way to share formulas, including descriptions and comments, between AmiBroker users. You will not only  be able to get the formula you need here but also contribute your own ideas.

http://www.amibroker.com/library/

h) Members Area.

This provides additional material for registered AmiBroker users, including the newest issues of Stocks & Commodities Traders tips for AmiBroker

http://www.amibroker.com/login.html

i) The online version of the AFL Function Reference (searchable).

http://www.amibroker.com/guide/afl/

j) AmiBroker Development Log.

http://www.amibroker.com/devlog/

k) Feedback Centre.

You can submit your ideas and suggestions and see the list of features planned for the next few releases (registered users only).

http://www.amibroker.com/todo.html

 

2) AmiBroker User’s Knowledge Base (UKB)

This site is owned and operated by AmiBroker but its contents are entirely contributed by users.

It was was created to save you time, raise your productivity, aid you in your Amibroker Formula Language (AFL) learning experience, and help you become a more successful trader.

http://www.amibroker.org/userkb/

UKB features:

          Automated Trading, by Herman van den Bergen

          Intelligent Optimization, by Fred Tonetti.

 

3) AmiBroker third party site

An unstructured FTP site, sponsored by AmiBroker.com, to facilitate  file sharing between Amibroker users.

http://www.amibroker.org/3rdparty/

 

4) The AmiBroker mailing list

The list is provided to help Amibroker users to share ideas, tips and other related information. This is the place where you can meet other AmiBroker users, ask questions and share ideas (searchable archive and file uploading).

http://finance.groups.yahoo.com/group/amibroker/

 

5) The AmiBroker Trading Systems mailing list.

This group is for discussion on developing and testing trading systems, using AmiBroker (searchable archive and file uploading).

http://finance.groups.yahoo.com/group/amibroker-ts/

 

6) The AmiBroker Automated Trading mailing list.

This group is for discussion on Automated Trading, using AmiBroker (searchable archive and file uploading).

http://finance.groups.yahoo.com/group/AmiBroker-at/

Using a GFX Include file

Important note: The AmiBroker 5.09.0 Beta introduced the following new GFX functions:

Status(“pxchartleft”) – returns x-coordinate of top-left corner of chart area
Status(“pxcharttop”) – returns y-coordinate of top-left corner of chart area
Status(“pxchartright”) – returns x-coordinate of bottom-right corner of chart area
Status(“pxchartbottom”) – returns y-coordinate of bottom-right corner of chart area
Status(“pxchartwidth”) – returns width chart area (right-left)
Status(“pxchartheight”) – returns width chart area (bottom-top)

Since this release appeared after this post was published these functions are not used in the examples below. This post has been left unchanged for educational purposes. For examples using the new functions please see the 5.09.0 Read Me file.

=====

While the post Creating GFX Chart-Overlays (v2) may have clarified a few of the more important aspects of using GFX functions, it doesn’t really give you a “quick Start” template to get started. Using a GFXInclude file can remove some of the burden of having to define pixel and charting parameters. The Include file at the bottom of this post contains most definitions as well as these common functions that you may want to call from your GFX application:

GetVisualBarIndex( ); // Returns array containing index for visible bars
gfxPlotHLineYPixelsColor ); // Plots horizontal line at level YPixels
gfxPlotVLineXPixelsColor ); // plots vertical line at level XPixels
GetYPixels); // Convert a vertical price number to the pixel equivalent
GetXPixels); // Convert a horizontal DateTime number value to the pixel equivalent

Of course you can, and should, add additional functions of your own. Here is an example of how to call the above functions to draw a GFX cross-hair cursor (Red in the capture):

gfxcrosshair.jpg

Here is the code that produced the above image:

GraphXSpace 5// See the AmiBroker help on how to init these variables
GfxSetBkModebkmode );
GfxSetOverlayModemode );
GfxSelectPencolorRed );
 
PlotC""colorBlackstyleLine ); // To define miny and maxy
#include <GFXInclude-001.afl> // Located in your default Include folder
 
// Example to draw cross-hair cursor
Yprice GetCursorYPosition(0);
XIndex SelectedValue(GetVisualBarIndex( ));
gfxPlotHLineGetYPixelsYPrice ), colorRed );
gfxPlotVLineGetXPixelsXIndex ), colorRed );

The include file listed below defines the following variables:

// pxwidth, pxheight, Miny, MinX, YRange, VisBarIndex, NumBarsVisible, pxPaneWidth, pxPaneheight, PixelsPerBar, PixelsPerPrice

You may want to copy the above comment line below the #include statement in your code to refresh your memory. You should copy the Include file to your default AmiBroker Include folder.

// GFXInclude-001.afl copy to default include folder

function gfxPlotHLineYPixelsColor )
{
    global pxwidth;
    GfxSelectPenColor ) ;
    GfxMoveTo0YPixels );
    GfxLineTopxwidthYPixels );
}
 
function gfxPlotVLineXPixelsColor )
{
    global pxheight;
    GfxSelectPenColor ) ;
    GfxMoveToXPixels);
    GfxLineToXPixelspxheight );
}
 
function GetVisualBarIndex( )
{
    lvb Status"lastvisiblebar" );
    fvb Status"firstvisiblebar" );
    bi BarIndex();
    StaticVarSet"NumberbarsVisible"Lvb fvb );
    return bi bi] - fvb;
}

function GetYPixels)
    {
    global PixelsPerPricepxTopAreaMaxY; 
    return (MaxY Y) * PixelsPerPrice pxTopArea;
    }

function GetXPixels)
    {
    global PixelsPerBarpxLeftArea;
    return PixelsPerBar pxLeftArea;
    }

_SECTION_BEGIN("GFX INITIALIZATION");
// These Parameters will change depending on screen layout/fonts
pxRightArea Param"Right Axis Area"930200); // Depends on font
pxDateArea Param"Date Axis Area"110100); // Depends on font
DateaxisOn ParamToggle"Date Axis""HIDE|SHOW");

pxLeftArea 5pxTopArea 5pxBottomArea 5; 
if ( DateaxisOn )
{
    pxBottomArea pxDateArea pxBottomArea;
    SetChartOptions2chartShowDates );
}
else
    SetChartOptions3chartShowDates );

pxwidth Status"pxwidth" );
pxheight Status"pxheight" );

// clalculate charting area width and height
Miny Status"axisminy" );
Maxy Status"axismaxy" );
YRange MaxY MinY;
VisBarIndex =  GetVisualBarIndex( );
NumBarsVisible StaticVarGet"NumberbarsVisible" );

// Calculate Pane width and height
pxPaneWidth pxwidth pxLeftArea pxRightArea;
pxPaneHeight pxHeight pxTopArea pxBottomArea;

// calculate conversion factors
PixelsPerBar     pxPaneWidth NumBarsVisible;
PixelsPerPrice pxPaneHeight YRange;
_SECTION_END();

Edited by Al Venosa.

Creating GFX Chart-Overlays (v3)

Important note: The AmiBroker 5.09.0 Beta introduced the following new GFX functions:

Status(“pxchartleft”) – returns x-coordinate of top-left corner of chart area
Status(“pxcharttop”) – returns y-coordinate of top-left corner of chart area
Status(“pxchartright”) – returns x-coordinate of bottom-right corner of chart area
Status(“pxchartbottom”) – returns y-coordinate of bottom-right corner of chart area
Status(“pxchartwidth”) – returns width chart area (right-left)
Status(“pxchartheight”) – returns width chart area (bottom-top)

Since this release appeared after this post was published these functions are not used in the examples below. This post has been left unchanged for educational purposes. For examples using the new functions please see the 5.09.0 Read Me file.

=====

Creating an exact chart overlay using GFX functions can be a daunting task for the non-professional programmer. The solutions presented here were derived through experimentation; if there is a better way, please make a comment. Once the pixel layout is fully understood, GFX becomes an extremely powerful tool and may just give you an additional trading edge. The first and most important step in using GFX functions is to understand how pixels make up your display. In AmiBroker the width and height of your charting pane can be retrieved using the following two functions:

pxwidth Status("pxwidth");
pxheight Status("pxheight");

Horizontal pixels count left to right, 1 to pxwidth; vertical pixels count top to bottom, 1 to pxheight. The area covered by these two numbers is shown in Yellow below. For a high resolution monitor this area may cover about 2000 (H) x 1000 (V) pixels. This pixel area includes the areas used by the X and Y axis, and the blank top and bottom margins.

pixelarea.png

Next is the standard charting area, which is the area where your price charts are located. This area excludes the surrounding blank margins and the areas used for axis labeling. If you want to keep your overlay within the standard chart boundaries, you have to place your images within the above boundaries. This area is highlighted in Blue in the image below:

chartingarea.png

The boundaries for this area can be determined by running the example code listed in Finding Pixel Boundaries. Eight parameters must be known to create pixel overlays:

pxwidth Status"pxwidth" );
pxheight Status"pxheight" );
pxLeftArea Param"Left Blank Margin"50100); // Constant
pxRightArea Param"Right Axis Area"930200); // Depends on font
pxTopArea Param"Top Blank Margin"50100); // Constant
pxDateArea Param"Date Axis Area"110100); // Depends on font
pxBottomArea Param"Bottom Blank Margin"50100); // Constant
DateaxisOn ParamToggle"Date Axis""HIDE|SHOW");

Since adding date labels to your DateTime axis changes the size of your pixel plotting area, you need to compensate for this:

if( DateaxisOn ) 
    {
    pxBottomArea pxDateArea pxBottomArea;
    SetChartOptions(2,chartShowDates);
    }
else 
    {
    SetChartOptions(3,chartShowDates);
    }

The pixel plotting area’s width and height can now be calculated:

pxPaneWidth pxwidth pxLeftAreapxRightArea;
pxPaneHeight pxHeight pxTopAreapxBottomArea;

The dimension of the Blue area shown earlier changes when you resize AmiBroker, open additional windows or panes, change the fonts in your axis, or turn On/Off date labels. When this happens you will have to recalibrate the boundaries. To convert prices to pixels, so that you can create an exact overlay, you also need to define the width and height of your regular chart pane. These are expressed in DateTime and Price units. They will change when you zoom your chart. When you have at least one price plot displayed, so that the values for miny and maxy are defined, you can calculate these boundaries as follows:

Miny Status("axisminy");
Maxy Status("axismaxy");
YRange MaxY MinY
BarsVisible Status("BarVisible");
NumBarsVisible Cum(BarsVisible);

You now have all the information needed to calculate the Pixels/Price and Pixels/Bar conversion factors:

PixelsPerBar     pxPaneWidth NumBarsVisible;
PixelsPerPrice pxPaneHeight YRange;

Putting it all together in a demo program (listed at the end of this post) produces the price-chart overlay shown below. The regular price plot is plotted using dots, so that the overlay is clearly visible. When you plot both traces in lines, you will see minor deviations that are probably due to rounding to the nearest pixel. The pixel price plot is shown in Red. The purpose of this exercise is to learn to work with pixels and be able to produce an exact overlay on the price chart. The Param window below the charts shows typical parameters; they will likely be different for your screen layout.

pricechartoverlay1.png

pricechartoverlayparam.png

function gfxPlotHLineYPixelsColor )
{
    global pxwidth;
    GfxSelectPenColor ) ;
    GfxMoveTo0YPixels );
    GfxLineTopxwidthYPixels );
}
 
function gfxPlotVLineXPixelsColor )
{
    global pxheight;
    GfxSelectPenColor ) ;
    GfxMoveToXPixels);
    GfxLineToXPixelspxheight );
}
 
function GetVisualBarIndex( )
{
    lvb Status"lastvisiblebar" );
    fvb Status"firstvisiblebar" );
    bi BarIndex();
    StaticVarSet"NumberbarsVisible"Lvb fvb );
    return bi bi] - fvb;
}

function GetYPixels)
    {
    global PixelsPerPricepxTopAreaMaxY; 
    return (MaxY Y) * PixelsPerPrice pxTopArea;
    }

function GetXPixels)
    {
    global PixelsPerBarpxLeftArea;
    return PixelsPerBar pxLeftArea;
    }
 
GraphXSpace 5;
SetChartOptions0chartHideQuoteMarker );

pxwidth Status"pxwidth" );
pxheight Status"pxheight" );

// These Parameters will change depending on screen layout/fonts
pxRightArea Param"Right Axis Area"930200); // Depends on font
pxDateArea Param"Date Axis Area"110100); // Depends on font
DateaxisOn ParamToggle"Date Axis""HIDE|SHOW");

// These Parameters appear constant and can probably be hardcoded
pxLeftArea Param"Left Blank Margin"50100); // Constant
pxTopArea Param"Top Blank Margin"50100); // Constant
pxBottomArea Param"Bottom Blank Margin"50100); // Constant
 
if ( DateaxisOn // Size of bottom boundary depends on whether dates are shown
{
    pxBottomArea pxDateArea pxBottomArea;
    SetChartOptions2chartShowDates );
}
else
    SetChartOptions3chartShowDates );
 
// Test Plots to help line up boundary lines
Color colorWhite;
gfxPlotVLinepxLeftAreacolor );
gfxPlotVLinepxwidth pxRightAreacolor );
gfxPlotHLinepxTopAreacolor );
gfxPlotHLinepxHeight pxBottomAreacolor );
 
// Calculate Pane width and height
pxPaneWidth pxwidth pxLeftArea pxRightArea;
pxPaneHeight pxHeight pxTopArea pxBottomArea;
 
// clalculate charting area width and height
PlotC""1styleDots );
Miny Status"axisminy" );
Maxy Status"axismaxy" );
YRange MaxY MinY;
VisBarIndex =  GetVisualBarIndex( );
NumBarsVisible StaticVarGet"NumberbarsVisible" );

// calculate conversion factors
PixelsPerBar     pxPaneWidth NumBarsVisible;
PixelsPerPrice pxPaneHeight YRange;

// For verification: Overlay pixel on price plot
FVB Status"firstvisiblebar" );
LVB Status"lastvisiblebar" );
GfxSelectPencolorRed );
for ( FVB 1<= LVB AND BarCountb++ )
{
    PrevPixelY GetYPixelsC[b-1] );
    PixelY GetYPixelsC[b] );
 
    PrevPixelX GetXPixelsVisBarIndex[b-1] );
    PixelX GetXPixelsVisBarIndex[b] );
 
    GfxMoveToPrevPixelXPrevPixelY );
    GfxLineToPixelXPixelY );
}

Edited by Al Venosa.

Setting prices with your cursor

To place LMT or STP orders quickly is easiest done by moving the horizontal cursor-line over the desired price and making a left mouse click. The demo code below shows how you can lock in prices this way. To Chart-Trade additional code must be added to make the click perform one task of many, and only do so when required. When you Apply the code below to an indicator you will see a dashed line at the cursor price. This dashed line refreshes only once per second with a local database, however, it will speed up when your chart is refreshed more frequently when working with live data.

Suppose you want to place a LMT order at $60.00 on the chart below. To do this you move the dashed line over the $60.00 price and click the Left mouse button. This will place a stationary solid line. You can now move your cursor to another price and click again, the solid line will move to this new location. In actual trading you can follow the price in real time and adjust your Limit prices so that they stay exactly where you want them. The chart produced by this demo code looks like this:

placingline.png

In Chart-Trading the first click that places the solid line would also place your order on the TWS. Each subsequent click would modify the order to the new price. This way you can adjust several prices on your chart, setting and moving around Entries, Targets, Stops, etc.

RequestTimedRefresh);
SetChartOptions2chartHideQuoteMarker );
LButtonDown GetCursorMouseButtons() == 9;
MousePrice GetCursorYPosition();

if ( MousePrice )
{
    StaticVarSet"MousePrice"MousePrice );
    if ( LButtonDown )
        StaticVarSet"ClickedMousePrice"MousePrice );
}

LB BarCount 1;
MousePrice NzStaticVarGet"Mouseprice" ), Null );
ClickedMousePrice StaticVarGet"ClickedMousePrice" );

PlotC""1128 );
PlotMousePrice""colorWhitestyleNoRescale styleDashed styleNoLabel00);
PlotText"CURSOR " NumToStrMousePrice[LB], 1.2 ), LB 5MousePrice[LB], colorBlackcolorWhite );
PlotClickedMousePrice""colorBlackstyleNoLabel styleNoRescale00);
PlotText"ORDER PLACED $" NumToStrClickedMousePrice[LB], 1.2 ), LB 5ClickedMousePrice[LB], colorBlackcolorWhite );

Parsing TWS Error messages

Here is some simple code you can use to enable you to start developing your own error-parsing functions. In this example, error messages are hard-coded to facilitate code development, i.e. you don’t have to be live with IB to test it. I suggest that you inspect the IB list of error messages, copy the errors that are important to you, and add them to the hard-coded list. Since IB doesn’t list the error messages in the exact format you will receive them, you may have to place some orders with errors to see their exact format. When Applied to an Indicator, the code can be exercised by selecting the Error Message with the slider in the Param window. It will display the current error and parsing results in the Chart Title. Be sure to set Parameter properties to wrap the Title.

parsingerrors.png

Param"Example Error Msg"0010);
switch ( )
{
case 0:
    TWSLastErrorMsg "ID=15071. Error 135. Can't find order with id =:15071";
    break;
case 1:
    TWSLastErrorMsg "ID=2094, Error 103, Duplicate ORder id";
    break;
case 2:
    TWSLastErrorMsg "ID=-1, Error 1102, Connectivity between IB AND TWS Has been restored - data MAintained";
    break;
case 3:
    TWSLastErrorMsg "ID=-1, Error 2100, New account data requested";
    break;
case 4:
    TWSLastErrorMsg "ID=-1, Error 1100, Connectivity between IB AND TWS has been lost";
    break;
case 5:
    TWSLastErrorMsg "Connection established OK, Next Order Id=2080";
    break;
case 6:
    TWSLastErrorMsg "ID=6125. Error 202. Order Canceled - reason:";
    break;
default:
    TWSLastErrorMsg "";
    break;
}

function GetErrorCodeTWSLastErrorMsg )
{
    StrFindTWSLastErrorMsg" Error " );
    ErrStr StrMidTWSLastErrorMsg6);
    ErrNum    StrToNumErrStr );
    ErrStr    NumToStrErrNum1.0False );
    return ErrStr;
}

function ExtractErrorIDTWSLastErrorMsg )
{
    StrFindTWSLastErrorMsg"ID=" );
    IDStr StrMidTWSLastErrorMsg2);
    IDNum    StrToNumIDStr );
    IDStr    NumToStrIDNum1.0False );
    return IDStr;
}

TWSErrorCode GetErrorCodeTWSLastErrorMsg );
TWSErrorOrderID ExtractErrorIDTWSLastErrorMsg );
Title "\n" +
        "Extracting OrderIDs and Error codes from TWS Error Messages" "\n" +
        "Example Error Msg: " TWSLastErrorMsg "\n" +
        "        Error Code: " TWSErrorCode "\n" +
        " TWS Error OrderID: " TWSErrorOrderID;

Introduction to Real-Time Chart-Trading

IMPORTANT NOTE: DO NOT USE ANY OF THE PROGRAMS PRESENTED HERE TO TRADE REAL MONEY. Most of the programs presented will only work with live data, execution reports, and market conditions. They will NOT work using the AmiBroker Bar Replay or the IB eDemo account.

Traditional Day-Trading by looking at TWS prices may still work for experienced traders who, probably over a period of many years, developed rapid interpretation skills. However, for less experienced traders graphical representations are much easier to interpret than spreadsheets or tables. Setting prices by clicking on your chart is much faster than entering prices in little windows. Trailing the price with your cursor is much faster than entering and modifying parameters in a GUI. All this is especially true when you are trading in real-time, where every second counts.

Posts in this category will introduce techniques that you can use to create your own custom designed Chart-Trading GUI. If you Google Chart-Trading you will find many examples of the many ways you can layout your personal interface. If you aim for maximum simplicity and automation you will end up with the most efficient GUI. Tentative topics that will be covered are:

1) Laying out buttons and controls (Param() Or Button())
2) House keeping functions (AutoTrading, CancelAll, CloseAll, etc.)
3) Automating setup for different trading methodologies (minimizing setup procedure)
4) Auto placement of orders
5) Auto Modification of Orders
6) Displaying order and system status
7) Performance reporting (Profits, Wins, etc.)

AmiBroker Custom Backtester Interface

by Wayne (GP)

Introduction

From version 4.67.0, AmiBroker provides a custom backtester interface to allow customising the operation of the backtester’s second phase which processes the trading signals. This allows a range of special situations to be achieved that aren’t natively supported by the backtester. AmiBroker tends to refer to this as the Advanced Portfolio Backtester Interface, but as it seems to be more widely referred to as the Custom Backtester Interface, I will use this latter terminology.

Due to the object model used by the backtester interface, a higher level of programming knowledge is required than for simple AFL or looping. This document starts by discussing that model, so is aimed at AFL programmers who are already proficient and comfortable with basic AFL use, array indexing, and looping. If you don’t understanding looping, then you almost certainly won’t understand the custom backtester interface.

The Object Model

The modern programming paradigm is called object-oriented programming, with the system being developed modelled as a set of objects that interact. The custom backtester interface follows that model.

An object can be thought of as a self-contained black-box that has certain properties and can perform certain functions. Internally it’s a combination of code and variables, where both can be made either private to the internals of the object only or accessible from outside for the benefit of users of the object. The private code and variables are totally hidden from the outside world and are of no interest to users of the object. Only developers working on the object itself care about them. Users of the object are only interested in the code and variables made accessible for their use.

Any variable made accessible to an object’s user is called a property of the object. For example, the Backtester object has a property (variable) called “Equity”, which is the current value of the portfolio equity during a backtest. Properties can be read and written much the same as any other variable, just by using them in expressions and assigning values to them (although some properties may be read-only). However, the syntax is a little different due to the fact they’re properties of an object, not ordinary variables.

An object’s code is made accessible to its users by providing a set of functions that can be called in relation to the object. These functions are called methods of the object. They are essentially identical to ordinary functions, but perform operations that are relevant to the purpose of the object. For example, the Backtester object has methods (functions) that perform operations related to backtesting. Methods are called in much the same way as other functions, but again the syntax is a little different due to them being methods of an object rather than ordinary functions.

The aim of the object model is to view the application as a set of self-contained and reusable objects that can manage their own functionality and provide interfaces for other objects and code to use. Imagine it as being similar to a home entertainment system, where you buy a number of components (objects) like a TV, DVD player, surround-sound system, and karaoke unit (if you’re that way inclined!). Each of those components manages its own functionality and provides you with a set of connectors and cables to join them all together to create the final application: the home entertainment system. The beauty of that arrangement is that each component provides a standard interface (if you’re lucky) that will allow any brands of the other components to be connected, without those components having to know the details of how all the other components work internally, and considerable choice in the structure of the final entertainment system constructed. Similarly, software objects have standard interfaces in the form of methods and properties that allow them to be used and reused in any software.

Accessing Oject Properties And Methods

To access the properties and methods of an object, you need to know not only the name of the property or method, but also the name of the object. In AmiBroker AFL, you cannot define or create your own objects, only use objects already provided by AmiBroker. AmiBroker help details all its objects, methods, and properties in the section “Advanced portfolio backtester interface”.
To use real AFL examples, the first object detailed in the help is the Backtester object. AmiBroker provides a single Backtester object to perform backtests. To use the Backtester object, you first have to get a copy of it and assign that to your own variable:

bo GetBacktesterObject();

The variable “bo” is your own variable, and you can call it whatever you like within the naming rules of AFL. However, to avoid a lot of verbose statements, it’s good to keep it nice and short. Previously you’ve only dealt with variables that are either single numbers, arrays of numbers, or strings. The variable “bo” is none of those, instead being a new type of variable called an object variable. In this case it holds the Backtester object (or really a reference to the Backtester object, but I don’t want to get into the complicated topic of references here). Now that you have the Backtester object in your own variable, you can access its properties and methods.

The syntax for referencing an object’s property is objectName.objectProperty, for example bo.InitialEquity,. That can then be used the same as any other variable (assuming it’s not a read-only property, which InitialEquity is not):

bo.InitialEquity 10000;
capital bo.InitialEquity;
gain = (capital bo.InitialEquity) / bo.InitialEquity 100;

From this you can see the advantage of keeping object variable names short. If you called the variable “myBacktesterObject”, then for the last example above you’d end up with:

gain = (capital myBacktesterObject.InitialEquity) / myBacktesterObject.InitialEquity 100;

===============
Here I’ve had to reduce the font size just to fit it all on a single line.
If a property is read-only, then you cannot perform any operation that would change its value. So, using the Equity property which is read-only:

currentEquity bo.Equity;    //  This is fine

but:

bo.Equity 50000;    //  This is an error!

The same syntax is used to access the methods of an object. The method name is preceded by the object name with a decimal point: objectName.objectMethod(). Any parameters are passed to the method in the same manner as to ordinary functions:

objectName.objectMethod(parm1parm2parm3).

For example, to call the Backtester object’s AddCustomMetric method and pass the two compulsory parameters Title and Value, a statement like this would be used:

bo.AddCustomMetric("myMetric"1000);

AmiBroker help indicates that this method returns a value of type “bool”, which means boolean and thus can only take the values True and False. However, it doesn’t detail what this return value means. A good guess would be that it returns True if the custom metric was successfully added and False if for some reason it failed to be added. However, that’s only a guess, but a common reason for returning boolean values. For some of the other methods that return values of type “long”, it’s more difficult to guess what they might contain.
Another example with a return parameter:

sig bo.GetFirstSignal(i);

Here the variable “sig” is another object variable, but this time of type Signal rather than Backtester. In other words, it holds a Signal object rather than a Backtester object. Unlike the single Backtester object, AmiBroker can have many different Signal objects created at the same time (one for each trading signal). As a Signal object holds the signal data for a particular symbol at a particular bar, the method needs to know the bar number, which would typically be specified using a loop index variable (‘i’ above) inside a loop:

for (0BarCounti++)
{
    . . . .
    sig bo.GetFirstSignal(i);
    . . . .
}

Once a Signal object has been obtained, its properties and methods can be referenced:

sig.PosScore 0;    //  Set position score to zero for this bar
if (sig.IsEntry())    //  If this bar's signal is entry (buy/short)
{
    . . . .
}

Note that the property sig.PosScore is a single number, not an array. While the AFL variable PositionScore is an array, the “sig” object only holds data for a single bar, so the property sig.PosScore is the position score value for that bar only, thus a single number.

Also note that AmiBroker help is not very clear on some topics. For example, the Signal object only has a few methods that indicate whether the current bar contains an entry, exit, long, or short signal, or has a scale in or out signal. However, it doesn’t indicate how you combine these to get the exact details. For example, how do you tell the difference between a scale-in and a scale-out? Is scaling in to a long position a combination of IsScale, IsEntry, and IsLong, or perhaps just IsScale and IsLong, or neither of those? In some cases you need to use trial and error and see what actually works (learn how to use the DebugView program with _TRACE statements: see Appendix B). Fortunately for this specific example, the Signal object also has a property called Type that indicates exactly what type the signal is.

Using The Custom Backtester Interface

To use your own custom backtest procedure, you first need to tell AmiBroker that you will be doing so. There are a few ways of doing this:

  1. By setting a path to the file holding the procedure in the Automatic Analysis Settings Portfolio page. This procedure will then be used with all backtests, if the “Enable custom backtest procedure” checkbox is checked.
  2. By specifying these same two settings in your AFL code using the functions SetOption(“UseCustomBacktestProc”, True) and SetCustomBacktestProc(““). Note that path separators inside strings need to use two backslashes, for example “c:\\AmiBroker\\Formulas\\Custom\\Backtests\\MyProc.afl”. Although why is not important here, it’s because a single backslash is what’s called an escape character, meaning the character(s) after it can have special meaning rather than just being printable characters, so to actually have a printable backslash, you have to put two in a row.
  3. By putting the procedure in the same file as the other AFL code and using the statement SetCustomBacktestProc(“”). This tells AmiBroker that there is a custom backtest procedure but there’s no path for it, because it’s in the current file. This option will be used throughout the rest of this document.

The next thing that’s required in all backtest procedures is to ensure the procedure only runs during the second phase of the backtest. That’s achieved with the following conditional statement:

if (Status("action") == actionPortfolio)
{
    . . . . 
}

And finally, before anything else can be done, a copy of the Backtester object is needed:

bo GetBacktesterObject();

So all custom backtest procedures, where they’re in the same file as the other AFL code, will have a template like this:

SetCustomBacktestProc("");
if (Status("action") == actionPortfolio)
{
    bo GetBacktesterObject();

    //  Rest of procedure goes here

}

If the backtests were using a procedure in the file:

c:\AmiBroker\Formulas\Custom\Backtests\MyBacktest.afl

then the first line above in your system AFL code would be replaced with:

SetOption("UseCustomBacktestProc"True);
SetCustomBacktestProc("c:\\AmiBroker\\Formulas\\Custom\\Backtests\\MyBacktest.afl");

and the rest of the procedure would be in the specified file. Or, if the same values were specified in the Automatic Analysis settings, the two lines above would not be needed in your AFL code at all, and the procedure would be in the specified file.

Custom Backtester Levels

The AmiBroker custom backtester interface provides three levels of user customisation, simply called high-level, mid-level, and low-level. The high-level approach requires the least programming knowledge, and the low-level approach the most. These levels are just a convenient way of grouping together methods that can and need to be called for a customisation to work, and conversely indicate which methods cannot be called in the same customisation because their functionality conflicts. Some methods can be called at all levels, others only at higher levels, and others only at lower levels. AmiBroker help details which levels each method can be used with. Naturally, the higher the level and the simpler the programming, the less flexibility that’s available.

This document will not detail every single method and property available, so the rest of this document should be read in conjunction with the AmiBroker help sections “Advanced portfolio backtester interface” and “Adding custom backtest metrics”.

High-Level Interface

The high-level interface doesn’t allow any customising of the backtest procedure itself. It simply allows custom metrics to be defined for the backtester results display, and trade statistics and metrics to be calculated and examined. A single method call runs the whole backtest in one hit, the same as when the custom backtester interface isn’t used at all.

AmiBroker help has an example of using the high level interface to add a custom metric. See the section called “Adding custom backtest metrics”. In essence, the steps are:

  1. Start with the custom backtest template above
  2. Run the backtest
  3. Get the performance statistics or trade details
  4. Calculate your new metric
  5. Add your new metric to the results display

That would look something like this:

SetCustomBacktestProc("");
if (Status("action") == actionPortfolio)
{
    bo GetBacktesterObject();    //  Get backtester object
    bo.Backtest();    //  Run backtests
    stats bo.GetPerformanceStats(0);    //  Get Stats object for all trades
    myMetric = <calculation using stats>;    //  Calculate new metric
    bo.AddCustomMetric("MyMetric"myMetric);    //  Add metric to display
}

As well as just using the built-in statistics and metrics, obtained from the Stats object after the backtest has been run, it’s also possible to calculate your metric by examining all the trades using the Trade object. As some positions may still be open at the end of the backtest, you may need to iterate through both the closed trade and open position lists:

for (trade bo.GetFirstTrade(); tradetrade bo.GetNextTrade())
{
    . . . .
}
for (trade bo.GetFirstOpenPos(); tradetrade bo.GetNextOpenPos())
{
    . . . .
}

In this example, “trade” is an object variable of type Trade, meaning it holds a Trade object. As with the Signal object, AmiBroker can have many Trade objects created at the same time, one for each closed or open trade. The first for loop iterates through the closed trade list, and the second through the open position trade list. The continuation condition “trade” theoretically means while the trade object is not zero, but in fact “trade” will be Null when the end of the list is reached. However, any conditional involving a null value is always false, so this will still work. The five Backtester object methods GetFirstTrade, GetNextTrade, GetFirstOpenPos, GetNextOpenPos, and FindOpenPos all return Null when the end of the list is reached or if no trade or open position is found.

The for loops are a little different to normal for loops in that they don’t have a standard loop index variable like ‘i’ that gets incremented at the end of each pass. Instead they call a Backtester object method to get the initial value (the first Trade object) and then another member to get the next value (the next Trade object). So the for loop conditions here are just saying start from the first Trade object, at the end of each pass get the next Trade object, and keep doing that until there are no more Trade objects (ie. “trade” is Null). The loops are iterating through the list of trades, not the bars on a chart. Each Trade object holds the details for a single trade.

Putting that code inside the custom backtest template looks like this:

SetCustomBacktestProc("");
if (Status("action") == actionPortfolio)
{
    bo GetBacktesterObject();    //  Get backtester object
    bo.Backtest();    //  Run backtests
    for (trade bo.GetFirstTrade(); tradetrade bo.GetNextTrade())
    {
        . . . .    //  Use Trade object here
    }
    for (trade bo.GetFirstOpenPos(); tradetrade bo.GetNextOpenPos())
    {
        . . . .    //  Use Trade object here
    }
    myMetric = <some result from Trade object calculations>;
    bo.AddCustomMetric("MyMetric"myMetric);    //  Add metric to display
}

As an example, say we want to calculate the average number of calendar days that winning trades were held for (there’s already a built-in Stats object value for number of bars, but we want number of calendar days). For that we’ll need a function that can calculate the number of calendar days between two dates. Let’s call it “DayCount”, a function that takes two parameters: the entry date and the exit date, both in AmiBroker date/time format. Since this document is about the custom backtester interface, I don’t want to go into how that function works right now. Let’s just assume it does, but the code for such a function is given in Appendix A if you’re interested. Then, for each trade we’ll need to know:

  1. If it was a winning or losing trade
  2. The entry date
  3. The exit date

And to calculate the average, we’ll need a total figure for the number of winning trade days and another total figure for the number of trades. The average is the total number of days winning trades were held divided by the total number of winning trades.
For the trade details, the Trade object has the following properties:

  1. EntryDateTime The entry date & time
  2. ExitDateTime The exit date & time
    and the following method:

  3. GetProfit() The profit for the trade

Before trying this example, the first time we’ve used this Trade object method, we make the assumption that the profit will be negative for losing trades and positive for winning trades, as AmiBroker help doesn’t clarify that detail (it could be some undefined value for losing trades). If trial and error proves that not to be the case, then we could alternatively try using the Trade object properties EntryPrice, ExitPrice, and IsLong to determine if it was a winning or losing trade. As it turns out upon testing, GetProfit does in fact work as expected.

Note that the Trade object also has a property called BarsInTrade, which looks like it could potentially be used instead of the dates, but that only gives the number of bars, not the number of calendar days.
So, to get the number of calendar days spent in a trade, we call our DayCount function passing the entry and exit dates:
DayCount(trade.EntryDateTime, trade.ExitDateTime);
and to determine if it was a winning trade, where break-even doesn’t count as winning:
trade.GetProfit() > 0;
The whole procedure would then be:

SetCustomBacktestProc("");
if (Status("action") == actionPortfolio)
{
    bo GetBacktesterObject();    //  Get backtester object
    bo.Backtest();    //  Run backtests
    totalDays 0;    //  Total number of winning days
    totalTrades 0;    //  Total number of winning trades
    for (trade bo.GetFirstTrade(); tradetrade bo.GetNextTrade())
    {    //  Loop through all closed trades
        if (trade.GetProfit() > 0)    //  If this was a winning trade
        {
            totalDays totalDays DayCount(trade.EntryDateTimetrade.ExitDateTime);
            totalTrades++;  
        }
    }    //  End of for loop over all trades
    avgWinDays totalDays totalTrades;    //  Calculate average win days
    bo.AddCustomMetric("AvgWinDays"avgWinDays);    //  Add to results display
}

Note that we only need to consider closed trades in this example, as counting open positions would not accurately reflect the number of days trades were typically held for. Also, the “totalTrades” variable only counts winning trades, not all trades, since we’re only averaging over winning trades.
When a backtest is run with this custom interface and a report generated, our new metric “avgWinDays” will be printed at the bottom of the report:
cbt1.GIF

And if we run an optimisation (using a different backtest to above), it will have a column near the right-hand end of the results:

cbt2.GIF

Note that the reason the “W. Avg Bars Held” column doesn’t seem to agree with the “AvgWinDays” column (ie. the former goes down while the latter goes up) is because the average bars figure includes open positions at the end of the backtest whereas we specifically excluded them.
As well as overall metrics per backtest, it’s also possible to include individual trade metrics in the backtester results. For this, the metric is added to each Trade object rather than the Backtester object and the trades are listed at the end of the procedure.
For example, to display the entry position score value against each trade in the backtester results, the following code could be used:

SetCustomBacktestProc("");
if (Status("action") == actionPortfolio)
{
    bo GetBacktesterObject();    //  Get backtester object
    bo.Backtest(True);            //  Run backtests with no trade listing
    for (trade bo.GetFirstTrade(); tradetrade bo.GetNextTrade())
        trade.AddCustomMetric("Score"trade.Score);    //  Add closed trade score
    for (trade bo.GetFirstOpenPos(); tradetrade bo.GetNextOpenPos())
        trade.AddCustomMetric("Score"trade.Score);    //  Add open pos score
    bo.ListTrades();            //  Generate trades list
}

The first for loop iterates through the closed trade list and the second through the open position list to get the entry score value for every trade listed in the results. Note that the bo.BackTest call is passed the value “True” in this case to prevent the list of trades being generated automatically by the backtester. Instead, they’re generated by the subsequent call to the bo.ListTrades method.
As another example, say we want to list for each winning trade how far above or below the average winning profit it was as a percentage, and similarly for each losing trade, how far above or below the average loss it was as a percentage. For this we need the “WinnersAvgProfit” and “LosersAvgLoss” values from the Stats object, and the profit from the Trade objects for each closed trade (for this example we’ll ignore open positions). Relative loss percentages are displayed as negative numbers.

SetCustomBacktestProc("");
if (Status("action") == actionPortfolio)
{
    bo GetBacktesterObject();    //  Get backtester object
    bo.Backtest(True);            //  Run backtests with no trade listing
    stat bo.GetPerformanceStats(0);    //  Get Stats object for all trades
    winAvgProfit stat.GetValue("WinnersAvgProfit");
    loseAvgLoss stat.GetValue("LosersAvgLoss");
    for (trade bo.GetFirstTrade(); tradetrade bo.GetNextTrade())
    {                    //  Loop through all closed trades
        prof trade.GetProfit();        //  Get trade profit in dollars
        relProf 0;            //  This will be profit/avgProfit as %
        if (prof 0)            //  If a winner (profit > 0)
            relProf prof winAvgProfit 100;    //  Profit relative to average
        else                //  Else if a loser (profit <= 0)
            relProf = -prof loseAvgLoss 100;    //  Loss relative to average
        trade.AddCustomMetric("Rel Avg Profit%"relProf);    //  Add metric
    }                    //  End of for loop over all trades
    bo.ListTrades();            //  Generate list of trades
}

Mid-Level Interface

To be able to modify actual backtest behaviour, the mid-level or low-level interfaces must be used. New metrics can also be calculated at these levels, but since that’s already covered above, this section will only look at what backtest behaviour can be modified at this level. Essentially this means using Signal objects as well as the Backtester object.
With the mid-level interface, each trading signal at each bar can be examined and the properties of the signals changed, based on the value of other Signal or Backtester object properties, before any trades are executed for that bar. For example, one Backtester object property is “Equity”, which gives the current portfolio equity, and one Signal object property is “PosSize”, the position size specified in the main AFL code, so the mid-level interface can allow, for example, position size to be modified based on current portfolio equity.
The custom backtester interface template for a mid-level approach, where all the signals at each bar need to be examined, is:

SetCustomBacktestProc("");
if (Status("action") == actionPortfolio) {
    bo GetBacktesterObject();    //  Get backtester object
    bo.PreProcess();    //  Do pre-processing (always required)
    for (0BarCounti++)    //  Loop through all bars
    {
        for (sig bo.GetFirstSignal(i); sigsig bo.GetNextSignal(i))
        {    //  Loop through all signals at this bar 
                . . . .
        }    //  End of for loop over signals at this bar
        bo.ProcessTradeSignals(i);    //  Process trades at bar (always required)
    }    //  End of for loop over bars
    bo.PostProcess();    //  Do post-processing (always required)
}

In this example, the variable “sig” is an object variable of type Signal, meaning it holds a Signal object. As with the Trade object in the earlier example, the inner for loop iterates through the list of signals at each bar, not through all bars on a chart. The for loop conditions are effectively saying start from the first Signal object for the current bar, at the end of each pass get the next Signal object for the same bar, and keep doing that until there are no more Signal objects for the bar (ie. “sig” is Null). Each Signal object holds the details of one signal at the current bar (ie. a buy, sell, short, cover or scale indication for one symbol).
The main differences between the mid-level and high-level approaches are:

  1. The Backtester object’s Backtest method is not called.
  2. The Backtester object’s ProcessTradeSignals method is called instead at each bar, after examining and possibly modifying some of the Signal object properties and/or closed or open Trade object properties.
  3. A loop is required to iterate through all bars of the chart.
  4. A nested loop is required inside that one to iterate through all the signals at each of those bars.

If a trading decision needs to be based on some other property of a particular stock, like it’s average daily trading volume for example, then the stock code symbol must be used to obtain that information. This is available in the Signal object’s “Symbol” property. However, since the backtester at this level is not run in the context of a particular symbol, the data must be saved to a composite symbol in the main code (or perhaps a static variable) and referenced in the custom backtest procedure with the Foreign function. For example, in the main AFL code:

AddToComposite(EMA(Volume100), "~evol_"+Name(), "V"atcFlagDefaults atcFlagEnableInBacktest);

Here the volume EMA array is saved to a separate composite symbol for each stock (ie. each composite consists of just a single stock). For this to work in backtests, the atcFlagEnableInBacktest flag must be used. Then in the custom backtest procedure:

evol Foreign("~evol_"+sig.Symbol"V");  //  Get symbol's volume array
evi evol[i];    //  Reference a value in the array

As a real example, to limit the number of shares purchased to a maximum of 10% of the 100 day EMA of the daily volume, and also ensure the position size is no less than $5,000 and no more than $50,000, the following mid-level procedure could be used:

SetCustomBacktestProc("");
if (Status("action") == actionPortfolio)
{
    bo GetBacktesterObject();    //  Get backtester object
    bo.PreProcess();    //  Do pre-processing
    for (0BarCounti++)    //  Loop through all bars
    {
        for (sig bo.GetFirstSignal(i); sigsig bo.GetNextSignal(i))
        {    //  Loop through all signals at this bar
            if (sig.IsEntry() && sig.IsLong())    //  If this signal is a long entry (ie. buy)
            {
                evol Foreign("~evol_"+sig.Symbol"V");   //  Get stock's composite volume array
                psize sig.PosSize;    //  Get position size specified in AFL code
                if (psize 0)    //  If it's negative (a percentage of equity)
                    psize = (-psize/100) * bo.Equity;  //  Convert to dollar value using current equity
                scnt psize sig.Price;    //  Calculate number of shares for position size
                if (scnt evol[i] / 10)    //  If number of shares is > 10% of volume EMA
                {
                    scnt evol[i] / 10;    //  Limit number of shares to 10% of EMA value
                    psize scnt sig.Price;    //  Calculate new position size
                }
                if (psize 5000)    //  If position size is less than $5,000
                    psize 0;    //  Set to zero so buy signal will be ignored
                else
               {
                    if (psize 50000)    //  If position size is greater than $50,000
                        psize 50000;    //  Limit to $50,000
                }
                sig.PosSize psize;    //  Set modified position size back into object
            }
        }    //  End of for loop over signals at this bar
        bo.ProcessTradeSignals(i);    //  Process trades at this bar
    }    //  End of for loop over bars
    bo.PostProcess();    //  Do post-processing
}

In this example, the statement psize = (-psize/100) * bo.Equity converts the percentage of equity value (which is negative) to its actual dollar value, using the Backtester object’s Equity property. The term -psize/100 (which doesn’t actually need to be inside brackets) converts the negative percentage to a positive fraction which is then multiplied by the current portfolio equity.
The statement if (sig.IsEntry() && sig.IsLong()) calls the two Signal object methods IsEntry and IsLong to determine if the current signal is an entry signal and a long signal (ie. a buy signal). Remember that the && operator is equivalent to AND. An alternative would be to check if the Signal object’s Type property was equal to one.
The array variable “evol” contains the whole EMA array realigned to the number of bars used by the custom backtest procedure. Padded bars don’t matter here as there won’t be any signals for the stock at any of those bars, and we’re only checking the volume on bars where there is a signal. As “evol” is an array, at each bar we’re only interested in the value for the current bar, hence the references to evol[i].
Finally, as detailed in the AmiBroker help, the Signal object’s Price property gives the price for the current signal, so there’s no need to use BuyPrice, SellPrice, etc., and the PosSize property is the signal’s position size value for the current bar. As this is not a read-only property, it can be both read and modified.
Another example, to prevent scaling in a position that already has $50,000 or more in open position value:

SetCustomBacktestProc("");
if (Status("action") == actionPortfolio)
{
    bo GetBacktesterObject();    //  Get backtester object
    bo.PreProcess();    //  Do pre-processing
    for (0BarCounti++)    //  Loop through all bars
    {
        for (sig bo.GetFirstSignal(i); sigsig bo.GetNextSignal(i))
        {    //  Loop through all signals at this bar
            if (sig.Type == 5)    //  If signal type is scale-in
            {
                trade bo.FindOpenPos(sig.Symbol);   //  Check for open position in stock
                if (trade)    //  Or could use "if (!IsNull(trade))"
                {
                    if (trade.GetPositionValue() >= 50000)  //  If open position value >= $50,000
                        sig.PosSize 0;    //  Set position size to zero to prevent purchase
                }
            }
        }    //  End of for loop over signals at this bar
        bo.ProcessTradeSignals(i);    //  Process trades at this bar
    }    //  End of for loop over bars
    bo.PostProcess();    //  Do post-processing
}

In this example, as each new scale-in signal is detected, the list of open positions is checked for an open position in the same stock as the new signal. If an open position exists, its current value is obtained, and if that value is $50,000 or more, the position size is set to zero to prevent the scale-in from happening.
The example combines use of the Backtester object, Signal objects and Trade objects to determine whether or not scale-in of a position should be permitted. Note that the Trade object is returned Null if no open position is found. As any comparison with a null value is always false, provided the test is for the True condition then the IsNull function is not needed: ie. “if (trade)” gives the same result as “if (!IsNull(trade))”. However, if the test is for the negative condition, IsNull is required: ie. “if (!trade)” won’t work (when “trade” is Null it will be treated as False rather than the desired True) and “if (IsNull(trade))” becomes necessary.

Low-Level Interface

The low-level interface provides the most flexibility to control backtester operation. As well as allowing signal properties to be modified, it also allows the entering, exiting, and scaling of trades even if no signal exists.
With the low-level interface, each trading signal at each bar can be examined, the properties of the signals changed, and trades entered, exited, and scaled. This could be used to implement special stop conditions not provided in the ApplyStop function, or to scale trades based on current portfolio equity or open position value and the like.
The custom backtester interface template for a low-level approach is:

SetCustomBacktestProc("");
if (Status("action") == actionPortfolio)
{
    bo GetBacktesterObject();    //  Get backtester object
    bo.PreProcess();    //  Do pre-processing
    for (0BarCounti++)    //  Loop through all bars
    {
        for (sig bo.GetFirstSignal(i); sigsig bo.GetNextSignal(i))
        {    //  Loop through all signals at this bar
            . . . .
        }    //  End of for loop over signals at this bar
        bo.HandleStops(i);    //  Handle programmed stops at this bar
        bo.UpdateStats(i1);    //  Update MAE/MFE stats for bar
        bo.UpdateStats(i2);    //  Update stats at bar's end
    }    //  End of for loop over bars
    bo.PostProcess();    //  Do post-processing
}

Note that this template currently has no trades performed in it, as there are a number of options there depending on the system. Typically, inside the signal loop (or possibly the trades loop) there will be a number of tests for various conditions and then trades entered, exited, and scaled accordingly.
The main differences between the low-level and mid-level approaches are:

  1. The Backtester object’s ProcessTradeSignals method is not called.
  2. The Backtester object’s EnterTrade, ExitTrade, and ScaleTrade methods are called instead at each bar, after examining and possibly modifying some of the signal properties and/or closed or open trade properties.
  3. The Backtester object’s HandleStops method must be called once per bar to apply any stops programmed in the settings or by the ApplyStop function.
  4. The Backtester object’s UpdateStats method must be called at least once for each bar to update values like equity, exposure, MAE/MFE, etc. The AmiBroker help is a little vague on how the TimeInsideBar parameter works (the values ‘1’ & ‘2’ in the sample above), but it must be called exactly once with that parameter set to two. It should also be called with it set to one to update the MAE/MFE statistics, but why it would be called with the value set to zero or more than once, I’m not sure.

As an example, let’s create a custom backtest procedure that scales in a long position by 50% of its injected capital (ie. excluding profit) whenever its open position profit exceeds its total injected capital, which means it’s sitting on 100% or more profit. The scale-in can be repeated whenever this condition occurs, as immediately after each scale-in, the injected capital will go up by 50%. The system doesn’t do any shorting and no other scaling occurs.

The required conditions therefore are:

  1. The profit must be greater than the injected capital to scale in.
  2. The scale-in position size is equal to half the injected capital.
  3. No signal is required to perform the scale-in.

The Signal object list is still needed to enter and exit all trades, as there’s no other mechanism to do that, but just the Trade object list is needed for scaling open positions. At each bar, each open long position in the trade open position list must be tested for scaling in, and a scale-in performed if the conditions are met.
The test for scale-in then looks like this:

trade.GetProfit() >= trade.GetEntryValue();    //  Entry value is injected capital

The scale-in position size is:

scaleSize trade.GetEntryValue() / 2;    //  Half of total injected capital

And the scale-in method call, using the closing price for scaling, is:

bo.ScaleTrade(itrade.SymbolTruetrade.GetPrice(i"C"), scaleSize);

Putting it all into our template gives:

SetCustomBacktestProc("");
if (Status("action") == actionPortfolio)
{
    bo GetBacktesterObject();    //  Get backtester object
    bo.PreProcess();    //  Do pre-processing
    for (0BarCounti++)    //  Loop through all bars
    {
        for (sig bo.GetFirstSignal(i); sigsig bo.GetNextSignal(i))
        {    //  Loop through all signals at this bar
            if (sig.IsEntry() && sig.IsLong())    //  Process long entries
                bo.EnterTrade(isig.SymbolTruesig.Pricesig.PosSize);
            else
            {
                if (sig.IsExit() && sig.IsLong())    //  Process long exits
                    bo.ExitTrade(isig.Symbolsig.Price);
            }
        }    //  End of for loop over signals at this bar
        bo.HandleStops(i);    //  Handle programmed stops at this bar
        for (trade bo.GetFirstOpenPos(); tradetrade bo.GetNextOpenPos())
        {    //  Loop through all open positions
            if (trade.GetProfit() >= trade.GetEntryValue())  //  If time to scale-in
            {
                scaleSize trade.GetEntryValue() / 2;    //  Scale-in the trade
                bo.ScaleTrade(itrade.SymbolTruetrade.GetPrice(i"C"), scaleSize);
            }
        }    //  End of for loop over trades at this bar
        bo.UpdateStats(i1);    //  Update MAE/MFE stats for bar
        bo.UpdateStats(i2);    //  Update stats at bar's end
    }    //  End of for loop over bars
    bo.PostProcess();    //  Do post-processing
}

Since we stated that the system doesn’t do any shorting, the tests for sig.IsLong aren’t really necessary.
The signal for loop processes all entry and exit signals generated by our buy and sell conditions in the main AFL code. As mentioned above, this is necessary since we’re not calling the ProcessTradeSignals method now, as that’s a mid-level method. The trade open position for loop checks for and processes all scaling in. When an exit signal occurs, the whole position is closed.
Extending this example now to include our custom avgWinDays metric from the high-level interface example:

SetCustomBacktestProc("");
if (Status("action") == actionPortfolio)
{
    bo GetBacktesterObject();    //  Get backtester object
    bo.PreProcess();    //  Do pre-processing
    for (0BarCounti++)    //  Loop through all bars
    {
        for (sig bo.GetFirstSignal(i); sigsig bo.GetNextSignal(i))
        {    //  Loop through all signals at this bar
            if (sig.IsEntry() && sig.IsLong())    //  Process long entries
                bo.EnterTrade(isig.SymbolTruesig.Pricesig.PosSize);
            else
            {
                if (sig.IsExit() && sig.IsLong())    //  Process long exits
                    bo.ExitTrade(isig.Symbolsig.Price);
            }
        }    //  End of for loop over signals at this bar
        bo.HandleStops(i);    //  Handle programmed stops at this bar
        for (trade bo.GetFirstOpenPos(); tradetrade bo.GetNextOpenPos())
        {    //  Loop through all open positions
            if (trade.GetProfit() >= trade.GetEntryValue())  //  If time to scale-in
            {
                scaleSize trade.GetEntryValue() / 2;    //  Scale-in the trade
                bo.ScaleTrade(itrade.SymbolTruetrade.GetPrice(i"C"), scaleSize);
            }
        }    //  End of for loop over trades at this bar
        bo.UpdateStats(i1);    //  Update MAE/MFE stats for bar
        bo.UpdateStats(i2);    //  Update stats at bar's end
    }    //  End of for loop over bars
    totalDays 0;    //  Total number of winning days
    totalTrades 0;    //  Total number of winning trades
    for (trade bo.GetFirstTrade(); tradetrade bo.GetNextTrade())
    {    //  Loop through all closed trades (only)
        if (trade.GetProfit() > 0)    //  If this was a winning trade
        {
            totalDays totalDays DayCount(trade.EntryDateTimetrade.ExitDateTime);
            totalTrades++;  
        }
    }    //  End of for loop over all trades
    avgWinDays totalDays totalTrades;    //  Calculate average win days
    bo.AddCustomMetric("AvgWinDays"avgWinDays);    //  Add to results display
    bo.PostProcess();    //  Do post-processing
}

Note that stops are handled before scale-in checking occurs, as there’s no point scaling in a trade if it’s about to get stopped out on the same bar (although it would be unlikely to satisfy the scale-in condition anyway if it was about to get stopped out).
Also note that the Trade object method GetEntryValue returns the total amount of injected capital, including all previous scale-in amounts. It’s not possible to get just the amount used in the initial purchase. It would actually be nice here if the Trade object had a few user-defined properties, to allow the user to persist any values they wanted to throughout the life of a trade (although this could also be done with static variables). For example, as mentioned above, the initial purchase amount before any scaling could be remembered, or perhaps the number of times scaling has occurred (your system may want to limit scaling in to a maximum of say three times).
Another similar example, but this time scaling out a position once it has doubled in value, removing the initial capital invested (approximately):

SetCustomBacktestProc("");
if (Status("action") == actionPortfolio)
{
    bo GetBacktesterObject();    //  Get backtester object
    bo.PreProcess();    //  Do pre-processing
    for (0BarCounti++)    //  Loop through all bars
    {
        for (sig bo.GetFirstSignal(i); sigsig bo.GetNextSignal(i))
        {    //  Loop through all signals at this bar
            if (sig.IsEntry() && sig.IsLong())    //  Process long entries
            {
                bo.EnterTrade(isig.SymbolTruesig.Pricesig.PosSize);
                trade bo.FindOpenPos(sig.Symbol);    //  Find the trade we just entered
                if (trade)    //  Or "if (!IsNull(trade))"
                    trade.MarginLoan 0;    //  On initial buy, zero margin loan property
            }
            else
            {
                if (sig.IsExit() && sig.IsLong())    //  Process long exits
                    bo.ExitTrade(isig.Symbolsig.Price);
            }
        }    //  End of for loop over signals at this bar
        bo.HandleStops(i);    //  Handle programmed stops at this bar
        for (trade bo.GetFirstOpenPos(); tradetrade bo.GetNextOpenPos())
        {    //  Loop through all open positions
            ev trade.GetEntryValue();    //  Entry value of trade (ie. initial capital)
            if (!trade.MarginLoan && trade.GetProfit() >= ev)   //  Only if MarginLoan is zero
            {
                trade.MarginLoan 1;    //  Indicate have scaled out once now
                bo.ScaleTrade(itrade.SymbolFalsetrade.GetPrice(i"C"), ev);    //  Scale out
            }
        }    //  End of for loop over trades at this bar
        bo.UpdateStats(i1);    //  Update MAE/MFE stats for bar
        bo.UpdateStats(i2);    //  Update stats at bar's end
    }    //  End of for loop over bars
    bo.PostProcess();    //  Do post-processing
}

In this example we only want to do the scale-out once, which introduces a new problem: how do we tell whether we’ve already done it or not? Trial and error shows that the entry value returned by the GetEntryValue method halves if you remove half of the value, so AmiBroker appears to treat a scale-out of half the value as being half profit and half original capital. As mentioned above, we really need a Trade object property here that we can write to with our own information. Since we’re not using margin, we can use the MarginLoan property, which fortunately is not read-only. I tried to use the Score property first, but that turned out to be read-only, despite AmiBroker help not mentioning that fact.
This example is mostly the same as the previous one, but instead of scaling in, we now scale out. Again, the trigger condition is the profit being greater than the entry value (injected capital), but we need to use a state variable to remember whether or not we’ve already scaled out the position so that we only do it once. As mentioned above, we can’t tell this from the entry value alone. While the MarginLoan property was available and writeable in this case, it would be much better, as already mentioned, if Trade objects had some user-definable properties.
And once again as a reminder, since I use C and C++ syntax rather than the syntax defined in AmiBroker help, !trade.MarginLoan is the same as NOT trade.MarginLoan and && is equivalent to AND. The statement !trade.MarginLoan just means if trade.MarginLoan equals zero.

Conclusion

That pretty much covers the use of the custom backtester interface at all three levels. While there are a number of object properties and methods I haven’t mentioned or used, this document is not intended to be a reference manual but rather an introduction to using the interface. There should be enough information here to allow you to figure out the rest for yourself with a bit of trial and error (as I’ve had to use myself while writing this document).

Computer programming in any language can be a rewarding, but at times extremely frustrating, experience. After many hours of trying to get your “simple” piece of code working properly, by which time you’re ready to swear on your grandmother’s grave that there has to be a problem with the language interpreter or compiler, almost invariably the problem is in your own code. It could be as simple as a missing semicolon, or as complex as a complete misunderstanding about how something is supposed to work. But as Eric Idle once said, always look on the bright side of life. The good thing about an extremely frustrating problem is that it feels SO good once you finally figure it out!

Appendix A – DayCount Function

The code for the DayCount function used to calculate the number of calendar days between two date/time values is below. This includes both entry and exit days in the count. It consists of two functions, the DayCount function itself, and a DayInYear function to calculate the current day number in a year for a particular date.
Firstly, the DayInYear function:

function DayInYear(ydayymonthyyear)
{
    doy yday;    //  Set number of days to current day
    for (1ymonthi++)    //  Loop over all months before this one
    {
        switch (i)    //  Sum number of days in each month
        {
            case 1:
            case 3:
            case 5:
            case 7:
            case 8:
            case 10:
            case 12:
                doy doy 31; break;    //  Months with 31 days
            case 4:
            case 6:
            case 9:
            case 11:
                doy doy 30; break;    //  Months with 30 days
            case 2:
            {
                doy doy 28;    //  February non-leap year
                if (!(yyear 4) && yyear != 2000)
                    doy++;    //  February leap year
                break;
            }
        }
    }
    return doy;    //  Return day in year, starting from 1
}

This gets called by the DayCount function for both the entry and exit days.
Now the DayCount function:

function DayCount(inDayoutDay)
{
    in DateTimeConvert(0inDay);    //  Convert entry to DateNum format
    out DateTimeConvert(0outDay);    //  Convert exit date

    iyy int(in 10000) + 1900;    //  Get entry year
    imm int((in 10000) / 100);    //  Month
    idd in 100;    //  Day
    doyi DayInYear(iddimmiyy);    //  Calculate entry day in year

    oyy int(out 10000) + 1900;    //  Get exit year
    omm int((out 10000) / 100);    //  Month
    odd out 100;    //  Day
    doyo DayInYear(oddommoyy);    //  Calculate exit day in year

    days 0;    //  Initialise days between to zero
    for (iyyoyyi++)    //  Loop from entry year to < exit year
    {
        if (!(4) && != 2000)    //  If is a leap year
            days days 366;    //  Has 366 days
        else
            days days 365;    //  Else has 365 days
    }
    days days doyo doyi 1;    //  Days/year plus exit minus entry day
    //  Plus one to include both dates
    return days;    //  Return total days between dates
}

Appendix B – Using DebugView

This appendix discusses the use of the Microsoft SysInternals program DebugView for debugging AFL applications. DebugView can be obtained from the Microsoft website here:
http://www.microsoft.com/technet/sysinternals/Miscellaneous/DebugView.mspx
When you run the program, you will get a window like this:

cbt3.GIF

The display area is where your AFL application can write to using _TRACE statements. Note though, as can be seen above, that your application may not be the only thing sending data to the viewer. DebugView captures all data sent to the viewer from all running applications.
The main toolbar controls are:

cbt4.GIF

The Clear Display button is the one you’ll likely use the most while debugging an application. And as with most applications, you can multi-select lines in the output display and use Edit->Copy (Ctrl+C) to copy them to the Clipboard for pasting into another application for further analysis.
Using The AFL _TRACE Statement
To output messages to the viewer from your AFL code, including from custom backtest procedures, you use the _TRACE statement:

_TRACE("Entered 'j' for loop");

You can concatenate strings simply by “adding” them together:

_TRACE("Processing symbol " trade.Symbol);

To include the value of parameters in the message, use the StrFormat function the same as for Plot statements:

_TRACE(StrFormat("Buying " sig.Symbol ", price = %1.3f"sig.Price));

A sample trace while testing the first low-level example given in this document:

That output was produced by the following code in the custom backtest procedure:

isAqp trade.Symbol == "AQP";
if (isAqp)
    _TRACE(StrFormat("Scaling in " trade.Symbol " at bar %1.0f, entry value = %1.3f"itrade.GetEntryValue()));
scaleSize trade.GetEntryValue() / 2;
if (isAqp)
    _TRACE(StrFormat("Profit = %1.3f, Value = %1.3f, price = %1.3f, scaleSize = %1.3f"trade.GetProfit(), trade.GetPositionValue(), trade.GetPrice(i"C"), scaleSize));
bo.ScaleTrade(itrade.SymbolTruetrade.GetPrice(i"C"), scaleSize);

Remember that as newlines are considered white space by the language, one statement can be spread over multiple lines for readability without affecting its operation. The only thing to be aware of is where a single string inside double quotes needs to span multiple lines. White space in a string is treated as exactly what it is, so if you put a line break in the middle of it, you will end up with a line break in your output (this is not true in all languages, but is with AFL as far as tracing goes). Instead, you can split it into two strings and concatenate them:

_TRACE(StrFormat("Profit = %1.3f, Value = %1.3f, price = %1.3f, " "scaleSize = %1.3f"trade.GetProfit(), trade.GetPositionValue(), trade.GetPrice(i"C"), scaleSize));

In the end though, this is only for readability purposes. As far as the language goes, it doesn’t matter if a single line is 1000 characters long and scrolls off the right-hand end of the screen. It just makes it more difficult to read the line when you’re working on that part of the code.

There’s little else that can be said about using DebugView for debugging your AFL code. Debugging is something of an art, and knowing what sort of information to trace at which parts of the code is something you’ll get better at the more you do it. Too little information and you can’t tell what’s happening. Too much and it can be like looking for the proverbial needle in a haystack.

Appendix C – Lichello AIM Algorithm

/*     The AIM algorithm is a monthly long position algorithm that buys as
        prices fall and sells as prices rise. It maintains a position control
        amount and then uses a buy safe and sell safe percentage to determine
        how much to buy or sell each month.

        Each stock is treated separately, with its own amount of allocated
        funds. This can be split into an initial purchase amount, which
        becomes the initial position control amount, and a cash component.
        These are the only funds available for the stock. On the last bar of
        each month, the algorithm below is followed:

        - Is the current portfolio value higher than the position control?
        - If so, check if sell any, if not, check if buy any.
        - If sell, calculate Value*(1-sellSafe)-positionControl. That is
          the dollar value of sale.
        - If buy, calculate positionControl-Value*(1+buySafe). That is
          the dollar value of purchase.
        - Check if the amount is greater than the minimum trade value.
        - If sell, check if already maximum cash balance and do vealie if so.
          Vealie is just add sellValue/2 to position control. Otherwise sell
          specified amount as scale-out.
        - If buy, check if sufficient funds available. If not, reduce
          purchase to remaining funds value (minus brokerage). If that is
          still greater than minimum trade, can still buy. Buy calculated
          value of shares as scale-in. Increase position control by buyValue/2.
        - Adjust cash balance of stock to allow for share sale or purchase,
          including brokerage.

        This implementation adds the following:

        - Buy signal for initial purchase based on relative positions of three
          EMAs and slope of the longest period one.
        - After initial purchase, scale-in and scale-out used for all trades.
        - Maximum loss stop used to sell out position if maximum loss reached.
        - Buy signals have random score to help with Monte Carlo testing.

        As the only sell signal is the maximum loss stop, once the initial
        equity has been used up in buying stocks and their cash balances, no
        more stocks will be bought until one of the purchased ones has been
        stopped out.

        This routine tracks the stock quantity and cash balance of each
        stock independantly of the backtester object. It will prevent
        purchases of new stocks if no cash is available even though the
        backtester may still show a positive overall cash balance, as the
        backtester cash balance includes the cash balances of each of the
        purchased stocks. In other words, the cash part of each stock
        position is still kept in the backtester's cash balance but is
        reserved for just that one stock until it gets sold if stopped out.

        The whole AIM algorithm is implemented in the low-level custom
        backtest procedure. The main AFL code just collects the parameters
        and passes them as static variables to the custom backtest procedure,
        and sets up the initial Buy array based on the EMA conditions.

        For more information on the AIM algorithm see the website:

        http://www.aim-users.com

        and follow the links "AIM Basics" and "AIM Improvements".
*/

//===============================================================

/*      Check if the last bar of a calendar month ....

        Checks if the passed bar is the last bar of a calendar month (which does not necessarily mean the last day of the month). It does this by checking if the next bar is in a different month. As it has to look ahead one bar for that, it cannot check the very last bar, and will
        always report that bar as not being the last bar of the month.

        "dn" is the DateNum array
        "bar" is the bar to check
*/

function IsEOM(dnbar)
{
    rc False;
    if (bar >= && bar BarCount-1)
    {
        mm Int((dn[bar] % 10000) / 100);    //  Month of passed bar
        mmn Int((dn[bar+1] % 10000) / 100);    //  Month of next bar
        rc mmn != mm;    //  End of month if not same
    }
    return rc;
}
SetCustomBacktestProc("");    //  Start of custom backtest procedure
if (Status("action") == actionPortfolio)
{
    bo GetBacktesterObject();
    bo.PreProcess();
    totalCash StaticVarGet("totalCashG");    //  Get global static variables
    iniPos StaticVarGet("iniPosG");
    iniCash StaticVarGet("iniCashG");
    buySafe StaticVarGet("buySafeG");
    sellSafe StaticVarGet("sellSafeG");
    minTrade StaticVarGet("minTradeG");
    maxCash StaticVarGet("maxCashG");
    maxLoss StaticVarGet("maxLossG");
    brok StaticVarGet("brokG");
    monteCarlo StaticVarGet("monteCarloG");
    dn DateNum();    //  Array for finding end of month
    for (0BarCount-1i++)    //  For loop over all bars
    {
        if (IsEOM(dni))    //  Scale trades only on last bar of month
        {
            for (trade bo.GetFirstOpenPos(); tradetrade bo.GetNextOpenPos())
            {
                qty StaticVarGet("qty"+trade.Symbol);    //  Current quantity for stock
                poCo StaticVarGet("poCo"+trade.Symbol);  //  Current position control for stock
                cash StaticVarGet("cash"+trade.Symbol);    //  Current cash balance for stock
                value trade.Shares*trade.GetPrice(i"C");  //  Current stock value
                profit trade.GetProfit();    //  Current trade profit
                bprice trade.GetPrice(i+1"C");    //  Potential buy price (tomorrow's price)
                sprice trade.GetPrice(i+1"C");    //  Potential sell price (tomorrow's price)
                if (profit / (iniPos iniCash) < -maxLoss)    //  If maximum loss reached
                {
                    bo.ExitTrade(itrade.Symbolsprice1);    //  Exit trade (stopped out)
                    exitVal cash qty*sprice brok;    //  Cash balance after sale
                    totalCash totalCash exitVal;    //  Update total system cash
                }
                else
                {
                    if (value poCo)    //  Increased in value, so look to sell
                    {
                        toSell value * (sellSafe) - poCo;    //  Value to sell
                        sshares Int(toSell sprice);    //  Number of shares to sell
                        if (sshares >= qty || toSell >= minTrade)  //  If more than min or all remaining
                        {
                            if (sshares qty)    //  Can't sell more than have
                                sshares qty;
                            sval sshares sprice;    //  Actual value to sell
                            if (cash maxCash)    //  If don't already have max cash
                            {
                                if (cash+sval maxCash)    //  If sale will give more than max cash
                                {
                                    sval maxCash cash;    //  Reduce sale to end with max cash
                                    sshares Int(sval sprice);
                                    sval sshares sprice;
                                }
                                ishares trade.Shares;    //  Number of shares have now
                                bo.ScaleTrade(itrade.SymbolFalsespricesval);    //  Sell the shares
                                soldShares ishares trade.Shares;  //  Number of shares sold
                                if (soldShares 0)    //  If actually sold some
                                {
                                    tval soldShares sprice;    //  Value of shares sold
                                    StaticVarSet("qty"+trade.Symboltrade.Shares);  // Store remaining qty
                                    StaticVarSet("cash"+trade.Symbolcash+tval-brok);  //  And cash
                                }
                            }
                            else    //  Have max cash already so do a vealie
                                StaticVarSet("poCo"+trade.SymbolpoCo+toSell/2);  //  The vealie
                        }
                    }
                    else    //  Decreased in value, so look to buy
                    {
                        toBuy poCo value * (buySafe);    //  Value to buy
                        if (toBuy cash-brok)    //  If don't have enough cash
                            toBuy cash-brok;    //  Reduce buy to remaining cash
                        if (toBuy >= minTrade)    //  If greater than minimum trade value
                        {
                            bshares Int(toBuy bprice);    //  Number of shares to buy
                            bpos bshares bprice;    //  Actual value of shares to buy
                            ishares trade.Shares;    //  Number of shares have now
                            bo.ScaleTrade(itrade.SymbolTruebpricebpos);  //  Buy the shares
                            boughtShares trade.Shares ishares;  //  Number of shares bought
                            if (boughtShares 0)    //  If actually bought some
                            {
                                tval boughtShares bprice;    //  Value of shares bought
                                StaticVarSet("qty"+trade.Symboltrade.Shares);  //  Store new quantity
                                StaticVarSet("poCo"+trade.SymbolpoCo+tval/2);  //  New pos control
                                StaticVarSet("cash"+trade.Symbolcash-tval-brok);  //  And cash
                            }
                        }
                    }
                }
            }    //  End of for loop over open positions
        }
        for (sig bo.GetFirstSignal(i); sigsig bo.GetNextSignal(i))    //  Check new buys
        {
            doBuy = !monteCarlo;    //  See if ignore for Monte Carlo testing
            if (monteCarlo)
            {
                 rand Random();
                 doBuy rand[i] >= monteCarlo;    //  "monteCarlo" is prob of ignoring buy
            }
                          if (doBuy && IsNull(bo.FindOpenPos(sig.Symbol)) && sig.IsEntry() && sig.IsLong() && sig.Price 0)    //  Can take initial entry signal for stock
                          {
                icash iniPos iniCash;    //  Initial cash value for stock position
                if (totalCash icash)    //  Ignore if not enough portfolio cash
                    break;
                ishares Int((iniPos-brok) / sig.Price);    //  Initial number of shares to buy
                ipos ishares sig.Price;    //  Value of shares to buy
                bo.EnterTrade(isig.SymbolTruesig.Priceipos);    //  Buy the shares
                trade bo.FindOpenPos(sig.Symbol);    //  Find trade for shares just bought
                if (!IsNull(trade))
                {
                    tval trade.GetEntryValue();    //  Value of shares
                    tshares trade.Shares;    //  Number of shares
                    StaticVarSet("qty"+sig.Symboltshares);    //  Store number of shares
                    StaticVarSet("poCo"+sig.Symboltval);    //  And position control (share value)
                    cash iniCash+iniPos-tval-brok;    //  Stock cash balance after purchase
                    StaticVarSet("cash"+sig.Symbolcash);    //  Store cash balance for stock
                    totalCash totalCash-iniCash-iniPos;    //  Subtract from portfolio cash
                }
            }
        }    //  End of for loop over buy signals
        bo.HandleStops(i);    //  Shouldn't be any stops
        bo.UpdateStats(i1);
        bo.UpdateStats(i2);    
    }    //  End for loop over bars
    for (trade bo.GetFirstOpenPos(); tradetrade bo.GetNextOpenPos())
    {    //  For all open positions at end of test
        qty StaticVarGet("qty"+trade.Symbol);    //  Number of shares remaining
        poCo StaticVarGet("poCo"+trade.Symbol);    //  And position control
        cash StaticVarGet("cash"+trade.Symbol);    //  And stock cash balance
        trade.AddCustomMetric("Shares"qty);    //  Add as metrics to trade list
        trade.AddCustomMetric("Value"qty*trade.GetPrice(BarCount-1"C"));
        trade.AddCustomMetric("PosCtrl"poCo);
        trade.AddCustomMetric("Cash"cash);
    }
    for (trade bo.GetFirstTrade(); tradetrade bo.GetNextTrade())
    {    //  For all closed (stopped out) trades
        poCo StaticVarGet("poCo"+trade.Symbol);    //  Final position control
        cash StaticVarGet("cash"+trade.Symbol);    //  And cash balance
        trade.AddCustomMetric("Shares"0);    //  Add as metrics to trade list
        trade.AddCustomMetric("Value"0);
        trade.AddCustomMetric("PosCtrl"poCo);
        trade.AddCustomMetric("Cash"cash);
    }
    bo.PostProcess();
}    //  End of custom backtest procedure
//================================================================

//      Start of main AFL code ....

totalCash Param("1. Total Cash (000)?"2010100010);    //  Get parameters
totalCash totalCash 1000;
iniPos Param("2. Initial Position (000)?"1001001);
iniPos iniPos 1000;
iniCash Param("3. Initial Cash (000)?"1001001);
iniCash iniCash 1000;
buySafe Param("4. Buy Safe?"1001001);
buySafe buySafe 100;
sellSafe Param("5. Sell Safe?"1001001);
sellSafe sellSafe 100;
minTrade Param("6. Minimum Trade?"500010000100);
maxCash Param("7. Maximum Cash (000)?"1000100010);
maxCash maxCash 1000;
maxLoss Param("8. Maximum Loss%?"2001001);
maxLoss maxLoss 100;
brok Param("9. Brokerage?"3001001);
monteCarlo Param("10. Monte Carlo%?"001001);
monteCarlo monteCarlo 100;

if (monteCarlo)    //  Probability of ignoring buy for Monte Carlo
    Optimize("monteCarlo"001001);    //  For running Monte Carlo test

SetOption("InitialEquity"totalCash);
SetOption("CommissionMode"2);
SetOption("CommissionAmount"brok);
SetTradeDelays(0000);

StaticVarSet("totalCashG"totalCash);    //  Set global static variables
StaticVarSet("iniPosG"iniPos);
StaticVarSet("iniCashG"iniCash);
StaticVarSet("buySafeG"buySafe);
StaticVarSet("sellSafeG"sellSafe);
StaticVarSet("minTradeG"minTrade);
StaticVarSet("maxCashG"maxCash);
StaticVarSet("maxLossG"maxLoss);
StaticVarSet("brokG"brok);
StaticVarSet("monteCarloG"monteCarlo);

e1 EMA(Close30);    //  EMA initial buy conditions
e2 EMA(Close60);
e3 EMA(Close180);
e3s LinRegSlope(e32);
bsig e3s && e1 e2 && e2 e3;

Buy bsig;
Sell False;    //  Only maximum loss stop to sell

PositionSize 0;    //  Calculated in custom routine
PositionScore Random();    //  Random position score for backtesting

The default parameters specified here are the AIM standard values, with $10K initial position, $10K cash, and 10% buy and sell safes. For vealies, the maximum cash balance for a stock defaults to $100K. To experiment with this algorithm in the manner it was intended, try it on individual stocks that have had significant swings but no overall trend. Strongly uptrending stocks will give the best results as the parameters approach buy and hold, with initial cash and buy safe of zero, and sell safe of 100%.

Note that the code uses trade.Shares*trade.GetPrice(i, “C”) for the current value, not trade.GetPositionValue. That’s because the latter function use’s the previous bar’s closing price to determine the current value, whereas we want the current bar’s price (it’s assumed that buy/sell tests are made after the close of trading). The actual prices then used are the next bar’s prices, to mimic making the trade the next trading day. Trade delays are set to zero to avoid confusion and conflict.

To run this code, copy everything in blue to an AFL file and then run it with the backtester. If you run it over a single stock, set the total cash value to be the sum of the initial position and initial cash values (the default setting), otherwise the backtest report won’t give a realistic result for the percentage return (most of the cash would never have been invested so would have zero gain for that component unless an annual interest rate was set). If running it over a portfolio, set the total cash value to be some multiple of the two initial values to allow that many positions to be entered simultaneously. Running it over a large watchlist of stocks will only pick a few positions, depending on the total cash available, with new positions subsequently only being opened if others are stopped out (note that the maximum loss stop is not part of the AIM algorithm, it’s my own addition).

If the backtester results report the trade list, there will only be one entry for each position, no matter how many times it scaled in and out. However, if it got stopped out and the same stock subsequently purchased again, that would show as two trades in the list. To see all the scale in and out trades, run the backtest in Detailed Log mode.

At the end of a backtest, the final quantity of shares, their value, the position control, and the cash balance figures are added to the Trade objects as custom metrics (one or two will be the same as existing metrics though). If the trade was closed, the quantity will be zero.
The parameters include a percentage for Monte Carlo testing. This is the probability of ignoring any particular new buy signal. A value of zero means all buys will be taken, subject to cash availability, while a value of 100 means none will be. The value shouldn’t be set too high otherwise the results might be unrealistic due to a sparsity of trades taken. I’d suggest a value up to 50%, with 25% being what I typically use myself. The less buy signals there are in the Buy array, the lower the value needs to be to avoid giving unrealistic results. To run a Monte Carlo test, set a percentage value and then run an optimisation. The random PositionScore array also helps with Monte Carlo testing.

Finally a disclaimer: while I’ve made every attempt to ensure this correctly implements the AIM algorithm as I have specified in the comments and accompanying text, I can’t guarantee that there are no errors or omissions or that this does in fact implement the algorithm correctly. I have presented it here primarily as a more advanced example of a custom backtest procedure, and all use is at your own risk. However, if you do find any errors, please let me know.

AFL Shapes Cheat Sheet

The plotshapes() can be used to plot shapes on your chart to indicate signals, stops, and other events and conditions. Figure 1 below gives a quick overview of the shapes that are available and includes a few undocumented ones. A PDF version suitable for printing is here: AFL Shapes Cheat Sheet

afl-shapes.png

Figure 1

Figure 2 shows the small AFL program that was used to explore all the built-in shapes and their numerical values.

for ( 0&ltBarCounti++ )
{
    O[i] = C[i] = i;
    H[i] = 5;
    L[i] = 5;

    if ( == )  {PlotTextNumToStri1.0False ), iL[i]-2colorDarkGrey );}
    else  {PlotTextNumToStri1.0False ), iH[i]+.5colorDarkGrey );}
}
PlotShapesCcolorRed0C, -10 );
PlotC""colorLightGreystyleBar );

Title "Hollow = " NumToStrshapeHollowCircle shapeCircle0) + "\n" + 
    "Small = " NumToStrshapeSmallCircle shapeCircle0);

aflshapesf2.png

Figure 2

With additions by Herman

Next Page »