Import from ASCII file

Import ASCII

AmiBroker has an easy-to-use and flexible quotation import feature. This document describes advanced concepts of the AmiBroker ASCII importer. Novice users should start with the ASCII Import Wizard.

How does it work?

Quotation data may come from various sources, so the format of the ASCII (i.e., text-based) file may be much different from one source to another. To handle all these differences, AmiBroker uses format definition commands that define the way text information is interpreted by the ASCII importer. The format definition commands are keywords that begin with a dollar sign ('$'). These commands may be embedded in the data file itself or may be stored in a separate format definition file for multiple use. Storing format definition commands in a separate file avoids the need to include the commands in every data file. The default format definition file name is "default.format". This file, all other ".format" files, and the "import.types" file (described later) should be stored in the \Formats subdirectory of AmiBroker's current working directory. The defaults are overridden by any commands included (embedded) in the data file itself.

So, when you use the "Import from ASCII" menu, AmiBroker first looks for the format definition stored in the "default.format" file and then parses the file you have chosen. If there is no "default.format" file, then it uses internal defaults (described below).

You can modify the "default.format" file to suit your needs. Moreover, using OLE Automation (Win32 version) or ARexx (Amiga) interface, you can specify the name of the format definition file which will be used instead of the "default.format" file.

Format definition commands

The command keywords begin with a dollar sign ('$'). Every line starting with a command is interpreted in a special way. Here is the list of commands recognized by AmiBroker's built-in importer. Bold letters mark keywords.

Command $ADDRESS Define company address

Arguments <string> address of company

Alias

Examples $ADDRESS "One Microsoft Way"

Command $AUTOADD Switch new ticker add mode.

Arguments <number> 0 - do not add; 1 - add a new stock when a non-existing ticker is detected (default = 0).

Alias

Examples $AUTOADD 1

Command $ALLOWNEG Allow negative numbers in prices

Arguments <number>
0 - do not allow negative values (default); 1 - allow negative values in prices. This additionally switches off any checking for OHLC relationships, so you can import any data into OHLC fields.

When $ALLOWNEG is NOT specified in the ASCII importer definition, AmiBroker performs the following range checking and fixup on open, low, and high prices:
if( open == 0 ) open = close;
if( high < max( open, close ) ) high = max( open, close );
if( low == 0 ) low = min( open, close )

Alias

Examples $ALLOWNEG 1

Command $ALLOW99SECONDS Convert invalid second stamp.

Arguments <onoff>
This flag works ONLY in conjunction with $TICKMODE 1 (see below for details).

$ALLOW99SECONDS set to 1 will convert all records with invalid seconds (i.e., greater than 59) to 59s. So a record stamped 16:29:70 will be treated as 16:29:59.

Alias

Examples $ALLOW99SECONDS 1

Command $APPENDNAME append string to the ticker name (useful when you need to join several fields together to make a unique stock symbol)

Arguments <string> string to append to the ticker symbol

Alias $APPENDTICKER

Examples

Command $BREAKONERR Define on-error behavior.

Arguments <number> 0 - to continue; 1 - to break import on error (default=0).

Alias

Examples $BREAKONERR 1

Command $CONT Define continuous quotations flag

Arguments <number> <0 or 1> - continuous quotations flag. This affects $AUTOADD 1 mode - if this is set, newly added stocks are switched to continuous quotation mode (this means enabling candlestick charts, for example).

Alias

Examples $CONT 1

Command $CURRENCY Define symbol's currency

Arguments <string> Defines the currency of the symbol.

Alias

Examples
$CURRENCY EUR

or

$FORMAT NAME, CURRENCY
$OVERWRITE 1
$AUTOADD 1

Command $DATE_DMY Define date

Arguments <number> The date in Canadian format (DD-MM-YY). If there is no argument given, the date is taken from the file name (without an extension).

Alias $DATE_CDN

Examples $DATE_DMY 12-05-99
$DATE_CDN 12-05-1999

Command $DATE_MDY Define date

Arguments <number> The date in US format (MM-DD-YY). If there is no argument given, the date is taken from the file name (without an extension).

Alias $DATE_USA

Examples $DATE_MDY 05/12/99
$DATE_USA 05/12/99

Command $DATE_YMD Define date

Arguments <number> The date in International format (YY-MM-DD). If there is no argument given, the date is taken from the file name (without an extension).

Alias $DATE_INT

Examples $DATE_INT 99-05-12
$DATE_CDN 1999.05.12

Command $DEBUG Switch logging (debug) mode

Arguments <number> 0 - no error logging; 1 - log errors to "import.log" file (default=0).

Alias

Examples $DEBUG 1

Command $FORMAT Define line format (sequence and types of fields)

Arguments DATE_MDY date in US format: MM-DD-YY (alias: DATE_USA)

DATE_DMY date in Canadian format: DD-MM-YY (alias: DATE_CDN)

DATE_YMD date in International format: YY-MM-DD (alias: DATE_INT)

TIME time in HH:MM:SS or HH:MM or HHMM or HHMMSS format

NAME ticker name (alias: TICKER)

ALIAS symbol alias ($AUTOADD and $OVERWRITE modes only)

FULLNAME symbol full name ($AUTOADD and $OVERWRITE modes only)

OPEN open price

HIGH high price

LOW low price

CLOSE close price

ADJCLOSE split-adjusted close

This is provided to read the adjusted close column from Yahoo. Works only in conjunction with the CLOSE field. When both CLOSE and ADJCLOSE are present in the ASCII format definition, the importer calculates the split factor by dividing ADJCLOSE by CLOSE. It then multiplies OPEN, HIGH, LOW, and CLOSE fields by this factor and divides the VOLUME field by this factor. This effectively converts unadjusted prices to split-adjusted prices. The split ratio gets locked once ADJCLOSE drops below 0.05.

OPENINT open interest

VOLUME volume

VOL1000 volume in thousands of shares

VOLMIL volume in millions of shares

VOLFACTOR volume factor (number of shares in a block) (default = 1)

TURNOVER turnover

AUX1 AUX1 field (auxiliary data)

AUX2 AUX2 field (auxiliary data)

SKIP skip (ignore) field

MARKET specify a field that contains market ID (affects $AUTOADD and $OVERWRITE modes only)

GROUP specify a field that contains group ID (affects $AUTOADD and $OVERWRITE modes only)

WATCHLIST specify a field that contains watch list number (0-31) (affects $AUTOADD and $OVERWRITE modes only)

INFO specify a field with additional information (WSE specific: nk, ns, rk, rs, ok, os, zd, bd).

REDUCTION specify a field with reduction rate in percentages (WSE specific)

ICB (new in 5.60)

Specify ICB code.

For example, if your file looks as follows:
(format is symbol, full name, ICB code)
AAN,AARON'S INC,5375

Then to import it using AmiBroker's import wizard, use the following:
$FORMAT Ticker,FullName,ICB
$OVERWRITE 1
$SEPARATOR ,
$CONT 1
$GROUP 255
$AUTOADD 1
$NOQUOTES 1

GICS
Specify GICS code.

For example, if your file looks as follows:
(format is symbol, full name, GICS sub-industry code)
AAN,AARON'S INC,25504060

Then to import it using AmiBroker's import wizard, use the following:
$FORMAT Ticker,FullName,GICS
$OVERWRITE 1
$SEPARATOR ,
$CONT 1
$GROUP 255
$AUTOADD 1
$NOQUOTES 1

INDUSTRY specify a field that contains industry ID (affects $AUTOADD and $OVERWRITE modes only)

INDUSTRYNAME
(new in 5.60) Specifies a field that contains the Industry Name. AmiBroker will check if a given industry name already exists and, if not, it will create a new Industry and assign the imported stock to the industry specified. Also, if SECTORNAME is specified, it will assign the newly added industry to the specified sector.

(affects $AUTOADD and $OVERWRITE modes only)

SECTORNAME
(new in 5.60) Specifies a field that contains the Sector Name. AmiBroker will check if a given sector name already exists and, if not, it will create a new Sector. Also, if INDUSTRYNAME is specified, it will assign the newly added industry to the specified sector.

(affects $AUTOADD and $OVERWRITE modes only)

APPENDTICKER specify a field that contains a string that should be appended to the ticker name (useful when you need to join several fields together to make a unique symbol).

MARGIN future contract margin deposit (positive value = dollars; negative value = percent of full value)

POINTVALUE future contract point value

ROUNDLOTSIZE round lot size (trading unit size)

TICKSIZE tick size

ADVISSUES number of advancing issues

ADVVOLUME volume of advancing issues

DECISSUES number of declining issues

DECVOLUME volume of declining issues

UNCISSUES number of unchanged issues

UNCVOLUME volume of unchanged issues

ADDRESS street address of company

CURRENCY specifies the currency of the symbol.

WEBID specifies the web ID.

DIV_PAY_DATE
EX_DIV_DATE
LAST_SPLIT_DATE
LAST_SPLIT_RATIO
EPS
EPS_EST_CUR_YEAR
EPS_EST_NEXT_YEAR
EPS_EST_NEXT_QTR
FORWARD_EPS
PEG_RATIO
BOOK_VALUE (requires SHARES_OUT to be specified as well)
BOOK_VALUE_PER_SHARE
EBITDA
PRICE_TO_SALES (requires CLOSE to be specified as well)
PRICE_TO_EARNINGS (requires CLOSE to be specified as well)
PRICE_TO_BV (requires CLOSE to be specified as well)
FORWARD_PE (requires CLOSE to be specified as well)
REVENUE
SHARES_SHORT
DIVIDEND
ONE_YEAR_TARGET
MARKET_CAP (requires CLOSE to be specified as well - it is used to calculate shares outstanding)
SHARES_FLOAT
SHARES_OUT
PROFIT_MARGIN
OPERATING_MARGIN
RETURN_ON_ASSETS
RETURN_ON_EQUITY
QTRLY_REVENUE_GROWTH
GROSS_PROFIT
QTRLY_EARNINGS_GROWTH
INSIDER_HOLD_PERCENT
INSTIT_HOLD_PERCENT
SHARES_SHORT_PREV
FORWARD_DIV
OPERATING_CASH_FLOW
FREE_CASH_FLOW
BETA fundamental data fields. For more info, read Using Fundamental Data

Alias

Examples $FORMAT TICKER DATE_MDY OPEN HIGH LOW CLOSE VOLUME
$FORMAT TICKER, DATE_INT, CLOSE, VOLUME
$FORMAT SKIP, TICKER, SKIP, SKIP, DATE_INT, OPEN, HIGH, LOW, CLOSE, TURNOVER

Command $FULLNAME Define full symbol name

Arguments <string> full symbol name

Alias

Examples $FULLNAME Apple Computer Inc.

Command $GICS Define GICS code (Global Industry Category System)

Arguments <number> This affects $AUTOADD 1 and $OVERWRITE 1 modes. If this is specified, symbols are assigned to a given GICS category.

Alias

Examples Now you can import GICS symbol-code assignments using the ASCII importer.
$FORMAT command now supports the GICS code
and there is $GICS command for single-symbol files.
For example, if your file looks as follows:
(format is symbol, full name, GICS sub-industry code)
AAN,AARON'S INC,25504060

Then to import it using AmiBroker's import wizard, use the following:
$FORMAT Ticker,FullName,GICS
$OVERWRITE 1
$SEPARATOR ,
$CONT 1
$GROUP 255
$AUTOADD 1
$NOQUOTES 1

Command $GROUP Define group ID

Arguments <number> This affects $AUTOADD 1 mode. If this is specified, newly added symbols are assigned to a group with a given number.

Alias

Examples

Command $HYBRID Switch hybrid mode on/off

Arguments <number> 0 (off) or 1 (on). When this flag is set, you can combine quotations from multiple files—for example, one file can contain only open prices and volume, and the other file can contain high/low/close data. This is especially useful for the Warsaw Stock Exchange for combining data from fixing and later continuous quotations.

Alias

Examples

Command $INDUSTRY Define industry ID

Arguments <number> This affects $AUTOADD 1 mode. If this is specified, newly added symbols are assigned to an industry with a given number.

Alias

Examples

Command $MARKET Define market ID

Arguments <number> This affects $AUTOADD 1 mode. If this is specified, newly added symbols are assigned to a market with a given number.

Alias

Examples

Command $NAME Define ticker name

Arguments <ticker> ticker name (symbol) (default = file name without path and extension)

Alias $TICKER

Examples $NAME AAPL
$TICKER MSFT

Command $NOQUOTES Switch quotation data mode

Arguments <number> 0 - (default) accept only quotation data (AmiBroker checks for non-zero prices and valid dates).
1 - switch off quotation data checking; this allows importing non-quotation data—for example, only ticker and full names.

Alias $TICKER

Examples $NAME AAPL
$TICKER MSFT

Command $OVERWRITE Switch overwrite mode on/off

Arguments <number> 0 - off; 1 - on. When overwrite mode is on, information provided by GROUP, MARKET, INDUSTRY, or FULLNAME fields is overwritten for existing symbols (not only for newly added).

Alias

Examples $OVERWRITE 1

Command $PRICEFACTOR Define price factor

Arguments <number> the factor by which price data are multiplied (default = 1).

Alias

Examples $PRICEFACTOR 100

Command $RAWCLOSE2OI Put Raw Close price into the OI field.

Arguments <number> 0 - off; 1 - on (off by default). This causes the OpenInterest field to be assigned the CLOSE (raw close) field value multiplied by 100.

Alias

Examples $RAWCLOSE2OI 1

Command $RECALCSPLITS Recalculate splits

Arguments <number> 0 - off; 1 - on (off by default). This causes splits to be recalculated by AmiBroker
by an algorithm that tries to construct a correct adjusted price, based on inaccurate information provided by Yahoo.
Note that Yahoo provides only 2 decimal digits in the adj. close field; therefore, the more the adjusted close approaches zero due to adjustments, the error grows. The option $RECALCSPLITS 1 is intended to address this problem (at least partially).
It works as follows:
1. For each bar, the ratio ADJCLOSE/CLOSE is calculated.
2. If the ratio changes in two consecutive bars by more than 10%, it means that a split happened in that bar.
3. Then, the true split ratio is guessed by matching a true fraction in the format of X/Y, where X and Y = 1..9, to the change in ratios.
3. Then the true split ratio is used to adjust all past bars until a new split is detected.

Works only in conjunction with ADJCLOSE

Alias

Examples $RECALCSPLITS 1

Command $RECALCVOL Switch automatic index volume recalculation

Arguments <number> 0 - off; 1 - on (base index only); 2 - on (all indexes). When this is on, AmiBroker calculates volumes for indexes based on assignments to markets and base indexes defined in the Categories window.

Alias

Examples $RECALCVOL 2

Command $RECALCAD Switch automatic advance/decline composite recalculation

Arguments <number>
0 - off; 1 - on. When this is on, AmiBroker calculates numbers and volumes of issues advancing, declining, and unchanged based on assignments to markets and base indexes defined in the Categories window.

Alias

Examples $RECALCVOL 2

Command $ROUNDADJ Round split-adjusted prices to a given number of decimal digits.

Arguments <decimaldigits>
decimaldigits - causes split-adjusted prices (see above) to be rounded to 'decimaldigits' precision. By default, no rounding is done.

Works only in conjunction with ADJCLOSE

Alias

Examples $ROUNDADJ 2

Command $SEPARATOR Define field separator character

Arguments <separator char> the character used to separate data fields (default = space)

Alias

Examples $SEPARATOR ,
$SEPARATOR ;

Command $SKIPLINES Define how many lines to skip (ignore)

Arguments <number> number of lines to skip (default = 0).

Alias

Examples $SKIPLINES 1

Command $STRICT Switches on/off strict checking of whether Open, High, Low prices are greater than zero.

Arguments <onoff> (default = 0)

Alias

Examples $STRICT 1

Command $TICKMODE Switches on/off tick mode

$TICKMODE is a special mode of the importer that allows importing quotes that have duplicate timestamps.
It makes two assumptions:
a) input data should come in ascending time order (i.e., OLDER records first, LATEST records last).
b) input data should consist of the entire tick history because the importer will DELETE any existing quotes (to avoid creating multiple copies of the same ticks).

Once again: Turning on
$TICKMODE 1
will DELETE ANY QUOTES that already exist in the database and then will import all ticks from the input data file.
You have been warned.

For example, data files like this:

MOL,0,20050606,162959,16400.0000,16400.0000,16400.0000,16400.0000,2MOL,0,20050606,162959,16400.0000,16400.0000,16400.0000,16400.0000,11MOL,0,20050606,162959,16400.0000,16400.0000,16400.0000,16400.0000,40

Can be imported using the following definition file:

$FORMAT Ticker, Skip, Date_YMD, Time, Open, High, Low, Close, Volume
$SKIPLINES 1
$SEPARATOR ,
$CONT 1
$GROUP 255
$AUTOADD 1
$DEBUG 1
$TICKMODE 1

Sometimes it happens that input files have invalid timestamps (seconds > 59).

For example:

MOL,0,20050606,162970,16400.0000,16400.0000,16400.0000,16400.0000,2

Please take a closer look at the first line shown in this example; it has time:16:29:70 (you see 70 seconds!).

So I had to add a special flag to the importer that works around such data errors.

It is called $ALLOW99SECONDS 1 and will convert all records with invalid seconds (i.e., greater than 59) to 59s.
So a record stamped 16:29:70 will be treated as 16:29:59.

Now, for tick mode to work with such incorrect records, you would need to add two lines to the ASCII importer definition:

$TICKMODE 1
$ALLOW99SECONDS 1

Arguments <onoff> (default = 0)

Alias

Examples $TICKMODE 1

Command $TIMESHIFT Define intraday time shift used during import

Arguments <number> number of hours to shift date/time stamps (can be fractional).

Alias

Examples
$TIMESHIFT 2
; will shift 2 hours forward

$TIMESHIFT -11.5
; will shift 11 and a half hours backward

Command $VOLFACTOR Define volume factor

Arguments <number> the factor by which volume data is multiplied (default = 1).

Alias

Examples $VOLFACTOR 10

Command $WATCHLIST Define watch list number

Arguments <number> This affects $AUTOADD 1 and $OVERWRITE 1 modes. If this is specified, newly added symbols are added to the watch list with a given number.

Alias

Examples

Command $CLEANSECTORS Clean (wipe) existing sector/industry structure

Arguments <number> If this is turned on (1), the existing sector/industry structure will be deleted and initialized with Sector 0, 1, 2, 3...63 / Industry 0...255.
This command should only be used in conjunction with SECTORNAME and INDUSTRYNAME $FORMAT fields to allow setting up a fresh industry structure.

Alias

Examples See example below (importing sector/industry structure)

Command $SORTSECTORS Sort sector/industry structure

Arguments <number> If this is turned on (1), the sector/industry structure will be sorted alphabetically after importing.
This command should only be used in conjunction with SECTORNAME and INDUSTRYNAME $FORMAT fields to allow setting up a fresh industry structure.

Alias

Examples See example below (importing sector/industry structure)

Command $USEONLYLOCALDB Switches "Use only local database" option for the symbol

Arguments <number> If data is fed by a database plugin, using the ASCII importer to add any symbol causes these newly added symbols to have the "Use only local database" flag turned on.
A new command:
$USEONLYLOCALDB 0
allows turning this off (so newly added symbols have "use only local database" turned off).

This flag does NOT affect existing symbols.

Alias

Examples

Command $WEBID Define web ID

Arguments <string> web ID

Alias

Examples $WEBID aapl

Notes:

for DATE_xxx, you can use ‘-‘, ‘/’ or ‘\’ as day/month/year separators. You can even omit separators entirely if you give a date in a 6-digit (YYMMDD, MMDDYY, DDMMYY) or 8-digit format (YYYYMMDD, MMDDYYYY, DDMMYYYY).
AmiBroker recognizes decimal as well as true fractions in price data. True fractions must follow the whole value after at least a single space. For example, you can specify: 5.33 or 5 1/3.

AmiBroker is not limited to any kind of fraction; if you wish, you can even write: 5 333/999

Comments

You can include comments in both format definition file and the data file(s). Each line starting with * (asterisk) or ; (semicolon) or # (hash) is treated as a comment and ignored by the ASCII importer.

Usage examples

What may look complicated from the command list will become quite clear after some examples. So I will give you four examples of how to write format definition files. The first example will show the definition for CSV (comma-separated values) quotes available from Yahoo's finance site. The second example will show the definition for the Metastock ASCII file format. The third example shows the definition for the Omega SuperCharts ASCII file format. And the fourth example will show the definition for s-files used by DM BOS (a Polish brokerage company).

Yahoo CSV

The data from Yahoo's site looks as follows:

Date,Open,High,Low,Close,Volume 1-Feb- 0,104,105,100,100.25,2839600 31-Jan- 0,101,103.875,94.50,103.75,6265000 28-Jan- 0,108.1875,110.875,100.625,101.625,3779900

The first line gives us a hint about the meaning of the comma-separated fields. The first field will hold the date. The remaining fields will hold open, high, low, close prices and volume. The importer should skip the first line and parse all the remaining lines that hold just comma-separated data. An appropriate format definition file would look like this:

$FORMAT Date_DMY,Open,High,Low,Close,Volume $SKIPLINES 1 $SEPARATOR , $DEBUG 1 $AUTOADD 1 $BREAKONERR 1

$DEBUG switches on error logging to the "import.log" file, and $BREAKONERR will cause the importer to stop after the first error is found. $AUTOADD ensures that a new ticker will be added to the database if it is missing. Well... you may ask: how does it know the ticker name? The answer is simple: if there is no field which defines the ticker name, the importer takes the file name (without path and extension) as a ticker. So, if you are importing the file "C:\My data\AAPL.CSV", AmiBroker will use "AAPL" as a ticker name.

Metastock ASCII

The data in Metastock ASCII format looks as follows:

<ticker>,<per>,<date>,<high>,<low>,<close>,<vol> AAP,D,1/17/2000,5483.33,5332.01,5362.3,0 AKS,D,1/17/2000,9868.45,9638.03,9687.62,0 FET,D,1/17/2000,3741.3,3540.2,3570.81,0

The first field will hold the ticker name, the second - time period ("D" means daily data), and the third - quotation date. The rest will hold high, low, close prices and volume. The importer should then skip the first line and parse all the remaining lines that hold just comma-separated data. An appropriate format definition file would look like this:

$FORMAT Ticker,Skip,Date_MDY,High,Low,Close,Volume $SKIPLINES 1 $SEPARATOR , $DEBUG 1 $AUTOADD 1 $BREAKONERR 1

Skip in $FORMAT defines a field that should be ignored by the importer.

Omega SuperCharts ASCII

The data in Omega SC ASCII format looks as follows:

ticker,date,open,high,low,close,vol AAP,20000117,5333.01,5483.33,5332.01,5362.3,3433450

This format is similar to previous ones; however, the date is in YYYYMMDD format without separators between the year, month, and day part. AmiBroker, however, can handle such dates with ease. An appropriate format definition file would look like this:

$FORMAT Name,Date_Int,Open,High,Low,Close,Volume $SEPARATOR , $DEBUG 1 $SKIPLINES 1 $AUTOADD 1 $BREAKONERR 1

Skip in $FORMAT defines a field that should be ignored by the importer.

DMBOS S-files

The data in this format looks as follows:

0,29-02-00,12:05,MIDWIG,1069.1,,,+1.2,336002000, 0,29-02-00,12:05,NIF,48.6,,,+0.8,1763000, 0,29-02-00,12:05,WIG20,2300.3,,,+1.1,336002000, 0,29-02-00,12:05,WIG,21536.8,,,+0.2,336002000, 0,29-02-00,12:05,WIRR,2732.8,,,+1.6,16373000, 1,29-02-00,12:05,AGORA,144.00,,,+4.7,15802000, 1,29-02-00,12:05,AGROS,40.00,nk,72,+5.0,840000, 1,29-02-00,12:05,AMERBANK,28.00,,,+3.7,22000, 1,29-02-00,12:05,AMICA,41.50,nk,99,+2.2,564000,

This format is a little more complicated. For us, useful fields are: 2nd - date, 4th - ticker, 5th - close price, and 9th - the turnover value (close * volume). The remaining fields hold other information that is not useful for us. An appropriate format definition file would look like this:

$FORMAT Skip,Date_DMY,Skip,Name,Close,Skip,Skip,Skip,Turnover $SEPARATOR , $DEBUG 1

Importing Sector/Industry Structure

Let's assume we have a text file with stock tickers, full names, sector names, and industry names listed line by line, as follows:

"DDD","3D Systems Corporation","Technology","Computer Software: Prepackaged Software"
"MMM","3M Company","Health Care","Medical/Dental Instruments"
"SVN","7 Days Group Holdings Limited","Consumer Services","Hotels/Resorts"
"AHC","A.H. Belo Corporation","Consumer Services","Newspapers/Magazines"
"AIR","AAR Corp.","Capital Goods","Aerospace"
"AAN","Aaron's, Inc.","Technology","Diversified Commercial Services"
"ABB","ABB Ltd","Consumer Durables","Electrical Products"

To import such a file, we use the following format definition:

$FORMAT Ticker, FullName,SectorName,IndustryName
$SEPARATOR ,
$AUTOADD 1
$NOQUOTES 1
$OVERWRITE 1
$CLEANSECTORS 1
$SORTSECTORS 1

$NOQUOTES 1 tells the importer that we will be importing non-quotation data. $AUTOADD 1/$OVERWRITE 1 is required to automatically add new symbols and overwrite existing symbol information. $CLEANSECTORS 1 wipes the existing stock/industry structure prior to importing, and $SORTSECTORS 1 sorts sectors/industries after importing so they will be listed in alphabetical order in the Symbol window. $FORMAT command just specifies the order and types of fields to import.

AmiBroker will read such an ASCII file one by one, then it will check whether a given sector name/industry name already exists and, if not, it will create a new sector/industry. Then it will assign the given symbol to the specified sector/industry.

The result will be a database with a new sector/industry structure being set up and symbols assigned to proper sectors and industries.

The described functionality is used to implement the Tools->Update US Symbol List and Categories tool.

Default behaviour

When importing ASCII files, AmiBroker attempts to open the "default.format" file (in the AmiBroker directory) to obtain the format definition. If such a file is missing, the following default format is applied:

$FORMAT DATE_USA, OPEN, HIGH, LOW, CLOSE, VOLUME $SEPARATOR

This means that, by default, the ASCII importer will use a space character as a separator and will parse the following fields: date, open, high, low, close, volume. The file name (without path and extension) will be used as a ticker name. All other import parameters ($DEBUG,$AUTOADD, etc.) are set to zero.

User-definable file types and formats

Now AmiBroker can use not only the default.format definition file but also other user-specified files. File types, filters, and format definition files are specified in the import.types file (an example is included in the update package). Now a user can prepare/modify the import.types file with a description of supported ASCII formats and filters to use. The format of the import.types file is:

<Descriptive name>|<File filter>|<definition file name>

Note the vertical line characters between these three fields. An example import.types file looks as follows:

Default ASCII (*.*)|*.*|default.format Yahoo's CSV (*.csv)|*.csv|yahoo.format Metastock ASCII (*.mst)|*.mst|metastock.format Omega SC ASCII (*.txt)|*.txt|omega.format S-Files (s*.*)|s*.*|sfile.format C-Files (c*.*)|c*.*|cfile.format Sharenet DAT (*.dat)|*.dat|dat.format

If such a file exists, you will see your types in the "Files of type" combo-box, and when you select one, an appropriate filter will be used. After selecting some files and clicking OK, the importer will use the specified ".format" file.

In that way, you can define as many text-based data formats as you like, and AmiBroker will be able to "understand" them all.

Ticker aliases

Now, each ticker can have an alias assigned, so AmiBroker's built-in importers can recognize that security by both its ticker symbol and alias names. This is useful when you are using two data sources that use slightly different symbol naming conventions or if you want to give the symbols a more intuitive name while retaining the ability to use importers without problems.

GICS Categorization

GICS is a global industry classification standard; see http://en.wikipedia.org/wiki/Global_Industry_Classification_Standard
for more details on the GICS system.

GICS codes are from 2 to 8 digits. Such as 10 for the energy sector or 351010 for the "Health Care Equipment & Supplies" industry.
The codes are fixed even if new classifications are added at some point in the future. It is important to understand that these codes work in a hierarchical way.

NOTE: Current databases DO NOT have GICS codes assigned to symbols.
As far as I know, PremiumData http://www.premiumdata.net/ is planning to release an AmiBroker-compatible database with GICS support.

AmiBroker now reads the GICS.txt file from its installation folder. It contains GICS categories listed one by one in the order of the GICS code in the following format:
GICS;Name;Description<CRLF>

GICS is a numeric code from 2 digits up to 8 digits.
Name is the GICS category name.
Description is the GICS category description.
These fields must be separated by a semicolon.
<CRLF> means carriage return/line feed characters (meaning "new line"—just press the ENTER/RETURN key if you are editing with a text editor).

There must be only one category per line in the GICS.txt file.

The default GICS.txt file is supplied already.

ICB Categorization

ICB stands for Industry Classification Benchmark (http://en.wikipedia.org/wiki/Industry_Classification_Benchmark).

AmiBroker also allows the ICB 4-level classification system, but the demo database does not have symbols classified according to that standard. You can find ICB classification codes in the ICB.txt file inside the AmiBroker folder.

NOTE: Current databases DO NOT have ICB codes assigned to symbols.
ICB classification for NYSE stocks can be imported from http://www.nyse.com/indexes/nyaindex.csv

AmiBroker now reads the ICB.txt file from its installation folder. It contains ICB categories listed one by one in the order of the ICB code in the following format:
ICB;Name<CRLF>

ICB is a numeric 4-digit code.
Name is the ICB category name.
These fields must be separated by a semicolon.
<CRLF> means carriage return/line feed characters (meaning "new line"—just press the ENTER/RETURN key if you are editing with a text editor).

There must be only one category per line in the ICB.txt file.

The default ICB.txt file is supplied already.

High-resolution timestamps (milli- and microsecond)

Starting from version 6.14, the ASCII importer supports microsecond resolution timestamps (HH:MM:SS.mmmuuu), where mmm - milliseconds 000..999, uuu - microseconds 000..999.

Command	$ADDRESS	Define company address
Arguments	<string>	address of company
Alias
Examples	$ADDRESS "One Microsoft Way"

Command	$AUTOADD	Switch new ticker add mode.
Arguments	<number>	0 - do not add; 1 - add a new stock when a non-existing ticker is detected (default = 0).
Alias
Examples	$AUTOADD 1

Command	$ALLOWNEG	Allow negative numbers in prices
Arguments	<number>	0 - do not allow negative values (default); 1 - allow negative values in prices. This additionally switches off any checking for OHLC relationships, so you can import any data into OHLC fields. When $ALLOWNEG is NOT specified in the ASCII importer definition, AmiBroker performs the following range checking and fixup on open, low, and high prices: if( open == 0 ) open = close; if( high < max( open, close ) ) high = max( open, close ); if( low == 0 ) low = min( open, close )
Alias
Examples	$ALLOWNEG 1

Command	$ALLOW99SECONDS	Convert invalid second stamp.
Arguments	<onoff>	This flag works ONLY in conjunction with $TICKMODE 1 (see below for details). $ALLOW99SECONDS set to 1 will convert all records with invalid seconds (i.e., greater than 59) to 59s. So a record stamped 16:29:70 will be treated as 16:29:59.
Alias
Examples	$ALLOW99SECONDS 1

Command	$APPENDNAME	append string to the ticker name (useful when you need to join several fields together to make a unique stock symbol)
Arguments	<string>	string to append to the ticker symbol
Alias	$APPENDTICKER
Examples

Command	$BREAKONERR	Define on-error behavior.
Arguments	<number>	0 - to continue; 1 - to break import on error (default=0).
Alias
Examples	$BREAKONERR 1

Command	$CONT	Define continuous quotations flag
Arguments	<number>	<0 or 1> - continuous quotations flag. This affects $AUTOADD 1 mode - if this is set, newly added stocks are switched to continuous quotation mode (this means enabling candlestick charts, for example).
Alias
Examples	$CONT 1

Command	$CURRENCY	Define symbol's currency
Arguments	<string>	Defines the currency of the symbol.
Alias
Examples	$CURRENCY EUR or $FORMAT NAME, CURRENCY $OVERWRITE 1 $AUTOADD 1

Command	$DATE_DMY	Define date
Arguments	<number>	The date in Canadian format (DD-MM-YY). If there is no argument given, the date is taken from the file name (without an extension).
Alias	$DATE_CDN
Examples	$DATE_DMY 12-05-99 $DATE_CDN 12-05-1999

Command	$DATE_MDY	Define date
Arguments	<number>	The date in US format (MM-DD-YY). If there is no argument given, the date is taken from the file name (without an extension).
Alias	$DATE_USA
Examples	$DATE_MDY 05/12/99 $DATE_USA 05/12/99

Command	$DATE_YMD	Define date
Arguments	<number>	The date in International format (YY-MM-DD). If there is no argument given, the date is taken from the file name (without an extension).
Alias	$DATE_INT
Examples	$DATE_INT 99-05-12 $DATE_CDN 1999.05.12

Command	$DEBUG	Switch logging (debug) mode
Arguments	<number>	0 - no error logging; 1 - log errors to "import.log" file (default=0).
Alias
Examples	$DEBUG 1

Command	$FORMAT	Define line format (sequence and types of fields)
Arguments	DATE_MDY	date in US format: MM-DD-YY (alias: DATE_USA)
	DATE_DMY	date in Canadian format: DD-MM-YY (alias: DATE_CDN)
	DATE_YMD	date in International format: YY-MM-DD (alias: DATE_INT)
	TIME	time in HH:MM:SS or HH:MM or HHMM or HHMMSS format
	NAME	ticker name (alias: TICKER)
	ALIAS	symbol alias ($AUTOADD and $OVERWRITE modes only)
	FULLNAME	symbol full name ($AUTOADD and $OVERWRITE modes only)
	OPEN	open price
	HIGH	high price
	LOW	low price
	CLOSE	close price
	ADJCLOSE	split-adjusted close This is provided to read the adjusted close column from Yahoo. Works only in conjunction with the CLOSE field. When both CLOSE and ADJCLOSE are present in the ASCII format definition, the importer calculates the split factor by dividing ADJCLOSE by CLOSE. It then multiplies OPEN, HIGH, LOW, and CLOSE fields by this factor and divides the VOLUME field by this factor. This effectively converts unadjusted prices to split-adjusted prices. The split ratio gets locked once ADJCLOSE drops below 0.05.
	OPENINT	open interest
	VOLUME	volume
	VOL1000	volume in thousands of shares
	VOLMIL	volume in millions of shares
	VOLFACTOR	volume factor (number of shares in a block) (default = 1)
	TURNOVER	turnover
	AUX1	AUX1 field (auxiliary data)
	AUX2	AUX2 field (auxiliary data)
	SKIP	skip (ignore) field
	MARKET	specify a field that contains market ID (affects $AUTOADD and $OVERWRITE modes only)
	GROUP	specify a field that contains group ID (affects $AUTOADD and $OVERWRITE modes only)
	WATCHLIST	specify a field that contains watch list number (0-31) (affects $AUTOADD and $OVERWRITE modes only)
	INFO	specify a field with additional information (WSE specific: nk, ns, rk, rs, ok, os, zd, bd).
	REDUCTION	specify a field with reduction rate in percentages (WSE specific)
	ICB	(new in 5.60) Specify ICB code. For example, if your file looks as follows: (format is symbol, full name, ICB code) AAN,AARON'S INC,5375 Then to import it using AmiBroker's import wizard, use the following: $FORMAT Ticker,FullName,ICB $OVERWRITE 1 $SEPARATOR , $CONT 1 $GROUP 255 $AUTOADD 1 $NOQUOTES 1
	GICS	Specify GICS code. For example, if your file looks as follows: (format is symbol, full name, GICS sub-industry code) AAN,AARON'S INC,25504060 Then to import it using AmiBroker's import wizard, use the following: $FORMAT Ticker,FullName,GICS $OVERWRITE 1 $SEPARATOR , $CONT 1 $GROUP 255 $AUTOADD 1 $NOQUOTES 1
	INDUSTRY	specify a field that contains industry ID (affects $AUTOADD and $OVERWRITE modes only)
	INDUSTRYNAME	(new in 5.60) Specifies a field that contains the Industry Name. AmiBroker will check if a given industry name already exists and, if not, it will create a new Industry and assign the imported stock to the industry specified. Also, if SECTORNAME is specified, it will assign the newly added industry to the specified sector. (affects $AUTOADD and $OVERWRITE modes only)
	SECTORNAME	(new in 5.60) Specifies a field that contains the Sector Name. AmiBroker will check if a given sector name already exists and, if not, it will create a new Sector. Also, if INDUSTRYNAME is specified, it will assign the newly added industry to the specified sector. (affects $AUTOADD and $OVERWRITE modes only)
	APPENDTICKER	specify a field that contains a string that should be appended to the ticker name (useful when you need to join several fields together to make a unique symbol).
	MARGIN	future contract margin deposit (positive value = dollars; negative value = percent of full value)
	POINTVALUE	future contract point value
	ROUNDLOTSIZE	round lot size (trading unit size)
	TICKSIZE	tick size
	ADVISSUES	number of advancing issues
	ADVVOLUME	volume of advancing issues
	DECISSUES	number of declining issues
	DECVOLUME	volume of declining issues
	UNCISSUES	number of unchanged issues
	UNCVOLUME	volume of unchanged issues
	ADDRESS	street address of company
	CURRENCY	specifies the currency of the symbol.
	WEBID	specifies the web ID.
	DIV_PAY_DATE EX_DIV_DATE LAST_SPLIT_DATE LAST_SPLIT_RATIO EPS EPS_EST_CUR_YEAR EPS_EST_NEXT_YEAR EPS_EST_NEXT_QTR FORWARD_EPS PEG_RATIO BOOK_VALUE (requires SHARES_OUT to be specified as well) BOOK_VALUE_PER_SHARE EBITDA PRICE_TO_SALES (requires CLOSE to be specified as well) PRICE_TO_EARNINGS (requires CLOSE to be specified as well) PRICE_TO_BV (requires CLOSE to be specified as well) FORWARD_PE (requires CLOSE to be specified as well) REVENUE SHARES_SHORT DIVIDEND ONE_YEAR_TARGET MARKET_CAP (requires CLOSE to be specified as well - it is used to calculate shares outstanding) SHARES_FLOAT SHARES_OUT PROFIT_MARGIN OPERATING_MARGIN RETURN_ON_ASSETS RETURN_ON_EQUITY QTRLY_REVENUE_GROWTH GROSS_PROFIT QTRLY_EARNINGS_GROWTH INSIDER_HOLD_PERCENT INSTIT_HOLD_PERCENT SHARES_SHORT_PREV FORWARD_DIV OPERATING_CASH_FLOW FREE_CASH_FLOW BETA	fundamental data fields. For more info, read Using Fundamental Data
Alias
Examples	$FORMAT TICKER DATE_MDY OPEN HIGH LOW CLOSE VOLUME $FORMAT TICKER, DATE_INT, CLOSE, VOLUME $FORMAT SKIP, TICKER, SKIP, SKIP, DATE_INT, OPEN, HIGH, LOW, CLOSE, TURNOVER

Command	$FULLNAME	Define full symbol name
Arguments	<string>	full symbol name
Alias
Examples	$FULLNAME Apple Computer Inc.

Command	$GICS	Define GICS code (Global Industry Category System)
Arguments	<number>	This affects $AUTOADD 1 and $OVERWRITE 1 modes. If this is specified, symbols are assigned to a given GICS category.
Alias
Examples	Now you can import GICS symbol-code assignments using the ASCII importer. $FORMAT command now supports the GICS code and there is $GICS command for single-symbol files. For example, if your file looks as follows: (format is symbol, full name, GICS sub-industry code) AAN,AARON'S INC,25504060 Then to import it using AmiBroker's import wizard, use the following: $FORMAT Ticker,FullName,GICS $OVERWRITE 1 $SEPARATOR , $CONT 1 $GROUP 255 $AUTOADD 1 $NOQUOTES 1

Command	$GROUP	Define group ID
Arguments	<number>	This affects $AUTOADD 1 mode. If this is specified, newly added symbols are assigned to a group with a given number.
Alias
Examples

Command	$HYBRID	Switch hybrid mode on/off
Arguments	<number>	0 (off) or 1 (on). When this flag is set, you can combine quotations from multiple files—for example, one file can contain only open prices and volume, and the other file can contain high/low/close data. This is especially useful for the Warsaw Stock Exchange for combining data from fixing and later continuous quotations.
Alias
Examples

Command	$INDUSTRY	Define industry ID
Arguments	<number>	This affects $AUTOADD 1 mode. If this is specified, newly added symbols are assigned to an industry with a given number.
Alias
Examples

Command	$MARKET	Define market ID
Arguments	<number>	This affects $AUTOADD 1 mode. If this is specified, newly added symbols are assigned to a market with a given number.
Alias
Examples

Command	$NAME	Define ticker name
Arguments	<ticker>	ticker name (symbol) (default = file name without path and extension)
Alias	$TICKER
Examples	$NAME AAPL $TICKER MSFT

Command	$NOQUOTES	Switch quotation data mode
Arguments	<number>	0 - (default) accept only quotation data (AmiBroker checks for non-zero prices and valid dates). 1 - switch off quotation data checking; this allows importing non-quotation data—for example, only ticker and full names.
Alias	$TICKER
Examples	$NAME AAPL $TICKER MSFT

Command	$OVERWRITE	Switch overwrite mode on/off
Arguments	<number>	0 - off; 1 - on. When overwrite mode is on, information provided by GROUP, MARKET, INDUSTRY, or FULLNAME fields is overwritten for existing symbols (not only for newly added).
Alias
Examples	$OVERWRITE 1

Command	$PRICEFACTOR	Define price factor
Arguments	<number>	the factor by which price data are multiplied (default = 1).
Alias
Examples	$PRICEFACTOR 100

Command	$RAWCLOSE2OI	Put Raw Close price into the OI field.
Arguments	<number>	0 - off; 1 - on (off by default). This causes the OpenInterest field to be assigned the CLOSE (raw close) field value multiplied by 100.
Alias
Examples	$RAWCLOSE2OI 1

Command	$RECALCSPLITS	Recalculate splits
Arguments	<number>	0 - off; 1 - on (off by default). This causes splits to be recalculated by AmiBroker by an algorithm that tries to construct a correct adjusted price, based on inaccurate information provided by Yahoo. Note that Yahoo provides only 2 decimal digits in the adj. close field; therefore, the more the adjusted close approaches zero due to adjustments, the error grows. The option $RECALCSPLITS 1 is intended to address this problem (at least partially). It works as follows: 1. For each bar, the ratio ADJCLOSE/CLOSE is calculated. 2. If the ratio changes in two consecutive bars by more than 10%, it means that a split happened in that bar. 3. Then, the true split ratio is guessed by matching a true fraction in the format of X/Y, where X and Y = 1..9, to the change in ratios. 3. Then the true split ratio is used to adjust all past bars until a new split is detected. Works only in conjunction with ADJCLOSE
Alias
Examples	$RECALCSPLITS 1

Command	$RECALCVOL	Switch automatic index volume recalculation
Arguments	<number>	0 - off; 1 - on (base index only); 2 - on (all indexes). When this is on, AmiBroker calculates volumes for indexes based on assignments to markets and base indexes defined in the Categories window.
Alias
Examples	$RECALCVOL 2

Command	$RECALCAD	Switch automatic advance/decline composite recalculation
Arguments	<number>	0 - off; 1 - on. When this is on, AmiBroker calculates numbers and volumes of issues advancing, declining, and unchanged based on assignments to markets and base indexes defined in the Categories window.
Alias
Examples	$RECALCVOL 2

Command	$ROUNDADJ	Round split-adjusted prices to a given number of decimal digits.
Arguments	<decimaldigits>	decimaldigits - causes split-adjusted prices (see above) to be rounded to 'decimaldigits' precision. By default, no rounding is done. Works only in conjunction with ADJCLOSE
Alias
Examples	$ROUNDADJ 2

Command	$SEPARATOR	Define field separator character
Arguments	<separator char>	the character used to separate data fields (default = space)
Alias
Examples	$SEPARATOR , $SEPARATOR ;

Command	$SKIPLINES	Define how many lines to skip (ignore)
Arguments	<number>	number of lines to skip (default = 0).
Alias
Examples	$SKIPLINES 1