{"id":1320,"date":"2007-10-21T06:47:42","date_gmt":"2007-10-21T06:47:42","guid":{"rendered":"http:\/\/www.amibroker.org\/userkb\/2007\/10\/21\/yahoo-faqs\/"},"modified":"2007-11-16T02:48:59","modified_gmt":"2007-11-16T02:48:59","slug":"yahoo-data-faqs","status":"publish","type":"post","link":"http:\/\/www.amibroker.org\/editable_userkb\/2007\/10\/21\/yahoo-data-faqs\/","title":{"rendered":"Yahoo Data FAQ’s"},"content":{"rendered":"

An unsorted, unqualified list of FAQ’s relating to anything and everything to do with Yahoo! Finance data.<\/p>\n

The list is dynamic and subject to change without notice.<\/p>\n

Comments, where included, are the opinion of the individuals concerned and are not necessarily correct. In some cases they may be ‘out of date’ as circumstances change with time. <\/p>\n

CAUTION – SOME OF THE LINKS CONTAIN COMMERCIAL REFERENCES OR LINK TO COMMERCIAL SITES.<\/font> <\/p>\n

NOTE: THE AUTHOR HAS NO COMMERCIAL AFFILIATIONS OF ANY KIND AND RECEIVES NO GRATUITIES, OR BENEFITS, FROM ANY COMMERCIAL ORGANIZATION OR INDIVIDUAL INVOLVED IN COMMERCIAL ACTIVITIES. NOR IS HE PERSONALLY ENGAGED IN ANY COMMERCIAL ACTIVITIES, RELATED TO TRADING, IN ANY WAY.<\/font> THE LINKS ARE PROVIDED AS A RESOURCE AND FOR EDUCATIONAL PURPOSES. THEY ARE NOT A RECOMMENDATION, ON THE PART OF THE AUTHOR, NOR DO THEY CONSTITUTE INVESTMENT ADVICE AND SHOULD NOT BE CONSTRUED AS SUCH. <\/font><\/p>\n

<\/font> <\/p>\n

Q7.<\/em><\/strong> I want to use Yahoo data for the ASX. Is it accurate?<\/p>\n

A7. The use of Yahoo data for ASX stocks is not good.  <\/p>\n

1.    No delisted history – any system testing you do based on Yahoo data is subject to population bias.  ie you can only test on the stocks that are currently listed. <\/p>\n

2.    Significant inaccuracies for any stock exchange that trades in increments less than 1c.  This includes the Australian Stock Exchange (ASX), Singapore Exchange, some NASDAQ stocks and most OTCBB & Pink Sheet stocks.  This happens because Yahoo’s data is rounded to the nearest cent.  For example, there are 14000 listed instruments on the ASX.  Over 8000 of them are currently trading below 20c which means they move in either 0.5 or 0.1c increments.  Since Yahoo only prices in cents this introduce VERY SIGNIFICANT innacuracies.  – eg. a 2.6c shown as 3c represents a 15% error).  <\/p>\n

3.    Normal dividends are adjusted by subtracting all historical data by the dividend amount.  Exchanges such as the ASX do not adjust price data for standard dividends.  By diluting it the way they do, any technical analysis or back-testing no longer shows a correct % return, nor does it show critical technical levels such as support and resistance. <\/p>\n

4.    No sector information – critical if you want to perform sector-based analysis. <\/p>\n

5.    Significant number of missing stock splits and other capital adjustments. <\/p>\n

Re the rounding, if the data is adjusted (for splits\/rights issues etc.) then you want plenty of decimal places so your testing shows the correct % gain.  In years past, the ASX had a different mechanism for determining the open and closing price of a stock, ending up with a weighted average of the bids\/asks to come up with one open\/close price (which would often be a fractional amount).  They changed this method a few years ago to more of a volume-matching method so that it’s a “normal” trading level. <\/p>\n

Richard Dale. <\/p>\n

Norgate Investor Services
– Premium quality Stock, Futures and Foreign Exchange Data for
  markets in Australia, Asia, Canada, Europe, UK & USA –
www.premiumdata.net<\/a> <\/p>\n

AmiBrokerYahooGroup message#68314 “Yahoo EOD data not always correct ?” http:\/\/finance.groups.yahoo.com\/group\/amibroker\/message\/68314<\/a> <\/p>\n

  <\/p>\n

Q6. I am downloading EOD data from Yahoo to an ASX 200 group on a daily basis and I am also using another source. I have noticed discrepancies in quotes between these sources. It appears that Yahoo is incorrect. Sometimes it can be 5 cents or more difference.<\/em><\/strong><\/p>\n

Differences may be because one source is adjusting for dividends while the other is not. AmiQuote\/Yahoo is adjusting, by default, for splits and
dividends but you can turn adjustment off by editing Formats\\aqh.format file (using Notepad) and removing ADJCLOSE field.<\/p>\n

Tomasz Janeczko – amibroker.com <\/p>\n

  <\/p>\n

 Q5. For two days in a row Yahoo data for US stocks retrieved with Amiquote has been very spotty – data comes through for some symbols but not
others?<\/strong><\/em><\/p>\n

A5. <\/em><\/strong>This is a Yahoo problem. For example if a stock is missing the LOW price of the day AmiQuote, by default, does not import data with missing LOW-HIGH prices.
You can however turn this off – however LOW price will be still missing – in fact it will use lowest of open\/close as a low).<\/p>\n

To accept it, despite missing LOW, you can go to aqd.format file (inside Formats subdirectory) and remove $STRICT 1 line. <\/p>\n

Tomasz Janeczko – AmiBroker.com<\/p>\n

AmiBrokerYahooGroup  message #116918     “Yahoo Data Very Spotty Last Two Days”  http:\/\/finance.groups.yahoo.com\/group\/amibroker\/message\/116918<\/a><\/p>\n

 <\/p>\n

Q4. AmiQuote does not automatically list tickers for the exchanges supported by Yahoo!Finance. Are there any other Yahoo down-loaders, compatible with AmiBroker, that do?<\/em><\/strong><\/p>\n

A4. Yes, there are at least two that the author is aware of. They are both commercially available.<\/p>\n

a) <\/em>Bodhi Freeway<\/em> (formerly Bodhi Gold<\/em>) is a down-loader, with Yahoo capability, that allow users to select the tickers they want to download. It is distributed by JustData and is free when bundled with their data subscription packages).<\/p>\n

http:\/\/www.justdata.com.au\/Products\/BodhiFreeway\/bg_yahoo.htm<\/a><\/p>\n

They produce data files in the Metastock <\/em>format that can be dynamically referenced in AmiBroker<\/em> using the Metastock<\/em> plugin<\/em>.<\/p>\n

http:\/\/www.justdata.com.au\/Training\/Tutorials\/Bodhi3\/amibroker.htm<\/a><\/p>\n

Sector classification is not available with Yahoo <\/em>international data.<\/p>\n

At the time of writing, Bodhi Freeway, to purchase as a stand-alone product is approximately twice the price of AmiQuote<\/em>.<\/p>\n

b) MLDownloader <\/p>\n

http:\/\/www.trading-tools.com\/<\/a><\/p>\n

Download End-of-Day stock quotes on 50+ exchanges worldwide. Forex, commodity futures, market indices, mutual funds, money market funds, ETF, corporate bonds, and metals are also supported.
Stores data in MetaStock <\/em>or in the form of ASCII files.
Supports all common MetaStock <\/em>formats (Version 6.52, 8.0, 9.0 and 10.0 (6000 securities per directory)).
Stock splits are performed automatically.
Notification when dividends were distributed.
Automatic update of indices and maintenance of your symbol lists.
Download attempts can easily be repeated – Detects symbol changes.
Downloading data can be scheduled.
Automatically connecting, downloading and disconnecting for analogue modem users.
MLDownloader<\/em> supports multiple stock quotes servers like Yahoo,<\/em> MSN.com<\/em> or Quote.com
<\/em>Easy way to create symbol list from existing MetaStock<\/em> database and continue updating historical EOD data.
Auto-re-download of missing data.
Option of Daily, Weekly and Monthly or Intraday data.<\/p>\n

At the time of writing, MLD, is approximately twice the price of AmiQuote<\/em>.<\/p>\n

 <\/p>\n

Q3. I am having problems with the US-Stocks database. There are 8630 symbols in the database. Yahoo has no historical data for over 300 of them. Amiquote puts ‘junk’ (html 404 error) in the data file for missing historical data.<\/strong><\/em><\/p>\n

A3. Authors comments – Not all download errors are caused by a lack of historical data. Sometimes this occurs because there have been symbol (name) changes or companies have restructured in some way (mergers, de-listings) and this is not reflected in the Yahoo database. Typically the majority of them are .OB (Bulletin Board<\/em>) stock. This is not an error caused by AmiQuote<\/em>, AmiBroker<\/em> or the US _Stocks<\/em> database.<\/p>\n

Within AmiBroker<\/em> they can be reported as NoQuotes <\/em>and sent to a Watch list<\/em> by using the Database Purify Tool<\/em> (they have been sorted into a NoQuotes<\/em> watch list by the developer of the US_Stocks<\/em> template). They can be deleted from the database by deleting them from the Watch list<\/em>, or alternatively users can check the Yahoo<\/em> site to find the reason for the ‘failure to download’ and make repairs to the database where necessary. <\/p>\n

Examples of how to use the Database Purification Tool<\/em>, and other a tips and tricks to help with quality control of databases, have been documented in this section of the Users’ Knowledge Base<\/em>.<\/p>\n

 <\/p>\n

Q2. I recently purchased AmiBroker & downloaded historical quotes from Yahoo. The volume display will not display values greater than 2,147,483,648.00 for indexes. All the recent volume values stop at this level. Is there a way to increase the display limit for volume?<\/em><\/strong><\/p>\n

A2. 2,147,483,648 is the upper limit for integers in AmiBroker<\/em>. The volume, as it is imported from Yahoo<\/em>, can be adjusted down using decimal factors.  Refer to the AmiBroker Knowledge Base<\/em> article http:\/\/www.amibroker.com\/kb\/index.php?s=Yahoo<\/a> by Tomasz Janeczko.  Also search the Yahoo message board archives, or the Help manual, for $volfactor to find additional information.<\/p>\n

AmiBrokerYahooGroup  message #116336     “Volume Display Stops at 2,147,483,648.00 on Indexes”    http:\/\/finance.groups.yahoo.com\/group\/amibroker\/message\/116336<\/a><\/p>\n

 <\/p>\n

Q1. Is there a template for downloading Yahoo data, plus sectors, for the international  markets like Germany or India?<\/em><\/strong><\/p>\n

A1. Authors comments – sorry but it appears not (unless someone comes forward and tells us otherwise). Generally the products and services available for the American market are superior to the other Yahoo<\/em> supported markets e.g. for the American market a database is maintained, including sector classification, for approximately 8000 stocks (the sector  information can be found on the Yahoo!Finance<\/em> Profile<\/em> page for each stock).<\/p>\n

By comparison, the Company<\/em> information, including the Profile<\/em>, is not available for the leading German or Indian stocks, let alone a range representative of those market as a whole.<\/p>\n

Germany – the top 30 company Adidas <\/em>has no active links in the Company <\/em>section.<\/p>\n

\"YFAQ003\" <\/p>\n

India – the top 100 company Bank of India<\/em> has no active links in the Company<\/em> section.<\/p>\n

\"YFAQ004\" <\/p>\n

Further to that, many of the Y!Finance international sites do not provide constituent lists for ‘local’ indices.<\/p>\n

The only way around this appears to be:<\/p>\n