Yahoo’s Data Providers (v2)

Yahoo sources data from a number of providers who are acknowledged at the bottom of the Y!Finance page: http://finance.yahoo.com/ 

Yahoo001

The partner page link in that section opens a list of delayed intra-day data suppliers: http://finance.yahoo.com/exchanges

As well as that, other providers are acknowledged in other places e.g. the S&P500 index is provided by Comstock and industry information, “on 7500 publicly traded U.S. companies”, is from Hemscott Americas etc.

The obvious point is that there are, at the least, two historical quotes databases, a fundamental database and several delayed intra-day data streams used, in different ways and in different places, at the site and there is no relationship between them in anyway. Some pages are a compendium, sourced from different providers, and it is difficult to be certain, in some cases, what is coming from where.

The main points to be aware of:

  • 1) The intra-day quotes, for listings on the major US exchanges, come directly from the exchanges.
  • 2) Intra-day quotes, for non-U.S. listings, are supplied by third party providers e.g. Comstock and Telekurs.
  • 3) Fundamental data, for American exchange listings, is supplied by Capital IQ, a division of Standard and Poor’s.

Note: The Capital IQ  database is the logical choice as the underlying database for two stock screeners, although this is not confirmed at the site:

Java screener – http://screener.finance.yahoo.com/newscreener.html  

HTML screener -  http://screener.finance.yahoo.com/stocks.html 

At the time of writing the HTML database contains 8467 listings, but sadly only allows users to view screened lists 20 at a time, while the newer Java screener, which has superior exporting abilities, only references NYSE, Nasdaq and Amex stock i.e. 4370 listings.

Bulletin Board (.OB) listings are included in the HTML screener database but Pink Sheets symbols (.PK) are not (to obtain all listings for the databases run a filter using price >= 0 as the criteria).

  • 4) Historical data, for the U.S. markets, is supplied by Commodity Service Inc (CSI) and is updated after hours (in a trial conducted by the author the update was logged at 8 PM EST (USA) i.e. four hours after the close).
  • 5) Historical and delayed intra-day data for the non-U.S. markets is supplied by Hemscott Americas (historical data is updated after hours).

Note: International listings require a suffix to identify the exchange.

For a list of suffixes for each exchange refer to: http://finance.yahoo.com/exchanges

  • 6) The stock listings in the CSI (historical) database are not the same as for the Capital IQ (fundamental) database, or any other database for that matter. 
  • 7) Only one of the ‘databases’ has a downloadable index, albeit a limited download, while access to other resources at the site is on a symbol by symbol basis.

There is, however, the ‘mother of all indexes’ at: http://biz.yahoo.com/i/ (it seems likely that this index is provided by CSI, based on it’s size and content e.g. it is mainly comprised of US and Canadian listings and the number of constituents also approximates the CSI universe.

The patchwork nature of the site sometimes produces surprising outcomes. 

One example:

Japanese financial services company Acom Co Ltd is an ADR traded on the OTC market. It is not a constituent of the HTML screened list, and it does not have a ‘Yahoo’ chart. A search for the company by name, using Symbol Lookup, sometimes returns a result ……. 

Yahoo002

……….and at other times it doesn’t (pity about that).

Yahoo004

 In spite of all that it does have quotation and historical data at the Yahoo site.

Note:  To go to Yahoo!Finance site and enter ACMUY.PK in the Enter Symbol(s) input box and click on GET QUOTES at any Y!Finance page.

Yahoo002

Clicking on the Historical Prices link, in the Quotes section of the left hand side bar, will open approximately two years of data for this symbol. (Yes – Amiquote will download the data).

This is a result that is typical for OTC ADR’s.

All in all a mixed bag from Yahoo!Finance.

  • Version 1 – Sept21/2007
  • Version 2 – Sept23/2007 – Corrected minor errors and added links.

Written using Windows XPHome and Internet Explorer v7.

1 Star2 Stars3 Stars4 Stars5 Stars (2 votes, average: 5.00 out of 5)
Loading ... Loading ...

4 Responses to “Yahoo’s Data Providers (v2)”

  1. brian_z
    October 5th, 2007 | 12:25 am

    ACMUY.PK is currently showing a daily chart at the summary page (contrary to the previous outcome reported in the post). I haven’t checked other ADR’s to see if the same applies. I will leave it to anyone who is interested to make their own enquiries (generally this is consistent with Yahoo’s ‘patchy’ delivery in some areas, but it doesn’t apply to all services, all of the time – IMO each Yahoo service should be judged on it’s merits).

  2. brian_z
    October 5th, 2007 | 11:01 pm

    Q. I use Tools > Auto Update Quotes (Amiquote) > Yahoo Historical to download
    EOD data on daily basis. I have about 1,000 stocks in my list (National Stock Exchange of India), it
    downloads data for one stock at a time, and takes roughly 2 hours to
    download the data. Whether data to be downloaded is for one day or 10 days
    or 100 days, there is not much of a difference in time taken.

    On the other hand, Yahoo Current downloads data for 200 stocks in one
    pass, and data for 100 stocks in 5 passes, in a total of less than 2
    minutes.

    Is there any similarly faster method for downloading daily EOD data?

    …Today at 6 pm when I downloaded from Yahoo Current for today’s EOD data, I
    got end-of-bar data, so the figures I got for open, high, low & close were
    the same. I did not get end-of-day data.

    A. The difference you see is due to the fact that current and historical data use different servers and the different way that they interact with data requests. You should consider downloading current daily and historical once a week on weekends. (Bill)

    AmiBrokerYahooGroup #115748 “Downloading Daily EOD Data”

    Authors comments: Intraday markets, historical USA and historical international all come from a different place so we can anticipate different formats and quality etc (It would help if people specify the country/exchange that they are referring to).
    Yahoo supports two Indian exchanges (Bombay by Interactive Data Real Time services – changed from Comstock? – and National Stock Exchange of India by NSEI) – .BO is 15 min delayed – .NS is unique in that it is real time. It seems that intraday, for one or both of those markets, is close price only (I am relying on locals to provide local knowledge by way of comments or posts at this site – I won’t necessarily continue to clip the message board – anyone can start a post/topic on Indian resources, or any other country/exchange for that matter).
    ***********************************************************************************************************
    Re: Bill’s suggestion – I agree with Bill’s comments about taking the current USA quotes at 4 PM and updating historically on the weekends (also suggested by Tomasz somewhere). luck!). Note: YahooFinance services for USA are generally superior quality to other services? My observations are: 1) US current data is frozen at the close +- a minute or two, and thereafter available until 4 hours before the next opening 2) Historical US is updated 4 hours after the close – it is superior quality to international historical and other services (because it is CSI?) 3) It appears that the historical index, to match the historical data, is provided by CSI and is at http://biz.yahoo.com/i/ (that is a ‘guess on my part’ – it contains a similar number of listings and is ‘biased’ to the USA and Canada as per CSI – unfortunately it doesn’t search or download too well?).

  3. brian_z
    October 6th, 2007 | 12:43 am

    Q. Anybody knows how to get Canadian stock (Toronto Stock Exchange)
    sectors and industries data and import them into AB? Is there a tool
    similar to Quotes Plus to get the job done?

    Thanks,
    Rolly

    AmiBrokerYahooGroup #115707 “Canadian stock sectors and industries”
    ***********************************************************************************************************
    Authors comments: Upcoming posts in this category will point to International resources (unfortunately the trend is towards ‘pay for play’ to access constituent lists e.g. Wiltshire index lists are only available by subscription) – already a search of this site for Canada or Canadian will bring up a link or two.

    In general (for international lists):

    1) Go to Yahoo international sites – http://world.yahoo.com/ and click on interactive world map (not all Yahoo international sites have finance section and it helps to be multi-lingual) or go to Y!F (USA) Homepage and scroll down to Yahoo!Finance Worldwide >> Canada – find the Canadian indices at the .ca site and download constituents list (index by index) – not all sites have index consituent downloads and usually they are limited to 50 per time (contractual agreement with their providers) – download sector indices, complete with constituent lists, for setting up sectors.
    2) MLDownloader – a commercial program that downloads international yahoo data and keeps symbol lists up to date: http://www.trading-tools.com/ – compatible with AmiBroker data plugin – it doesn’t specialise in sectors/industries and doesn’t do anything special re the US market – it can be helpful in producing ticker lists for ‘hard to interpret’ international sites e.g. those sites that use numerical codes for stock symbols – for international traders who want highly liquid international stock lists the international index (with constituents) can do the same job.
    3) CSI – a commercial EOD provider who specialises in Canadian stocks, as well as US (plus UK?).
    4) Exchanges – some exchanges provide constituent lists – in this case the Toronto exchange http://www.tsx.com/ doesn’t appear to accomodate in this regard.
    5) Indexes – some of the index sites include international or global indexes with constituent lists (as a rule of thumb the indexes are limited to larger caps but if traders are looking for liquidity that is not a bad way to go – an upcoming post will show why that is the case). S&P are generally the most friendly in this regard – for Canadian sector lists go to the S*P home site >> indices >> equity indices >> Canada for downloadable constituent lists for TSX composites with denominated sectors – GIC’s classification only – (totals 200 -300 stock – presumably the major caps).
    http://www2.standardandpoors.com/portal/site/sp/en/us/page.home/home/1,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0.html

    It all involves a good deal of digging around, and work, which is probably why some traders prefer to pay the pros to do it for them.

    If anyone comes across any good hits, for data resources, please post here via comments.

    P.S. search http://biz.yahoo.com/i/ using .V or .TO and you will see that Yahoo has quite a few listings for the Canadian exchanges (you can only see the first 150 or so listings though – contractual restrictions once again) – as a rule of thumb for Yahoo historical – get your symbol lists from somewhere else, append Yahoo exchange suffixes and try to download using Yahoo Current exchange null quotes stock can be filtered out in AB – (a future post will include a way(s) to quickly find and delete redundant symbols) – Yahoo is not too helpful about coughing up complete symbols lists so just ask and you might receive data.

  4. brian_z
    October 8th, 2007 | 10:15 pm

    Q. I am considering trying out Yahoo as an EOD data provider. Before I do take the time to setup AB with a
    new database, what are the pro’s and con’s of using Yahoo?

    Do you get what you pay for? Free – so it means you have a lot of
    database maintenance, etc.

    Dave

    AmiBrokerYahooGroup #115903 Re: New Yahoo US-Stocks Database

    A1.Well, it is true you get what you pay for. I guess a lot of it depends
    upon what you are trying to do with it. Personally, I use it mostly
    for scanning and isolating those securities that I have interest in
    long term. I don’t trade off of it at all.

    Then when things look like what I want, I use AB with IB or TS to
    verify my findings. Then I take the trade… I like the idea of using
    a couple of data sources to ensure what I am trading.

    Best of luck!
    Jim
    A2. Yahoo’s quotes are posted by a third party data supplier. So, the key question is where does Yahoo get its data. As they note:

    “Historical chart data and daily updates provided by Commodity Systems, Inc. (CSI).”

    Therefore, as far as data quality goes how good is CSI? If they are no good, then Yahoo is no good. If they are good, then Yahoo is good. There is a chart on CSI’s website from a 1999 Futures article that shows CSI had the lowest error rate. Is there more recent data that indicates otherwise?

    Bill

    I think it depends on what your indicators are computing. I have seen many posts in the past which imply that people such as myself who only use Yahoo EOD are asking for problems. My indicators compute many averages of the last 100, 200, and sometimes the last 300 days of data. These indicators are looking for the core consensus of what is happening with a particular symbol.

    Therefore, in my case, any errors that may exist have essentially ZERO effect on these massive number of averages.

    If your indicators look at only the last 10 to 20 days, and do very little averaging, then errors that may exist in the data could have an effect.

    Also, I totally ignore volume and use ossillating indicators that are derived only from price action. Ron D

    *********************************************************************************************************************
    Authors comments: My intention for the Yahoo series, aside from providing an entry point for newcomers to AB, is to dig around the Yahoo site and report back, without bias, on what ‘tools’ it provides for traders. When the dust settles I believe the two points, made by Bill and Jim, will emerge as prime considerations i.e. it’s “horses for courses” and USA historical data (CSI) is the jewel in the crown. Re horses for courses – it’s a different cup of tea for long term traders versus short term traders (long termers have time to compare Yahoo’s prices to their broker’s prices etc) – or for new traders (with general interest cf old hands with a specific data requirements). Some creative thinking can find ‘interesting’ uses for Yahoo e.g. initial testing of new ideas before committing to a paid data contract (trying a different market etc) – for data that your current provider doesn’t include (international or ADR’s?).
    Re CSI – We (traders) have no independent measure for data quality that we can refer to – not like the international metre etc – the best we can do is make relative comparisons – we believe that CSI provides quality data – if a trader wants to ‘trade off’ Yahoo they could satisfy themselves by subscribing to CSI and cross-checking one against the other, for a period – others are satisfied, based on their experience, that they can successfully trade, the way they trade, by using Yahoo data alone while another group won’t touch it at all.

    In future posts I ask the question, “What is the best independent, or relative, measure we can apply to data to qualify it?”. I don’t expect to reach a perfect conclusion but at least I will make a start – “The beginning of a good answer is a good question”. In some ways I am just setting the stage for that enquiry – it requires some preparation first (It’s the only content in the ‘Yahoo series’ that might be of interest to traders – the ‘series’ is mainly for newcomers).
    FTR beyond the basic training material my ‘style’ is built around starting discussion and enquiry – it is up to others to take it further if they want.
    Nothing is free – time is money (personally I only regard money as potential energy!).