Copyright 2014 Altova GmbH
Licensed under the Apache License, Version 2.0 (the 'License');you may not use this file except in compliance with the License.You may obtain a copy of the License at
There are numerous ways to download multiple Company files from the SEC web site, although it is not something they make simple to select specific companies. Probably the simplest is from the current and historical RSS feeds, accessible from XBRL.SEC.GOV, from which you can access multiple zip files based on filing date. The Financial Statement Data Sets below provide numeric information from the face financials of all financial statements. This data is extracted from exhibits to corporate financial reports filed with the Commission using eXtensible Business Reporting Language (XBRL). Mar 11, 2014 Join Alexander Falk, CEO of Altova as he demonstrates how you can pull data direct from the SEC database, process it and perform financial analysis. This one hour webinar will spark ideas that you.
Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an 'AS IS' BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.
XBRL.US Webinar: How to download and process SEC XBRL Data Directly from EDGAR
These are the supporting Python files for the XBRL.US Webinar that is availbleon YouTube: https://www.youtube.com/watch?v=2Oe9ZqXVGME as well as the slidesavailable here on SlideShare: http://www.slideshare.net/afalk42/xbrl-us-altova-webinar
See also my recent blog post: http://www.xmlaficionado.com/2014/09/how-to-download-and-process-sec-xbrl.html
Please watch the YouTube video and review the slides to see how these Pythonscripts are intended to be used. Also note that these scripts were written withPython 3.3.3 so they may require modifications if you use them with a differentversion of Python.
To use this approach you will need to download and install RaptorXML+XBRL Server fromthe Altova website: http://www.altova.com/download-trial-server.html and thenrequest a 30-day free evaluation license key.
These sample Python scripts available here on GitHub were tested with a the MacOSversion of RaptorXML+XBRL Server, but should function with the Windows or Linux versionas well. You may need to change the file-paths pointing to the RaptorXML+XBRL Serverexecutable in the Python script, though, or add them to the global PATH environmentvariable on your system.
In addition to the standard Python libraries, you also need to install the Pythonfeedparser module/library available here: https://pypi.python.org/pypi/feedparser
For more information on RaptorXML, please see here: http://www.altova.com/raptorxml.html
Usage Information
Sec Xbrl Viewer
These scripts now require RaptorXML+XBRL v2015r3 or newer.
loadSECfilings
These creates a subdirectory sec/ and then subsequent year-based directories and monthsunderneath and downloads all SES XBRL filings from the EDGAR system to your local harddisk for further processing. Please use only during off-peak hours in order to notoverload the SEC servers. This downloads the ZIPped XBRL filings, so you'll have oneZIP file per filing submitted to the SEC on your drive. If you call this scriptagain for the current or any previous month at a later day, it will only downloadany files that are new and have not yet been downloaded before.
Examples
This will load all SEC filing for September 2014.
This will load all SEC filing for the start of the XBRL pilot program in 2005 until 2014.WARNING: If you download all years available (2005-2014) this will be about 127,000 filesand take about 18GB of data on your hard disk, so please use with caution, especiallywhen you are on a slow Internet connection.
![How to download xbrl data from sec How to download xbrl data from sec](/uploads/1/2/6/1/126123732/545486204.png)
valSECfilings
This will call RaptorXML+XBRL Server to validate the SEC filings for a specified yearand month or for a range of years. It assumes that the files have been downloaded bythe script above into a local sub-directory sec/. You can restrict the filings to justthose for a particular company or for a list of companies by providing their respectiveCIKs or ticker symbols. Optionally you can pass a Python script to RaptorXML+XBRL Serverwith the -s parameter, which will then be executed by the built-in Python interpreterinside of RaptorXML+XBRL Server to perform additional post-validation processing ofthe XBRL files. As an example, there is a Python script extractRatios.py in this projectthat demonstrates how to extract common financial ratios (quick ratio, cash ratio) fromthe XBRL filings.
Examples
This will validate all downloaded SEC filings for the month of September 2014. If a largenumber of files is passed to the Python script, it will create batches of about 20 jobseach and pass those to RaptorXML+XBRL Server in sequential batches.
This will validate all SEC filings submitted by Apple, Microsoft, and Oracle for theyears 2013 and 2014. Positive validation messages as well as any errors or warningsare output to the console window.
This will validate all Oracle XBRL filings for the years 2013-2014 and then performpost-validation analysis of the filings using the supplied Python script extractRatios.pythat gets passed to RaptorXML+XBRL Server and executed by its built-in Python interpreter.This particular example script prints document and entity information and then extractsvarious balance sheet facts to calculate current ratio, quick ratio, and cash ratio asand example of how to do post-validation XBRL processing. Furthermore, it appends thoseratios to an output file ratios.csv in the same directory.
Reminder
To see these scripts and a lot more in-depth explanation, please watch theYouTube video of the webinar here: https://www.youtube.com/watch?v=2Oe9ZqXVGME
Securities and Exchange Commission (SEC) EDGAR database. EDGAR containsregulatory filings from publicly-traded US corporations including their annualand quarterly reports:
All companies, foreign and domestic, are required to file registrationstatements, periodic reports, and other forms electronically through EDGAR.Anyone can access and download this information for free. [from the SECwebsite]
Human Interface
See http://www.sec.gov/edgar/searchedgar/companysearch.html Best hindi movies download torrent sites.
Bulk Data
Each company in EDGAR gets an identifier known as the CIK which is a 10 digitnumber. You can find the CIK by searching EDGAR using a name of stock marketticker. Sky high blu ray download torrent.
For example, searching for IBM by ticker shows us thatthe the CIK is
0000051143
.Note that leading zeroes are often omitted (e.g. in the ftp access) so thiswould become
51143
.Next each submission receives an 'Accession Number' (acc-no). For example,IBM's quarterly financial filing (form 10-Q) in October 2013 had accessionnumber:
0000051143-13-000007
.HTTPS File Paths
Given a company with CIK (company ID) XXX (omitting leading zeroes) anddocument accession number YYY (acc-no on search results) the path would be:
File paths are of the form:
For example, for the IBM data above it would be:
EDGAR has retired HTTP services. Instead use the HTTPS equivalent.
Note, if you are looking for a nice HTML version you can find it at in theArchives section with a similar URL (just add -index.html):
Indices
If you want to get a list of all filings you'll want to grab an Index. As the help page explains:
The EDGAR indices are a helpful resource for HTTPS retrieval, listing thefollowing information for each filing: Company Name, Form Type, CIK, DateFiled, and File Name (including folder path).
Four types of indexes are available:
- company — sorted by company name
- form — sorted by form type
- master — sorted by CIK number
- XBRL — list of submissions containing XBRL financial files, sorted by CIKnumber; these include Voluntary Filer Program submissions
URLs are like:
That is, they have the following general form:
So for XBRL in the 3rd quarter of 2010 we'd do:
![Sec Sec](/uploads/1/2/6/1/126123732/814969637.jpg)
CIK lists and lookup
There's a full list of all companies along with their CIK code here: http://www.sec.gov/edgar/NYU/cik.coleft.c
If you want to look up a CIK or company by its ticker you can do the following query against the normal search system:
Then parse the atom to grab the CIK. (If you prefer HTML output just omit output=atom).
There is also a full-text company name to CIK lookup here:
(Note this does a POST to a 'text' API at http://www.sec.gov/cgi-bin/cik.pl.c)
Parsing XBRL Data
See
scripts
and README file there.References
Xbrl Sec Filing
- CorpWatch have an excellent API and DB dump covering a lot of EDGAR info - see the CorpWatch DataHub Entry