How to Gather Data Using a Web Crawler: An Application Using SAS to Search Edgar
14 Pages Posted: 9 Oct 2007
Date Written: 2007
Empirical researchers in economics, finance and accounting grapple with a fundamental problem of gathering a representative sample of firms to test various theories that abound in the literature. Several well established data vendors provide a wealth of information toward this endeavor. However, data providers only provide a portion of information available. In this paper we provide a simple SAS program that can search for particular phrases in any Form filed by a registrant with the SEC. This allows researchers to "crawl" the web and access a large trove of data disclosed by managers in their public filings.
Keywords: Data gathering, Web crawling
JEL Classification: G00, M40, M41, M46
Suggested Citation: Suggested Citation