Free for a Fee

The practice of search engine crawling and caching of Web site content has infrequently been litigated. (The Perfect 10 case is a significant exception.) This may be because most Web site operators want their content to be indexed and available on search engines. Those Web site operators that do not want their content copied and indexed can stop crawling and caching by deploying a robots.txt file. By generally accepted convention, a robots.txt file is consulted by most search engine crawlers for instructions from a Web site operator as to whether, and to what extent, Web site content may be copied and indexed. The inclusion of the “noindex” metatag in a robots.txt file instructs a crawler that the content may not be copied and indexed.

In Parker v. Yahoo!, Inc., 2008 U.S. Dist. LEXIS 74512 (E.D. Pa. Sep. 26, 2008), the district court held that a Web site operator’s failure to deploy a robots.txt file containing instructions not to copy and cache Web site content gave rise to an implied license to index that site. The court did say, however, that the implied license in the case may have, at some point, been terminated by the operator.

more

The content in this post was found at http://newmedialaw.proskauer.com/2008/10/articles/internet/parker-v-search-engines-part-ii-challenge-to-search-engine-caching-dismissed-on-most-but-not-all-grounds/ and was not authored by the moderators of freeforafee.com. Clicking the title link will take you to the source of the post.