Robots.Txt

Robots.txt is common name of a text file that is uploaded to a Web site’s root directory and linked in the html code of the Web site. The robots.txt file is used to provide instructions about the Web site to Web robots and spiders. Web authors can use robots.txt to keep cooperating Web robots from accessing all or parts of a Web site that you want to keep private.

Vangie Beal
Vangie Beal
Vangie Beal is a freelance business and technology writer covering Internet technologies and online business since the late '90s.

Top Articles

Huge List Of Texting and Online Chat Abbreviations

From A3 to ZZZ we list 1,559 text message and online chat abbreviations to help you translate and understand today's texting lingo. Includes Top...

How To Create A Desktop Shortcut To A Website

This Webopedia guide will show you how to create a desktop shortcut to a website using Firefox, Chrome or Internet Explorer (IE). Creating a desktop...

The History Of Windows Operating Systems

Microsoft Windows is a family of operating systems. We look at the history of Microsoft's Windows operating systems (Windows OS) from 1985 to present...

Hotmail [Outlook] Email Accounts

  By Vangie Beal Hotmail is one of the first public webmail services that can be accessed from any web browser. Prior to Hotmail and its...

Abacus Definition & Meaning

An abacus, also known as a counting frame, is a mechanical device used...

Legacy Code Definition &...

Legacy code refers to source code that has been inherited from a previous...

Unregulated Power Supply Definition...

An unregulated power supply is a system that transforms input voltage into direct...