Webopedia on Google+Webopedia on TwitterWebopedia on FacebookTech Bytes Blog
Main » TERM » S »

spider

A program that automatically fetches Web pages. Spiders are used to feed pages to search engines. It's called a spider because it crawls over the Web. Another term for these programs is webcrawler.

Because most Web pages contain links to other pages, a spider can start almost anywhere. As soon as it sees a link to another page, it goes off and fetches it. Large search engines, like Alta Vista, have many spiders working in parallel.

Also see How Web Search Engines Work in the Did You Know . . . ? section of Webopedia.







TECH RESOURCES FROM OUR PARTNERS
DID YOU KNOW?
Who's Moving Ahead in Cloud Computing?

The future remains, well, cloudy. But either way: Amazon, look out. Microsoft is gaining fast. Read More »

Hype Versus Action in the Developer's World

Often times technologies start as hype but with time become adopted. As a developer or technologist, it is worth reading the hype and knowing the... Read More »

Microsoft Hyper-V Network Virtualization Q&A

The top 5 Hyper-V questions with answers provided by Nirmal Sharma, a MCSEx3, MCITP and Microsoft MVP in Directory Services. Read More »

QUICK REFERENCE
Webopedia Polls

Webopedia Polls: Is iPhone 6 or iWatch on your list? Read More »

How to Create a Desktop Shortcut to a Website

This Webopedia guide will show you how to create a desktop shortcut to a website using Firefox, Chrome or Internet Explorer (IE). Read More »

Flash Data Storage Vendor Trends

Although it is almost impossible to keep up with the pace of ongoing product releases, here are three recent highlights in the flash data storage... Read More »