Scrape

Scraping is the process of using automated tools to collect large amounts of data output from an application, website or application programming interface (API). The most common tool used for scraping is bots. These bots extract HTML code and other data stored inside databases.

Data scraping is used for both legitimate purposes, as well as with illegal, nefarious intentions. Digital businesses may use scraping to harvest data for legal purposes, such as auto-fetching prices for price comparison sites or to perform market research from forums and social media platforms. On the other hand, malicious actors may use scraping to steal copyrighted content or undercut unlisted competitor prices.

How scraping works

Legitimate scraping typically uses pre-built bots, scripts, or scraping-as-a-service providers. 

Malicious parties often create their own scripts for data scraping that don’t abide by certain restrictions, such as disguising themselves as real users. 

The following are the typical steps in the malicious scraping process:

  1. Identify the target website or application.
  2. Malicious actors will then limit the possibility of detection by creating fake user accounts and obfuscate source IP addresses.
  3. Bots are deployed across the resource. In addition to scraping, these illegal bots use can sometimes overload servers, leading to slow website performance and possibly crashing it entirely.
  4. Finally, content and database information is extracted and stored in the actor’s own database.

Types of data scraping

There are three primary types of data scraping.

Content scraping

Content scraping refers to when bots scrape the content present on a website. This information can then be replicated to mirror the unique advantages of products and services that rely on site content.

Price scraping

Price scraping using bots to pull data on prices. This can be used for legitimate purposes for comparison sites but can also be used to undercut competitor prices or create unique advantages over pricing plans.

Contact scraping

Contact scraping pulls user data, such as email addresses and phone numbers. Spammers and scammers use this information for bulk email lists, robocalls and social engineering attacks.

Data scraping vs. data crawling

Crawling

Crawling is used to index content. The most common example of this is Google using Googlebots to crawl website content to inform search engine results. Crawler bots make no attempt to hide their identity when crawling sites.

Scraping

Scraping specifically pulls data and stores it in other databases. Scraper bots typically hide their identity by pretending to be web browsers or users. They take more advanced actions than crawler bots, such as filling out form fields.

How to protect against data scraping

The following steps can be used to protect against data scraping:

  1. Monitor new and existing user accounts with high levels of activity but who haven’t made any purchases.
  2. Look for unusually high traffic to particular assets.
  3. Look at competitors for signs of price and catalog matching.
  4. Use software tools that use behavioral analysis to identify malicious activity in identifying bad bots.
Kyle Guercio
Kyle Guercio has worked in content creation for six years contributing blog posts, featured news articles, press releases, white papers and more for a wide variety of subjects in the technology space.

Top Articles

The Complete List of 1500+ Common Text Abbreviations & Acronyms

Text Abbreviations reviewed by Web Webster   From A3 to ZZZ we list 1,559 SMS, online chat, and text abbreviations to help you translate and understand...

Windows Operating System History & Versions

The Windows operating system (Windows OS) refers to a family of operating systems developed by Microsoft Corporation. We look at the history of Windows...

How to Create a Website Shortcut on Your Desktop

Website Shortcut on Your Desktop reviewed by Web Webster   This Webopedia guide will show you how to create a website shortcut on your desktop using...

Generations of Computers (1st to 5th)

Reviewed by Web Webster Learn about each of the 5 generations of computers and major technology developments that have led to the computing devices that...

Telecommunication

Telecommunication refers to telephony and cellular network technology. However, the broader definition includes...

Spoofing

What is spoofing? As it pertains to cybersecurity, spoofing is when a person disguises...

How to Indent in...

Microsoft Word is a graphical word...