Question
This article answers:
- What is Web Scraping?
- Do I need Silverline Web Scraping Protection?
- What are Benefits of Silverline Web Scraping Protection (Shape Defense)?
- When should I use Web Scraping Protection?
This article describes
- negative effects of malicious bot traffic to your application
- benefits of enabling a mitigation service
- use cases to decide which endpoints to protect.
Environment
- Silverline Shape Defense
- Silverline Web Scraping Protection
- Bot defense
- Bot protection
- Anti-bot
Answer
- What is Web Scraping?
- What are the Negative Impacts of Malicious Web Scraping?
- What are the Benefits of Silverline Web Scraping Protection?
- When Should I Use Silverline Web Scraping Protection? Which Endpoints Should I Protect?
- How to Turn On Silverline Web Scraping Protection
What is Web Scraping?
OWASP (Open Web Application Security Project® ), a non-profit organization that works to improve the security of software, describes Scraping as collecting accessible data and/or processed output from the application.
- Scraping may use fake or compromised accounts, or the information may be accessible without authentication.
- The scraper may attempt to read all accessible paths and parameter values for web pages and APIs, collecting the responses and extracting data from them.
- Scraping may occur in real time or be more periodic in nature.
- Some Scraping may be used to gain insight into how it is constructed and operates - perhaps for cryptanalysis, reverse engineering, or session analysis.
- Diagram of Scraping:
Web Scraping is done by bots, software programs that run automated tasks.
- Good Bots (like Google or travel booking sites) aggregate data across sites to improve your user’s experiences and increase revenue.
- Bad Bots have malicious intent and can negatively affect the user experience by slowing or denying service, or impact revenue with malicious activity like card cracking.
More info on Bots:
- Q&A: What are bots?
- Q&A: Does Shape Defense allow Good Bots (like Google) to crawl pages?
- Q&A: What Are The Parameters That Shape Defense Uses To Identify A Bot?
- Good Bots, Bad Bots, and What You Can Do About Both (F5 Labs)
- Are Bots Skewing Your Business? (F5.com)
What are the Negative Impacts of Malicious Web Scraping?
The negative effects of malicious (non-sanctioned) Web Scraping:
- Negative User Experience
- Can impact your user’s ability to interact with your site
- Reduced Revenue
- Malicious schemes can siphon revenue
- Skewed Data
- Malicious traffic can skew data metrics
- Performance Degradation
- Increased traffic can slow site performance
- Compromised Assets
- User credentials can be compromised
- Intellectual property (IP) can be compromised
What are the Benefits of Silverline Web Scraping Protection?
- Improved Customer Experience
- Increased Site Profitability
- Clean Data
- Secure Assets (IP)
When Should I Use Silverline Web Scraping Protection? Which Endpoints Should I Protect?
Q&A: Which Endpoints to Protect with Silverline Web Scraping Protection (Shape Defense)?
How to Turn On Silverline Web Scraping Protection
You must be a Shape Defense Customer.
Silverline Web Scraping Protection must be turned on by the Silverline SOC: How to Contact the SOC
Related Articles
Silverline Web Scraping Protection
- Getting Started with Silverline Shape Defense Web Scraping
- How to Configure Silverline Shape Defense Web Scraping Protection
- How to Validate Silverline Shape Defense Web Scraping Protection
- How to Configure Shape Defense Allowlists via iRule Data Tables
- Does Silverline Web Scraping Protection (Shape Defense) support redirects?
Silverline Shape Defense
- Getting Started with Silverline Shape Defense (Onboarding)
- How to Configure Shape Defense (SSD)
- How to View Reports / Filter Metrics in Shape Defense Summary
Bots
- Q&A: What are bots?
- Q&A: Does Shape Defense allow Good Bots (like Google) to crawl pages?
- Q&A: What Are The Parameters That Shape Defense Uses To Identify A Bot?
- Good Bots, Bad Bots, and What You Can Do About Both (F5 Labs)
- Are Bots Skewing Your Business? (F5.com)